A Multilinear (Tensor) Framework for Computer Vision and Graphics

Abstract

Natural images are the consequence of multiple factors related to scene structure, illumination, and imaging. Multilinear algebra, the algebra of higher-order tensors, offers a potent mathematical framework for explicitly dealing with the multifactor nature of image formation. I will present a multilinear model that computes (nonlinear) manifold representations of image ensembles in which the multiple constituent factors (or modes) are disentangled and analyzed
explicitly. This nonlinear model is computed via a tensor decomposition, known as the N-mode SVD, which is an extension to tensors of the conventional matrix singular value decomposition (SVD). I will demonstrate the potency of our model in the context of facial image ensembles, where the relevant factors include different facial geometries, expressions, lighting conditions, and viewpoints. When applied to the difficult problem of automated face recognition, our
multilinear representation, called TensorFaces, yields significantly improved recognition rates relative to eigenfaces, the standard, linear, principle components analysis (PCA) approach. Our multilinear framework is also valuable in the context of image synthesis in computer graphics. I will present TensorTextures, a novel, multilinear approach to image-based rendering. Given a sparse set of sample images of a texture, TensorTextures learns the interaction between viewpoint, illumination, and geometry that determines surface appearance, including complex details due to surface mesostructure, such as self-occlusion and self-shadowing. The TensorTextures algorithm provides a parsimonious, explicitly multi-factor approximation to the bidirectional texture function (BTF). Time permitting, I will also summarize the application of tensor analysis to human motion capture data, leading to decompositions that we call "human motion signatures", which are useful in human action recognition and graphical animation synthesis.

Speaker Biography

M. Alex O. Vasilescu (www.mrl.nyu.edu/~maov) was educated at the Massachusetts Institute of Technology (MIT) and the University of Toronto, where she is currently a PhD candidate under the supervision of Prof. Demetri Terzopoulos. She is also a research scientist in the Computer Science Department of New York University's Courant Institute of Mathematical Sciences. She has done research at the MIT Artificial Intelligence Lab and has interned at research centers of IBM, Intel, Compaq, and Schlumberger corporations. She has published research papers in computer vision and computer graphics, particularly in the areas of face recognition, human motion analysis/synthesis, image-based rendering, and physics-based modeling (deformable models). She has given several invited talks about her work and has three patents pending. Her face recognition research, known as TensorFaces, is being funded by the TSWG, the Department of Defense's Combating Terrorism Support Program. She has been named by MIT's Technology Review Magazine to their 2003 TR100 List of Top Young Innovators.