Technology name:

3D Reconstruction of Faces and Other Objects

Technology description: Scientists at the Weizmann Institute of Science have developed a novel approach to 3D shape reconstruction from single images (i.e., "image based modeling"). The underlying assumption behind this effort is the availability of shape and appearance information of one or more example 3D objects, all from a specific class, similar to the work of a sculptor.

Two new methods for single view 3D reconstruction were applied. The first method pieces together examples of feasible appearance-to-depth mappings, extracted from a pre-collected set of 3D objects, to produce reconstructions of novel objects belonging to a variety of classes (e.g., hands, human figures). On-the-fly example synthesis scheme is used to cope with unknown viewing conditions and large example sets. The flexibility of this method is demonstrated by producing depth estimates for a variety of objects from different classes, including estimates for the occluded back side of imaged objects. This method can also be used for the inverse problem of estimating novel colors for existing shapes (i.e., automatic object colorization). Visit this webpage for some examples and related publications.

The second method focuses on human faces, allowing detailed and fast 3D reconstruction from a single image and uses only a single reference model to guide the reconstruction process (in contrast to other approaches that use hundreds of stored example models). The underlying idea is to use shading information (the distribution of intensities in the image) to recover the 3D shape of face, while using the reference model to extract information that is essential for the recovery process, i.e., lighting, pose, rough albedo, and appropriate boundary conditions. This method was experimentally shown to produce accurate reconstructions of novel faces overcoming significant shape differences from the reference face due to gender, race, and facial expressions. It can potentially overcome some of the most critical problems in producing 3D models of unknown individuals. Visit this webpage for some examples related publications.

Each of these methods can be used independently and the benefits of the two methods can be combined to form a single shape from single image reconstruction pipeline.

Face recognition using accurate 3D reconstructions from single images.

Rapid modeling and prototyping - producing rough 3D models from images as a starting point for future refinement by artists.

Automatically producing novel appearances (texture maps) to massive numbers of stock 3D shapes.

Enhanced visualization of imaged objects, using their estimated 3D shape.

Technologuy holder: Yeda R&D Ltd @ The Weizmann Institute of Science
Contact: ruth.ben-yakar@weizmann.ac.il
Tag: technology transfer with human face
License Contact: Shay.Marcus@weizmann.ac.il
Info. URL: http://www.yedarnd.com/Technology.aspx?t=1406