Computer-generated images after 3D depth-map extraction from 2D face images. Joseph and Harrison's more accurate method is shown on the right.
Edmonton—An ECE professor and his graduate student have developed a method to more accurately estimate the 3D depth of objects presented in 2D images — a process they have demonstrated with face images from a Yale University database.
Dr. Dileepan Joseph and PhD student Adam Harrison recognized the limitations of existing solutions for mapping the depth of 2D images while working on their virtual reflected-light microscopy (VRLM) system, and saw an opportunity to develop a better method for computers to extract 3D data from flat images.
While humans are able to easily separate illumination from the shape and texture of an object, computers can have a difficult time doing this from 2D images. Existing methods for computer vision depth mapping use a series of images of an object taken from the same viewpoint, but illuminated from different directions. This reveals surface geometry and reflective properties. However, image noise — an unavoidable phenomenon in digital imaging — can cause computers to poorly estimate depth.
“There are non-idealities in the imaging process, such as imperfections of digital image sensors, which leads to uncertainty in models for estimating depth,” Joseph explained. “One of the biggest problems is ‘noise’ in the image, whether perceptible or not. Other literature recognized the problem of noise, but didn’t address it correctly.
“We found a way to statistically model the uncertainty, so we estimate the shape most likely given a specific set of assumptions built into our algorithm.”
By assuming random noise and modeling how known images are generated from known illumination interacting with unknown shape and texture, the researchers worked backward mathematically to derive the most likely shape, or depth map, and texture. What they found is that their new method is insensitive to image noise and produces a more realistic end result than previous methods. Better yet, it’s just as efficient as the methods it outperforms.
While the new process helped the duo improve their VRLM system, improved 3D depth mapping has various other applications and their research was recently published in the July 2012 issue of the prestigious journal IEEE Transactions on Pattern Analysis and Machine Intelligence. Facial recognition processes and other artificial intelligence applications can benefit from their findings.
While the new method has a promising future, Joseph is quick to point out that there’s still work to do.
“We were inspired by an applied problem [VLRM] to solve a science problem. We answered important questions in computer vision that had dogged people, but this brought new ones into focus.”
He specifically cited that cast shadows — shadows on the object cast by its very features — still pose problems, though they do not cause the method to fail. Specular, or shiny, surfaces can also trick the method into interpreting depth improperly. Lastly, the researchers would like to produce accurate models using fewer 2D images, which would further improve the system’s applicability.