3-D Depth Reconstruction from a Single Still Image

何開暘 2010.6.11

Visual Cues for Depth Perception

Monocular Cues Texture variations, texture gradients, interposition, occlusion,

known object sizes, light and shading, haze, defocus

Stereo Cues Motion Parallax and Focus Cues

image → feature → depth

Chose features that capture 3 types of cues: texture variations, texture gradients, and color

Model conditional distribution of depths given monocular image features p(d|x)

Estimate parameters by maximizing conditional log likelihood of training data

Given an image, find MAP estimate of depths

Outline

Introduction Feature Vector Probabilistic Model Experiments Reference

Feature vectors

Two types of features Absolute depth features―used to estimate absolute depth at a p

articular patch Relative features―used to estimate relative depths

Capture three types of cues Texture variation―apply Law’s masks to intensity channel Haze―apply a local averaging filter to color channels Texture gradient―apply six oriented edge filters to intensity cha

nnel

Features for Absolute Depth

Compute summary statistics of a patch i in the image I(x,y) as follows Use the output of each of the 17 (9 Law’s masks, 2 color channe

ls and 6 texture gradients) filters Fn, n=1,…,17 as:

(dimension 34)

To estimate absolute depth at a patch, local image features centered on the patch are insufficient

Use more global properties

More Global Properties

Use image features extracted at multiple spatial scales (three scale)

Features used to predict depth of a particular patch are computed from that patch as well as 4 neighboring patches (Repeated at each of the three scales)

Add to features of a patch additional summary features of the column it lies in

(5*3+4)*34=636 dimensional

Features for Relative Depth

To learn the dependencies between two neighboring patches

Compute a 10-bin histogram of each of the 17 filter outputs , giving a total of 170 features yis for each patch i at scale s

Relative depth features yijs for two neighboring patches i and j at scale s will be the differences between their histogram, i.e., yijs=yis-yjs

Outline

Introduction Feature Vector Probabilistic Model Experiments Reference

Gaussian Model

Laplacian Model

Outline

Introduction Feature Vector Probabilistic Model Experiments Reference

Result

Improving Performance of Stereovision using Monocular Cues

The average errors as a function of the distance from the camera

Reference

A.Y. Ng A. Saxena, S.H. Chung. 3-d depth reconstruction from a single still image. In International Journal of Computer Vision (IJCV), 2007.

Michels, J., Saxena, A., & Ng, A. Y. (2005). High speed obstacle avoidance using monocular vision and reinforcement learning. In 22nd international conference on machine learning (ICML).