Stereo SLAM - Politecnico di...
Transcript of Stereo SLAM - Politecnico di...
![Page 1: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/1.jpg)
Stereo SLAM
Davide Migliore, PhD [email protected]
Department of Electronics and Information, Politecnico di Milano, Italy
Monday, 15 June 2009
![Page 2: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/2.jpg)
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Do you remember the pin-hole camera?
2
Monday, 15 June 2009
![Page 3: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/3.jpg)
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
- Each camera has a P matrix
3
Monday, 15 June 2009
![Page 4: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/4.jpg)
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
4
Monday, 15 June 2009
![Page 5: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/5.jpg)
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
5
Monday, 15 June 2009
![Page 6: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/6.jpg)
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
6
Monday, 15 June 2009
![Page 7: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/7.jpg)
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Error modeling problem
7
Monday, 15 June 2009
![Page 8: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/8.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣The idea - Use the Unified Inverse Depth parametrization (Montiel et al.
2006)
- Rectify images and initialize the point using
8
Monday, 15 June 2009
![Page 9: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/9.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Measurement Equations
9
Monday, 15 June 2009
![Page 10: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/10.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Measurement Equations
10
Monday, 15 June 2009
![Page 11: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/11.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)11
Monday, 15 June 2009
![Page 13: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/13.jpg)
Slide n°
PhD Davide Migliore - [email protected]
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Monday, 15 June 2009
![Page 14: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/14.jpg)
Slide n°
PhD Davide Migliore - [email protected]
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
![Page 15: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/15.jpg)
Slide n°
PhD Davide Migliore - [email protected]
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
![Page 16: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/16.jpg)
Slide n°
PhD Davide Migliore - [email protected]
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Data Association
DA
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
![Page 17: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/17.jpg)
Slide n°
PhD Davide Migliore - [email protected]
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Data Association
DA
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
![Page 18: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/18.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Data Association Trouble
13
Monday, 15 June 2009
![Page 19: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/19.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Data Association Trouble
14
Monday, 15 June 2009
![Page 20: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/20.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Data Association Trouble
15
Monday, 15 June 2009
![Page 21: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/21.jpg)
Slide n°
Davide Migliore
Compatibility 16
Monday, 15 June 2009
![Page 22: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/22.jpg)
Slide n°
Davide Migliore
NN Data Association 17
Monday, 15 June 2009
![Page 23: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/23.jpg)
Slide n°
Davide Migliore
NN Data Association 18
Monday, 15 June 2009
![Page 24: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/24.jpg)
Slide n°
Davide Migliore
Joint Compatibility 19
Monday, 15 June 2009
![Page 25: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/25.jpg)
Slide n°
Davide Migliore
JCBB 20
Monday, 15 June 2009
![Page 26: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/26.jpg)
Slide n°
Davide Migliore
JCBB 21
Monday, 15 June 2009
![Page 27: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/27.jpg)
Slide n°
Davide Migliore
Demo Time‣Switch on Matlab
22
Monday, 15 June 2009
![Page 28: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/28.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Joint Compatibility Branch & Bound Results
23
Monday, 15 June 2009
![Page 29: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/29.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Results
24
Monday, 15 June 2009
![Page 30: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/30.jpg)
Slide n°
Davide Migliore
Scaling problem 25
Monday, 15 June 2009
![Page 31: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/31.jpg)
Slide n°
Davide Migliore
Scaling problem 26
O(n2)
Monday, 15 June 2009
![Page 32: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/32.jpg)
Slide n°
Davide Migliore
Solution: local maps 27
‣Switch to matlab again
Monday, 15 June 2009
![Page 33: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/33.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Results
28
Monday, 15 June 2009
![Page 34: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/34.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Results
29
Monday, 15 June 2009
![Page 35: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/35.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
30
Monday, 15 June 2009
![Page 36: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/36.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
30
Monday, 15 June 2009
![Page 37: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/37.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
31
Monday, 15 June 2009
![Page 38: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/38.jpg)
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
32
Monday, 15 June 2009
![Page 39: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/39.jpg)
Slide n°
Davide Migliore
Inverse Scaling?‣ Is it possible to use the inverse scaling?‣Yes
‣Results? Coming soon!!
33
Monday, 15 June 2009
![Page 41: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/41.jpg)
Slide n°
Davide Migliore PhD - [email protected]
Thanks for your attention34
Questions
Monday, 15 June 2009
![Page 42: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/42.jpg)
Omnidirectional SLAM
Davide Migliore, PhD [email protected]
Department of Electronics and Information, Politecnico di Milano, Italy
Monday, 15 June 2009
![Page 43: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/43.jpg)
Slide n°
Davide Migliore
What is an Omni Camera? Omnidirectional sensors come in many varieties, but
by definition must have a wide field-of-view.
~180º FOV
wide FOV dioptric cameras (e.g. fisheye)
~360º FOV
polydioptric cameras (e.g. multiple overlapping cameras)
>180º FOV
catadioptric cameras (e.g. cameras and mirror systems)
Monday, 15 June 2009
![Page 44: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/44.jpg)
Slide n°
Davide Migliore
(Poly-)Dioptric solutions
Pros: - High resolution
per viewing angle
Cons:- Bandwidth
- Multiple cameras
One to two fish-eye cameras or many synchornized cameras
Monday, 15 June 2009
![Page 45: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/45.jpg)
Slide n°
Davide Migliore
(Poly-)Dioptric solutionsOne to two fish-eye cameras or many synchornized
cameras
Homebrewed polydioptric cameras are cheaper, but require calibrating and synchronizing; commercial designs tend to be expensive
Monday, 15 June 2009
![Page 46: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/46.jpg)
Slide n°
Davide Migliore
Catadioptric solutionsUsually single camera combined with convex mirror
Cons:- Blind spots
- Low resolution
Pros: - Single image
Monday, 15 June 2009
![Page 47: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/47.jpg)
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
![Page 48: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/48.jpg)
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
![Page 49: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/49.jpg)
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
![Page 50: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/50.jpg)
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
![Page 51: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/51.jpg)
Slide n°
Davide Migliore
Catadioptric cameras
Camera Models
Monday, 15 June 2009
![Page 52: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/52.jpg)
Slide n°
Davide Migliore
Catadioptric cameras• mirror
Camera Models
Monday, 15 June 2009
![Page 53: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/53.jpg)
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
![Page 54: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/54.jpg)
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
![Page 55: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/55.jpg)
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
![Page 56: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/56.jpg)
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
![Page 57: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/57.jpg)
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
![Page 58: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/58.jpg)
Slide n°
Davide Migliore
Central catadioptric cameras
• mirror
• camera
Camera Models
Monday, 15 June 2009
![Page 59: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/59.jpg)
Slide n°
Davide Migliore
Central catadioptric cameras
• mirror
• camera
• single effective viewpoint
Camera Models
Monday, 15 June 2009
![Page 60: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/60.jpg)
Slide n°
Davide Migliore
Central catadioptric cameras
• mirror
• camera
• single effective viewpoint
(surface of revolution of a conic)
Camera Models
Monday, 15 June 2009
![Page 61: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/61.jpg)
Slide n°
Davide Migliore
F1
F2
Types of central catadioptric cameras 43
Monday, 15 June 2009
![Page 62: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/62.jpg)
Slide n°
Davide Migliore
• hyperbola + perspective camera
F1
F2
Types of central catadioptric cameras 43
Monday, 15 June 2009
![Page 63: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/63.jpg)
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
![Page 64: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/64.jpg)
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
![Page 65: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/65.jpg)
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
![Page 66: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/66.jpg)
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
• ...F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
![Page 67: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/67.jpg)
Slide n°
Davide Migliore
Other types of central cameras 44
Monday, 15 June 2009
![Page 68: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/68.jpg)
Slide n°
Davide Migliore
Other types of central cameras 44
Monday, 15 June 2009
![Page 69: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/69.jpg)
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Why do we need calibration? 45
Monday, 15 June 2009
![Page 70: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/70.jpg)
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
![Page 71: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/71.jpg)
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
![Page 72: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/72.jpg)
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
![Page 73: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/73.jpg)
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
![Page 74: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/74.jpg)
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
![Page 75: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/75.jpg)
Slide n°
Davide Migliore
u
v
X Y
Z
What?
Monday, 15 June 2009
![Page 76: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/76.jpg)
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image
What?
Monday, 15 June 2009
![Page 77: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/77.jpg)
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image • Camera focal length
Focal length
What?
Monday, 15 June 2009
![Page 78: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/78.jpg)
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image • Camera focal length• Orientation and position between camera & mirror
Focal length
R, T
What?
Monday, 15 June 2009
![Page 79: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/79.jpg)
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image • Camera focal length• Orientation and position between camera & mirror• Mirror shape
Focal length
R, T
What?
Monday, 15 June 2009
![Page 80: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/80.jpg)
Slide n°
Davide Migliore
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
![Page 81: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/81.jpg)
Slide n°
Davide Migliore
1. Mirror and camera axes are aligned =>
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
![Page 82: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/82.jpg)
Slide n°
Davide Migliore
1. Mirror and camera axes are aligned =>
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
![Page 83: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/83.jpg)
Slide n°
Davide Migliore
1. Mirror and camera axes are aligned =>
2. x-y mirror axes coincide with u-v camera axes =>
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
![Page 84: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/84.jpg)
Slide n°
Davide Migliore
Reflected rays do not intersect in a point but are tangent to a “caustic”
And how about non-central cameras?
Monday, 15 June 2009
![Page 85: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/85.jpg)
Slide n°
Davide Migliore
Reflected rays do not intersect in a point but are tangent to a “caustic”
And how about non-central cameras?
Monday, 15 June 2009
![Page 86: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/86.jpg)
Slide n°
Davide Migliore
Visual Odometry (Scaramuzza et al. 2009)49
Monday, 15 June 2009
![Page 87: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/87.jpg)
Slide n°
Davide Migliore
Omni SFM (Lhuillier et al. 2008)50
Monday, 15 June 2009
![Page 88: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/88.jpg)
Slide n°
Davide Migliore
Omni SFM (Lhuillier et al. 2008)51
Monday, 15 June 2009
![Page 89: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/89.jpg)
Slide n°
Davide Migliore
Omni SFM (Lhuillier et al. 2008)52
Monday, 15 June 2009
![Page 91: Stereo SLAM - Politecnico di Milanohome.deib.polimi.it/matteucc/lectures/3DSFVM/StereoOmniSLAM.pdf · Slide n° Davide Migliore What is an Omni Camera? Omnidirectional sensors come](https://reader034.fdocument.pub/reader034/viewer/2022050310/5f722bf78ca2a9428a05d4b3/html5/thumbnails/91.jpg)
Slide n°
Davide Migliore PhD - [email protected]
Thanks for your attention53
Questions
Monday, 15 June 2009