3D Computer Vision Winter Term 2005/06

Administrative Info Lecture by Prof. Nassir Navab Exercises by Martin Groher, Daniel Pustka, Marco Feuerstein 4+2 SWS, 8 ECTS, Theoretische Informatik, Wahlpflichtfach

Time & Location Tuesday 10:30 - 12:00 MI 00.13.009A Thursday 10:30 - 12:00 MI 00.13.009A Exercises: Thursday 13:00 - 14:30 MI 00.13.009A First Lecture: 18th Oct 2005

Site Content Overview Content Course Schedule Exercises Readings Lecture Script Draft Programming Assignment

Announcements Lecture Script Draft Online You do NOT have to register for the first intermediate test! Your participation is the registration Chapter on Singular Value Decomposition added to script draft Demo Session about Marker Tracking & Error Visualization on Thu, 24th Nov 10.30am Exam: 9th of February. Please be there at 10.00am already!! You may bring only the slides, no books, laptops, etc. are allowed! The exam will be one part ''open book'' (with slides), one part closed book (without slides) The deadline for the programming assignment is postponed to February 24th midnight! The results of the exam including programming assignment and intermediate exams have been launched

Overview

Making a computer see was something that leading experts in the field of Artificial Intelligence thought to be at the level of difficulty of a summer student's project back in the sixties. Forty years later the task is still unsolved and seems formidable. A whole field, called Computer Vision, has emerged as a discipline in itself with strong connections to mathematics and computer science and looser connections to physics, the psychology of perception and the neuro sciences.

Over the past decade there has been a rapid development in the understanding and modelling of the geometry of multiple views in computer vision. The theory and practice have now reached a level of maturity where excellent results can be achieved for problems that were unsolved a decade ago, and often thought unsolvable. These tasks and algorithms include problems like:
Given two/three/multiple images, and no further information, compute/estimate:

• matches between the images
• the 3D position of the points that generate these matches
• the cameras that generate the images

adapted form Hartley & Zisserman's "Multiple View Geometry in Computer Vision"

This tasks and algorithms as well as the methods which allow us to reason about the quality of our results are the core of the lecture 3D Computer Vision.

Content

1. Intro, motivation & Overview
2. 2D Transformations
3. Projective 2D Geometry
4. 3D Transformations
5. Projective 3D Geometry
6. Parameter Estimation
7. Camera Models
8. Camera Calibration
9. Single View Geometry
10. Stereo Vision I – Image Rectification & Disparity maps
11. Linear Algebra I - Singular Value Decomposition
12. 3D reconstruction I: Factorization Method
13. Epipolar Geometry : Essential & Fundamental Matrices
14. Stereo Vision II – 3D Reconstruction
15. Planes & Homographies
16. Basics of Image Mosaicing
17. Representation and Motion of Lines & Cylinders
18. Three View Geometry of Lines & Cylinders
19. The Trifocal Tensor
20. 3D reconstruction II: Multi-view Reconstruction
21. 3D reconstruction III: Motion and Structure from Motion
22. Mixed Camera Models (Perspective and Orthographic)
23. Combined Othrographic and Perspective (COP) Images: Calibration & Reconstruction
24. 3D reconstruction IV: Fusion of 3D Maps (ICP)
25. Image-based Rendering I
26. Image-based Rendering II
27. Conclusion & Discussions

Course Schedule

Date	Topic	Material
October 18, 2005	Intro & Motivation	Slides
October 20, 2005	Transformations & Projective Geometry Lecture continues in Exercises	Slides 1 , Slides 2
October 25, 2005	Transformations & Projective Geometry	Slides 3
October 27, 2005	Exercises instead of lecture Basic math, Introduction to MatLAB, Hom. Coordinates	Exercise Sheet 1 Solution 1 Reminder on Products in Linear Algebra MatLAB demo script
November 1, 2005	No Lecture	Allerheiligen
November 3, 2005	1st intermediate test	Results of the 1st intermediate test
November 8, 2005	Linear Parameter Estimation, DLT	Slides 4
November 10, 2005	Parameter Estimation: Statistical error models	(continued)
November 15, 2005	Parameter Estimation: Sampson Error, normalization	(continued)
November 17, 2005	Parameter Estimation: Using homography estimation for Tracking (SelimBenhimane)	Slides 7
November 22, 2005	Error Analysis	Slides 8
November 24, 2005	Siemens Field Trip For those not going to Siemens: Demos on marker tracking and error propagation	Marker Tracking Visualize Pose Error
November 29, 2005	Non-linear Parameter Estimation	Probability Reminder Slides 9
December 1, 2005	No Lecture	Dies academicus
December 6, 2005	2nd Intermediate Test	Results of the 2nd intermediate test
December 8, 2005	Error Analysis	see Slides 8
December 13, 2005	Camera Models I	Slides 10
December 15, 2005	Camera Models II	see Slides 10
December 20, 2005	Camera Models III	see Slides 10
December 22, 2005	Computation of the Projection Matrix	Slides 11
December 27, 2005	No Lecture	Christmas Holidays
December 29, 2005	No Lecture	Christmas Holidays
January 3, 2006	No Lecture	Christmas Holidays
January 5, 2006	No Lecture	Christmas Holidays
January 10, 2006	Epipolar Geometry I	Slides 12
January 12, 2006	Epipolar Geometry II	Slides 13
January 17, 2006	Epipolar Geometry III	Slides 14
January 19, 2006	Epipolar Geometry IV	Slides 15
January 24, 2006	Demo session: Introduction to teaching/research events of the Chair for Computer Aided Medical Procedures and Augmented Reality
January 26, 2006	Invited Talk Dr. Mirko Appel, Siemens Corporate Technology: Co-registered Orthographic and Perspective (COP) Images
January 31, 2006	Invited Talk Dr. Vincent Lepetit, Ecole Polytechnique Federal de Lausanne (EPFL): 3D Object Tracking and Detection for Augmented Reality	Slides
February 2, 2006	Invited Talk Dr. Adrien Bartoli, LASMEA Laboratory, France	Slides
February 7, 2006	The Trifocal Tensor	Slides 16
February 9, 2006	Final Exam	Results

Exercises

Date	Topic	Material
October 20, 2005	Transformations & Projective Geometry Lecture, NO Exercises
October 27, 2005	Transformations in Projective Space	Exercise Sheet 2 Solution 2 Reminder on Conics and Quadrics MatLAB Ex 6b Conics in Maple Quadrics in Maple
November 3, 2005	Field Trip to BrainLab
November 10, 2005	Multivariate Calculus Reminder, Homography computation, Pluecker lines	Reminder on Multivariate Calculus Exercise Sheet 3 Solution 3
November 17, 2005	SVD, Linear Parameter Estimation, Errors	Reminder on SVD Exercise Sheet 4 Solution 4
November 24, 2005	Siemens Field Trip
December 1, 2005	No Exercises	Dies Academicus
December 8, 2005	Mosaicing, non-linear parameter estimation, error propagation	Exercise 5 Solution 5 Jacobian of Reprojection
December 15, 2005	Error Propagation	Exercise 6 Solution 6
December 22, 2005	Camera Models and Projection Matrices	Exercise 7 Solution 7 MingPaper
December 29, 2005	No Exercises	Christmas Holidays
January 3, 2006	No Exercises	Christmas Holidays
January 12, 2006	Camera Calibration Algorithms	Exercise 8 Zhang Paper
January 19, 2006	Self-calibration of the distortion of a zooming camera	Exercise 9 Solution 9
January 26, 2006
February 2, 2006
February 9, 2006

Readings

• Primary Reading
  • Multiple View Geometry in Computer Vision by Richard Hartley & Andrew Zisserman

• General Introduction to 3D Computer Vision
  • Three-Dimensional Computer Vision by Olivier Faugeras
  • Computer Vision: A Modern Approach by David A. Forsyth & Jean Ponce
  • Introductory Techniques for 3-D Computer Vision by Emanuele Trucco & Alessandro Verri

• More Specific Readings
  • The Geometry of Multiple Images: The Laws That Govern the Formation of Multiple Images of a Scene and Some of Their Applications by Olivier Faugeras, Quang-Tuan Luong, Theodore H. Papadopoullos; MIT Press; 2001

Lecture Script Draft

In the following we present some outtakes from a lecture script draft we are currently working on. This is still work in progress, meaning that it is incomplete and error prone. Specifically it does not cover the whole material presented in the lecture.

So far, we've been trying to concetrate on the mathematical basics needed for the lecture and some basic concepts.

We hope the stuff is useful to some extent. Check here regularly for new versions!

We are grateful for all comments! Please contact Martin () or Darko ().

Topic	script draft
Short geometrically motivated introduction to 2D Projective Space	3DCV_projective_geometry_000.pdf
math basics reminder: multivariate calculus	3DCV_reminder_calculus_000.pdf
math basics reminder: conics and quadrics (more on conics still, however)	3DCV_reminder_conics_000.pdf
math basics reminder: singular value decomposition	3DCV_svd_000.pdf
math basics reminder: statistics, probability	3DCV_reminder_statistics_000.pdf
math basics reminder: maximum likelihood estimation	3DCV_reminder_mle_000.pdf

Programming Assignment

• Text
  • Overview slide
  • General remarks
  • Detailed description
  • Zhang paper
• Code
  • corner_detector.m (with courtesy of Marc Pollefeys)
  • levmar.m
  • for the lazy bones: hcross.m (cross product of 2 hom. points), hline.m (draw a line given in hom. coordinates), plotPoints.m (plot points in an image)

• Pix
  • here are some test images. some are more suitable than others. but all quite boring. so feel encouraged to make your own images. the cooler the better...