A stereo camera pair estimates depth by finding the same scene point in two images and measuring its horizontal shift, or disparity. For rectified cameras, depth is approximately focal length times camera baseline divided by disparity. That relationship made stereo reconstruction appealing in the 2000s: ordinary digital cameras could produce dense 3D estimates. But a one-pixel matching error has a much larger effect on a distant surface, where disparity is small, than on a nearby one.
Researchers tackled the problem at several stages: camera calibration, correspondence matching, surface construction and identifying results that could not be trusted. Systems included passive camera pairs, moving cameras and projector-assisted rigs. The distinctions mattered because a convincing 3D rendering could hide substantial measurement errors.
From two images to measured depth
Calibration estimates each camera’s lens properties and the cameras’ relative positions. Rectification then warps the images so corresponding points lie along the same image row. A matcher can search one line instead of an entire image. Calibration errors can put a point on the wrong row; poor synchronization can leave a moving object in different positions in the two frames.
Once correspondences are found, triangulation recovers 3D coordinates. Camera separation, or baseline, involves a trade-off. A wider baseline can improve depth precision at a distance, but the views become less alike, and more points appear in only one camera. Narrow-baseline pairs are easier to match, yet their small disparities make distant depth less certain. Lens distortion, exposure differences and errors in the estimated camera positions add uncertainty before any surface is built.
How correspondence methods changed
Local matching and dense disparity
A common method compared small neighborhoods of pixels across rectified images. Sum-of-absolute-differences and related costs let systems test many candidate disparities efficiently. Window size brought a familiar compromise: a large window gathered more evidence on a plain surface but blurred depth boundaries; a small one preserved boundaries but was more vulnerable to noise and repeated patterns. Image pyramids and coarse-to-fine searches helped handle larger shifts without testing every possibility at full resolution.
Other methods treated the entire disparity map as an optimization problem. Smoothness terms favored similar disparities at neighboring pixels while allowing jumps at object edges. Dynamic programming, graph cuts and belief propagation were prominent research tools during the decade, with different computational costs and assumptions about a scene. They could fill weakly textured regions, though a smooth-looking answer was not necessarily a measured one. A featureless wall offers little direct evidence of its exact depth.
Some systems did not seek depth at every pixel. Sparse stereo matched distinctive corners or features tolerant of changes in scale and rotation, then triangulated those points. It was useful for camera pose estimation and mapping, particularly when a moving camera supplied multiple views. The right output depended on the task: a robot might need dependable obstacle positions and landmarks, while surface modeling needed enough coverage to describe the shape between them.
Occlusion and the checks that mattered
At an object boundary, one camera may see background that the other cannot. Those pixels have no correct match. Repeated textures, reflective objects, transparent materials and low-light images cause other ambiguities. Practical pipelines used checks such as:
- Left–right consistency: compare matches computed in both directions and flag disagreements.
- Confidence or uniqueness tests: reject a match when several candidate locations score similarly.
- Subpixel refinement: estimate disparities between pixel positions when the matching cost supports it.
- Post-processing: remove isolated matches or fill small gaps while distinguishing inferred values from observations.
These checks reduced visible artifacts, but they could not reveal a surface hidden from view. A disparity map is best read alongside its invalid-pixel mask, not just as a color-coded picture.
Beyond a fixed passive camera pair
Multi-view stereo combined images taken from more than two camera positions. Extra views could reveal surfaces hidden in one pair and provide independent evidence for a proposed 3D point. They also brought more calibration, image registration and computation. With moving cameras, pose had to be estimated as well; pose errors could bend or duplicate reconstructed surfaces. Bundle adjustment refined camera parameters and 3D point positions together by reducing image reprojection errors, although a low reprojection error did not guarantee complete geometry.
Structured-light systems changed the matching task by projecting a known pattern onto a surface. A camera could match observed pattern elements to their projected locations instead of relying entirely on natural texture. Controlled lighting helped with otherwise blank objects, but shiny surfaces, ambient light and movement could disrupt the pattern. The need for a projector also made structured light less suitable than passive stereo for many outdoor or long-range settings. It belongs in the history of stereo reconstruction because projector and camera form a triangulation pair, even though only the camera records an image.
What a reconstructed surface actually contained
A disparity map, a point cloud and a polygon mesh represent different stages of the result. The map holds image-aligned estimates; triangulation turns valid estimates into 3D points; meshing connects those points into a surface. Removing outliers and combining overlapping views could improve a model, but meshing might bridge a real hole or hide an occluded region. A nearby obstacle’s position could matter more to a robot than an attractive mesh. Cultural-heritage recording and industrial inspection placed greater weight on surface coverage and dimensional accuracy.
Benchmarks made methods easier to compare, especially on rectified image pairs with reference disparities. A headline error rate still depended on which pixels were counted, how occlusions were handled and how closely the test scenes resembled the intended application. Runtime figures needed context too: image size, disparity range, processor and whether calibration or refinement time was included. A strong laboratory ranking did not automatically make a method suitable for a mobile robot or a real-time camera.
When examining a reported result from the 2000s, pick an object boundary in the input photographs and follow it through the disparity map and invalid-pixel mask. If one view loses sight of the background behind that object, a gap in the depth estimate may be the geometrically honest result. A smooth mesh can erase the gap without adding any evidence for the surface it draws.