Dynamic 3D reconstruction from stereo videos
Author(s)
Jing, Junpeng
Type
Thesis or dissertation
Abstract
We live in a dynamic world where objects and scenes continually change over time. Enabling machines to perceive and interpret such changes is the goal of dynamic 3D reconstruction, which seeks to recover scene structure and motion from visual data for applications in robotics, autonomous driving, and augmented reality.
This thesis studies dynamic 3D reconstruction from stereo videos, with a focus on estimating temporally consistent disparity from synchronized camera pairs. Stereo vision provides a geometrically grounded solution, yet most deep stereo methods are designed for static images and often produce temporally inconsistent predictions when applied to videos. Combined with the scarcity of high-quality stereo video datasets, this has limited progress towards deployable stereo systems. This thesis addresses these challenges through new datasets, architectures, and training paradigms that improve accuracy, temporal consistency, and efficiency.
To address the data bottleneck, we introduce two datasets: Infinigen SV, a large-scale synthetic dataset of photorealistic natural scenes with dense ground truth, and SouthKen SV, a real-world stereo video dataset captured across diverse environments. Building on these datasets, we propose BiDAStereo, a framework that introduces bidirectional alignment to improve temporal consistency. Stereo Any Video further exploits frozen foundation priors for robust zero-shot disparity estimation. We also explore Stereo Video Stabilization and propose a lightweight temporal module that can be attached to pretrained image-based models to improve consistency while preserving image-level accuracy. Finally, Lite Any Stereo achieves efficient zero-shot stereo matching through a compact backbone and a scalable training strategy, enabling real-time deployment on resource-limited hardware.
Collectively, these contributions advance stereo-based dynamic 3D reconstruction and provide a foundation for consistent, generalizable, and efficient stereo perception.
This thesis studies dynamic 3D reconstruction from stereo videos, with a focus on estimating temporally consistent disparity from synchronized camera pairs. Stereo vision provides a geometrically grounded solution, yet most deep stereo methods are designed for static images and often produce temporally inconsistent predictions when applied to videos. Combined with the scarcity of high-quality stereo video datasets, this has limited progress towards deployable stereo systems. This thesis addresses these challenges through new datasets, architectures, and training paradigms that improve accuracy, temporal consistency, and efficiency.
To address the data bottleneck, we introduce two datasets: Infinigen SV, a large-scale synthetic dataset of photorealistic natural scenes with dense ground truth, and SouthKen SV, a real-world stereo video dataset captured across diverse environments. Building on these datasets, we propose BiDAStereo, a framework that introduces bidirectional alignment to improve temporal consistency. Stereo Any Video further exploits frozen foundation priors for robust zero-shot disparity estimation. We also explore Stereo Video Stabilization and propose a lightweight temporal module that can be attached to pretrained image-based models to improve consistency while preserving image-level accuracy. Finally, Lite Any Stereo achieves efficient zero-shot stereo matching through a compact backbone and a scalable training strategy, enabling real-time deployment on resource-limited hardware.
Collectively, these contributions advance stereo-based dynamic 3D reconstruction and provide a foundation for consistent, generalizable, and efficient stereo perception.
Version
Open Access
Date Issued
2025-01-09
Date Awarded
2026-04-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
Mikolajczyk, Krystian
Publisher Department
Department of Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)