Robot manipulation through the lens of pose estimation
File(s)
Author(s)
Dreczkowski, Kamil Ryszard
Type
Thesis
Abstract
Object pose estimation is a versatile and useful tool for robot manipulation, enabling robots to reason explicitly about spatial relationships and adapt their movements to accommodate variations in object placement. Whether grasping objects, opening doors, or performing precision assembly, a robot's ability to reason about poses is crucial for successful manipulation.
This thesis advances both the application of pose estimation to robot manipulation and pose estimation techniques themselves. On the applications side, we introduce Trajectory Transfer, a framework that models one-shot imitation learning as an unseen object pose estimation problem, enabling robots to instantly reproduce manipulation tasks following a single demonstration. We further extend Trajectory Transfer to Multi-Task Trajectory Transfer (MT3), which combines retrieval with cross-instance pose estimation to achieve both spatial and category-level generalization of demonstrated tasks. Given MT3’s high learning efficiency, we find that we are able to teach a robot a thousand distinct tasks from a single demonstration each, in less than 24 hours.
On the pose estimation side, we propose Hybrid ICP, a novel variant of the Iterative Closest Point (ICP) algorithm that dynamically optimizes key ICP design choices based on the visible surface geometry of a target object and the current pose estimate. This method achieves superior accuracy and robustness compared to standard ICP variants when estimating poses of objects. We also investigate eye-in-hand camera calibration through the lens of pose estimation, and propose a learning-based method that achieves accurate calibration from a single RGB image by directly regressing the pose of the robot's end-effector.
This thesis advances both the application of pose estimation to robot manipulation and pose estimation techniques themselves. On the applications side, we introduce Trajectory Transfer, a framework that models one-shot imitation learning as an unseen object pose estimation problem, enabling robots to instantly reproduce manipulation tasks following a single demonstration. We further extend Trajectory Transfer to Multi-Task Trajectory Transfer (MT3), which combines retrieval with cross-instance pose estimation to achieve both spatial and category-level generalization of demonstrated tasks. Given MT3’s high learning efficiency, we find that we are able to teach a robot a thousand distinct tasks from a single demonstration each, in less than 24 hours.
On the pose estimation side, we propose Hybrid ICP, a novel variant of the Iterative Closest Point (ICP) algorithm that dynamically optimizes key ICP design choices based on the visible surface geometry of a target object and the current pose estimate. This method achieves superior accuracy and robustness compared to standard ICP variants when estimating poses of objects. We also investigate eye-in-hand camera calibration through the lens of pose estimation, and propose a learning-based method that achieves accurate calibration from a single RGB image by directly regressing the pose of the robot's end-effector.
Version
Open Access
Date Issued
2025-01-04
Date Awarded
2025-11-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
Johns, Edward
Sponsor
Engineering and Physical Sciences Research Council
Grant Number
DTP Reference Number 2297064
Publisher Department
Department of Computing
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)