Cloud4D: Estimating cloud properties at a high spatial and temporal resolution
File(s)
Accepted version
OA Location
Author(s)
Lin, Jacob
Gryspeerdt, Edward
Clark, Ronald
Type
Conference Paper
Abstract
There has been great progress in improving numerical weather prediction and climate models using machine learning. However, most global models act at a
kilometer-scale, making it challenging to model individual clouds and factors such as extreme precipitation, wind gusts, turbulence, and surface irradiance. Therefore, there is a need to move towards higher-resolution models, which in turn require high-resolution real-world observations that current instruments struggle to obtain. We present Cloud4D, the first learning-based framework that reconstructs a physically consistent, four–dimensional cloud state using only synchronized ground-based cameras. Leveraging a homography-guided 2D-to-3D transformer, Cloud4D infers the full 3D distribution of liquid water content at 25 m spatial and 5 s temporal resolution. By tracking the 3D liquid water content retrievals over time, Cloud4D additionally estimates horizontal wind vectors. Across a two-month deployment comprising six skyward cameras, our system delivers an order-of-magnitude improvement in space-time resolution relative to state-of-the-art satellite measurements, while retaining single-digit relative error (< 10%) against collocated radar measurements. Code and data are available on our project
page https://cloud4d.jacob-lin.com/.
kilometer-scale, making it challenging to model individual clouds and factors such as extreme precipitation, wind gusts, turbulence, and surface irradiance. Therefore, there is a need to move towards higher-resolution models, which in turn require high-resolution real-world observations that current instruments struggle to obtain. We present Cloud4D, the first learning-based framework that reconstructs a physically consistent, four–dimensional cloud state using only synchronized ground-based cameras. Leveraging a homography-guided 2D-to-3D transformer, Cloud4D infers the full 3D distribution of liquid water content at 25 m spatial and 5 s temporal resolution. By tracking the 3D liquid water content retrievals over time, Cloud4D additionally estimates horizontal wind vectors. Across a two-month deployment comprising six skyward cameras, our system delivers an order-of-magnitude improvement in space-time resolution relative to state-of-the-art satellite measurements, while retaining single-digit relative error (< 10%) against collocated radar measurements. Code and data are available on our project
page https://cloud4d.jacob-lin.com/.
Date Acceptance
2025-09-18
Citation
NeurIPS Proceedings
Journal / Book Title
NeurIPS Proceedings
Copyright Statement
Subject to copyright. This paper is embargoed until publication.
Source
9th Conference on Neural Information Processing Systems (NeurIPS 2025)
Publication Status
Accepted
Start Date
2025-12-05
Finish Date
2025-12-07
Coverage Spatial
San Diego, CA, USA