We have developed a population-level method for dietary assessment using low-cost wearable cameras. Our approach, EgoDiet, employs an egocentric vision-based pipeline to learn portion sizes, addressing the shortcomings of traditional self-reported dietary methods. To evaluate the functionality of this method, field studies were conducted in London (Study A) and Ghana (Study B) among populations of Ghanaian and Kenyan origin. In Study A, EgoDiet’s estimations were contrasted with dietitians’ assessments, revealing a performance with a Mean Absolute Percentage Error (MAPE) of 31.9% for portion size estimation, compared to 40.1% for estimates made by dietitians. We further evaluated our approach in Study B, comparing its performance to the traditional 24-Hour Dietary Recall (24HR). Our approach demonstrated a MAPE of 28.0%, showing a reduction in error when contrasted with the 24HR, which exhibited a MAPE of 32.5%. This improvement highlights the potential of using passive camera technology to serve as an alternative to the traditional dietary assessment methods.