Better understanding of atopic dermatitis’ effect on quality of life could enhance current management and therapeutic strategies. Studies investigating factors related to the health-related quality of life (HRQOL) of children with atopic dermatitis and their caregivers are limited. This cross-sectional study included 559 children (<16 years) with atopic dermatitis and their caregivers. Disease severity was associated with infants’ HRQOL (moderate: IRR: 1.42, 95% CI 1.20–1.67; severe: IRR: 1.72, 95% CI 1.32–2.24). Age and disease severity were associated with children’s HRQOL (age: IRR: 0.99, 95% CI 0.98–1.00; moderate: IRR: 1.08, 95% CI 1.02–1.14). Quality of life subdomains itching/scratching, emotional distress and sleep disturbance were most reported and increased with higher disease severity. Both caregivers’ mental and physical health were negatively affected by children’s HRQOL (physical: IRR: 0.99, 95% CI 0.99–1.00; mental: IRR: 0.98, 95% CI 0.97–0.99). Sociodemographic characteristics (gender, ethnicity, educational attainment of carers, number of children) did not demonstrate significance in children’s HRQOL model. In conclusion, current atopic dermatitis diagnostics and treatment have to be extended to the factors influencing both children’ as their caregivers’ quality of life and adapting management accordingly. Itching/scratching, emotional distress and sleep disturbance deserve attention. Sociodemographic characteristics in children’s HRQOL models also merit attention in further research.