We analyse an institutional curriculum change initiative from the perspective of academics in a physics department to identify the barriers faced when curriculum change is used as a process to increase equity, diversity, and inclusion (EDI). We explore what curriculum means in physics, how power relationships within the university affect the response to top-down change, and ultimately who has control over the curriculum. We find that the different conceptualisations of curriculum in physics compared to educational research mean that a directed top-down approach stalls when concepts that fall outside of the physics curriculum, such as EDI, are introduced. Nevertheless, initiatives that are in response to EDI concerns that would be considered to be curriculum change from an educational research perspective are enacted within the department on a localised scale. Understanding the different conceptualisations of curriculum and how they interact to support or hinder change should help guide future institutional change efforts.