Linear optical circuits that can provide reconfigurable interference between several optical modes are emerging as a powerful tool for both classical and quantum optics. In this work, we propose and demonstrate a modular architecture for building these circuits. Each module contains independent phase-controlled Mach–Zehnder interferometers, and several modules can be interfaced to build large reconfigurable interferometers. We demonstrate our approach by fabricating three modules in the form of ultraviolet-written silica-on-silicon chips. We characterize these chips, connect them to each other, and implement a wide range of linear optical transformations. Our work facilitates the development of large-scale linear optical circuits for quantum optics.