Mechanisms of protein recognition have been extensively studied for single-domain proteins1, but are less well characterized for dynamic multidomain systems. Ubiquitin chains represent a biologically important multidomain system that requires recognition by structurally diverse ubiquitin-interacting proteins2,3. Ubiquitin chain conformations in isolation are often different from conformations observed in ubiquitin-interacting protein complexes, indicating either great dynamic flexibility or extensive chain remodelling upon binding. Using single-molecule fluorescence resonance energy transfer, we show that Lys 63-, Lys 48- and Met 1-linked diubiquitin exist in several distinct conformational states in solution. Lys 63- and Met 1-linked diubiquitin adopt extended ‘open’ and more compact ‘closed’ conformations, and ubiquitin-binding domains and deubiquitinases (DUBs) select pre-existing conformations. By contrast, Lys 48-linked diubiquitin adopts predominantly compact conformations. DUBs directly recognize existing conformations, but may also remodel ubiquitin chains to hydrolyse the isopeptide bond. Disruption of the Lys 48–diubiquitin interface changes conformational dynamics and affects DUB activity. Hence, conformational equilibria in ubiquitin chains provide an additional layer of regulation in the ubiquitin system, and distinct conformations observed in differently linked polyubiquitin may contribute to the specificity of ubiquitin-interacting proteins.