Bright solitary waves in a Bose-Einstein condensate contain thousands of identical atoms held together despite their only weakly attractive contact interactions. They nonetheless behave like a compound object, staying whole in collisions, with their collision properties strongly affected by intersoliton quantum coherence. We show that separate solitary waves decohere due to phase diffusion, dependent on their effective ambient temperature, after which their initial mean-field relative phases are no longer well defined or relevant for collisions. In this situation, collisions occur predominantly repulsively and can no longer be described within mean-field theory. When considering the timescales involved in recent solitary wave experiments where nonequilibrium phenomena play an important role, these features could explain the predominantly repulsive collision dynamics observed in most condensate soliton train experiments.