In order for bus operators and/or their respective authorities to understand where service quality can improve, it is useful to systematically compare performance with organizations displaying similarities in types of services offered, operational characteristics and density of the service area. These similar characteristics enable peer organizations to benchmark performance once their operational data are normalized for differences in scale of operations. The most commonly used normalization factors for the demand side output are passenger boardings and passenger kilometres. For the supply side output these are vehicle kilometres and vehicle hours. Through twelve years of experience in the International Bus Benchmarking Group (IBBG) a better understanding of differences in service characteristics between ‘similar’ peers has been achieved, which highlight a challenge for the interpretation of normalized performance. It became clear that relative performance should often not be concluded from performance indicators normalized in a single dimension. Variety between peers in commercial speed, trip length, vehicle planning capacity, vehicle weight and network efficiency result in the need for a bi-dimensional or balanced approach to data normalization. This paper quantifies the variety within these operational characteristics and provides examples of the interpretation bias this may lead to. A framework is provided for use by bus organization management, policymakers and benchmarking practitioners that suggests applicable combinations of denominators for a balanced normalization process, leading to improved understanding of relative performance.