The performance of solar cells based on molecular electronic materials is limited by relatively high nonradiative voltage losses. The primary pathway for nonradiative recombination in organic donor-acceptor heterojunction devices is believed to be the decay of a charge-transfer (CT) excited state to the ground state via energy transfer to vibrational modes. Recently, nonradiative voltage losses have been related to properties of the charge-transfer state such as the Franck-Condon factor describing the overlap of the CT and ground-state vibrational states and, therefore, to the energy of the CT state. However, experimental data do not always follow the trends suggested by the simple model. Here, we extend this recombination model to include other factors that influence the nonradiative decay-rate constant, and therefore the open-circuit voltage, but have not yet been explored in detail. We use the extended model to understand the observed behavior of series of small molecules:fullerene blend devices, where open-circuit voltage appears insensitive to nonradiative loss. The trend could be explained only in terms of a microstructure-dependent CT-state oscillator strength, showing that parameters other than CT-state energy can control nonradiative recombination. We present design rules for improving open-circuit voltage via the control of material parameters and propose a realistic limit to the power-conversion efficiency of organic solar cells.