Nonfullerene acceptors (NFAs) in blends with highly crystalline donor polymers have been shown to yield particularly high device voltage outputs, but typically more modest quantum yields for photocurrent generation as well as often lower fill factors (FF). In this study, we employ transient optical and optoelectronic analysis to elucidate the factors determining device photocurrent and FF in blends of the highly crystalline donor polymer PffBT4T-2OD with the promising NFA FBR or the more widely studied fullerene acceptor PC71BM. Geminate recombination losses, as measured by ultrafast transient absorption spectroscopy, are observed to be significantly higher for PffBT4T-2OD:FBR blends. This is assigned to the smaller LUMO-LUMO offset of the PffBT4T-2OD:FBR blends relative to PffBT4T-2OD:PC71BM, resulting in the lower photocurrent generation efficiency obtained with FBR. Employing time delayed charge extraction measurements, these geminate recombination losses are observed to be field dependent, resulting in the lower FF observed with PffBT4T-2OD:FBR devices. These data therefore provide a detailed understanding of the impact of acceptor design, and particularly acceptor energetics, on organic solar cell performance. Our study concludes with a discussion of the implications of these results for the design of NFAs in organic solar cells.