The non-fullerene acceptors (NFAs) employed in state-of-art organic photovoltaics (OPVs) often exhibit strong quadrupole moments which can strongly impact on material energetics. Herein, we show that changing the orientation of Y6, a prototypical NFA, from face-on to more edge-on by using different processing solvents causes a significant energetic shift of up to 210 meV. The impact of this energetic shift on OPV performance is investigated in both bilayer and bulk-heterojunction (BHJ) devices with PM6 polymer donor. The device electronic bandgap and the rate of non-geminate recombination are found to depend on the Y6 orientation in both bilayer and BHJ devices, attributed to the quadrupole moment-induced band bending. Analogous energetic shifts are also observed in other common polymer/NFA blends, which correlates well with NFA quadrupole moments. This work demonstrates the key impact of NFA quadruple moments and molecular orientation on material energetics and thereby on the efficiency of high-performance OPVs.