Understanding degradation mechanisms of organic photovoltaics (OPVs) is a critical prerequisite for improving device stability. Herein, the effect of molecular structure on the photostability of non-fullerene acceptors (NFAs) is studied by changing end-group substitution of ITIC derivatives: ITIC, ITIC-2F, and ITIC-DM. Using an assay of in situ spectroscopy techniques and molecular simulations, the photodegradation product of ITIC and the rate of product formation are identified, which correlates excellently to reported device stability, with ITIC-2F being the most stable and ITIC-DM the least. The choice of acceptor is found to affect both the donor polymer (PBDB-T) photostability and the morphological stability of the bulk heterojunction blend. Molecular simulations reveal that NFA end-group substitution strongly modulates the electron distribution within the molecule and thus its quadrupole moment. Compared to unsubstituted-ITIC, end-group fluorination results in a stronger, and demethylation a weaker, molecular quadrupole moment. This influences the intermolecular interactions between NFAs and between the NFA and the polymer, which in turn affects the photostability and morphological stability. This hypothesis is further tested on two other high quadrupole acceptors, Y6 and IEICO-4F, which both show impressive photostability. The strong correlation observed between NFA quadrupole moment and photostability opens a new synthetic direction for photostable organic photovoltaic materials.