In this work, we investigate the low-rank approximation of elliptic problems in heterogeneous media by means of Kolmogrov $n$-width and asymptotic expansion. This class of problems arises in many practical applications involving high-contrast media, and their efficient numerical approximation often relies crucially on certain low-rank structure of the solutions. We provide conditions on the permeability coefficient $\kappa$ that ensure a favorable low-rank approximation. These conditions are expressed in terms of the distribution of the inclusions in the coefficient $\kappa$, e.g., the values, locations, and sizes of the heterogeneous regions. Further, we provide a new asymptotic analysis for high-contrast elliptic problems based on the perfect conductivity problem and layer potential techniques, which allows deriving new estimates on the spectral gap for such high-contrast problems. These results provide theoretical underpinnings for several multiscale model reduction algorithms.