This study uses data acquired from three-dimensional discrete element method simulations to reconsider what measure of state can be used to predict stiffness in granular materials. A range of specimens with linear and gap-graded particle size distributions are considered and stiffness is measured using small amplitude strain probes. Analysis of the data firstly confirms that the void ratio, which is typically used as a measure of state in experimental soil mechanics, does not correlate well with shear stiffness. However, the empirical expressions developed by Hardin and his colleagues can capture variations in stiffness, provided an appropriate state variable is used. The study then highlights that the contribution of individual contacts to the overall stiffness is highly variable, depending on both the contact force transmitted and the particle size. Analyses explore how the stress transmission both within and between the different size fractions affects the overall stiffness. This heterogeneity in stiffness relates to the heterogeneity in the stress transmission amongst the different fractions. By considering the heterogeneity of stress distribution amongst different particle size fractions, a new semi-empirical stress-based state variable is proposed that provides insight into the factors that influence stiffness.