Despite recent advancements in using machine learning (ML) techniques to establish the microstructure-property linkage for composites’ representative volume elements (RVEs), challenges persist in effectively characterising the effect of microstructural randomness on material properties. This complexity arises from the difficulty of expressing randomness as definitive variables and its intertwined relations with other factors, such as material constituents. Such complexities result in limitations in generalising ML models across different material constituents. Conventional solutions to these challenges usually necessitate large datasets, which require considerable computational resources, for an accurate and generalisable ML models to be trained. This paper presents an innovative approach to tackling these challenges by integrating a high-accuracy convolutional neural network (CNN) with a novel microstructure-factored constitutive model (MCM). The MCM, rooted from classic empirical constitutive modelling, effectively segregates the microstructural and constituting material effects, extending the generalisability and thus significantly enhancing the efficacy of the CNN. This new approach enabled a CNN trained on the transverse stress-strain curves of one set of material constituents (CF/PEEK at 270 °C) to be generalised for the rapid prediction of various sets of material constituents at different temperatures, unseen by the CNN during training, with an average mean absolute percentage error around 3 %.