Human-in-the-loop design with machine learning

Abstract

Deep learning methods have been applied in design fields to randomly generate design images. However, these AI generation methods do not consider human cognitive aspects, and the randomly generated results have no human cognitive input. This leads to AI-driven design applications not considering human perception, especially in design applications where perceived qualities are important, such as fashion, graphics, advertising and product appearance. In this thesis, we seek to advance an AI method for generating design images through deep-learning-based brain decoding. We expect this could help to generate design images with human cognition interactions. We propose a human-in-the-loop design framework with machine learning for generating design images with human cognition represented by brain activity, and ultimately laying the technical foundations of a new design method that optimises human perceptions in the loop. Specifically, three cognitive mechanisms common in design have been explored: mental association, preferences and conceptual blending. Through the proposed human-in-the-loop AI design image generation framework, three types of cognitive models have been investigated for these three cognitive mechanisms. Design cases have been applied by utilising the proposed models to demonstrate the technical potential in generating design images with human cognition. In this thesis, we aim to add human cognitive aspects to AI design image generation processes. The thesis makes the following contributions: 1) We propose a human-in-the-loop AI design image generation framework with human cognition integrated during AI design generation processes. 2) To provide evidence supporting the proposed AI design framework, three cognitive models – ‘cognitive categorisation model’, ‘cognitive transformation model’ and ‘cognitive combination model’ have been demonstrated with design cases. 3) We have collected six datasets for different design tasks, including three brain signal datasets (two EEG datasets and one EEG and fMRI simultaneous recording dataset) and three corresponding images datasets, for the model training and testing of the model.