In this paper we aim to predict dialogue suc-cess and user satisfaction as well as emo-tion on a turn level. To achieve this, we in-vestigate the use of spectrogram representa-tions, extracted from audio files, in combina-tion with several types of convolutional neuralnetworks. The experiments were performed onthe Let’s Go V2 database, comprising 5065 au-dio files and having labels for subjective andobjective dialogue turn success, as well as theemotional state of the user. Results show thatby using only audio, it is possible to predictturn success with very high accuracy for allthree labels (90%). The best performing inputrepresentation were 1s long mel-spectrogramsin combination with a CNN with a bottleneckarchitecture. The resulting system has the po-tential to be used real-time. Our results signif-icantly surpass the state of the art for dialoguesuccess prediction based only on audio.