Transmission of a Gaussian source over a time-varying multiple-input multiple-output (MIMO) channel is studied under strict delay constraints. Availability of a correlated side information at the receiver is assumed, whose quality, i.e., its correlation with the source signal, also varies over time. A block-fading model is considered for the states of the time-varying channel and side information; perfect state information at the receiver is assumed, while the transmitter knows only the statistics. The high signal to noise ratio performance, characterized by the distortion exponent, is studied for this joint source-channel coding problem. An upper bound is derived and compared with several lower bounds based on list decoding (LD), hybrid digital-analog transmission, as well as multi-layer schemes, which transmit successive refinements of the source, relying on progressive or superposition transmission with LD. The optimal distortion exponent is characterized for the single-input multiple-output and multiple-input single-output scenarios by showing that the distortion exponent achieved by multi-layer superposition encoding with joint decoding meets the proposed upper bound. In the MIMO scenario, the optimal distortion exponent is characterized in the low bandwidth ratio regime, and it is shown that the multi-layer superposition encoding performs very close to the upper bound in the high bandwidth ratio regime.