Grazing incidence ultrasonic microscopy (GIUM) is an experimental method for visualising the microstructures of polycrystals with local preferential orientations. It has previously been demonstrated on an austenitic stainless steel weld, exposing grains much smaller than the propagating wavelength, but the physical mechanism of the method has only been proposed as a hypothesis. In this paper, we use grain-scale finite element simulations based on the EBSD measurements to verify the principles behind GIUM images further and to assess how deep does the method penetrate the component under examination. The simulations indicate that while lateral contraction of grains contains microstructure signatures, the free surface effect is the crucial factor contributing to the generation of the images. Further, we show that only features up to the depth in the order of the average grain size in that direction can be visualised.