Objective
To investigate and compare the effect of simulator training on quantitative scores for ultrasound‐related skills for trainees with novice level ultrasound experience and expert ultrasound operators.

Methods
Three novice (comprising of 11, 32, 23 participants) and one expert (10 participants) subgroups undertook an ultrasound simulation training session. Pre‐ and post‐training test scores were collected for each subgroup. Outcome measures were as follows: mean accuracy score for obtaining the correct anatomical plane, percentage of correctly acquired target planes, mean number of movements, time to achieve image, distance travelled by probe and accumulated angling of the probe.

Results
The novices showed improvement in image acquisition after completion of the simulation training session with an improvement in the rate of correctly acquired target planes from 28–57% to 39–83%. This was not replicated in the experts. The novice’s individual ratios based on pre‐ vs. post‐training metrics improved between 1.7‐ and 4.3‐fold for number of movements, 1.9‐ and 6.7‐fold for distance, 2.0‐ and 5.2‐fold for time taken and 1.8‐ and 7.3‐fold for accumulated angling. Among the experts, there was no relationship between pre‐training simulator metrics and years of ultrasound experience.

Conclusions
The individual simulation metrics suggest the sessions were delivered at an appropriate level for basic training as novice trainees were able to show demonstrable improvements in both efficiency and accuracy on the simulator. Experts did not improve after the simulation modules, and the novice scores post‐training were similar to those of experts, suggesting the exercises were valid in testing ultrasound skills at novice but not expert level.