Phantoms are common substitutes for soft tissues in biomechanical research and are usually tuned to match tissue properties using standard testing protocols at small strains. However, the response due to complex tool-tissue interactions can differ depending on the phantom and no comprehensive comparative study has been published to date, which could aid researchers to select suitable materials. In this work, gelatin, a common phantom in literature, and a composite hydrogel developed at Imperial College, were matched for mechanical stiffness to porcine brain, and the interactions during needle insertions within them were analyzed. Specifically, we examined insertion forces for brain and the phantoms; we also measured displacements and strains within the phantoms via a laser-based image correlation technique in combination with fluorescent beads. It is shown that the insertion forces for gelatin and brain agree closely, but that the composite hydrogel better mimics the viscous nature of soft tissue. Both materials match different characteristics of brain, but neither of them is a perfect substitute. Thus, when selecting a phantom material, both the soft tissue properties and the complex tool-tissue interactions arising during tissue manipulation should be taken into consideration. These conclusions are presented in tabular form to aid future selection.