We develop the recent proposal by the authors to exploit the isomonodromic tau function defined by Jimbo, Miwa and Ueno (JMU) to solve the accessory parameter problem in conformal mapping theory. We focus here on mappings of Schwarz-Christoffel type: in particular, the mapping from the upper half plane to a 4-sided polygon where the sides are all straight lines. We show that one can obtain the relevant accessory parameters -- the pre-image of the polygonal vertices -- via a special ``zero curvature limit'' in which the radius of curvature of some of the edges tends to zero. We apply the procedure to rectangular domains where the JMU tau function is given by a ratio of Riemann theta functions, known as the Picard solution, and take the zero curvature limit to recover the accessory parameter obtained by Nehari using quite different methods. We then turn to trapezoids, deriving new asymptotic formulas for the accessory parameters in the limit of large and small aspect ratios. Our work lends a new geometrical perspective to problems of isomonodromy that we believe provides theoretical insight, while also showing how classical problems in conformal mapping can benefit from new ideas emerging from isomonodromic deformation theory.