Several metres below the coastal ocean surface there are areas of high ecological activity that contain thin layers of concentrated motile phytoplankton. Gyrotactic trapping has been proposed as a potential mechanism for layer formation of bottom-heavy swimming algae cells, especially in flows where the vorticity varies linearly with depth (Durham et al., Science, vol. 323(5917), 2009, pp. 1067–1070). Using a continuum model for dilute microswimmer suspensions, we report that an instability of a gyrotactically trapped cell layer can arise in a pressure-driven plane channel flow. The linear stability analysis reveals that the equilibrium cell-layer solution is hydrodynamically unstable due to negative microswimmer buoyancy (i.e. a gravitational instability) over a range of biologically relevant parameter values. The critical cell concentration for this instability is found to be Nc≃104 cells cm−3 , a value comparable to the typical maximum cell concentration observed in thin layers. This result indicates that the instability may be a potential mechanism for limiting the layer’s maximum cell concentration, especially in regions where turbulence is weak, and motivates the study of its nonlinear evolution, perhaps, in the presence of turbulence.