The central nervous system contains a vast array of cell types that are produced along the length of the rostrocaudal axis. This diversity in cell identity is established during embryonic development, and ensures that physiologically distinct cell types develop in the appropriate position in the body. Understanding how this cellular diversity arises remains a major challenge central to the field of developmental biology. In more recent years, approaches using pluripotent embryonic stem cells (ESCs) as in vitro models of development have revealed many insights into nervous system regionalisation. Here, we outline advances in the directed differentiation of ESCs, focusing on the generation of the spinal cord. We discuss the regionalisation events that impact the caudal part of the nervous system, highlighting general principles underpinning rostrocaudal differences within the mammalian body plan.