Networks provide powerful and flexible models for many biological systems. Gene regulatory networks model the complex biomolecular interactions that drive genetically identical cells to adopt distinct functional identities. Gene regulation is complicated and multifactorial, involving different biological mechanisms that are often well-characterized individually, but whose combinatorial effects on a large scale are poorly understood. Recent technological advances mean that highly-informative single cell data are being generated on an unprecedented scale, but these large, unwieldy datasets contain technical noise and biological heterogeneity, and must be analyzed carefully. Here we make use of information theory to address the complexities and nonlinearities of these data, drawing on recent developments in quantifying information within multivariate systems. We first provide a comprehensive review of single cell analysis and the multiple definitions of gene regulatory networks. Then we develop a network inference algorithm using multivariate information, and use it to learn networks from several developmental systems. We next present a method for quantifying significance in a large-scale information theoretic context, and use it to control the accuracy of our networks for different applications. Finally we infer a network from over 11,000 expressed genes and, using graph theoretic and bioinformatic analyses, characterize the biological processes occurring during early neural development. Throughout this work, we take a data-driven approach, favouring statistical methods that make minimal assumptions. Alongside our analyses, we provide efficient, reusable software, culminating in a collection of free, open source software libraries.