We introduce substantial advancements to the field of hand reconstruction and modelling. We observe that the current state-of-the-art methods for hand pose estimation do not generalize well to samples captured in a non-laboratory environment and cannot provide real-time mobile inference. We also identify issues with the existing deformable models of the human hand that have limited shape variability and cannot express texture variations. We effectively address these problems from multiple directions. The first contribution is a weakly-supervised mesh-convolutional neural network capable of reconstructing a 3D hand mesh from a single image in the wild. It outcompetes the existing systems while working in real-time on mobile devices. We make this possible by proposing an automated data collection system and novel framework for non-linear latent representation of 3D hand meshes and image alignment. We also lead a data collection effort to generate high-resolution 3D hand scans with texture on a previously unseen scale in terms of subjects and pose diversity. Our dataset allows us to build the most diverse deformable hand model with shape and texture variations capable of representing children, adults, and the elderly from a broad range of ethnic groups. We show that our systems obtain state-of-the-art performance and enable novel industrial applications.