Machine Learning (ML) is often described as a process of learning about patterns and relationships. Structure is an example of the relationship between spaces of things; in this work, we provide a definition of learning in machines written as the process of learning unknown structure. This produces a unified view of ML that affords us concrete notions of spaces of tasks, and how they relate to chosen models.

Using such a view, we define what it means to transfer between ML problems, and how to learn to transfer. Our definition embodies the notion that transfer is tightly coupled with biases, in that to transfer is to assume biases. Further, we define transfer in the same language of structure as we did vanilla learning; the key difference manifests as the structure that is learnt. This definition highlights differences between learning to transfer, and learning by transfer.

We provide a framework, based on the theory of foliations that expresses our notions of transfer in the context of structure. We express popular methods of transfer in ML using our framework, and discuss how our framework informs us about the benefits of transfer.

The primary goal of this thesis is to introduce the mathematical and philosophical frameworks by which learning and transfer in machines can be expressed and interpreted consistently in terms of structure.