The most obvious points of contact between linear and matrix algebra and statistics are in the area of multivariate analysis. We review the way that, as both developed during the last century, the two influenced each other by examining a number of key areas. We begin with matrix and linear algebra, its emergence in the nineteenth century, and its eventual penetration into the undergraduate curriculum in the twentieth century. We continue with a similar account for multivariate analysis in statistics. We pick out the year 1936 for three key developments, and the early post-war period for three more. We then turn to some special results in linear algebra that we need. We briefly discuss four of the main contributors, and close with thirteen ‘case studies’, showing in a range of specific cases how these general algebraic methods have been put to good use and changed the face of statistics.