We propose a procedure for optimising the friction matrix of underdamped Langevin dynamics when used for continuous time Markov Chain Monte Carlo. Starting from a central limit theorem for the ergodic average, we present a new expression of the gradient of the asymptotic variance with respect to friction matrix. In addition, we present an approximation method that uses simulations of the associated first variation/tangent process. Our algorithm is applied to a variety of numerical examples such as toy problems with tractable asymptotic variance, diffusion bridge sampling and Bayesian inference problems for high dimensional logistic regression.