We consider the problem of an overdamped Brownian particle moving in multiscale potential with N+1 characteristic length scales: the macroscale and N separated microscales. We show that the coarse-grained dynamics is given by an overdamped Langevin equation with respect to the free energy and with a space-dependent diffusion tensor, the calculation of which requires the solution of N fully coupled Poisson equations. We study in detail the structure of the bifurcation diagram for one-dimensional problems, and we show that the multiscale structure in the potential leads to hysteresis effects and to noise-induced transitions. Furthermore, we obtain an explicit formula for the effective diffusion coefficient for a self-similar separable potential, and we investigate the limit of infinitely many small scales.