In this paper we consider systems of weakly interacting particles driven by colored noise in a bistable potential, and we study the effect of the correlation time of the noise on the bifurcation diagram for the equilibrium states. We accomplish this by solving the corresponding McKean--Vlasov equation using a Hermite spectral method, and we verify our findings using Monte Carlo simulations of the particle system. We consider both Gaussian and non-Gaussian noise processes, and for each model of the noise we also study the behavior of the system in the small correlation time regime using perturbation theory. The spectral method that we develop in this paper can be used for solving linear and nonlinear, local and nonlocal (mean field) Fokker--Planck equations, without requiring that they have a gradient structure.