Multi-dimensional covariance analysis and its validity for correlation of processes leading to multiple products are investigated from a theoretical point of view. The need to correct for false correlations induced by experimental parameters which fluctuate from shot to shot, such as the intensity of SASE X-ray Free Electron laser pulses, is emphasized. Three-fold covariance analysis based on simple extension of the two-variable formulation is shown to be valid for variables exhibiting Poisson statistics. In this case false correlations arising from fluctuations in some unstable experimental parameter that scale linearly with signals can be eliminated by three-fold partial covariance analysis, as newly defined here. Four-fold covariance based on the same simple extension is found to be invalid in general. Where fluctuations in an unstable parameter induce non-linear signal variations, a technique of contingent covariance analysis is proposed here to suppress false correlations. In this paper we also show a method to eliminate false correlations associated with fluctuations of several unstable experimental parameters.