We propose an efficient algorithm to visualise symmetries in neural networks.
Typically, models are defined with respect to a parameter space, where
non-equal parameters can produce the same input-output map. Our proposed
method, GENNI, allows us to efficiently identify parameters that are
functionally equivalent and then visualise the subspace of the resulting
equivalence class. By doing so, we are now able to better explore questions
surrounding identifiability, with applications to optimisation and
generalizability, for commonly used or newly developed neural network
architectures.