We study binocular rivalry in a competitive neural network with synaptic depression. In particular, we consider two coupled hypercolums within primary visual cortex (V1), representing orientation selective cells responding to either left or right eye inputs. Coupling between hypercolumns is dominated by inhibition, especially for neurons with dissimilar orientation preferences. Within hypercolumns, recurrent connectivity is excitatory for similar orientations and inhibitory for different orientations. All synaptic connections are modifiable by local synaptic depression. When the hypercolumns are driven by orthogonal oriented stimuli, it is possible to induce oscillations that are representative of binocular rivalry. We first analyze the occurrence of oscillations in a space-clamped version of the model using a fast-slow analysis, taking advantage of the fact that depression evolves much slower than population activity. We then analyze the onset of oscillations in the full spatially extended system by carrying out a piecewise smooth stability analysis of single (winner-take-all) and double (fusion) bumps within the network. Although our stability analysis takes into account only instabilities associated with real eigenvalues, it identifies points of instability that are consistent with what is found numerically. In particular, we show that, in regions of parameter space where double bumps are unstable and no single bumps exist, binocular rivalry can arise as a slow alternation between either population supporting a bump.