Recent work demonstrated the value of multi clusters of spiking Random Neural Networks (RNN) with dense soma-to-soma interactions in deep learning. In this paper we go back to the original simpler structure and we investigate the power of single RNN cells for deep learning. First, we consider three approaches with the single cells, twin cells and multi-cell clusters. This first part shows that RNNs with only positive parameter can conduct convolution operations similar to those of the convolutional neural network. We then develop a multi-layer architecture of single cell RNNs (MLSRNN), and show that this architecture achieves comparable or better classification at lower computation cost than conventional deep-learning methods.