After discussions of a few relevant patents of neural networks, a model description of moment neuronal
networks with context units is given by introducing intra-layer inputs. The dynamics of homogeneous networks derived
from the intra-inputs are explored, including the input-output relationships and the stability of such network. It is shown
how the spontaneous activity is propagated across the homogeneous feed-forward networks with context units. Due to a
more biologically reasonable context unit, such network offers a significant advantage over the recent moment neuronal
networks in that it can enhance or weaken the dynamics of network by the adjustment of the parameters from context unit,
based on those from network itself, and it can lead to some unexpectedly dynamic properties. In this paper we highlight
the key and sophisticated role played by the context unit in dynamics of such network.