Laboratory of Neural Networks

Headed by Yakov Kazanovich, Dr. Sci.

Directions of Research

Artificial neural networks represent mathematical or technical constructions built from neuron-like elements. Neural networks are used for mathematical modeling in neurobiology and for designing artificial intelligence systems. Accordingly, investigations on neural networks can be divided into two subfields: mathematical (computational) neurobiology and neurocomputing. Mainly, the research work of the laboratory is directed to the modeling of biological neural systems with the aim to understand neurophysiological principles of information processing in the brain.

Experimental studies of the brain show that the dynamics of electrical activity play an important role in the interaction of brain structures. In particular, rhythmical activity and its synchronization may represent one of the general mechanisms of information processing in the brain. The work of the brain is characterized by a wide spectrum of rhythms which correlate with external stimulation and internal psychological state of the organism. Stable patterns of rhythmic spiking have been discovered in various parts of the brain in the recordings of the activity of single neurons, neural populations, and brain structures. Such experimental data were obtained in the studies of primary areas of visual and olfactory cortices, sensorymotor cortex, thalamus, hippocampus, and other brain structures.

The study of the role of oscillations and their synchronization in the work of the brain led to the development of a new paradigm in the theory of neural networks which is called oscillatory neural networks [1-3]. It is assumed that the theory of oscillatory neural networks should be able to answer to the following questions:

  • What are the conditions when oscillatory activity appears and can be synchronized? How to explain particular forms of oscillatory neural activity of brain structures?
  • What kind of information processing can be implemented by using oscillations and synchronization? What is the role of neural dynamics in realization of cognitive functions?

The work of the laboratory is based on the theory of oscillatory neural networks and is directed to the modeling of such cognitive phenomena as feature binding, attention, memory, and novelty detection. The brain structures on which the research is concentrated include the neocortex and the septo-hippocampal system. The main instruments of research are the theory of dynamical systems, bifurcation analysis, and computer simulations of multidimensional stochastic processes (stochastic fields) and the systems of ordinary differential equations.

Main Results

  • Mathematical models of short term memory are developed based on the theory of phase transitions [4, 5]. The models presented a new approach to memory formation and emphasized the role of metastable states and collective behavior in the prosess of memorization. The theory of interacting Markov processes and fields has been applied to the analysis of dynamical regimes in stochastic neural networks. It is shown that despite stochastic behavior of individual neurons cooperative effects such as phase transitions and synchronization of neural activity are typical for the behavior of neural networks. The model has been used for simulations of low frequency oscillations in the septum, the habituation in the hippocampus, and metastable states in the neocortex.
  • Conditions for oscillatory neural activity and the regimes of oscillation synchronization have been described for neural networks of various connection architectures and types of elements (pacemaker neurons of Hodgkin-Huxley [6] and Hindmarsh-Rose [7], integrate-and-fire neurons [8], Wilson-Cowan oscillators [9, 10], phase oscillators [11-14]). The stability of synchronous regimes has been studied for biologically adequate architectures and interaction parameters.
  • The types and role of oscillatory activity in the hippocampus has been studied [15-18]. A model of working memory in the hippocampus is developed which is able to memorize and associatively recall the sequences of events. Spatio-temporal patterns of neural activity in the hippocampus have been described. A new mechanism of theta rhythm generation is suggested based on mutual inhibition of neurons in the medial septum and hippocampus.
  • A model of autonomous control of six-legged locomotion of stick insects has been developed that allowed the simulation of different types of walking and showed high stability of walking [19]. The training of the model was based on a genetic algorithm.
  • The synchronization principle has been applied to develop a series of models of attention focus formation and switching [20, 21] and of novelty detection [22]. The process of attention focus formation has been studied as a function of system parameters. The results of these studies have been used to develop a complex model of visual perception that combines feature binding, novelty detection, and consecutive selection of objects in the focus of attention [23].
  • An oscillatory model of single and multiple object tracking is worked out [24]. The model can track visual targets surrounded by distractors with the probability of errors that complies with experimental evidence.
  • A conception of inclusive sensory characteristics is introduced as a basic notion to describe the hierarchy of object features. [25. 26]. The hierarchy begins from elementary features (determined in the primary regions of the visual system) and spreads to the features of higher levels (determined in the associative cortex) that bind perceptual representation of objects into meaningful patterns and scenes. Specific patterns of neural activity, which map inclusive characteristics, are relayed from upper to lower neuronal levels. This stimulates those neuronal populations whose signals correspond to the highest inclusive characteristic. Stimulation from above reduces the time of response of selected neurons to milliseconds. As a result, a fast sensory pathway, single and unique for each act of perception, is formed. The multi-level hierarchical selection of sensory features and adaptively meaningful entities provides for the transmission to superior cerebral structures of only those sensory data that meet the organism?s requirements and objectives.
  • A collection of mathematical and computational methods for the analysis of neurophysiological recordings has been developed [27, 28]. A method for the estimation of the parameters of neural interaction has been suggested that works on the bases of recordings of a pair of interacting neurons. The method was tested by using a simulated neural network model and showed the high quality of estimation of the connection strength between the neurons. Also the principles of visualization of neurophysiological data for multichannel recordings have been worked out.

Main Publications

  1. Borisyuk, G.N., Borisyuk, R.M., Kazanovich, Y.B., Luzyanina, T.B., Turova, T.S., and Cymbalyuk, G.S. (1992). Осцилляторные нейронные сети. Математические результаты и приложения. Математическое моделирование, 4(1), 3-43 (in Russian).
  2. Borisyuk, G.N., Borisyuk, R.M., Kazanovich, Y.B., and Ivanitskii, G.R. (2002). Models of neural dynamics in brain information processing - the developments of 'the decade'. Physics - Uspekhi, 45 (10), 1073-1095.
  3. Kazanovich, Y.B. (2007). Nonlinear dynamics modeling and information processing in the brain. Optical Memory & Neural Networks, 16 (3), 111-124.
  4. Kryukov, V.I., Borisyuk, G.N., Borisyuk, R.?., Kirillov, A.K., and Kovalenko, Ye.I. (1990). Metastable and unstable states in the brain. In Stochastic cellular systems: ergodicity, memory, morphogenesis, R.L. Dobrushin, V.I. Kryukov, and A.I. Tom (eds.), pp. 225-358. Manchester Univ. Press.
  5. Borisyuk, R. and Cooke, T. (2007) Metastable states, phase transitions, and persistent neural activity. BioSystems, 89:30-37
  6. Chik, D., Borisyuk, R., and Kazanovich, Y. (2009) Selective attention model with spiking elements. Neural Networks (in press).
  7. Cymbalyuk, G.S., Nikolaev, E.V., and Borisyuk, R.M. (1994). In-phase and anti-phase self-oscillations in a model of two electrically coupled pacemakers. Biological Cybernetics, 71, 153-160.
  8. Borisyuk, R. (2002). Oscillatory activity in the neural networks of spiking elements. BioSystems, 67,3-16.
  9. Borisyuk, R.M. and Kirillov, A.B. (1992). Bifurcation analysis of a neural network model. Biol. Cybern. 66, 319-325, 1992.
  10. Borisyuk, G.N., Borisyuk, R.M., Khibnik, A.I., and Roose, D. (1995). Dynamics and bifurcations of two coupled neural oscillators with different connection type. Bull. Math. Biol., 57, 809-843.
  11. Kazanovich, Y.B. and Borisyuk, R.M. (1994). Synchronization in a neural network of phase oscillators with the central element, Biol. Cybern., 71, 177-185.
  12. Kazanovich, Y.B. and Borisyuk, R.M. (1999). Dynamics of neural networks with a central element. Neural Networks, 12(3): 441-454.
  13. Kazanovich, Y.B. and Borisyuk, R.M. (2003). Synchronization in oscillator systems with phase shifts. Progr. Theor. Phys., 110, 1047-1058.
  14. Luzyanina, T.B. (1995). Synchronization in an oscillator neural network model with time delayed coupling. Network, 6, 43-59.
  15. Borisyuk, R.M. and Hoppensteadt, F. (1998). Memorizing and recalling spatial-temporal patterns in an oscillator model of the hippocampus. Biosystems, 48, 3-10.
  16. Borisyuk, R. and Hoppensteadt, F. (1999). Oscillatory models of the hippocampus: A study of spatio-temporal patterns of neural activity. Biological Cybernetics, 81, 359-371.
  17. Borisyuk, R., Denham, M., Denham, S., and Hoppensteadt, F. (1999). Computational models of predictive and memory-related functions of the hippocampus. Rev. Neurosci., 10, 213-232.
  18. Denham, M. and Borisyuk, R. (2000). A model of theta rhythm production in the septal-hippocampal system and its modulation by ascending brain stem pathways. Hippocampus, 10, 698-716.
  19. Cymbalyuk, G., Borisyuk, R., Muller-Wilm, U., and Cruse, H. (1998). Oscillatory network controlling six-legged locomotion. Optimization of model parameters. Neural Networks, 11, 1449-1460.
  20. Kazanovich, Y. and Borisyuk, R. (2002). Object selection by an oscillatory neural network. BioSystems, 67(1-3), 103-111.
  21. Borisyuk, R.M. and Kazanovich, Y.B. (2003). Oscillatory neural network model of attention focus formation and control. BioSystems, 71, 29-38.
  22. Borisyuk, R., Denham, M., Kazanovich, Y., Hoppensteadt, F. and Vinogradova, O. (2001). Oscillatory model of novelty detection. Network, 12, 1-20.
  23. Borisyuk, R. and Kazanovich, Y. (2004). Oscillatory model of attention-guided object selection and novelty detection. Neural Networks, 17, 899-915.
  24. Kazanovich, Y.B. and Borisyuk, R.M. (2006). An oscillatory neural model of multiple object tracking. Neural Computation, 18, 1413-1440.
  25. Sergin, V.Ya. (2002). Perceptual binding of sensory events: The hypothesis of inclusive characteristics. J. High Nervous Activity, 52 (6), 645-655 (in Russian).
  26. Sergin, A.V. and Sergin, V.Ya. (2008). Model of perception: The hierarchy of inclusive sensory characteristics and top-down cascade transfer of excitation. Neural Network World, 18, 227-244.
  27. Borisyuk, G.N., Borisyuk, R.?., Kirillov, A.K., Kovalenko, Ye.I., and Kryukov, V.I.. (1986). New methods of analysis of neural activity. Pushchino.
  28. Stuart, L, Walter, M., and Borisyuk, R. (2005). The Correlation Grid: Analysis of Synchronous Spiking in Multi-dimensional Spike Train Data and Identification of Feasible Connection Architectures. BioSytems, 79:223-233.