Understanding of the human brain functioning currently represents a challenging problem. In contrast to usual serial computers and complicated hierarchically organized artificial man-made systems, decentralized, parallel and distributed information processing principles are inherent to the brain. Besides adaptation and learning, which play a crucial role in brain functioning, oscillatory neural activity, synchronization and resonance accompany the brain work. Neural-like oscillatory network models, designed by the authors for image processing, allow to elucidate the capabilities of dynamical, synchronization-based types of image processing, presumably exploited by the brain. The oscillatory network models, studied by means of computer modeling and qualitative analysis, are presented and discussed in the book. Some other problems of parallel distributed information processing are also considered, such as a recall process from network memory for large-scale recurrent associative memory neural networks, performance of oscillatory networks of associative memory, dynamical oscillatory network methods of image processing with synchronization-based performance, optical parallel information processing based on the nonlinear optical phenomenon of photon echo, and modeling random electric fields of quasi-monochromatic polarized light beams using systems of superposed stochastic oscillators. This makes the book highly interesting to researchers dealing with various aspects of parallel information processing.

M.G. Kuzmina, Keldysh Inst. of Applied Maths RAS; E.A. Manykin, NRC Kurchatov Inst.; E.S. Grichuk, NRC Kurchatov Inst.

Contents

1. INTRODUCTION

1.1 Brain: structure and function principles.

1.2 Artificial neural networks.

1.2.1 Neuron models.

1.2.2 Feed-forward neural networks.

1.2.3 Recurrent networks.

1.3 Multilayer networks and function approximation.

1.4 Learning of neural networks.

1.4.1 Learning algorithms based on gradient descent.

1.4.2 Supervised and unsupervised learning. Relation to optimization problems.

1.4.3 Hebbian learning. Competitive learning.

1.4.4 Back-propagation learning algorithm.

1.4.5 Statistical learning theory.

1.4.6 Learning algorithms for recurrent networks. Relation to controllable dynamical systems.

1.5 Neural networks of associative memory (attractor networks).

1.5.1 Network memory as a controllable set of stable attractors of network dynamics. Extraneous memory.

1.5.2 Memory storage capacity.

1.5.3 Problem of recall from network memory.

1.5.4 Hopfield network model. The model relation to spin systems.

1.5.5 Sparsely encoded neural networks.

1.6 Dynamical neural networks.

1.6.1 Neurons with complicated internal dynamics.

1.6.2 Neural oscillators. Oscillatory networks.

1.6.3 Oscillatory systems with chaotic dynamics. Communication via chaos synchronization.

1.7 Modern extensions of neural network paradigm.

1.7.1 Multi-agent systems.

1.7.2 Cognitive dynamical systems.

2. NEURAL, OSCILLATORY AND PHASOR NETWORKS OF ASSOCIATIVE MEMORY

2.1 Neural networks of associative memory.

2.1.1 Networks of auto-associative and hetero-associative memory.

2.1.2 Statistical macrodynamics approach for large scale associative memory networks.

2.1.3 Noise reduction by hetero-associative memory networks.

2.1.4 The analysis of recall process from memory in macrodynamical approach.

2.2 Oscillatory networks of associative memory.

2.2.1 Oscillatory and phasor networks of associative memory.

2.2.2 Features of associative memory oscillatory networks.

2.2.3 Controllable memory of phasor networks.

2.2.4 Choice of oscillatory network parameters providing the best memory characteristics.

3. OSCILLATORY NETWORKS FOR MODELING OF THE BRAIN STRUCTURE FUNCTIONS

3.1 The motivations for creation of oscillatory network models.

3.2 Types of collective dynamics for large systems of coupled oscillators.

3.2.1 Structural portrait of governing dynamical system.

3.2.2 Critical phenomena in large systems of coupled oscillators.

3.3 Oscillatory networks for modeling of olfactory and auditory brain system performance.

3.4 Overview of oscillatory network models for image processing problems.

3.4.1 Model by C.Malsburg.

3.4.2 Model LEGION by D.Wang and D.Terman.

3.4.3 Model by Z.Li.

3.4.4 Model by Y.Choe and R.Miikkulainen.

3.4.5 Model by B.Yu and L.Zhang.

3.4.6 Model by A.Labbi et al.

3.4.7 Model by R.Opara and F.Wörgötter.

3.4.8 Model by R.Borisyuk and Y.Kazanovich.

3.4.9 Model by M.B.H.Rhouma and H.Frigui.

3.4.10 Model by L.Zhao and E.Macau.

3.5 An oscillatory network model of the brain visual cortex.

3.5.1 Some known facts on the brain visual system.

...