Abstract
This dissertation describes the physical reality of how mental objects, such as perceptions, inner visions, thoughts and feelings exist in the brain in the form of patterns of neural activity.

The text describes the medium in which the patterns exist, i.e. the "hardware" of the brain:
- the main functional component, the neural network, and its properties, and
- the interconnection between neural networks in the brain.

The text further examines the operation of pattern processing in the brain's assembly of networks - the storage and recall of patterns in the networks - and how this relates to familiar properties of the mental process of "thinking," such as the nature of the transition from one perception to another, and the drawing of a conclusion from several perceptions that are, or recently have been, active in the brain.

The essay shows the experimental results of how a mental object, that appears in the brain and in consciousness, is accompanied by a set of EEG oscillations. Based on the notion of "ensemble coding" of mental objects, it is argued that the properties of the oscillations can be used to form a "signature" that uniquely corresponds to the mental object.

This suggests an interesting possibility. By comparing ongoing EEG to known signatures of mental objects, it can be possible to externally determine what mental objects pass through the consciousness of a person, and that it can be possible to externally display what a person is "thinking" - like a film on a screen.

Introduction:
1. The Format of Information in the Central Nervous System
1.1 Patterns of Neural Activity
1.2 The Input of Information to the Brain is in the Form of Patterns
1.3 The Output from the Brain is in the Form of Patterns
1.4 Mental Activity as the Processing of Neural Patterns

2. The Hardware of the Brain
2.1 The Neural Network
2.2 Neural Network Property 1: Capability to Store and Recall Activity Patterns
2.3 The Biological Reality of Neural Activity Patterns
2.4 Cortex Contains Many Neural Networks
2.5 Neural Network Property 2: A Neural Network Goes from Pattern to Pattern
2.6 The Hierarchy of Networks in Cortex

3. The Dynamics of Patterns in Cortex
3.1 Flow of Activity
3.2 Mental Objects Occupy Cortex and Consciousness One by One

4. The Structure of Information in the Brain
4.1 The Representation of Objects and Concepts in Cortex
4.2 Ensemble Coding

5. Activity Pattern Processing Mechanisms
5.1 Attention
5.2 Chains of Related Perceptions
5.3 Deduction (Drawing of Conclusions)

6. Brain Research Progress
6.1 Overview of Methods
6.2 EEG Oscillations
6.3 EEG Oscillation Profiles

7. The Utilization of EEG Analysis to Externally Display the Information Contents of the Brain

8. Final Notes

References

Introduction
The input to the brain, the sensory impressions, come to the brain as patterns of neural activity.

The output from the brain also has the format of neural activity patterns, which in ordered sequences are fed to the pyramidal neurons of the primary motor cortex to run the motor system.

It is possible to describe the intermediate processes, such as visions, emotions, perceptions, and the combining of perceptions to form deductions ("thinking"), as the processing of neural activity patterns. The paradigm promises to lead to a full grasp of the physical mechanisms that underlie the kaleidoscope of experiences that make up our mental life.

The aim here is to describe the mechanisms of pattern processing in neural networks as physical processes, which would give an understanding of the workings of the Central Nervous System with the same precision as we can describe how a television picture is produced by the electronics in the TV set, or how the constellation of logical gates in a computer brings about the calculations and actions of the computer.

1. The Format of Information in the Central Nervous System
1.1 Patterns of Neural Activity
If we look at a TV screen, up close with a magnifying glass, we see dots. Red, blue and green dots. The dots vary in luminosity, they flicker. However, watching the dots as such does not say much of what is going on. If, however, if we take a step back the dots merge into shapes and images. The collection of dots come together to form meaningful information. They combine to represent faces, or sceneries, or text. They become information with content.

But, we can note, the ultimate constituents of the information are dots, small elementary pieces of data that as such do not have much meaning. The pattern of the dots constitutes information.

The same thing can be seen in a computer. On a microlevel a computer contains small pieces of information that have the values 0 or 1. Nothing else. The computer information, seen up close, consists of "dots" of 0 and 1. However, the 0s and 1s, bits, are by convention collected into 8-bit groups, bytes. A byte can contain 256 different combinations of 0s and 1s, and so we can, in a byte, have each of the 52 small and capital letters of the alphabet represented.

Several letters can form a word. Several words can form a sentence. And so the computer can, in the combination of bits, contain meaningful information. The pattern of the bits constitutes the information in a computer.

We have the same kind of arrangement in the brain. Information is contained in the brain as patterns of elements of neural activity.

We can immediately realize this, if we have a look at the machinery of the eye. The retina, at the back of the eye, contains 5 million cones and 120 million rods as sensors of light. It is immediately obvious that an image that falls on the retina is converted into 125 million neural signals. The image is converted in to a pattern of 125 million "digits" of neural activity. Together they represent the image, and information, that the eye receives.

1.2 The Input of Information to the Brain is in the Form of Patterns
All information that the brain receives, arrives as combinations of activity in the neural axons that are projected to the brain, that is, as patterns of neural activity.

For example, the primary visual cortex, the palm sized area at the lower rear part of the cortex, receives a set of neural signals from the retina that is topographically organized as the retina. The image in the eye projects to the cortex of the brain. The neural signals from the retina have, albeit, already been processed so that the neural signals in cortex represent edges and lines in the image. But the fact remains, the input to the primary visual cortex has the form of patterns of neural activity.

The cochlear apparatus of the ear separates the constituent frequencies in a sound that is received by the ear, and transmits these as neural signals to the primary auditory cortex in the brain. The primary auditory cortex is a strip of cortex at the upper rim of the temporal lobe, in which neurons sensitive to different frequencies are tidily organized, from front to back, in rising frequency order. As a distinct sound is made up of distinct frequencies, a sound that is picked up by the ear will be represented by a distinct set of active neurons in the primary auditory cortex, i.e. a pattern of activity.

The same applies for the sense of touch. Sensations received by the skin project to the brain as patterns in the primary somatosensory cortex of the brain.

In the sense of smell different olfactory receptors react to different molecules in air, and the chemical composition of substances present in air, passing through the nose, is reflected in patterns in the olfactory neural tract projecting to the brain.

Likewise different taste receptors in the tongue react to different molecules of what has been put in the mouth, and the neural activity patterns from the gustatory receptors give the brain information of what substances are about to be ingested.

In summary, the information that the brain receives from the various sensory apparatuses all arrive in the form of neural activity patterns.

1.3 The Output from the Brain is in the Form of Patterns
The muscle system of the body is run through the primary motor cortex, an elongated area which lies as a band in front of the central sulcus (fissure, divide) of the brain.

The figure shows how the motor cortex is connected to the body. Distinct points, distinct pyramidal neurons, in the primary motor cortex activate distinct muscle groups in the body.

It is quite clear to see that organized movements of the body are brought about by the brain feeding organized sequences of neural activity patterns into the primary motor cortex.

1.4 Mental Activity as the Processing of Neural Patterns
As the input to the brain and the output from the brain are in the form of neural activity patterns, it can easily be presumed that also the intermediate processes - which, experienced from the "inside," appear as mental visions, perceptions and feelings - operate with patterns of neural activity.

Different branches of neural science have, during the recent years, produced a wealth of new findings, and these begin to form a critical mass from which a picture of how the mechanism operates, with patterns, is beginning to emerge.

2. The Hardware of the Brain
2.1 The Neural Network
The central piece of the hardware of the brain is the neural network. The network performs the fundamental brain operations, somewhat akin to the transistor in the electronic circuits of a television or a computer. In these pieces of machinery combinations of transistors, together with capacitors and resistors, form the circuits that produce complex meaningful operations. In the brain the combination of several neural networks, together with other Central Nervous System components, produce complex meaningful operations which make up purposeful human behaviour. (The analogy between transistor and network is somewhat stretched but, in essence, the neural network is a unit that performs processing of activity patterns.)

There is reason to recap the central properties of a neural network.

A neural network can be modelled in a simple physical mechanism, which can be studied in computer simulations. This model captures the essential properties of a biological network, and the simplified mechanism shows the fundamental properties of pattern processing. These form the core of the here presented hypothesis.

The properties are phenomena which represent a new paradigm of scientific thinking. They are somewhat counterintuitive, which can explain why a theory of the operation of the brain has been slow in forming.

Teuvo Kohonen (research professor, presently emeritus, at the Technical University of Helsinki, Finland) has worked with simulated neural networks and has demonstrated the intriguing properties of the networks, [2]. When several Kohonen-networks are assembled to circuits, mechanisms with very interesting properties and capabilities emerge.

A neural network, as encapsulated in the theoretical model by prof. Kohonen, can be defined with the following "axiomatic" properties:

• A neural network is made up of large set of neural columns, organized as a flat matrix. A neural column (so named, to correspond to the neural column of cortex) is a unit that can attain various levels of activity.


• Each neural column in the matrix is connected to each other neural column in the matrix. (In biological networks a neural column is made up of several neurons, and each neuron is connected to several thousand other neurons. The connections within a cortical network are shown in fig. 3, as the bands of Ballinger and the plexiform axon layer.)


• The connections between the neural columns are made up of synapses, which are malleable connections, which alter their conductivity according to "Hebb's law." Cursorily Hebb's law can be expressed as "A synapse increases its conductivity in proportion to the combined activity of the sending and the receiving neuron." The quantitative mathematical version of the law can be found in Kohonen's work [2].


• An active column in the network influences its neighbours in such a way that the nearest neighbours are excited and the ring-shaped set of columns outside the excitement zone are inhibited. Fig. 4. (This has been called a Sombrero, or Mexican hat, neighbour interaction function.) The local neural interaction is an essential component of the neural network. It has the effect of setting up a competition between patterns that are stored in the network, to allow one of them to emerge as the active constellation of columns, the active pattern, as described below.

2.2 Neural Network Property 1: Capability to Store and Recall Activity Patterns
The central exceptional property of a neural network is, that it can store and recall patterns of activity. When a specific combination of activities in the neural columns - a specific pattern - is active in the network, the pattern can be imprinted into the network.

This combination of activities, this pattern, becomes something with an identity of its own (as in a specific memory, or a vision). It becomes a unit of information. It can later be made to re-emerge in the network, and so we have, in a network, a mechanism that can store and recall information.

A pattern is stored in the network when the pattern is fed to the network through an afferent tract of axons, which causes a set of neural columns to become activated, and the conductances of the synapses are allowed to adjust according to Hebb's law.

The remarkable, slightly counterintuitive thing is, that several patterns can be fed to and stored in a network. The network can contain several imprints of patterns.

The neural network has the property of being able to recall a pattern that has been stored. For this the network is placed in a state where the synapses do not adjust, which in biological networks can be taken to be brought about through an altered chemical state.

Recall then works as follows:

• When the network is subjected to a pattern that is part of, or approximates, one of the stored patterns, the whole of the stored pattern emerges as activity in the network. The inciting pattern could, for the following discussion, be termed a key pattern.


• The pattern that is recalled into the network is the one that is most like the key pattern. "Most like" can, in the ideal case of the model, be precisely defined as the pattern for which the sum of the square of the differences between column activities, of the stored patterns and the key pattern, is a minimum.


• The pattern that is recalled is not an exact replica of the original pattern. Some cross-over effects between patterns stored in the network occur, but if the stored patterns are approximately orthogonal (constituting a unique set of activities), the recalled pattern can be distinguished as the original pattern, and will in subsequent neural processing act as the original pattern.

2.3 The Biological Reality of Neural Activity Patterns
Fig. 6, from [35], illustrates the biological reality of a neural activity pattern. The figure shows a section of rat cortex where the positions of the active neural columns are displayed.

The pattern in the figure can perhaps be assumed to be typical for biological networks. If so, one can observe that the pattern is relatively sparse, and this would give an indication of the predisposition for uniqueness (orthogonality) of patterns in biological networks, and an explanation the large storage capacity of biological systems.

2.4 Cortex Contains Many Neural Networks
Cortex is a 2mm thick sheet of neural elements, which in a human has the spatial extent of some 2500 cm2, i.e. approximately the size of four A4 paper sheets. Cortex contains neural columns, "tubes" extending through cortex, which have a diameter of approximately 10 micrometers, and are built up of several neurons, of different kinds. The neural columns each operate as a unit, and can attain various states of activity. In pattern processing they correspond to a dot on the TV screen, or a bit in a computer.

The neural columns are organized into neural networks, subunits of cortex, which approximately correspond to the Brodmann areas of cortex.

A neural network receives and sends out patterns of activity through tracts of axons, to other cortical areas/networks, or to subcortical structures, such as the thalamus or the basal ganglia, Fig. 3.

2.5 Neural Network Property 2: A Neural Network Goes from Pattern to Pattern
One particular property of the neural network is of special significance. When all the metabolic processes are running in a cortical area (e.g. the ion pumps in the membranes of the neurons are working to maintain electrical potential differences) a pattern, that is active in a neural network, represents an energy dip for the network. Consequently it is a stable state for the network, and a state that the network will gravitate to.

Each active pattern represents a local energy minimum, and for this reason, when the network is subjected to a "disturbance" flowing in through the afferent (input) axon tracts, the network can be "pushed" to shift away from one pattern, but will after this settle in another pattern in the network. The network will go from pattern to pattern.

2.6 The Hierarchy of Networks in Cortex
Cortex contains many networks. Cortex can be divided into distinct areas based on cytoarchitectural and axon tract connection criteria, and in 1909 Brodmann made an initial determination of approximately 50 distinct areas for each cortical hemisphere, i.e. some 100 areas in all. Fig. 2. The definition of areas has since been refined and to some extent subdivided. All the same, cortex consists of areas, forming neural networks, that can receive, store and emit neural activity patterns.

Felleman & van Essen [3] have charted the connections in the visual areas of the macaque monkey, and found various types of connections, which run from distinct layers in cortex to end up in specific layers in cortex. When the areas were arranged according to the types of connections, it turned out that the areas settled into a hierarchy, and connections could be characterized as ascending (forward), lateral and descending (feedback) connections.

The forward connections run from the primary visual cortex, through 8 levels of areas in the temporal and parietal cortices, up to areas in hippocampus and the frontal lobe.

3. The Dynamics of Patterns in Cortex
3.1 Flow of Activity
Halgren et al. [36] have, with MagnetoEncephaloGraphy, MEG, recorded the progression of activity over cortex after a visual impression arrives to the brain, i.e. when a person is shown a word, and it connects to, and activates, an object/concept in cortex. The progression of activity is pictured in figure 8. (The time axis runs from left to right, and the various horizontal rows show different views of the brain and different experimental conditions.)

We can see that the activity starts in the primary visual cortical areas, and then flows out "all over" cortex, approximately along the forward connections in the Felleman & van Essen hierarchy of cortical areas. The activity is (in this case) especially strong in the ventral section of the temporal lobe. It also flows over into the frontal lobe.

We can further observe the rate of the progression of activity. It reaches a maximum at approximately 250 ms after presentation of the stimulus. MEG registers the electrical current in the pyramidal neurons flowing perpendicular to the surface of cortex, and this can be assumed to reflect the establishment of patterns in areas of cortex.

I have, myself, carried out experiments of a similar nature with EEG, and seen similar results. In an experiment at the University of Cape Town (UCT) a test person was presented with a number on a computer screen, and EEG was recorded at the event. A 10Hz oscillation band was extracted from the EEG, and the intensity of the 10Hz oscillation was plotted over time, for EEG recorded in 128 different points on the scalp. The result is shown in figures 15a-e.

The 10 Hz oscillation is presumed to reflect the presence of a pattern in a network. The development of activity, in Figs. 15a-e, is very much like the progression in Fig. 8.

(Fig. 8 description: The activity is recorded with MagnetEncephaloGraphy, MEG. Increasing time runs from left to right. The various horizontal rows show different perspectives of the brain and different objects presented to the test person. One can observe how the activity begins in the primary visual cortex, and then spreads out over "the whole of cortex", reaching also to the frontal lobe. Maximum MEG activity occurs at approximately 250 ms.)

3.2 Mental Objects Occupy Cortex and Consciousness One by One
An important point to note, apparent in the evidence of the figures 8 and 15a-e, is that an object/concept that is registered by the brain essentially fills up the cortex.

This is also in congruence with an observation that one can reach through introspection, namely that objects/concepts follow each other in consciousness one by one. If an object/concept, when it is active in cortex and present in consciousness, essentially occupies the whole of cortex, then objects/concepts have to be processed sequentially, one by one.

This, combined with the observation in 2.5, that a network goes from one pattern to another, leads to the presumption that cortex basically goes from one global pattern to another - from one perception grabbing the whole of cortex, to the next perception, occupying "all" of cortex.

4. The Structure of Information in the Brain
4.1 The Representation of Objects and Concepts in Cortex
An object/concept, that is present in cortex, has different representations in different parts of cortex. Different features of an object/concept are present as patterns in different cortical areas.

At the lowest level in the visual system, in the primary visual cortex V1, elementary features of an image that falls on the retina, such as lines and edges and the direction of the lines in the image, are represented by activity in distinct neural columns. When we move up the Felleman & van Essen hierarchy an abstraction of features takes place, so that, in the following visual areas, columns in V2 indicate the presence of combinations of lines, such as corners or T-junctions, in V3 neural activities represent object form and depth, in V4 color and in V5 motion. [42]

Different types of features are assembled along different pathways in the area hierarchy. Basically three pathways have been distinguished - an upper pathway, into the dorsal parietal cortex, a middle pathway, along the lateral parietal cortex, and a lower pathway, into the ventral temporal cortex. The upper pathway has been determined to extract and represent the position of an object, in body-relative coordinates. The middle pathway represents the relative positions of objects in a visual field, and the lower pathway has been found to generate patterns that indicate the type and identity of an object [34].

It is quite remarkable, and somewhat counterintuitive, that an object that is projected to the retina, in all the shapes and forms it can have as an image, can activate a distinct pattern in the ventral temporal lobe that represents the type of object that is seen, say a shoe or a building. This, however, has empirically been proven to be the case - Figs. 9, 10, ref. [4].

This can be explained with network mechanisms. Several patterns, in a lower level sending network, can project to a specific set of neural columns in a receiving network, and this will serve as a common identifier for an object. See [31] p. 107 for a more detailed account of the mechanism. Many images of cat can activate a pattern higher up in the cortical hierarchy that in the brain system indicates that the object "cat" has been seen.

The emergence of such a setup can fairly easily be explained. At some point in the lifetime of an individual, a specific object will be the focus of interest. Say, a cat. At this point a pattern in a low level area, representing the current image of the cat, and a pattern in a high level temporal area, representing an advanced feature composition for the cat, will be active, and the patterns in the two areas will be linked through synaptic adjustment.

At some later point in life the same object will again be the focus of attention, and the two patterns in the high and low level areas will be active again. At this point in time additional images, with accompanying low level patterns relating to the same object, will be active and, as the high level pattern for the object has been activated, the new active low level patterns will be linked to the high level pattern. Through Hebb's law. The high level pattern will come to serve as a common object indicator for many images of the object, i.e. an object type abstraction for a cat.

(Additional reflection: It is short mental leap to imagine that the object type patterns are linked to words, as words are labels for types of objects (house, cat, food, rain,...) As we know, words are channelled to the muscle system of the mouth/larynx for verbal expression through Wernicke's area, which resides at the junction of the temporal and parietal lobes. One can easily imagine that objects/concepts that are present in consciousness and that are about to be verbally expressed, are present in the temporal lobe as patterns representing the type of object, and that these feed patterns into Wernicke's area for forwarding to verbal expression of the object.)

4.2 Ensemble Coding
The findings of Haxby et al. ([4], figs. 9, 10) support the hypothesis of "ensemble coding," which supposes that an object in cortex exists as a collection of features in the object. Fig. 11 illustrates the idea.

The object "Granny" exists in cortex as the simultaneous occurrence of the features of her face shape, her grey hair, her glasses, her customary floral dress, etc. As different types of objects activate specific areas prominently, one can suppose that the component features in Granny (objects in their own right) activate specific loci especially distinctly. Granny consists of a macro-pattern of feature activities - her distinct features.

One could perhaps even see the saccadic scanning of a object under scrutiny as supporting ensemble coding. When studying an object, the eyes skip from feature to feature. In a face, say, the they move from eye to eyebrow to hair to mouth, and so on, absorbing each feature by direct gaze, through the fovea (the area with the high resolution in the centre of the retina). Each prominent image detail receives a moment's special attention, and, obviously, activates its own cortical pattern. And, in line with the findings in [4], and as illustrated by figs. 9 and 10, each of these can be supposed to be represented by an activity pattern of special prominence in a distinct area of cortex. Granny then, when regarded, becomes a macro-pattern of her features.

5. Activity Pattern Processing Mechanisms
5.1 Attention
Most people are familiar with the so-called cocktail-party effect. In the din of a cocktail party one can follow and concentrate on a specific conversation - on the theme, and/or on the distinguishing sound of the voices having that specific conversation.

And when it comes to attention, more specifically, attention can be directed to a certain modality, to a section of the visual field, to a particular object in the field, or to a particular aspect/feature of a perceived object (form, color etc.), and, as said, to the feature subject-of-conversation.

Attention can, with volition, be directed to a distinct aspect of a perception.

This fits nicely in with ensemble coding. If it can be assumed that different features of a perception are represented by patterns in different cortical networks, one can very easily come to the conclusion that the process of focusing attention by volition in physical terms means, that the Central Nervous System increases the neural activity of a certain area in the cortex.

It is known that the CNS can, by volition, increase activity in specific areas of cortex. An overview of activation effects reported in literature can be found in [1], pp. 254-300.

There are structures in the brain that can serve as attention focusing, and cortical area activity elevating, agents. The prime candidate is the prefrontal cortex, which, through descending pathways to the ventral brain stem, and directly to thalamus, selectively can activate regions of cortex through ascending projections in the thalamus [38]. The basal ganglia could also play a role, in that they collect neural patterns from all over the cortex, and, through the projections to the substantia nigra, can activate sections of the thalamus and ascending projections to cortex.

The conundrum of how volition is generated, is a separate interesting question, which as a physical process has not received much attention yet. However, indications are that selection of volitional focus takes place by the projection of patterns, that have been activated in cortex (objects, goals, visions), to the limbic, emotional, system for "evaluation", and stimulation of the organism through nucleus accumbens if a subject that is passed through cortex, by the limbic system is found worth to pursue.

This, again, is a simple network mechanism. It operates precisely as the mechanism for determining the type of an object, as described in 4.1. This time, however, the projection from the pattern to evaluate is directed to nuclei in the CNS that causes a general arousal in the CNS system, and the whole individual gets activated, and "motivated."

5.2 Chains of Related Perceptions
From introspection, and from general psychological observation, we know that thoughts tend to run in chains. An object/concept in consciousness tends, after some time, to be replaced by another object/concept in consciousness, and the new perception nearly always seems to be related, in some way, to the previous "thought." This can be seen to derive naturally from the ensemble coding of objects and network key activation mechanisms.

The cortical areas and networks are extensively interconnected. Fig. 7. When an object/concept is active in cortex, various features are active as patterns in different areas of cortex. These feature-patterns become, in the development of a human, interconnected, through the fact that they are simultaneously activated, and trans-area connections become established through synapse adjustments in linking axon tracts. An object that is present in cortex will come to form a macro-pattern, that stretches over areas containing the object features.

When attention is directed towards a specific feature of a perception, the activity of a specific section of cortex is raised. A specific feature of a perceived object is activated. Say Granny's glasses. The activity in the network for the glasses-feature becomes stronger, and dominant. One can now envisage that the area for glasses begins to emit patterns through the trans-area connections, and that various features of glasses (shape, material, color, ..), through key-activation in networks, become active in cortical areas. The macro-pattern for the object of glasses in totality rises in activity, and becomes the dominant object in cortex, and in consciousness. The object Granny smoothly links on to the object glasses.

The hypotheses of ensemble-coding and neural network storage/recall provide a natural physical explanation for the phenomenon of the progression of one perception to another related perception, i.e. to the phenomenon of chains of thought.

5.3 Deduction (Drawing of Conclusions)
Humans are capable of deduction. From a multitude of perceptions gathered from the environment, they are capable of drawing conclusions, and devising behavioural strategies.

This is a valuable capability. The animal species Homo Sapiens clearly has this capacity especially well developed, and the capacity has made this particular species exceptionally successful in an evolutionary, Darwinistic, sense.

A suggestion for how the process of deduction works can be devised with neural networks. A moments reflection will make it obvious that the process of deduction operates according to the scheme that the brain collects facts and operational rules into working memory, and that these generate mental objects/concepts, which constitute deductions.

For example:

• "3", "+", "4" => "7"
• "All men are mortal," "Socrates is a man" => "Socrates is mortal"
• "6", "next-larger-number" => "7"

In the process of deduction, two or more items are brought into consciousness and into "working memory", and, as a result of this, a new item appears in consciousness and working memory. In physical terms this would correspond to the process that two or more patterns (global patterns) are, in sequence, brought to activity in cortex and, following this, a new pattern emerges into cortex.

It can be noted that this process, in the simple form it now is examined, is deterministic. The objects "3", "+" and "4" bring up "7", every time. "3", "+" and "4" together form a key that in the cortical networks recall "7".

This is obviously a consequence of memorization, working just like the mechanism of storage and recall shown in fig. 5. However, here the objects "3", "+" and "4" collectively recall "7."

We can, at the present initial stage, draw up a crude suggestion for how this works.

The brain contains interdigitated networks. These networks receive afferent axon inputs from different sources, and the axons from the different sources project to different columns in the receiving network. One example of such a network has fairly well known, the striate primary visual cortex, V1. It receives input from the two eyes to different columns residing in the same network.

In literature we find also other interdigitated networks described:

• [40]: "In certain common target areas, as for example the cingulate cortices, frontal and parietal efferents terminate in an array of interdigitating columns, an arrangement much like that observed for callosal and associational projections to the principal sulcus."


• [41]: "The projections from posterior parietal and prefrontal cortices interdigitate as separate columns in the cingulate and parahippocampal cortices."

This suggests an interesting mechanism.

Assume that A and B are networks, and that A contains a pattern X, and B contains a pattern Y. A and B have axon projections to the interdigitated network I. In an arrangement like this the network A will feed an extract of the pattern X to the interdigitated network I, and the network B will feed an extract of Y into I. The combination of input patterns in I can be labelled the union pattern U.

One can now in principle envisage the effect that, when A and B contain X and Y, the interdigitated network I will contain U, but not otherwise. If A contains pattern Z, then the union pattern U will not appear in I, but some other pattern V.

This elementary simple circuit could be the arrangement, and the mechanism, for simple automated decisions, for example for the case that one network contains the representation of body position and a second network contains the representation of the intended position for a hand. The resultant pattern can produce a neural activation that via the motor system initiates an appropriate hand movement.

The arrangement can also be the basis for more complex neural operations, such as deduction. For this, however, there are still additional issues to consider.

Assume a network P that contains the abstraction of the procedure "next". (Here we, naturally, have something that is difficult to visualize - a neural pattern that has the meaning of the operation "next". But it is conceivable. For an average individual the operation next-in-a-series has been well rehearsed, and ingrained in a (presumably) prefrontal network, in school and in life. One can reasonably realistically assume the existence of a neural pattern that has the significance of "next", and this, in our example, is situated in P.)

Secondly we assume a pattern representing the object "6" in a network O, which e.g. has filtered in from the visual perception of the number 6.

The networks P and O feed the extracts of "next" and "6" into the interdigitated network I and the combination, the resultant pattern, will be a pattern that, after a delay, running through a transfer circuit, will bring up the object of the number "7" into the areas where it can be perceived, and where it is brought into consciousness.

The "6" and the "7" obviously occupy the same areas, for the reasons that:

• the "6" is extinguished from consciousness when "7" appears,


• the cycle of fetch-the-next-larger-number can be performed any number of times, i.e. we can have an endless row of numbers appearing in the areas of cortex and in consciousness (i.e. no memory limit),


• the object produced by deduction, the "7," can be used for mental operations precisely as the original "6," i.e. for deduction, for memorization, or for output.

It is not yet clear what the transfer circuit is, but one candidate can be the axon tracts from cortex to thalamus, which in turn, through the connections running from thalamus to cortex, can feed in the deduced object into cortex, somewhat in the same way as the sensory patterns through thalamus are fed to cortex, via the primary sensory areas.

There are other circuits in the brain that also could be the medium for the cyclical process of deduction, e.g.

• cortico-cortical connections


• the loop cortex - basal ganglia - thalamus - cortex


• the loop cortex - cerebellum - cortex

and within these groups different channels and variations. It can be expected that, through experiments and careful analysis, as described in the following, it will be possible to pin down the structure(s) that forms the loop in which deduction runs.

6. Brain Research Progress
6.1 Overview of Methods
In the last decades several methods for studying the living brain have been developed, e.g.

• Positron Emission Tomography, PET


• Magnetic Resonance Imaging, MRI


• functional Magnetic Resonance Imaging, fMRI


• MagnetoEncephaloGraphy, MEG


• ElectroEncephaloGraphy, EEG, charting Event-Related Potentials, ERP

6.2 EEG Oscillations
Christina Krause (professor, Helsinki University) has published some works on EEG oscillations. In [38] we find an illustration, Fig. 12 below, which shows the progression of EEG oscillations in different frequency bands, in ongoing mental activity, registered with an electrode over occipital cortex.

If we have a closer look at the oscillations for the 7-10 Hz frequency band, we can see that "waves" of oscillations come and go, with each "wave" lasting some 0.5 to 1 seconds. One immediately gets the impression that we here see patterns of activity in a network (under the electrode) come and go, each accompanied by a surge of ~10 Hz electrical oscillations.

Krause ventures a guess that the 7-10Hz oscillations stem from thalamocortical network connections. It is easy to visualize a circuit in thalamus where excitatory nuclei rhythmically interact with the inhibitory reticular nucleus, to produce oscillating neural activity. An oscillator in thalamus would give a natural explanation to a phenomenon that many researchers have observed, the fact that synchronized oscillations can be recorded in spatially widely separated areas in cortex. A centrally situated oscillator, in thalamus, can emit synchronized oscillations in different directions, to different areas in cortex.

One could also note that it intuitively seems advantageous that a pattern is accompanied by an oscillation in activity, generated by a central oscillator in thalamus, as this would have the effect of "pumping up" patterns in the cortical networks, to spread out the patterns into all the neural columns in the respective networks.

This, I believe, merits a closer investigation. In 2004-2005 I undertook experiments at the University of Cape Town (UCT), with the aim of seeing if areas of oscillations could be distinguished, which (presumably) would correspond to the intermittent activation of thalamocortical connections to different areas, and to active patterns.

6.3 EEG Oscillation Profiles
The setup of the experiments was designed to produce simple elementary operations in the brain of a person, to bring out the most simple cortical activity and pattern constellations possible. In the experiment the test person sees a number on a computer screen, and mentally generates the next larger number. After this a new number is shown on the screen, and if this is the correct next-larger number, the test person confirms this with a click of a button.

The relevant sections of the experiment are: 1) the visual perception of an object, 2) the mental operation of generating the next-larger number, and 3) the appearance of the deduced object in cortex. The confirmation only has the purpose of keeping up the attention of the test person.

EEG was recorded at 128 points over the scalp. The seeing of the prompt number, the mental operation and the appearance of the next-larger number in cortex occur in approximately 800 ms. The EEG data was analyzed for frequencies in a 200 ms interval in the experiment. The 200 ms interval was in 10 ms steps "pushed" along the 800 ms, to find the frequencies occurring in the different phases of the experiment. For each step the frequencies were extracted with a Fourier transformation (fft, Matlab).

One can see quite a fascinating kaleidoscope of frequencies come and go. Fig. 13 shows a typical obtained Fourier frequency power diagram.

One striking feature emerges. The 10Hz oscillation power peak occurs very prominently in a large number, the majority, of the diagrams.

10 Hz data was then collated and displayed as activity in all the electrodes simultaneously, in snapshots along the 800 ms interval, to give a picture of how the 10Hz activity fluctuates over the scalp in the course of the mental processing. The figures 15a-e show the result. (Black - no activity, white - maximum oscillation.)

Observations:

• At 20 ms after stimulus (first number) presentation activity is low all over cortex, (15a)


• Activity then rises


• The activity spreads out over nearly all of cortex, and reaches a peak at approximately 200 ms, (15b). This, approximately, is the time that the perception of the number appears in consciousness.


• Then, interestingly, the 10Hz activity goes down over most of cortex. The 10Hz oscillation appears to be actively suppressed, (15c).


• After this the next-larger-number begins to appear in cortex, and the 10Hz activity for this object reaches its peak at 420 ms, (15d).


• In the activity that is recorded one can observe that the object/number, that is produced by the mental operation into cortex, activates a distinct set of electrodes. Activity occurs in distinct areas of the left parietal and the right frontal lobes.


• At 800 ms the same oscillation configuration still lingers on, (15e). This obviously is a manifestation of the "working memory" activity of the object, as it is retained in cortex/consciousness in anticipation for use at result verification later in the experiment.

7. The Utilization of EEG Analysis to Externally Display the Information Contents of the Brain
There is a large amount of data, besides 10 Hz oscillation profiles, that can be extracted from EEG. Briefly summarized we can, for each object/concept that passes through the brain, determine

• all the occurring frequencies of oscillations, in all electrodes. (It is a fairly simple matter to record EEG in 128 points over the skull, and obtain a fairly dense measurement matrix.) There are many frequencies present.
  
  Fig. 13 shows an example of frequencies registered by one electrode at a specific point in time.


• the amplitude of the various frequencies


• the progression of the amplitudes (rising, falling, development profile)


• determine the phase synchronization of oscillations of different frequencies in different points. (Ref. [44] describes some applicable measurement methods.)


• determine the temporal sequence of area activations.

All these measurements form a "signature" of brain activity for each object/concept that gets activated into the brain. It would be a fairly simple matter to chart the signatures for various elementary objects and concepts, and set up a database with signatures for various perceptions / mental objects.

The potential uses of such a database would be very very interesting:

1. With such a database it could be possible to set up a system which would be able to display, as images on a computer screen, the objects/concepts pass through cortex and through the consciousness of a person.
   
   The process of "thinking" could be displayed as a film on a computer screen. And, in this we would have at our disposal a tool with, just simply, immense potential.


2. The database could contain a large number of "traces of thinking" of clinically normal individuals, and then oscillation maps of mental processing from individuals with known mental deficiencies.
   It is almost certain that the maps of normal and pathological cases differ. If so, we would have a
   tool for strictly objective evidence of pathology,
   for clinical use.


3. The Psychology Department at the University of Witwatersrand (South Africa) works with remedial programs for children that have learning difficulties. From Wits has been suggested that EEG oscillation tests could be used for testing school maturity for learners. We know that the myelination of the axons in the frontal lobes mature in the early school years, and an EEG oscillation test could most probably easily reveal the maturity of a child's mental processing equipment. With an early assessment of a child's school readiness it would be possible to avoid costs and unnecessary frustration with children who, just simply, would need to wait a year or so to attend school successfully.


4. Explanation of the Efficacy of Neurofeedback:
   It is an empirical fact that volitional regulating of brain EEG frequencies can lead to beneficial health effects, such as reduced anxiety or alleviated depression.
   
   This is the principle for the therapy method of Neurofeedback.
   
   (The method has fairly recently become possible, in that electrical brain oscillations with a computer can be displayed on a monitor screen.)


5. However, it has not been possible, yet, to explain why and how Neurofeedback has its beneficial effects. With a more full study of:
   - the dynamics of neural activity patterns,
   - oscillations,
   - the interaction of patterns with subcortical CNS structures, and
   - how these match up with mental sensations,
   - a way to understanding how Neurofeedback works seems to be opening up.

8. Final Notes
8.1 Brain Data and Oscillation Signatures
We do not yet know from what neural circuits and oscillators, the oscillations, of all the various frequencies in which they occur, stem. They can originate in patterns that are active in areas, in functional connections between areas, in connections between cortex and deeper CNS structures, and also come from the deeper CNS structures. This will, by and by, become clear when oscillation data is collected.

However, we can note that a perception of an object is a distinct well defined piece of information, that can be evoked time and time again. An apple is perceived as a round, red, fragrant, edible fruit yesterday, today and tomorrow.

An apple that is present in consciousness constitutes a distinct reproducible state of cortex, and this state is almost certainly reflected in a distinct set of properties in the oscillations that can be recorded in EEG.

Because of this there is all reason to undertake an attempt to set up a database of EEG profiles for various perceptions.

8.2 Experiment and Measurement Conditions
As we can see from Fig. 15b, the processing of sensory visual input activates a large number of areas. The transformation of the visual signals to the high-level object representations is a process that requires a large portion of the brain machinery.

On the other hand, when an object is activated into consciousness and cortex through mental processing, we will, obviously, have basically only the high-order patterns, which constitute the core representation of the object, activated. Fig. 15d.

The EEG recordings for building the database of EEG signatures are naturally best done with perceptions of objects that are generated through mental activity (through reasoning, or recollection). This will yield bare essential macro-patterns for mental objects, stripped of activity for sensory processing.

The measurement methods to obtain data for the database have to be designed accordingly. (I have some experience in this. I will be happy to talk with you about this, should you embark on setting up a EEG signature - perception database.)

References
[1] Cognitive Neuroscience - The Biology of the Mind
Michael S. Gazzaniga, Richard B. Ivry, George R. Mangun
W. W. Norton & Company, 2002

[2] Self-Organization and Associative Memory
Teuvo Kohonen
Springer Verlag, 1984

[3] Distributed Hierarchical Processing in the Primate Cerebral Cortex
Daniel J. Felleman, David C. van Essen
Cerebral Cortex Jan/Feb 1991; 1047-3211/91

[4] Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex
James V. Haxby, M. Ida Gobbini, Maura L. Furey, Alumit Ishai, Jennifer L. Schouten, Pietro Pietrini
Science, Vol 293, 28 September 2001

[5] Spatiotemporal localization of cortical word-repetition effects in a size judgement task using combined fMRI/MEG
Dale, A.M., Halgren, E., Lewinw, J.D., Bruckner, R.L., Paulson, K., Marinkovic, K., & Rosen, B.R.
Neuroimage, 5: S592 (1997)

[6] Artificial Intelligence
Patrick Henry Wilson
Addison-Wesley Publishing Company, Inc., 1984

[7] Self-Organizing Maps
Teuvo Kohonen
Springer Verlag, Berlin, 2001

[8] The Cerebral Code
William H. Calvin
MIT Press, 2000

[9] Consciousness / How matter becomes imagination
Gerald Edelman, Guilio Tononi
Penguin Books, 2001

[10] From Neuron to Brain
Stephen W. Kuffler, John G. Nicholls, A. Robert Martin
Sinauer Associates Inc. Publishers, 1984

[11] Cognitive Contextual Integration
Eric Halgren et al.
Center for Advance Medical Technology - LabCogNeuro
Salt Lake City, UT 84108, USU
Internet website, 2003

[13] Regional and cellular fractionation of working memory
Patricia S. Goldman-Rakic
Proc. Natl. Acad. Sci. USA
Vol. 93, pp. 13473-13480, Nov 1996

[14] A Computational Model of How the Basal Ganglia Produce Sequences
Gregory S. Berns, Terrence J. Sejnowski
Massachusettss Institute of Technology
Journal of Cognitive Neuroscience 10:1, pp. 108-121, 1998

[15] Computational Neuroscience
Website: www.geocities.com/dougnbr/neural1.html 2002

[16] Scripts, Plans, Goals and Understanding
Schank, R. C., & Abelson, R.
Lawrence Erlbaum Associates, Hillsdale, New Jersey, USA. 1977

[17] Model of Cortical-Basal Ganglionic Processing: Encoding the Serial Order of Sensory Events
David G. Beiser, James C. Houk
Derpartment of Physiology, Northwestern University Medical School
Chicago, Illinois 60611, USA
The American Physiological Society, 1998

[18] Sequence Learning in Model Frontostriatal Circuits
P.A. Simen, E.G. Freedman, R.L. Lewies, T.A. Polk
University of Michigan, USA
Website: www.eecs.umich.edu/~psimen
~2002

[19] A Computational Model of How the Basal Ganglia Produce Sequences
Gregory S. Berns, Terrence J. Sejnowski
Journal of Cognitive Neuroscience 10:1, pp. 108-121
Massachusetts Institute of technology, 1998

[20] A computational model of action selection in the basal ganglia.
K. Gurney, T.J. Prescott, P. Redgrave
Biologicl Cybernetics 84, pp. 401-410, 2001

[21] Parallel neural networks for learning sequential procedures
Okihide Hikosaka, Hiroyuki Nakahara, Miya K. Rand, Katsuyuki Sakai, Xiaofeng Lu, Kae Nakamura, Shigehiro Miyachi, Kenji Doya
Trends neuroscience 22, pp. 464-471, 1999

[22] The Role of the Neostriatum in the Execution of Action Sequences
John Randall Gobbel
PhD Dissertation, University of California, San Diego, USA, 1997

[23] The detection and generation of sequences as a key to cerebellar function.
Experiments and theory.
Valentino Braitenberg, Detlef Heck, Fahad Sultan
Behavioral and Brain Sciences 20 (2): pp. 229-277, 1997

[24] Parallel organization of functionally segregated circuits linking basal ganglia and cortex.
Alexander G. E., DeLong M. R., Strick P. L.
Annual Review of Neuroscience, 9, 357-381, 1986

[25] Mapping the Mind
Rita Carter
Phoenix, Orion Books Ltd, 1998

[26] A user's guide to the brain
John J. Ratey
Little, Brown and Company, London, 2001

[27] Cerebellum: Movement Regulation and Cognitive Functions
James C Houk, Lee E Miller
Encyclopedia of Life Sciences / Nature Publishing Group / www.els.net, 2001

[28] Self-Organizing Maps
Teuvo Kohonen
Springer-Verlag, Berlin Heidelberg, third edition, 2001

[29] Environmentally mediated synergy between perception and behavior in mobile robots.
Paul F. M. Verschure, Thomas Voegtlin, Rodney J. Douglas

[30] The Language Instinct
Steven Pinker
Penguin Books, 1995

[31] How the Mind Works
Steven Pinker
Penguin Books, 1999

[32] The Blank Slate
Steven Pinker
Penguin Books, 2003

[33] The Human Connectome: A Structural Description of the Humane Brain
Olaf Sporns, Guilio Tononi, Rolf Kötter
PLoS Computational Biology | www.ploscompbiol.org
September 2005 | Volume 1 | Issue 4 | e42

[34] Introducing Mind and Brain
Angus Gellatly, Oscar Zarate
Icon Books Ltd, 2005

[35] Attractor dynamics of network UP states in the neocortex
Rosa Cossart, Dmitriy Aronov & Rafael Yuste
Nature, Vol 423, 15 May 2003

[36] Declarative Memory
LabCogNeuro, E. Halgren et al.
Center for Advanced Medical Technology
729 Arapeen Dr., Salt Lake City UT 84108, USA

[37] Artificial Intelligence
Patrick Henry Winston
Addison-Wesley Publishing Company, 1984

[38] The Executive Brain - Frontal Lobes and the Civilized Mind
Elkhonon Goldberg
Oxford University Press, 2001

[38] Brain electric oscillations and cognitive processes
Christina M. Krause
Kluwer Academic Publishers, 2003

[39] Structural (Operational) Synchrony of EEG Alpha Activity During an Auditory Memory Task
Andrew Fingelkurts, Alexander Fingelkurts, Christina Krause, Alexander Kaplan, Sergei Borisov, and Mikko Sams
NeuroImage, 2003, V. 20. No 1.P. 529-542

[40] How the brain works
Paul Bush, 1996
Website: http://keck.ucsf.edu/~paul/brain.htm

[41] Brain architecture - understanding the basic plan
Larry W. Swanson (Professor, University of Southern California, Los Angles, USA)
Oxford University Press, Inc., 2003

[42] How the brain works
John McCrone
Dorling Kindersley Limited, London 2002

[43] The Cognitive Correlates of Human Brain Oscillations
Michael J. Kahana
The Journal of Neuroscience, 8 Feb 2006, 26(6):1669-1672

[44] Theta Oscillations in Human Cortex During a Working-Memory Task: Evidence for Local Generators
S. Raghavachari, J. E. Lisman, M. Tully, J. R. Madsen, E. B. Bromfield, M. J. Kahana
J Neurophysiol 95: 1630-1638, 2006.

Author Details
Bertil Osterberg
MSc in Engineering, 1970, from Åbo Akademi, Turku, Finland
BSc In Economics, 1977, from Svenska Handelshögskolan, Helsinki, Finland

20 years professional experience of computers and information processing - programming and building IT systems.

Post-graduate studies at the Technical University of Helsinki, 1985-7
Subjects: Artificial Intelligence, Neural Networks, The Physiology of the Brain

Post-graduate studies and research at the University of Cape Town, 2004
Subject: The physical mechanism for the information processing of the Brain.

Post:
PO Box 891404
Lyndhurst 2106
South Africa

E-mail:
bertil@icon.co.za

Mobile tel:
+27-82-461 3158

Tel:
+27-11-869 9826