Time and the Brain (Conceptual Advances in Brain Research, Vol 3)

Time and the Brain

Conceptual Advances in Brain Research A series of books focusing on brain dynamics and information processing systems of the brain. Edited by Robert Miller, Otago Centre for Theoretical Studies in Psychiatry and Neuroscience, New Zealand (Editor-in-chief), Günther Palm, University of Ulm, Germany and Gordon Shaw, University of California at Irvine, USA.

Volume 1 Brain Dynamics and the Striatal Complex edited by R.Miller and J.R.Wickens Volume 2 Complex Brain Functions—Conceptual Advances in Russian Neuroscience edited by R.Miller, A.M.Ivanitsky and P.Balaban Volume 3 Time and the Brain edited by R.Miller Forthcoming Volume Sex Differences in Lateralization in the Animal Brain edited by V.L.Bianki and E.B.Filippova, translated by T.Ganf Volumes in Preparation Cortical Areas: Unity and Diversity The Female Brain Functional Memory and Brain Oscillations This book is part of a series. The publisher will accept continuation orders which may be cancelled at any time and which provide for automatic billing and shipping of each title in the series upon publication. Please write for details.

Time and the Brain Edited by

R.Miller Otago Centre for Theoretical Studies in Psychiatry and Neuroscience New Zealand

harwood academic publishers Australia • Canada • France • Germany • India • Japan Luxembourg • Malaysia • The Netherlands • Russia • Singapore Switzerland

This edition published in the Taylor & Francis e-Library, 2005. "To purchase your own copy of this or any of Taylor & Francis or Routledge's collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk." Copyright © 2000 OPA (Overseas Publishers Association) N.V. Published by license under the Harwood Academic Publishers imprint, part of The Gordon and Breach Publishing Group. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, without permission in writing from the publisher. Printed in Singapore. Amsteldijk 166 1st Floor 1079 LH Amsterdam The Netherlands British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library. ISBN 0-203-30457-8 Master e-book ISBN

ISBN 0-203-34338-7 (Adobe eReader Format) ISBN: 90-5823-060-0 (Print Edition) ISSN: 1029-2136

CONTENTS

Series Preface Preface List of Contributors 1 Empirical Evidence about Temporal Structure in Multi-unit Recordings Alessandro E.P.Villa 2 Cross-correlograms for Neuronal Spike Trains. Different Types of Temporal Correlation in Neocortex, their Origin and Significance Lionel G.Nowak and Jean Bullier 3 The Space-Time Continuum in Mammalian Sensory Pathways Asif A.Ghazanfar and Miguel A.L.Nicolelis 4 Information Flow along Neocortical Axons Harvey A.Swadlow 5 Psychophysics of Human Timing Thomas H.Rammsayer and Simon Grondin 6 Cortical Processing by Fast Synchronization: High Frequency Rhythmic and Non-rhythmic Signals in the Visual Cortex Point to General Principles of Spatiotemporal Coding Reinhard Eckhorn 7 EEG Alpha and Cognitive Processes Wolfgang Klimesch 8 Theta Frequency, Synchronization and Episodic Memory Performance Wolfgang Klimesch 9 Distributed Assemblies, High Frequencies and the Significance of EEG/MEG Recordings Friedemann Pulvermüller 10 Cell Assemblies, Associative Memory and Temporal Structure in Brain Signals Thomas Wennekers and Günther Palm 11 The Relation between EEG and Evoked Potentials Erol Başar , Sirel Karakas , Elke Rahn and Martin Schürmann 12 Coherence and Phase Relations between EEG Traces Recorded from Different Locations Peter Rappelsberger , S.Weiss and Baerbel Schack 13 Temporal Structure of Neural Activity and Models of Information Processing in the Brain Galina N.Borisyuk , Roman M.Borisyuk and Yakov B.Kazanovich

vii ix xi 1 63

112 150 181 196

236 262 281

293 320 346

389

Discussion Section POSTLUDE The Neuroanatomy of Time Valentino Braitenberg Index

410 454 460

SERIES PREFACE The workings of the brain, including the human brain are a source of endless fascination. In the last generation, experimental approaches to brain research have expanded massively, partly as a result of the development of powerful new techniques. However, the development of concepts which integrate and make sense of the wealth of available empirical data has lagged far behind the experimental investigation of the brain. This series of books entitled Conceptual Advances in Brain Research (CABR) is intended to provide a forum in which new and interesting conceptual advances can be presented to a wide readership in a coherent and lucid way. The series will encompass all aspects of the sciences of brain and behaviour, including anatomy, physiology, biochemistry and pharmacology, together with psychological approaches to defining the function of the intact brain. In particular, the series will emphasise modern attempts to forge links between the biological and the psychological levels of describing brain function. It will explore new cybernetic interpretations of the structure of nervous tissue; and it will consider the dynamics of brain activity, integrated across wide areas of the brain and involving vast numbers of nerve cells. These are all subjects which are expanding rapidly at present. Subjects relating to the human nervous system as well as clinical topics related to neurological or psychiatric illnesses will also make important contributions to the series. These volumes will be aimed at a wide readership within the neurosciences. However, brain research impinges on many other areas of knowledge. Therefore, some volumes may appeal to a readership, extending beyond the neurosciences. Books suitable for the series are monographs, edited multiauthor collections or books deriving from conferences, provided they have a clear underlying conceptual theme. In order to make these books widely accessible within the neurosciences and beyond, the style will emphasise broad scholarship comprehensible by readers in many fields, rather than descriptions in which technical detail of a particular speciality is dominant. The next decades promise to provide major new revelations about brain function, with far-reaching impact on the way we view ourselves. These great breakthroughs will require a broad interchange of ideas across many fields. We hope that the CABR series plays a significant part in the exploration of this important frontier of knowledge.

PREFACE Since the days of Galileo, time has been a fundamental variable in scientific attempts to understand the natural world. It has been clear since the first recordings of electrical activity in the brain, by Caton, and later by Danielewski, that electrical signals from the brain consist of very complex temporal patterns. This perspective has continued to the present day. It is now richly demonstrated by recordings of electrical activity at both the single unit level and by the methods of electroencephalography. When recording of the electrical activity from single nerve cells in the brain became possible, the emphasis was initially on events occurring on a more brief time scale than the fastest of psychological processes. However, in more recent years, single unit activity has also been widely studied on a larger time scale, corresponding to many psychological processes. In particular, in the last decade there has been a major shift in the way we view single unit activity in several brain structures: A generation ago, the emphasis was on mapping space (for instance in a sensory receptor surface) against spatial location in the brain. Nowadays we are more concerned with the temporal structure detectable in neuronal impulse trains. Temporal structure in impulse trains may thus be part of the way in which timing of sensory or motor events is represented in the brain; and indeed, it is possible that external events and inner cognitive processes may be represented in the brain in part using temporal coding, even when these events and processes, in themselves, lack detailed temporal structure. Analysis of temporal structure in the activity of large populations of nerve cells in the brain has also undergone major advances in the last ten or fifteen years. Functional brain imaging using metabolic activity gives good spatial resolution of brain processes. In contrast, mapping based on brain activity, based on the EEG or the MEG is superior in revealing the temporal structure of dynamic processes in the brain. For the EEG, the time scale needed to describe the basic rhythms of brain electrical activity corresponds roughly to that of the more rapid of psychological processes. That electrical activity is also known to correlate in many ways with psychological or behavioural events observed at the level of the whole organism. This book, the third in the series Conceptual Advances in Brain Research, explores modern approaches to these temporal aspects of brain electrical activity. The earlier chapters focus mainly on temporal structure revealed from trains of impulses recorded from single nerve cells, or from several such nerve cells recorded at the same time, and their possible relation to behaviour and psychological processes. In the later chapters, the emphasis shifts to temporal structure in the EEG, again with several essays exploring correlations with behaviour and psychology. There are also two chapters of a more theoretical nature. The final chapter forms an edited discussion between contributors, delving further into a variety of issues which arose during editing of the main chapters. In this, a significant question is also touched on—the relation (if any) between temporal structure discernible at the single unit level, and that seen in the EEG.

Study of the electrical activity of the brain by EEG recording and by the methods of single unit electrophysiology are often not well integrated: The two approaches tend to be adopted in different laboratories, and indeed in different countries. By bringing together within a single book modern information acquired using both approaches it is hoped that these two approaches can become better integrated as complementary windows on the information processing achieved by the brain. As editor of this book, I have to say that I am greatly honoured to have received such a fine set of chapters to edit, and I hope that my own contributions to the book are at the same standard as those of the chapter authors. R.Miller.Dunedin. December. 1999

LIST OF CONTRIBUTORS Erol Başar Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Galina N.Borisyuk Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region, 142292 Russia Roman M.Borisyuk Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region, 142292 Russia Valentino Braitenberg Max Planck Institute for Biological Cybernetics Spemann Strasse 38 D-7400 Tübingen Germany Jean Bullier INSERM U371 Cerveau et Vision 18 Avenue du Doyen Lépine 69675 Bron Cedex France Reinhard Eckhorn Department of Physics

Neurophysics Group Philipps-University Renthof 7 D-35032 Marburg Germany Asif A.Ghazanfar Department of Neurobiology Box 3209 Duke University Medical Centre 101 Research Drive Bryan Research Building Durham, NC 27710 USA Simon Grondin Institute of Psychology University of Goettingen Gosslerstrasse 14 D-37073 Goettingen Germany Sirel Karakas TÜBITAK Brain Dynamics Research Unit Ankara Turkey Yakov B.Kazanovich Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region 142292 Russia Wolfgang Klimesch Department of Physiological Psychology Institute of Psychology University of Salzburg Hellbrunnerstrasse 34 A-5020 Salzburg Austria

Miguel A.L.Nicolelis Department of Neurobiology Box 3209 Duke University Medical Centre 101 Research Drive Bryan Research Building Durham, NC 27710 USA Lionel G.Nowak Section of Neurobiology Yale University School of Medicine C303 Sterling Hall of Medicine 333 Cedar Street New Haven, CT 06510 USA Günther Palm Abteilung Informatik Fakultät für Informatik Universität Ulm Oberer Eselberg D-7900 Ulm Germany Friedemann Pulvermüller Department of Psychology Fachgruppe Psychologic Sozialwissenschaftliche Fakultät Universität Konstanz Postfach D23 D-78457 Konstanz Germany Elke Rahn Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Thomas H.Rammsayer Institute of Psychology University of Goettingen

Gosslerstrasse 14 D-37073 Goettingen Germany Peter Rappelsberger Institute for Neurophysiology University of Vienna Waehringer Strasse 17 A-1090 Vienna Austria B.Schack Institute for Medical Statistics, Informatics and Documentation Friedrich Schiller University Jahn Strasse 3 D-07740 Jena Germany Martin Schürmann Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Harvey A.Swadlow Department of Psychology (U-20) The University of Connecticut Storrs, CT-06269 USA Alessandro E.P.Villa Laboratory of Neuro-heuristics Institute of Physiology University of Lausanne 7 Rue du Bugnon CH-1005 Lausanne Switzerland S.Weiss Institute for Neurophysiology University of Vienna Waehringer Strasse 17 A-1090 Vienna

Austria Thomas Wennekers Abteilung Informatik Fakultät für Informatik Universität Ulm Oberer Eselberg D-7900 Ulm Germany

1 Empirical Evidence about Temporal Structure in Multi-unit Recordings Alessandro E.P.Villa Laboratory of Neuro-heuristics, Institute of Physiology, University of Lausanne, 7 Rue du Bugnon, CH-1005 Lausanne, Switzerland Tel: ++41+21–692–5532/X.5516/X.5500; FAX ++41–21–692.5532 e-mail: [email protected] URL: www-lnh.unil.ch

The brain is a highly interconnected network of neurones, in which the activity in any neurone is necessarily related to the combined activity in the neurones that are afferent to it. Due to the widespread presence of reciprocal connections between brain areas, reentrant activity through chains of neurones is likely to occur. Certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements as a consequence of developmental and/or learning processes. In cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring, ordered, and precise (in the order of few ms) interspike interval relationships are referred to as “spatiotemporal patterns” of discharges. This term encompasses both their precision in time, and the fact that they can occur across different neurones, even recorded from separate electrodes. This chapter introduces the fundamental assumptions and algorithms that lead to the detection of complex patterns of neural discharges and introduces a way of interpreting this activity within the framework of non-linear dynamics. Empirical results of experimental and simulation studies are provided in different sections of the chapter. KEYWORDS: Spatiotemporal firing patterns; Neural dynamics; Brain theory; Time code; Frequency code; Multi-unit recordings; Non-linear dynamics; Sensorimotor association

1. BRIEF HISTORICAL INTRODUCTION In 1753–1755 the physiologist Albrecht von Haller published in Göttingen an historical essay, the “Dissertation on the Irritable and Sensitive Parts of Animals” (original title: De partium corporis humani sensibilius et irritabilus). This work was based on numerous experiments of vivisection, and on stimulation of organs using the new knowledge

Time and the brain

2

offered to physiology by physics, chemistry and natural history. With a rudimentary technique of stimulation, Von Haller classified the parts as irritable, sensible or elastic and noted that the reactions varied between different parts of the brain. The historical importance of the work by Von Haller is not so much related to the results obtained, but rather in systematically developing a transdisciplinary approach to brain research using the new technologies of his time. In 1791 the Italian physician Luigi Galvani started the publication of a remarkable series of studies that demonstrated muscle twitch in a frog by touching its nerves with electrostatically charged metal, and later using two dissimilar uncharged metals. These observations led Galvani to postulate that the circulation of a particular body fluid, that exists naturally in the nerves in a state of disequilibrium, provided the stimuli for the muscle fibres to contract. In addition, normal muscular contraction without a source of electrostatic electricity was, in Galvani’s view, evidence for the existence of an additional and “natural” form of electricity, that he called “animal electricity”. This latter statement set the scene for the famous Galvani-Volta controversy. Alessandro Volta, a friend of Galvani and an Italian physicist who, in 1775, invented the electrophorus—a device to generate static electricity—gave an opposed and experimentally valid explanation of Galvani’s experiment. The electricity did not come from the animal tissue but was generated by the contact of different metals, brass and iron, in a moist environment. The interest in this controversy resides in the confrontation of two basic, irreducible interpretations of the same observation, derived from each scientist’s different background: Galvani saw the frog phenomenon as the work of biological organs, Volta as that of a physical apparatus. The outcome of the controversy was exceptional. On the one hand the challenge of Volta’s opinion led Galvani to perform a new series of experiments that demonstrated muscular contraction by touching the exposed muscle of one frog with the nerve of another frog, thus showing for the first time the existence of bioelectric forces. On the other hand, Volta focused his research efforts upon the study of electric fluids between dissimilar metals, and in 1800 he presented the first electric battery, providing future researchers with a stable source of electricity not dependent on electrostatic forces. With the introduction of currents of “Galvanic fluids” into the brain, a way was opened to making new discoveries in physiology at the beginning of the XIXth century. Electricity was used not just as an experimental intervention, which formed the basis of electrophysiology (the study of the connection between living organisms and electricity), but in addition, the proper characteristics of propagation and generation of this type of energy became the basis of fertile hypotheses. In the last quarter of the XIXth century, Eduard Hitzig and Gustav Fritsch discovered the localization of cortical motor areas in the dog, using electrical stimulation, and Richard Caton was the first to record electrical activity from the brain. Electrophysiology started to develop rapidly, and Edgar D.Adrian published a seminal study suggesting the all-or-none principle in nerve (1912). In the late 1920s Hans Berger in Germany demonstrated the first human electroencephalogram and opened the way to clinical applications of electrophysiology. Nevertheless, the English School led investigations in electrophysiology in the first part of our century, and for his specific research on the function of neurones, Adrian shared the 1932 Nobel Prize for Medicine with Sir Charles Sherrington. Although most remembered for his scientific contributions to neurophysiology, Sherrington’s research focused on spinal reflexes as

Empirical evidence about temporal structure

3

well as on the physiology of perception, reaction and behaviour. His approach was transdisciplinary, in the sense that disciplines were used not only “one next to the other”, but were really intermingled in his protocols, an extraordinary example of a nonreductionist view of neurophysiology. At present, and still based upon the work performed by Adrian (1934), most neurophysiologists make their deductions by observing mean frequencies of nervous discharges (spikes), i.e. whether there are a lot or only a few spikes over a relatively prolonged time interval. Important concepts and findings have been clarified by using the overall mean rate as a measure of neuronal activity, at both peripheral and central levels of the nervous system. In particular it is important to remind ourselves of the influence of the findings of Mountcastle et al. (1963) indicating that the relationship between frequency of firing of selected neurones in the ventrobasal thalamus of deeply anaesthetized monkeys and the angle of extension of the contralateral knee was a perfect power function. Hence, the relationship between log frequency and log angle is virtually a perfect linear function. Although this point will not be developed further in the present chapter, it is important to note that the effect, and the type of anaesthesia determine dramatically the neuronal discharge pattern, especially at thalamic and cortical levels (e.g., see Mukhametov et al., 1970; Zurita et al., 1994). This effect has not always been adequately taken into account by investigators eager to develop theoretical models of “neuromimetic” circuits. The slow integration time of nerve cells, operating in the range of a few milliseconds, roughly a million times slower than presently available supercomputers, and the huge number of connections established by a single neurone (for review, see Braitenberg and Schüz, 1991) has suggested that information in the nervous system might be transmitted by simultaneous discharge of a large set of neurones. Multiple dimensions of stimuli relevant to sensory function and behaviour are processed by thousands of neurones distributed over many areas of the brain. Indeed, the hypothesis that neurones (both individually and jointly) process information over time, and following precise time relationships, has pervaded the neurosciences ever since the nervous system was conceptualized as a set of dynamic networks of interacting neurones (McCulloch and Pitts, 1943; for review, see MacGregor, 1987). That neurones convey a temporal code has been well-known since electrophysiological studies in the 1960s led to the recognition that spike trains—the time series formed by the sequences of time intervals between spikes—were related to meaningful physiological variables (Bullock, 1961; Segundo et al., 1963; Perkel and Bullock, 1965; Segundo et al., 1966; Nafe, 1968). Subsequent work made an important contribution, by recognizing temporal coding in different experimental circumstances and animal species, and by proposing worthwhile quantification procedures (Perkel et al., 1967a,b; Segundo and Perkel, 1969; Legéndy, 1975; Eckhorn et al., 1976; Brillinger et al., 1976; Klemm and Sherry, 1981; Abeles, 1982; Tsukada et al., 1982; Rosenberg et al., 1989; Bialek et al., 1991; Rapp et al., 1994; Friston, 1995; Martignon et al., 1995; Rieke et al., 1995; de Ruyter van Steveninck et al., 1997). This provided new insights into how the association between simultaneously recorded spike trains, in the time and frequency domains, may reflect the degree of coordinated activity within a cell assembly. The integrative capabilities of neurones (for review, see Segundo, 1986) on the one hand, and parallel processing and redundancy

Time and the brain

4

inherent to neural pathways on the other hand, have become more obvious during the past two decades, thanks also to the advances in neuroanatomical methods using retrograde and/or anterograde tracers (e.g. horseradish peroxidase, fluorescent dyes, biocytin, etc.). However, for most functional systems the available electrophysiological data demonstrate merely the presence of temporally organized neural activity, so that during recent years investigations have mainly focused on establishing causal relations between the occurrence of precise temporal relationships and cognitive or motor processes (Fetz, 1997). Therefore, two hundred years after the Galvani-Volta controversy, two main classes of theories explaining information processing in the brain have been proposed: In one, neurones convey a precise temporal code (Abeles, 1982; Abeles, 1991), while the other is based on noisy rate coding (Shadlen and Newsome, 1994). According to the rationale of a noisy rate code, the question has been raised whether the variability of spike intervals carries information. This variability may depend on the role of the decay of the post-synaptic potential (PSP) in determining the prevailing operating mode of network processing (König et al., 1996). Softky and Koch (1993) have shown that, in order to find the same degree of variability seen in vivo, coincidence detection with fast integration of small excitatory PSPs is required in models. Alternatively, Shadlen and Newsome (1994) have proposed that the balance between inhibition and excitation plays a critical role in the variability of network behaviour with membrane time constants of 8–20 ms. If an irregular interspike interval results from integration of excitatory and inhibitory postsynaptic potentials, then the timing of postsynaptic spikes is random and can no longer reflect the timing of presynaptic events. Precise patterns of spikes—their intervals and coincidences—would fail to propagate (Shadlen and Newsome, 1994). König et al., (1996) have provided a summary of these two viewpoints and proposed that the interplay of excitation and inhibition (E-I) effectively reduces the time constant for synaptic integration, thus providing coincidence detection in the cortex, although the actual synaptic decay is relatively long. Indeed, a balance condition of excitatory and inhibitory post-synaptic potentials plays an important role in the activity of many network models (Wilson and Cowan, 1974; Douglas and Martin, 1991; Tsodyks and Sejnowski, 1995; Usher and Stemmler, 1995; Xing and Gerstein, 1996) and has many complex spatial and temporal influences (Thomson and Deuchars, 1994). Changes in cellular excitability modify the time to firing, thus altering the E-I dynamics. Some specific examples of time-varying factors influencing the balance of E-I levels are the cell threshold potential, PSP kinetics, receptor activity such as NMDA, and calcium currents (Edmonds et al., 1995). The inability to preserve information about the time of a spike suggests that the discharge rate cannot be modulated in a time-locked fashion to specific inputs. In a random walk model (Gerstein and Mandelbrot, 1964) this assumption may be wrong because the discharge rate can follow the activity of inputs, but, given a base rate, the time to the next spike would appear as random. Then, the average instantaneous discharge rate through an ensemble of neurones belonging to a functional assembly would be capable of transmitting changes in spike rate with a precision of 10–50 ms (Shadlen and Newsome, 1998). Recordings performed in several cortical areas of behaving monkeys support the hypothesis of a connection between fluctuations in neural discharge rate and behaviour (Georgopoulos et al., 1993; Celebrini and Newsome, 1994). However, the existence of the rate code

Empirical evidence about temporal structure

5

mechanism does not imply that temporal codes do not also exist in the brain. The cerebral cortex is a highly interconnected network of neurones, in which the activity of each cell is necessarily related to the combined activity in the neurones that are afferent to it. Due to the presence of reciprocal connections between cortical areas, reentrant activity occurs, through chains of neurones. Furthermore, certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements, as a consequence of developmental and/or learning processes. According to the rationale of a precise temporal code, in cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring and ordered interspike intervals, with a precision of the order of a few milliseconds, are referred to as “spatiotemporal patterns” of discharges. For true demonstration, such temporal firing patterns must occur to a statistically significant level (see Figure 1). If functional correlates of spatiotemporal neural coding exist, one would expect that whenever the same information is presented, the same temporal pattern of firing would be observed. Several lines of evidence exist showing Spatiotemporal patterns in vivo (Bair and Koch, 1996) and in vitro (Mainen and Sejnowski, 1995). Recent studies on active propagation of action potentials in dendrites have provided additional results supporting the existence of precise neuronal timing (Stuart and Sakmann, 1994). A synaptic response increases if the presynaptic spike precedes the postsynaptic spike, but if the order is reversed, the synaptic response decreases. The window for synaptic

Time and the brain

6

Figure 1 . Outline of the general procedure followed by pattern detection algorithms, (a) Analysis of a set of simultaneously recorded spike trains. Three cells, labelled A, B, and C, participate in patterned activity. Three occurrences of a precise pattern are detected. Each occurrence of the pattern has been labelled by a specific marker in order to help the reader to identify the corresponding spikes, (b) Estimation of the statistical significance of the detected pattern, (c) Display of pattern occurrences as a raster plot aligned on the start of the pattern.

plasticity is 100 ms wide, and a difference in spike timing of only 10 ms near coincidence switches plasticity from potentiation to depression (Markram et al., 1997). Neurophysiological measurements in human subjects performing a visual go/no-go categorization task indicate that speed of processing of visual information in human is in

Empirical evidence about temporal structure

7

the order of 150 ms (Thorpe et al., 1996). Taking into account the firing rates of neurones in the visual pathway, only few spikes at most can be generated by each neurone in performing this highly demanding task, thus providing strong arguments for the importance of precise temporal coding. Indeed, electrophysiological experiments in primates have established that correlated firing between single neurones recorded simultaneously in the frontal cortex may evolve within tens of milliseconds, in systematic relation to behavioural events, without modulation of the firing rates (Vaadia et al., 1995). Furthermore, in multiple electrode recordings performed in the primary motor cortex of monkeys trained in a delay-pointing task, spike synchronization occurred in relation to purely internal events (stimulus expectancy), during which modulations of firing rate were distinctly absent (Riehle et al., 1997). These findings provide an important support for the general view that a stronger synaptic influence is exerted by multiple converging neurones firing in coincidence, thus making synchrony of firing ideally suited for highlighting responses and for expressing relations among neurones with high temporal precision.

2. DETECTION OF SPATIOTEMPORAL FIRING PATTERNS An influential and remarkable model based on the assumptions of high temporal precision in brain processing is the “synfire” chain hypothesis. This model suggests how precise timing can be sustained in the central nervous system by means of feed-forward chains of convergent/divergent links and re-entrant loops between interacting neurones forming an assembly (Abeles, 1982). Structures like synfire chains may exhibit attractors in which a group of neurones excite themselves, maintaining elevated firing rates for long periods and allowing the same neurone to participate in many different synfire chains. A fundamental prediction of such a model is that simultaneous recording of activity of cells belonging to the same assembly and involved repeatedly in the same process should be able to reveal repeated occurrences of such spatiotemporal firing patterns. Note that the term “firing pattern” encompasses both their precision in time, and the fact that they can occur across different neurones, even when recorded from separate electrodes. The following example (Figure 2) illustrates schematically the recurrence of patterns in a network connected according to the “synfire” model. A chain of 56 model neurones is formed by 8 sets of 7 neurones connected by diverging/converging links. This means one neurone in a set (in black in Figure 2a) receives 7 inputs from the previous set and projects to all 7 neurones of the next set. In the example shown in Figure 2a the links are subdivided into 5 excitatory (open circle) and 2 inhibitory connections (gray circles). This network is embedded into a larger network (200 neurones) with random connections. Note that the last link of the chain is connected to the first link. The model neurones follow simple integrate-and-fire dynamics (Villa and Tetko, 1995; Hill and Villa, 1997). Each neurone integrates all post-synaptic potentials. If the integrated depolarization passes the threshold, the neurone fires, becoming refractory, whereas if firing does not take place, the membrane potential tends to return to the resting level. Synaptic strengths and time constants have been selected in

Time and the brain

8

order to allow “spontaneous activity” to propagate but also to prevent “epileptic” spikes to self-generate in a simulation run. Figure 2b shows 28 successive frames of the neuronal chain activity, in arbitrary time units. Let us assume that three selected neurones, i.e. c10, c36, and c46, are recorded by virtual electrodes. The corresponding spike trains are shown in Figure 2c, and a firing pattern formed by four spikes was detected in these spike trains. More generally, let the list of cell labels that appear in a pattern of c spikes be noted as S c =(i 1 ,...,i j ,...i c ). In this, any label of the recorded neurones can be assigned to i j . In particular, for a 4-tuple pattern of spikes (quadruplet) obtained from a simultaneous recording of ten spike trains, let us assume that we find S 4 =(c36, c36, c10, c46), meaning all patterns of complexity c equal to 4, formed by a spike recorded from cell c36, followed by another event of cell c36, then a spike recorded from cell c10, which in turn is followed by a spike recorded from cell c46. One specific example of such patterns with delays of 6 time units between the first two spikes from

Figure 2 . Example of spatiotemporal pattern detected in a simulated chain of diverging/converging links. (a) Basic structure of one set of the chain, formed by 7 neurones. One neurone in a set (black circle) receives 5 excitatory (white circles and solid lines) and 2 inhibitory (gray circles and dashed lines) inputs from the previous chain. The output projections of one neurone on the cells of the next set follow the same pattern. (b) The results of the simulation of the activity in a network formed by eight successive sets from time 61 to 88 in arbitrary time steps. The large dots indicate that a neurone is firing.

Empirical evidence about temporal structure

9

The events belonging to the pattern detected in panel (c) are enclosed by squares, (c) A significant pattern formed by four cells is detected. The pattern was: cell 36; 6 time units (t.u.) later cell 36 again; 8 t.u. later cell 10 and 11 t.u. later cell 46.

c36, 2 time units between c36 and c10 and 5 time units between c10 and c46 is notated as (see Figure 2). In addition, this notation assumes that the same time jitter was found for all events forming the pattern. An apparent weak point of the “synfire” theory is the requirement for neural mechanisms able to support a precise timing of spike patterns even after large time delays, and in the presence of various neuromodulators. For instance, a prominent effect of cholinergic modulation is to reduce adaptation of spike frequency. This results in an increase of neuronal excitability and a shortening of inter-spike intervals, so that the overall effect of such modulation can be a modification of spike timing. Differential actions of acetylcholine on the excitability of two subtypes of inhibitory cortical interneurones exist, so that cortical cholinergic activation may change the direction of information flow within cortical circuits (Xiang et al., 1998). Despite this, patch-clamp experiments in rat neocortical slices have shown that timing of spikes produced in response to fluctuating current injections may be preserved during cholinergic modulation (Tang et al., 1997). Transmission across neocortical synapses depends on the frequency of presynaptic activity and studies of synaptic depression have demonstrated that different aspects of the firing patterns of their afferents are transmitted depending on the average presynaptic frequency (Abbott et al., 1997; Tsodyks and Markram, 1997). When a single sensory stimulus drives many neurones to fire at elevated rates, the spikes of these neurones may become tightly synchronized, which could be involved in “binding” together individual firing-rate feature representations into a unified object percept. However, the response to elevated rates of firing may be weak, if any, and population coding based on relative spike timing can systematically signal stimulus features following the stimulus time course even where mean firing rate does not change (deCharms and Merzenich, 1996). These properties determine the fact that little information about steady-state frequency is transferred across synapses. This result is in contradiction with the rate code hypothesis, assuming that a higher input rate corresponds to a higher output rate. In this respect, information theory applied to the coding problem in the frog and house fly (Bialek and Rieke 1992; Rieke et al., 1995) also suggests that spike timing is important for reliable information transfer from receptors to brain, and may explain the subsequent behavioural events. In summary, several lines of evidence are in favour of precise temporal activity in the brain, but the development of analytical methods to detect reliably the transient temporal relations in sequences of spike intervals therefore represents a critical step to establish their function as a temporal code. One proposed technique aimed at detecting “favoured patterns”—i.e. patterns that occur more often than can reasonably be expected at random—consists of identifying template patterns even if the detailed timing of the patterns varies slightly (Dayhoff and Gerstein, 1983a). The advantages of this method, referred to as Favoured Pattern Detection (FPD), is that it can be used to detect other favoured patterns whose occurrences may have extra or missing spikes, and the firing patterns can be tested for

Time and the brain

10

significance by Monte Carlo algorithms. Unfortunately, this technique can be applied only for analysis of temporal patterns of activity generated by a single neurone, and for fixed jitter—i.e. the fluctuation in temporal precision allowed for each spike. Moreover, the choice for settings, e.g. the threshold level parameter, may have discouraged its use among electrophysiologists, and limited its application to experimental data (Dayhoff and Gerstein, 1983b). The use of multiple microelectrode recordings, and of large artificial neural networks of spiking neurones has challenged investigators to search for methods allowing the analysis of higher complexity firing patterns, formed by several spike trains and involving several events within a pattern. An approach to the exact calculation of the probability of randomly obtaining each individual recurring pattern was proposed for three and more distinct spike trains (Frostig et al., 1984, 1990). In this method the statistical evaluation is based on the use of a 2×2 contingency tables and the application of Fisher’s exact test. At the first step, the detection stage, all possible intervals between three spike trains are tested for significance, and at a second step, the expansion stage, all significant patterns found previously are checked for higher-order associations among spike trains. The main limitations of this method consist of its inability to detect recurring patterns involving less than three different spike trains and patterns of more than three spikes, if not all subpatterns formed by a triplet of spikes are statistically significant. Several investigators have pointed out that in most experimental preparations presently available the chance of observing patterns of discharges of higher-order complexity is very low, and specific analytical methods aimed at analyzing patterns formed by three spikes (triplets) were developed. The principles of three-fold correlation among spikes are illustrated in Figure 3. Suppose that the firing times of three neurones A, B, and C are recorded simultaneously (Figure 3a). One could ask whether the firing of cells B and C depends on the time elapsed from a spike of cell A. Let us consider the time delays tAB of the discharges of cell B after neurone A fired and similarly the time delays tAC. The joint distribution histogram may be constructed as shown in Figure 3b. At each discharge of cell A the spike train of neurone B is plotted in the X-axis and the spike train of neurone C is plotted in the Y-axis of the graph. The crossings of columns and rows, falling within a defined bin size, that correspond to the spike trains of neurones B and C recorded simultaneously are counted and a Z-axis, perpendicular to the plan of drawing, is plotted. The value of the compound rate plotted on this Z-axis may be coded in shades of gray (or on a colour scale). A graphic possibility for plotting all compound joint distributions between three neurones is based on a triangular coordinate system because the three ). The X-Y graph can be variables tAB, tAC, tCB, are not independent (since skewed by 60° (Figure 3b, right panel) so that all three joint distributions can be plotted together (Figure 3c). A practical application of this method in practice is shown in Figure 3d. In this example the data were recorded from the thalamus of an anaesthetized cat during spontaneous activity (Villa, 1988, 1990). Cell 1 (discharging at a rate of 2.8 spikes/s) was recorded in the posterior group of the thalamus, cell 3 (1.7 spikes/s) in the dorsal nucleus of the medial geniculate body and cell 7 (5.7 spikes/s) in the brachium of the inferior colliculus. A black triangular bin indicates a significant event corresponding to a spike of cell 3, then 8 ms later a spike from cell 7 and a spike of cell 1 occurring 116 ms after the onset of the pattern. The time accuracy of this estimation was limited to 15

Empirical evidence about temporal structure

11

ms so that spikes of cells 7 and 1 might be considered synchronous. Detection of triplets was first achieved using the “snowflake” triple-spike renewal histogram (Perkel et al., 1975; Abeles, 1983). However, triple-spike renewal histograms do not provide a reliable estimation of the significance of the detected patterns, and to solve this problem the method was developed further into the Joint PeriStimulus Time Histogram (JPSTH) (Aertsen et al., 1989). A number of applications of this technique to

Figure 3 . Outline of the general procedure followed to study three-fold correlation among spikes, (a) Three spike trains, labelled A, B and C, are recorded simultaneously. The time of occurrence of spikes A are

Time and the brain

12

successively taken as onset time for calculation of delays of simultaneously recorded spikes of cell B (tAB) and C (tAC). Three markers indicate three successive spikes of cell A. (b) A plot of all possible patterns formed by B and C spikes as a function of the A spikes marked on the top panel is presented in form of a X-Y plot where the abscissa represents the delays of B spikes after A firing and the ordinate the delays of C spikes after A firing. The left panel shows the same plot with the Y-axis skewed by 60°. (c) A three-cell correlogram is constructed by joining three graphs in order to represent all possible time relations among the firing of three neurones. The three intervals (tAB, tAC, tCB) associated with any point on the graph can be read by projecting the point onto the three external axes, (d) The activity of three neurones, labelled 1, 3 and 7, were recorded from three different electrodes in the cat thalamus. A significant pattern is detected as shown by the black triangular bin. The pattern was: cell 3; 8 ms later cell 7 and 108 ms later cell 1. The shades of gray correspond to the number of counts per bin on a relative scale.

the study of simultaneously recorded spike trains in behaving monkeys have been reported (Abeles et al., 1993b,c). Since then, the algorithm has been improved for statistical evaluation, in order to take into account the existence of significant crosscorrelation between cells. Analyses of multiple spike trains recorded in behaving monkeys using this most recent algorithm, referred to as Joint Triplet Histogram (JTH), have been published recently (Prut et al., 1998). A careful analysis of the compound activity of three-cells may suggest interesting hypotheses about functional connections between distinct nuclei. Let us consider the result illustrated in Figure 3d. One could argue that cells in the brachium of the inferior colliculus and cells in the dorsal nucleus of the medial geniculate body receive common afferent activation from the ascending auditory pathway, and synchronous activation of these nuclei is easy to accept. However, the cross-correlation between the neurones recorded in these nuclei was flat, as indicated by the absence of dark diagonal bands centered on time zero in the snowflake plot, and the quasi-synchronous activation of cells 3 and 7 was only detected when a spike in cell 1 occurred more than 100 ms later. This delay cannot be due to direct connections through the thalamus (according to the present knowledge of thalamic organization). The observation of a complex pattern of firing in different thalamic nuclei suggests that these anatomical regions may form distributed cell assemblies that are functionally connected by some kind of reverberating activity, probably involving re-entrant activity from the cerebral cortex. In chains of diverging/converging links there is no need for each of the synaptic connections to be particularly strong, since they become effective through coactivation with others. Then, several synapses can depolarize the postsynaptic cell to reach its firing threshold, by spatial and temporal summation. In the case of pure excitatory feed-forward chains of neurones (Abeles, 1982, 1991) it has been demonstrated that a strong stimulation (e.g. input fibres firing at a high asynchronous rate) applied to all the cells of a set in the chain would elicit a spike in all of them synchronously, so that the next set of neurones down the chain would also be activated synchronously. This synchronous

Empirical evidence about temporal structure

13

activity would then propagate along the whole chain, with a limited time jitter. Thus, interconversions from firing rate code to synchronous activity, and back, can be performed by a chain of diverging/converging links. This observation is related to the origin of the “synfire” chain model (Abeles, 1982). If these chains include excitatory and inhibitory links, the analysis of the propagating activity is not so simple. In addition when one considers that distinct sets of the chain can be activated independently , then the picture of a propagating wave becomes less clear. However, these chains will tend to produce unitary events, i.e. precise spike synchronization between neurones, which occur significantly more often than expected by chance. The temporal spacing between the spikes should not be regarded as corresponding to the relative location of the parent cells along the chain. Such characteristics of the model are illustrated by Figure 4. In this example the very same simulation results as in Figure 2 are considered, but different cells were sampled by the virtual electrodes. It is important to note that cells involved in unitary events would not necessarily show signs of synchronization in crosscorrelograms, because they do not receive either common excitatory inputs or common inhibitory inputs. In Figure 4a it can be seen that, due to the presence of other inputs in some occurrences of the temporal pattern, a spike belonging to the pattern may be missing because a cell is in a refractory state. However, due to the peculiar properties of diverging/converging links the pattern is sustained by the other neurones and may persist in the network. An increasing number of studies are now investigating unitary events and preliminary experimental evidence has been reported (Riehle et al., 1997; Prut et al., 1998). It is therefore possible that unitary events in the primary motor cortex are related to pattern generation, for instance as starting points for delayed synchronous patterned activity related to the precise activation of sequences of muscles involved in motor output.

3. EMPIRICAL EXAMPLES The Pattern Detection Algorithm (PDA) developed by Abeles and Gerstein (1988) allows one to perform a comprehensive search for firing patterns and to test if there is a statistical excess of patterns. According to the complexity of the patterns to be analyzed,

Time and the brain

14

Figure 4 . Example of a unitary event detected in a simulated chain of diverging/converging links, (a) Two events corresponding to the synchronous activation of cells 1, 17, 23 and 33 have been detected, plus an event corresponding to the synchronous activation of cells 1, 17 and 33. (b) Same simulation results of Figure 2b. The large dots indicate that a neurone is firing. The events belonging to the unitary events are enclosed by squares.

two methods (“ad-hoc” and “bootstrap” algorithms tested against simulated multineurone data) are used for estimation of the expected number of repeated patterns of spikes, regardless of their complexity. Improved and computationally effective versions for estimation of the significance of patterns of spikes in this algorithm, based on probabilistic and combinatorial algorithms, were recently elaborated (Tetko and Villa, 1997a). The main restriction of PDA when compared to the other methods is that it does not allow one to establish the statistical significance of an individual pattern which repeats a number of times, but only the existence of an overall excess of precise patterns beyond expectations. This limits its main applicability to cases when pattern occurrence is very rare. Assuming a Poisson distribution of point processes, the significance of the excess of detected patterns Z, of complexity c, and repeating exactly r times in the record,

Empirical evidence about temporal structure

15

over the expected number X is estimated by pr{Z,X}=P(Z,X) where P(Z,X) represents the incomplete gamma function (Press et al., 1994). This estimation is simplified to if only one firing pattern is detected. Let us fix the level of significance to be p 0 =0.001. For example, if the number of patterns detected by PDA is Z=6, and only one pattern (i.e., X=1) is expected by chance, then a significant excess of detected patterns is observed because . However, this estimation cannot indicate which ones out of the six detected patterns are significant. In some cases PDA allows one to detect an excess of patterns when all sequences of spike intervals are significant. Consider an example where the expected number of patterns is as low as X=0.001. If one pattern is detected, its significance is equal to thus indicating that the pattern is highly significant. If in the above example we detect six patterns instead of one, then any of these six can be considered as highly significant. (In such a case each of the six pattern can be considered significant by itself.) Routine studies using the classical PDA have demonstrated that this optimal case is exceptional. Most often, patterns that could be significant by themselves under appropriate values of jitter are detected by PDA as distinct patterns if the search is performed with maximum accuracy, i.e. with time jitter equal to 1 ms. The application of PDA to several experimental models, and to recordings from various brain areas, has revealed that complex Spatiotemporal firing patterns do indeed occur (Vaadia et al., 1989; Villa and Abeles, 1990; Abeles et al., 1993a; Villa and Fuster, 1992). In these studies the algorithm was usually set to find any repeating pattern of three or more spikes, provided that the entire pattern lasted not more than 900 ms at most, and was repeated with accuracy of 1 ms. Since high frequency bursts (200 spikes/s or more) may produce patterns which relate to intracellular processes, burst filtering should be performed. A filtering procedure was originally proposed by Abeles and Gerstein (1988). A good filtering algorithm should provide a minimal loss of information, for an acceptable rate of errors in the estimation method. This would require fine tuning of this procedure that includes several combinations of three independent parameters: the time window for burst detection, the time window and the number of counts in high-frequency filtering. An alternative filtering method based only on one parameter, the filtering frequency, has been recently proposed (Tetko and Villa, 1997a). This method, with filtering frequency ranging between 200 and 400 spikes/s, provides better estimates of the number of patterns, for a comparable loss of information. A record was considered as having an excess of repeating firing patterns only when the probability of finding as much (or more) repetitions by chance was less than the significance level p 0 . Significant patterns that repeated exactly, more than four times, may be used as templates for searching approximate patterns with jitter of ±5 ms. Figure 5 shows an example of the activity of six neurones recorded in the cat thalamus during spontaneous activity (Villa, 1990; Villa and Abeles, 1990). It illustrates a pattern of 3 spikes, repeating 25 times, starting with spike 2, then after 89 ms spike 4, and then after 305 ms spike 5. On the left panel (Figure 5a) the rasters are aligned on pattern start and on the right panel (Figure 5b) the corresponding histograms (we may call peri-pattern histograms) are computed. In addition to the pattern itself, lasting hundreds of ms and formed by spikes recorded from different electrodes, this example is important because it shows an additional feature. The histogram of Figure 5b shows that two bursty neurones, cells 1 and 6, do not

Time and the brain

16

discharge in correspondence to the last spike of the triplet (cell 5 at 394 ms), but both cells show a significant increased tendency to discharge, in bursts, near 410 ms after the pattern

Figure 5 . (a) Raster display of the activities of six neurones during spontaneous activity recorded in the auditory thalamus of the cat. The rasters are aligned by displaying the first spike in the pattern at time 0. The pattern was: cell 2; 89±5 ms later cell 4, and then 305±5 ms later cell 5. The units were recorded by four microelectrodes. The first, in the thalamic reticular nucleus, picked-up units 1–2; the second, in the medial geniculate body, unit 3; the third, also in the medial geniculate body, picked-up units 4–5 and the last, in thalamic reticular nucleus, unit 6. (b) Histograms of the activities of the units triggered by the pattern onset. The abscissa full scale is 800 ms and the ordinate full scale is 85 spikes/s. Note that units 1 and 6 had a significant higher firing rate 410 ms after the pattern onset. The dashed lines correspond to the 99% confidence levels and a solid line is plotted at the average firing level. The curves are smoothed by a moving Gaussian bin shape of 10 ms.

onset. Therefore we could say that Figure 5 illustrates in fact a very complex pattern that should be noted . This is true, but no algorithms are yet available to detect such a pattern because the firing of the bursty cells is only loosely,

Empirical evidence about temporal structure

17

yet significantly, synchronized with the pattern onset. Furthermore, the selective inhibition of cells 1 and 6 at the same time as the third event of the triplet represents per se a valuable piece of information, related to the highly precise temporal processing occurring in the thalamus. Unfortunately, complex temporal firing patterns mixing spike occurrences and “missing spikes” are not detected by the usual algorithms. This is partly due to the enormous increase in computations required by taking recursively the “negative” of one spike train with all other spike trains recorded simultaneously, and partly by the fact that it is not correct to simply take the “negative”. However, there is no doubt that selective absence of spikes at specific timings would contribute to a better understanding the temporal processing in the brain, and it is reasonable to forecast that in the next few years methods will appear to deal with this problem.

4. PATTERN RECURRENCE IN RELATION TO STIMULI AND RECORDING CONDITION The next two examples illustrate some relations of firing patterns elicited by sensory stimuli, in the sense that they would not be observed during spontaneous activity (Villa 1988; Villa et al., 1991). Recordings were made in the auditory thalamus of the cat. Figure 6 shows a triplet, occurring selectively during the presentation of a stimulus which was a white noise burst lasting 200 ms, delivered to both ears. The firing pattern always started after the stimulus onset, but it always ended before the stimulus offset. However, the pattern was not time locked with the stimulus onset, as clearly shown by the irregular alignment of the stimulus onset and the start of the pattern. Similarly, a pattern could be elicited by a stimulus, but occurring after the stimulus offset and without a tight locking to it. Such an example is illustrated by the triplet in Figure 7, repeating 12 times, starting by spike 3, then after 285 ms spike 1 and then after 189 ms spike 2. Note that spikes 1 and 2 were recorded from the same electrode, thus indicating that these cells lay within a distance of few tens of microns of each other. Nevertheless the delay between these correlated discharges was near to 200 ms, and suggested long reentrant loops from the cortex to the thalamic reticular nucleus or, alternatively, from the substantia nigra and/or from modulatory mesencephalic nuclei, that are known to project to this thalamic region. In order to establish the significance of firing patterns as signposts of distributed network activity related to specific brain processes, it is crucial to demonstrate that whenever the network is working in the same state, the same temporal pattern of firing is observed. Several above-mentioned examples have suggested that cortical activity may affect the recurrent activity in the thalamus, as shown by several corticofugal studies, either experimental (Villa et al., 1991; Payne et al., 1996; Villa et al., 1999b) or theoretical (Tetko and Villa, 1997b). The next examples report data recorded in the auditory thalamus of the rat during reversible cortical deactivation by cooling (Villa et al., 1999b). This type of study illustrates an additional problem that may arise in studies of temporal coding. The number of spikes necessary for a fair statistical evaluation of the

Time and the brain

18

Figure 6 . Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.0 mm. The lowermost raster shows the onset of the stimulus, as a thick tick, and the gray bars represent the time of stimulus duration. The rasters are aligned by displaying the first spike in the pattern <2,3,1; 49,83> at time 0. The pattern is repeated 19 times with a jitter of ±3 ms. The abscissa full scale is 600 ms.

significance of the patterns may not be met in all data-sets because the experimental protocol, in this case the cortical cooling, may produce a massive decrease in firing rates of selected single units, or alternatively a massive increase of discharges, and a major change in the pattern of discharges (i.e. from regular to bursty). In case of a decrease in firing rate a partial methodological compensation might be obtained by increasing the recording time. Figure 8 shows an example of a pattern formed by three spikes belonging to two single units recorded simultaneously from two different electrodes in the rat auditory thalamus. The pattern is formed by cell 7 firing, then after 350 ms by a discharge of cell 11 and then after 21 ms cell 11 firing again. This exact pattern was used as template for searching approximate matching patterns with a jitter of ±2ms. Note that several spikes may occur in between the spikes belonging to the pattern without affecting the precise intervals separating the occurrences of the cell discharges. In this example, the significant pattern was observed 9 times in 400 s of spontaneous activity recorded during the control condition. No patterns were observed in 300 s of recording during cooling, but the very

Empirical evidence about temporal structure

19

same precise pattern was observed 5 times in 300 s during the recovery period. Note that between the first and the last occurrences of the pattern more than 90 minutes had passed. Another example (Figure 9) illustrates that significant patterns observed during cortical cooling might be completely absent prior to and after cortical deactivation. This case

Figure 7 . Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.4 mm. The rasters are aligned by displaying the first spike in the pattern <3,1,2; 285,474> at time 0. The pattern is repeated 12 times with a jitter of ±2 ms. The abscissa full scale is 1000 ms.

shows a pattern of 3 spikes repeating 9 times (with a jitter of ±2ms) starting by a spike of cell 8, followed by a spike of cell 5 180 ms later, then after 280 ms by a spike of cell 6. Note that only cells 5 and 6 were recorded from the same electrode. This second example also illustrates the tendency observed in firing patterns during cortical cooling to exhibit a higher complexity than patterns observed during control and recovery conditions (Villa and Abeles, 1990).

5. PATTERNS IN RELATION TO BEHAVIOURAL STATES Although the previous examples have provided solid evidence for the existence of precise temporal structure in neural activity, this does not yet establish its function as a temporal code. What is needed is some demonstration that Spatiotemporal firing patterns occur

Time and the brain

20

reliably under particular behavioural conditions, and are related to cognitive activity. The following example shows experimental results recorded in the primate inferotemporal cortex, which is considered one of the last in an ascending hierarchy of cortical processing stages that begins in the striate cortex. Studies with behaving monkeys have led to the conclusion that the inferotemporal cortex is not only a higher order stimulusanalyzer, but is also involved in retention of visual information, and is modulated by attention (Fuster, 1990). In the data presented here, the monkey performed a visual delayed matching-to-sample task (Villa and Fuster, 1992). A trial in one such task consisted of the following:

Figure 8 . Spatiotemporal pattern of spikes detected in the rat auditory thalamus 9 times prior to and 5 times after recovery from cooling of the auditory cortex, but absent during cortical deactivation. The firing pattern was formed by three spikes, starting by spike 7, then after 359 ms spike 11 and then after 21 ms spike 11 again. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated by thick ticks in the raster displays (right panels) aligned by displaying the first spike in the pattern at time 0. Note that spikes 7 and 11 were recorded from different electrodes. The analyzed records corresponded to 400 s of spontaneous activity

Empirical evidence about temporal structure

21

during control condition, 300 s during cortical cooling and 300 s in the recovery condition. Each block of 100 s is represented by a thick mark along the time axis of the experimental protocol, and the dots indicate the start event of the pattern.

(1) sample stimulus (a colour or a geometric figure on a luminescent button with a diameter of 25 mm); (2) 10–20 seconds of delay (retention period); and (3) choice of one stimulus (among 2 or 4) that matched the sample by a given feature (colour or symbol). Correct responses were reinforced with a squirt of juice. About 40% of all single units recorded in the inferotemporal cortex showed a sustained elevation of firing during the delay after the sample stimulus (which was retained in short-term memory for performance of the task). This sample was further subdivided into selective units (14% of all units) showing an activation during the delay which was selectively and significantly higher after one particular tested stimulus, and non-selective units (26%) which were unspecifically activated by all tested stimuli. The distribution of firing rates for both

Figure 9 . Triplet of spikes detected 9 times in the rat auditory thalamus, exclusively during cortical deactivation, starting by spike 8, then after 180 ms spike 5 and then after 280 ms spike 6. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated as in Figure 8. Note that only spikes 5 and 6

Time and the brain

22

were recorded from the same electrode. The analyzed records corresponded to 300 s of spontaneous activity during control condition, 500 s during cortical cooling and 300 s in the recovery condition.

groups in spontaneous discharge was skewed and broad, but the median firing rate for non-selective units (4.4 spikes/s) was higher than that for selective units (2.5 spikes/s). In this study, only one spike train at a time was recorded and thus the temporal firing patterns were formed by multiple spikes of one neurone. Figure 10 illustrates two patterns formed by three spikes observed in two different neurones. The triplet of cell 14 repeated 6 times during the intertrial periods, which did not reach statistical significance, whereas its repetitions were significant (19 times out of 68 trials) during the delay period. The accuracy of the pattern was ±3 ms. The triplet of cell 47 repeated 7 and 19 times during the intertrial and delay periods, respectively, out of 75 trials. One pattern involved long intervals, of the order of hundreds of ms, whereas the other pattern involved relatively short intervals. However, the striking similarity between these two patterns is that both patterns appeared to a significant extent during the retention period of the sample stimulus, and that only one occurrence of the pattern (seldom two) was observed per trial (although in only 20–25% of all trials).

Figure 10 . Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of two non-selective units recorded in the

Empirical evidence about temporal structure

23

primate inferotemporal cortex during intertrial intervals (left panels) and during the delay (right panels). The measuring time was equal in both conditions. Cell 14 shows a pattern of 3 spikes repeating 6 and 19 times during the intertrial and delay periods, respectively: The second spike of the triplet occurred 165 ms after the pattern start and the third 195 ms after the second spike. Cell 47 shows a triplet repeating 7 and 19 times during the intertrial and delay periods, respectively. The first spike of cell 47 occurred at time 0 (aligning the rasters), the second 42 ms later, and the third 48 ms later. In these rasters the pattern is allowed an accuracy of ±3 ms.

In spike trains recorded in the inferotemporal cortex it was difficult to observe firing patterns composed of 4–6 spikes recurring more than twice within a record. Figure 11 shows an example of such a complex pattern formed by spikes of a selective unit during intertrial periods, and during the delay after the preferred stimulus (the only periods characterized by an increase in firing rate). The pattern was composed of 4 spikes: a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. This quadruplet repeated 6 times during the intertrial periods. The four subpatterns corresponding to the triplets included in the original quadruplet are also displayed in Figure 11. The incidence of the highly organized pattern decreased when the cell was activated by its preferred stimulus, irrespective of an increase in the rate of discharges during the delay. However, it is important to note that one specific subpattern, i.e. was detected at a statistically-significant level during the intertrial and delay periods. This triplet occurred 5 and 9 times during the intertrial and delay periods, respectively. The example of Figure 11 is important, because it illustrates several additional characteristics of precisely time-structured activity in the cortex. Firstly, it illustrates the contribution of subpatterns to a higher order pattern, thus suggesting the robustness of a complex pattern even if at some occurrences a spike is missing. Secondly, it illustrates that subpatterns may express information processing different from that involving the superpattern (as shown by the disappearance of the event at a delay of 95 ms in Figure 11, and at the same time a relative increase in occurrences of the triplet ). On a more general level, the examples of Figures 10–11 raise an interesting problem. Notably, the incidence of firing patterns tended to be inversely related to the selectivity of the units for the stimulus that the animal held in short-term memory (Villa and Fuster, 1992). During the delay periods, and in comparison with intertrial periods, selective units showed a decrease of firing patterns, whereas non-selective units showed an increase. On the one hand, the more patterned activity of non-selective units may reflect their involvement in wide networks representing multiple and general attributes of that particular stimulus held in memory. On the other hand the term “selective unit” shows the long-held belief and

Time and the brain

24

Figure 11 . Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of a selective unit (cell 61) recorded in the primate inferotemporal cortex during intertrial intervals (25 lines) and during the delay (17 lines). The pattern was composed of 4 spikes, a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. The occurrences of the exact subpatterns formed by 3 out of 4 spikes of the original pattern are also displayed on this raster. The quadruplet is allowed an accuracy of ±3 ms, whereas the subpatterns ±1 ms. Note that during the delay the average firing rate of the cell increases, while the quadruplet pattern disappears, although one particular subpattern remains visible.

prejudicial influence of the rate code on our assumption of what is “selective”. In fact, selective units were defined on the basis of an increased firing rate during the delay period that followed a particular stimulus of the tested set. It is not possible to rule out the possibility that in another paradigm or experimental condition the “selective” units would become “non-selective” and vice versa. Thus, one should bear in mind that the term “selective” should be restricted to a particular experimental situation. However, it is interesting to note that both populations of cells increased their rate of discharges during the retention period, but only the non-selective units increased the patterned activity. An attractive interpretation of these data could be the following: to retain all stimulus characteristics, some transfer of information occurs between the selective units, which encode the specific stimulus feature being retained, and the non-selective units, which are part of a wider network encoding contextual attributes of the stimulus that may be used for binding with other features. This model supports the hypothesis of conversion from asynchronous activity to synchronous activity (and back) through diverging/converging

Empirical evidence about temporal structure

25

connections, as proposed by Abeles (1982, 1991). One important limitation of the PDA method is its application to patterns significant by themselves, as explained above. An additional limitation is the use of a fixed jitter. A new method, called Pattern Grouping Algorithm (PGA), has recently been developed, aiming to identify and evaluate the statistical significance of temporal patterns of spikes formed by three or more different events (Villa and Tetko, 1999; Tetko and Villa, 1999). In particular, this new algorithm is able to recognize patterns of spikes with slight differences in spike timing, and to cluster them as one single group of patterns. Firstly, PGA considers those patterns detected by the classic pattern detection algorithm as templates for search of similar patterns in the spike train data. There are three adjustable parameters in PGA: (i) the maximal duration of the pattern, measured as a delay between the first and the last spike in the sequence of spikes (i.e. the window duration); (ii) the level of significance to be used for detection of significant groups; and (iii) the maximum allowed jitter for timing accuracy of time delays in a pattern group. Based on the idea developed in the template search method (Dayhoff and Gerstein, 1983a) and on selection of optimal jitter, PGA is able to minimize the jitters allowed for each spike in the template independently, according to the actual distribution of spikes. Thus, the probability of detection of significant patterns of spikes is optimized. PGA is not restricted to the analysis of patterns formed by only one cell, and may be applied to the identification of precise temporal patterns regardless of their complexity and number of different cells in a pattern. Therefore, PGA can estimate the significance of each spatiotemporal pattern of firing. The method (described in detail in Tetko and Villa, 1999) consists of an independent estimation by the previously developed PDA, FPD and JTH methods, that have been updated in order to consider the variable jitters. Only those patterns that are significant by all methods are considered for the final step of PGA. Eventually, the final output of PGA consists of a central template pattern with variable jitters corresponding to previously detected patterns grouped into one class. Several tests and validation analyses have demonstrated that PGA can detect true patterns and avoid the detection of spurious patterns, even in the presence of several non-stationarities in the spike trains. These non-stationarities are likely to occur in recordings from behaving and freely-moving animals. Even so, it must be kept in mind that for most of the functional systems the available data demonstrate the presence of precisely timed neural activity whose relationship with behaviour is not always evident. To prove that these temporal patterns do indeed serve for response selection, data are required that establish causal relations between the occurrence of precise temporal relationships and cognitive or motor processes. The next examples will provide solid evidence about these processes. Spike trains from up to 15 single neurones were recorded simultaneously in the temporal cortex of freely moving rats, while animals waited for acoustic cues in a Go/NoGo task (Villa et al., 1999c). The auditory stimulus contained two types of information: pitch (“high” or “low”), and position (“left” or “right”). During the first phase of training, the location was kept constant and the rats had to discriminate between tones of high and low pitch, with one signalling “Go” and the other “NoGo”. Reinforcement was given only after correct “Go” trials. In the second phase, pairs of tones were delivered (one tone from each location), with four possible tone-position combinations. We have shown that, despite the lack of reward for correct NoGo

Time and the brain

26

performance, and the lack of painful punishment for incorrect trials, all rats learn the task to a satisfactory level (performance in the range 70–90%). The paradigm allows analysis of activity during two functional epochs: the “wait” period prior to the stimulus, during which the animal must remain at the back of the cage, and restrain responding despite an increasing probability of imminent stimulus delivery; and the “processing” time, from stimulus delivery until beginning of movement, during which the information content of the signal is processed and motor output organized. Only the “wait” period is considered here for analysis. Significant patterns were detected using PGA from a data-set of 13 hours of recording, involving over one million spikes (Villa et al., 1998a, 1999a). “Go” responses resulted both from correct movements to the feeder in response to low pitch sound at the right speaker and from incorrect movements when a high pitch was delivered to this speaker. Conversely, “NoGo” responses could be correct (failure to move in response to the high pitch from the right speaker) or incorrect (failure to move in response to low pitch from right speaker). Of particular interest were the patterns that were significantly associated with the type of response the animal made later, independent of whether the response was that prompted by the cue. Figure 12 illustrates one such pattern, a triplet formed by the spikes of two cells recorded from distinct electrodes placed in the temporal cortex within the same hemisphere. Note that there were more than twice as many trials (n=27) with patterns that were followed by “Go” responses as there were trials (n=12) with patterns followed by “NoGo” responses, despite the fact that the total number of “Go” and “NoGo” responses was nearly the same (n=283 and 269, respectively). An alternative display of the spikes belonging to the triplets is shown in Figure 12b. In this illustration, the rasters are aligned to an external event (the stimulus onset) instead of being aligned to the start of the pattern. Of these “behaviour-predicting” patterns, half were associated with an enhanced tendency to “Go” in response to the stimulus, and for about 20% of these patterns, trials including the pattern were associated with a faster or slower reaction time than those lacking the pattern (Villa et al., 1998a, 1999a). Figure 13 illustrates one such pattern, and the corresponding reaction times. The triplet shown here is formed by three different neurones, two of which were recorded from the same electrode. It is important to notice that the patterns were associated in almost half (22/37) of the “Go” responses and that pattern occurrence during the wait period led to an accelerated reaction time, by 355 ms on average. Although the patterns could start at any time during the waiting period, we

Empirical evidence about temporal structure

27

Figure 12 . (a) Raster display of the activities of two neurones recorded from two different electrodes in the temporal cortex of a freely-moving behaving rat. The rasters are aligned by displaying the first spike in the pattern at time 0. The pattern was: cell 5; 320±3 ms later cell 11, then 342±3 ms later cell 5 again. The data shown include trials recorded on two consecutive days and in which both Go (n=27) and NoGo (n=12) responses occurred, (b) Raster display of the data corresponding only to the Go responses shown in (a) but on a compressed time scale, and aligned to the time of stimulus onset instead of to time of the first spike of the pattern. Spikes involved in generating instances of the pattern during the waiting period (time to the left of the stimulus onset) are displayed as bars instead of ticks. Three spikes constituting one instance of the pattern are picked out by empty circles. Note the stability of this pattern recorded over two consecutive days.

Time and the brain

28

observed that in several cases the time course of pattern appearance was related to the reaction time. Figure 14 illustrates this finding on a cumulative display of reaction time vs. timing of pattern occurrence. The reaction times of the subjects were normalized to 800 ms on average, in order to cumulate eleven examples on the same curves. Notice that the individual timing of each pattern was by itself significantly correlated with the reaction time on a quadratic regression analysis . The left

Figure 13 . Raster display of the activities of three rat cortical neurones participating in a spatiotemporal firing pattern, aligned by the occurrence of the first spike in the pattern. The time of stimulus presentation is marked by horizontal ticks in the lower window. The pattern was formed by spike #12, 867±2 ms later cell #11, and then after 29±1 ms cell #5. The timing of the three spikes forming the pattern is indicated by the arrows below the stimulus raster. Note that this pattern repeated 22 times (p<0.001) before the onset (i.e. during the waiting period) of a stimulus which triggered a Go response (either correct or incorrect). On average, the subject responded significantly faster after stimulus onset if a pattern was detected in the waiting period.

panel of Figure 14 shows that the earlier the pattern occurred in the waiting period, the slower the reaction time. A pattern occurring just before the stimulus presentation elicited a fast reaction time, thus suggesting a kind of matching operation. The right panel of Figure 14 illustrates the opposite case, where the occurrence of the pattern just before the stimulus provoked a slower reaction.

Empirical evidence about temporal structure

29

Figure 14 . Reaction time, measured as the delay between stimulus onset and the first infra-red beam crossing, vs. the timing of patterns before the stimulus onset. The timing occurrence of five different patterns (labelled 1–5) are cumulated on the left panel, showing a faster reaction time if the pattern occurred just before the stimulus. The right panel shows six other patterns (A-F) cumulated on the same curve and showing a slower reaction time when the pattern occurred just before the stimulus. For all patterns the reaction time of the subjects was normalized to 800 ms on average in order to cumulate the data.

The remaining behaviour-predicting patterns were associated with an enhanced “NoGo” tendency. Overall mean rates of activity did not vary significantly across trials, and the patterns could not be related to the prompting auditory cues, since these occurred later and were chosen randomly. Hence, these data demonstrate for the first time that particular spatiotemporal patterns predict future behavioural responses. The information coded by means of precise spike timings has predictive value, and can therefore allow forecasts of the animal’s future response, regardless of whether or not these responses are ones demanded by subsequent unpredictable cues. The impact of the presence of specific spatiotemporal patterns on the responses of the animal suggests that such patterns participate in set-related activities that prime a particular response, and thus bias the subject’s response choice, regardless of appropriateness. Alternative, and not mutually exclusive hypotheses, can be proposed, whereas the underlying dynamics can be only a matter of conjecture. Firstly, patterns may be considered as parts of “templates” generated by preparatory processes activated in order to extract the useful information from forthcoming sensory cues (auditory “objects” in cognitive contexts requiring difficult discriminations). Templates would be activated selectively prior to the stimulus—the “Go” or the “NoGo” sound—and allow faster match-mismatch recognition; consequently, they are comparable to the “preconceived ideas” that influence human perception. “Go patterns”, for instance, would reflect states wherein signals are more likely to be interpreted as “Go” prompts. Such fine structures of brain activity, coined in more precise terms than overall mean rates, could be related to thalamo-cortical information processing. Several lines of evidence indicate that the auditory cortex exerts

Time and the brain

30

a dynamic control over the functional segregation of signals transmitted via the thalamocortical loops, and activity in such loops may generate precisely repeating interspike intervals (Villa et al., 1991; McCormick and Bal, 1994; Miller, 1996; Tetko and Villa, 1997b).

6. PATTERNS IN RELATION TO MOTOR PROGRAMMING The next examples investigate a second hypothesis which cannot be ruled out, that patterns may be parts of motor programs, each pre-prepared for active performance of either a “Go” or a “NoGo” response. The activity of multiple neurones, recorded simultaneously from the monkey primary motor cortex during the performance of a delayed-pointing task, was analyzed by PDA and PGA (Villa et al., 1995, 1998b). Each trial was initiated by the monkey pressing a lever. Following a fixed interval (predelay period), a preparatory signal and then, after a second delay, an imperative (response) signal were presented (Figure 15a). This protocol was designed to manipulate the temporal aspects of preparation for visually guided pointing movements by varying the interval between these two signals. On any one trial, this interval (the preparatory period) could take one of 4 values (600, 900, 1200, or 1550 msec). Thus, during trials with the longest delay, there were three earlier times at which a response signal could have occurred, and which might have been expected by the animal. These moments are referred to as expected signal times. Spatiotemporal firing patterns were indeed detected preferentially in relation to the expectancy of behaviourally relevant events. The timing of several of these spatiotemporal patterns was found to be related to the preparatory signal. Figure 15b shows such an example, in which a pattern of three spikes occurred 20 times, started by neurone 4, then after 113±1.0 ms neurone 6 and then after

Empirical evidence about temporal structure

31

Figure 15 . (a) Diagram of the time course of a trial in the delayed-pointing task. Each trial was initiated by the monkey pressing a lever (“start”). Following a fixed interval (pre-delay period), a preparatory signal (PS) and then, after a second delay, an imperative (response) signal (RS) were presented. The preparatory periods were randomly selected out of four possible fixed durations (600,900, 1200, and 1500 ms). For the longest preparatory period, three expected signal delays (ESI at 600 ms, ES2 at 900 ms, and ES3 at 1200 ms) are identified, (b) Raster displays of spiking activities of a group of four neurones recorded in the primate motor cortex. In the last row of the panel, the markers indicate the time of occurrence of the corresponding events. The rasters are aligned by the first spike in the pattern. The scale bar corresponds to 500 ms. The pattern was: cell 4; 113±1.0 ms later cell 6, and then 294±2.0 ms later cell 6 again. In

Time and the brain

32

three trials this pattern occurred twice. These trials can be recognized by the labels 1 and 2 near the corresponding PS. Note that the majority of patterns precede the onset of the preparatory signal. PS: preparatory signal; RS: response signal; ES: expected occurrence of RS.

294±2.0 ms neurone 6 again. In 15 out of 20 occurrences this firing pattern ended a few hundreds of ms before the onset of the preparatory signal. This result suggests that neurones in the primary motor cortex can undergo temporally patterned activity in relation to the expectancy of signal occurrence.

Empirical evidence about temporal structure

33

Figure 16 . Raster displays of spiking activities a group of four neurones (different from Figure 15) recorded in the primate motor cortex. The rasters are aligned by the last spike in the pattern. The pattern was: cell 4 and cell 5 firing almost synchronously (within ± 1.0 ms); 55±3.5 ms later cell 5, and 223±4.5 ms later cell 4. All types of trials were pooled and the patterns were equally distributed among expected signal times (ES1: n=4; ES2: n=3) and response signals (RS1: n=3; RS2: n=2; RS3: n=1; RS4: n=1). Note that when the pattern was preceding an expected signal (labeled by empty solids) the time of occurrence of the corresponding response signal is also marked. The response signals are labeled by black dots. The question mark indicates the brain process associated with the expected signal.

Of particular interest are several patterns that occurred preferentially prior to the preparatory signal and/or expected signal times. A complex pattern formed by four spikes was observed fourteen times (each time in a different trial) and always ended over a

Time and the brain

34

narrow interval (20-230 ms) before the actual or expected signal time (Figure 16). For example, in the case of the longest preparatory period (i.e. 1500 ms) a response signal could have been presented at any of four moments. Since trials were presented at random, the animal could not know in advance the duration of PP. The timing of the pattern shown in Figure 16, which predicts with reasonable accuracy the times of possible responses signals, suggests that the occurrences of the pattern correspond to some brain process associated with expectancy of upcoming signals. Furthermore, patterned activity might be initiated by synchronous volleys of spike discharges, as shown by the two quasi-synchronous events shown in Figure 16, and ended before the occurrence of the response signal. This pattern is interesting also because it exemplifies a typical result of PGA, since the optimal time jitters were all unequal. An alternative display to show the relationship between the preparatory period and the pattern occurrence is based on a raster plot of the spike trains triggered by onset of the PS. In addition, this raster should be restricted to those trials characterized by an occurrence of a specific pattern during the preparatory period. Figure 17a shows an example of a brief pattern (lasting less than 50 ms) formed by three spikes from two different neurones: the first two spikes are synchronous, and the third occurs 48 ms later. The histogram of the 28 occurrences of the end-times of the pattern (Figure 17b) demonstrates that this distribution is not uniform. Half of the patterns (14/28) ended within 100 ms before the actual or expected end of the preparatory period. Both neurones involved in the pattern showed only a slight modulation in firing rate during the preparatory period compared to the predelay period, as shown in the peri-stimulus time histograms for the longest preparatory periods (Figure 17c). However, the most interesting feature of this pattern is its predictive value for the subsequent behaviour. The total response time, i.e. the interval between signal onset and touching the target, was 512±12 ms (mean and SEM) in the 22 trials containing a spiking pattern, whereas in the 101 trials without pattern, response time was 569±13 ms. In this particular example, reaction time, i.e. the interval between signal onset and the movement onset, was significantly faster as well (248±9 vs. 297±9 ms).

7. SUMMARY OF EMPIRICAL EXAMPLES In summary, experimental results of the coordinated timing of spikes, and the synchronous firing or network synchronization have provided several conceptual frameworks in which temporal coding in spiking activity is important. Temporal codes do not preclude a rate code being superimposed on it simultaneously. The presence of sensory stimuli or motor output often correlates with single-unit firing rates averaged over many trials and/or with asynchronous compound activity of a neuronal population (Riehle and Requin, 1995; Georgopoulos, 1995). In addition, it has been well established that hippocampal place cells signal by their rates whether the animal is within the place field, independently of whether the animal is entering or leaving that field (Wilson and McNaughton, 1993). However, as such studies increase in complexity, a simple rate code may be rendered inadequate as a predictor of behaviour. In this section a number of

Empirical evidence about temporal structure

35

bodies of experimental evidence for the existence of precise neuronal timing has been provided. Spatiotemporal patterns do in fact occur spontaneously, but occur recurrently, and in a more stable way, in particular temporal relationships to stimuli, or movement, and during the retention period of a delayed response task; and under particular experimental conditions “behaviour-predicting” patterns have been detected. It has been proposed (Ferster and Spruston, 1995) that clear proof for a

Figure 17 . (a) Raster display of the spiking activities of two neurones recorded simultaneously in the primate motor cortex. The spike trains are aligned on the onset of the preparatory signal (PS). Only trials characterized by the presence of a pattern are displayed. The pattern was: cell 1 and cell 6 firing almost synchronously (within ±1.0 ms) and 48±5.0 ms later cell 1 again. In one trial the three spikes participating in the pattern are indicated by circles. In the second row (cell 1) thick ticks indicate the last spiking event of each pattern. In

Time and the brain

36

the third row the ticks indicate the response stimuli (RS) associated with each pattern (i.e., RS1, RS2, RS3, and RS4 after 600, 900, 1200, and 1500 ms, respectively). Note that pattern occurrences were distributed in 22 trials, i.e. in few cases one pattern occurred in relation to the expected stimulus (ES) and one in relation to the response stimulus in the same trial, (b) Histogram of the end-times of pattern occurrences. The bin size is 100 ms. Note that the last spike of this pattern preceded either an expected stimulus (12 times, empty bins) or RS (16 times, filled bins), (c) Peri-stimulus time histograms of the activity of cells 1 and 6 for 31 trials characterized by preparatory periods lasting 1500 ms (PP4). The abscissa is time (ms) and the ordinate is scaled in rate units (spikes/s). The vertical full scale for cell 1 and cell 6 is 27 and 45 spikes/s, respectively. The curve is convoluted by a Gaussian shape bin of 22 ms.

temporal code would require that distinct stimuli could reliably produce different temporal spike patterns. However, this restrictive view, to assimilate the temporal code to a kind of “Morse alphabet”, cannot account for the complex dynamics existing in the brain described in the next section.

8. CHAOS AND ATTRACTORS An alternative way to investigate neural dynamics in spike trains assumes that the whole time series of spike occurrences is an expression of some fundamental process governing the activity of the neurones being recorded. When a specific input pattern activates a cell assembly, it is assumed that the neurones will be activated either in a timely organized mode or in an asynchronous mode. Then, a mode of activity defines how information is processed within a neural network and how it is associated with the output pattern of activity that is generated. This assumption does not exclude the possibility that the same cell would respond in an opposite mode to other stimuli. Furthermore, the same neural network may subserve several modes of activity through modulation of its connectivity (for instance during learning or pathological processes) or by modulation of its excitability, e.g. by modulation of the resting potential or of the synaptic time constants. In this framework the state of the neural network is defined by a set of parameters characterizing the neural network at a certain time. Then, the state of the network at any given time is represented by the values of these parameters, and a network state is fully determined if all parameters are known for each neurone. A state, in general, changes over time. Neurones are embedded in an inextricably large neural networks whose coordinated activities determine the timing of action potentials. Ab absurdo if we were able to set the same initial conditions for all elements of the neural networks we would obtain the same spike trains. Such a dynamical system as a whole is said to be deterministic, as it is possible to predict precisely the evolution of the system in time if one knows exactly the initial conditions. However, a slight change or incorrect measurement in the initial conditions results in a seemingly unpredictable evolution of

Empirical evidence about temporal structure

37

the system. Such sensitive dependence on initial conditions is a hallmark of chaotic systems. A process is defined by the passage of a state over time. The simplest definition of a chaotic process, or simply chaos, is that it is completely deterministic at each step of its temporal evolution, yet over the long term, its response is unpredictable. An equivalent definition of a process is a “path” over time, or trajectory, in the state space. The attractor is defined by the set of points approached by the trajectory as time increases to infinity. If the evolution in time of the system is described by a trajectory forming a closed loop (also referred to as a periodic orbit), then the system is said to have a limit cycle, also referred to as a periodic attractor. In general, brief initial perturbations applied to any combination of the governing set of parameters move such a system away from the closed loop, but, with passage of time, the trajectory collapses asymptotically to the same attractor. If, on the other hand, the system is deterministic, yet sensitive to small initial perturbations, the trajectory defining its dynamics is an aperiodic orbit, then the system is said to have a chaotic attractor, often referred to as a strange attractor.

9. DYNAMICAL SYSTEM ANALYSIS Let us consider a simple example of a point process, whose dynamics is characterized by discrete steps in time. Let be a time series with K points, where x represents the state of the system. In a dynamical system the subsequent state of the system is determined by its present state (Addison, 1997). The simplest expression would be to consider a map defined by where α is a control parameter. The biological systems, and the brain in particular, are often characterized by feedback mechanisms. The expression known as the logistic map, illustrates a simple dynamical system with a negative non-linear feedback, defined for x [0,l]. It is clear from this expression that the time arrow is non-reversible, because it is always possible for each x i to obtain a value x i+1 but there are two possible x i for each x i+1 . The behaviour of this equation is very interesting. For all values 0≤α≤1 the iterated series decay towards zero. Conversely, for some other values of α , e.g. 1.7 or 2.1, the series converges to a fixed point, equal to 0.52381 and 0.41176 respectively. Conversely, for values α equal to 3.2 and 3.52, the series converge to two alternating fixed points, i.e. and four alternating fixed points, i.e. respectively. The trajectories of the three last examples are periodic orbits, with periods equal to 1,2, and 4, respectively. However, it is possible to find other intersesting cases. With a control parameter equal to 4.0 and an initial condition x 0=0.6 the system tends to decay to zero, but with an initial condition x 0=0.4 the dynamics never produces a repeating sequence of states. This aperiodic behaviour is different from randomness, or white noise, because an iterated value x i can occur only once in the series, otherwise, due to the deterministic dynamics of the system, the next value must be also a repetition and so on for all subsequent values. In formal terms, let us define a function ƒ which describes how a state x is mapped into the space of state. Such a function is the probability function f of the mode of activity of

Time and the brain

38

the network. If the function is set by a control parameter µ we can write . A dynamical system x’ is a subset of the space of state and can be obtained by taking the gradient of the probability function with respect to the state variable, that is . Mathematically speaking, the space of state is a finite dimensional smooth manifold, assuming that ƒ is continuously differentiable and the system has a finite number of degrees of freedom (Smale, 1967). For periodic activity the set of all possible perturbations define the inset of the attractor or its basin of attraction. In case of the logistic map, for a=3.2 all initial conditions in the interval end up approaching the period 2 attractor. This interval is known as the basin of attraction for the period 2 attractor, whereas the value x 0=0.6875 is a fixed point. If the activity is generated by chaotic attractors, whose trajectories are not represented by a limit set either before or after the perturbations, the attracting set may be viewed through the geometry of the topological manifold in which the trajectories mix. The function ƒ corresponding to the logistic map, is the parabolic curve containing all the possible solutions for x . This function belongs to the single humped map functions which are smooth curves with single maxima (in here the single maximum is at x =0.5). It is easy to conceive that in large neural networks the complexity of the system is such that several attractors may appear, moving in space and time across different areas of the network. Such complex Spatiotemporal activity may be viewed more generally as an attracting state, instead of simply an attractor. In particular, simulation studies demonstrated that a neural circuit activated by the same initial pattern tends to stabilize into a temporally-organized mode or into an asynchronous mode if the excitability of the circuit elements is adjusted to the first order kinetics of the postsynaptic potentials (Villa and Tetko, 1995; Hill and Villa, 1997).

10. DETECTION OF CHAOS IN EXPERIMENTAL DATA SETS It is obvious that it is not possible to know all variables determining brain dynamics, yet the progress made by computational and statistical physics during the two past decades has brought a number of methods allowing one to differentiate between random (i.e. unpredictable) and chaotic (i.e. seemingly unpredictable) time series. It is difficult to prove that biological systems generate chaotic attractors, owing primarily to their shortlived and apparently nonstationary behaviour (Mpitsos, 1989). Several lines of experimental observation suggest that chaotic processes may be generated by brain activity (Babloyantz et al., 1985; Rapp et al., 1985). In particular, chaotic discrete dynamics observed in spike trains allow simple networks of neurones to perform complicated tasks that would require considerably more complex networks to perform if the signals were generated by non-chaotic discrete or continuous processes (Mpitsos and Burton, 1992). Among several methods available for classification of discrete time series the Grassberger-Procaccia algorithm (Grassberger and Procaccia, 1983) has been widely applied to theoretical and experimental cases. This algorithm is based on the following

Empirical evidence about temporal structure

39

idea. Let { x 1 ......x k } be a time series corresponding to interspike interval sequences generated by K +1 number of spikes (Cutler and Kaplan, 1997). Delay coordinates { y 1 ....y n } in a d -dimensional embedding space are defined by setting y 1 = ( x 1 .....x d ), y and Y≡ { y 1 ....y N } is the 2 = { x 2 ....x d+1),....y N ={ x N ....x K ), where set of delay vectors. One fixes an embedsding space with dimension d and computes the values of the so-called correlation integrals, corresponding to the average of the number of points falling inside an hypersphere (with arbitrary small radius) around each point of the original time series. These quantities are plotted against the radius for several values d of the dimension of the embedding space (Figure 18). When the curves are definitely parallel, the system is said to be deterministic and their slope defines the correlation dimension, often referred to as D 2 , of the attractor underlying the chaotic structure of the system (Figure 18b). The value of d at which the convergence starts is the embedding dimension. In practical applications, the correlation dimension D 2 must be calculated in a meaningful range of values of the radius. This interval, denoted as the scaling region, must have an amplitude of at least a decade (Eckmann and Ruelle, 1992). Moreover, the number K of points defining the time series should be significantly large in order to get reliable estimates of the correlation dimension. In particular, for typical experimental time series with 800≤ K≤10,000 points the embedding spaces should have a realistic dimension d≤6. Higher dimensional systems cannot be reliably detected over relatively

Figure 18 . Correlation integrals of two spike trains, (a) No attractor is detected in the embedding space of dimensions d up to d=8 for this spike train (n=1094 spikes), (b) This spike train (n=876 spikes) is characterized by a correlation dimension D 2=2.1 embedded in a 4-dimensional space. The thick black bars indicate the scaling region. Note that for time series with less than 900 points the computation of the correlation integrals is stopped at an embedding space of dimension d=7.

short time series and their dynamics cannot be distinguished from random noise Rapp et al., 1993). Furthermore, the embedding dimension of the interspike interval attractor

Time and the brain

40

reconstruction must be greater than or equal to the correlation dimension of the attractor in order to obtain a meaningful result (Castro and Sauer, 1997). Misapplications of the Grassberger-Procaccia algorithm led, in the 1980s, to unreliable estimations of deterministic structures for time series (see Rapp et al., 1993; Theiler and Rapp, 1996, for details). Nonetheless, its careful use in recent years has indicated that numerous neurones (up to 25% in some samples) recorded in the substantia nigra and thalamus of anaesthetized rats can be characterized by deterministic dynamics (Celletti and Villa, 1996; Celletti et al., 1997,; Celletti et al., 1999). The analysis of neuronal activity recorded in the primate motor cortex during the delayed-pointing task (see Section §6) revealed the existence of chaotic dynamics in and across paired spike trains (Villa et al., 1998b). The attractors were generally embedded in a space of low dimension, ranging from 2 to 5, and were characterized by a correlation dimension between 0.15 and 3.34 (Figure 19). In the intertrial periods, the dynamical system tended to be characterized by a correlation dimension near 1, and by an attractor embedded in a 4-dimensional space. Conversely, during the preparatory period the geometric scaling properties of the neural dynamics were broadly distributed in the sense that no clear clustering around a specific value of the correlation dimension could be observed (Figure 19). This might suggest the disruption of a common state of activity by the ongoing motor preparation. Attractors observed during the preparatory period were not related to the activity associated with anticipation of the response signal, but rather to preparatory motor processes. Thus, the question is raised whether such dynamics bear some message,

Figure 19 . Neuronal activity recorded in the primate motor cortex (see Figure 15a). Scattergram and marginal distributions of the embedding and

Empirical evidence about temporal structure

41

correlation dimensions of the attractors determined by the GrassbergerProcaccia algorithm in 34 spike trains. The filled symbols and the black bars refer to observations during the preparatory period whereas empty circles and light gray bars refer to the control period.

or may represent a “side effect” of some global self-organizing dynamics within the cerebral cortex. Indeed, deterministic dynamics in the cortex may be interpreted as a collective phenomenon arising from the expression of local attractor dynamics, sustained by a synaptic matrix formed in the process of learning (Herrmann et al., 1993; Amit et al., 1994; Amit, 1995; Amit et al., 1997). However, with respect to information processing in the brain, the body of observations derived from the application of these methods strongly depends upon the dynamics of the underlying neural circuitry. The detection of deterministic dynamics necessarily requires the stability of the generating processes over a relatively long period of time. Thus, significant deterministic dynamics across spike trains may reveal neuronal interactions involved in long-term processes (e.g. memory traces, learned motor programs), whereas the previously cited methods could reveal short-term operational processes at attentional or feature-binding level. Therefore, pattern detection algorithms and dynamical systems analysis may both be characterized by repetitive dynamics of neuronal activity. The question can be raised if there are some relations between these two approaches.

11. SPATIOTEMPORAL FIRING PATTERNS AND CHAOTIC ATTRACTORS PGA was applied to two records from a large scale simulated network described elsewhere (Amit and Brunel, 1997) with maximum allowed jitter equal to 7 ms. The network was composed of 15,000 integrate-and-fire cells, from which 12,000 were excitatory and 3,000 were inhibitory. Each cell in the network had a probability 0.15 of a direct contact with other cells and received, on average, 1,800 excitatory and 450 inhibitory synapses from neurones belonging to the network and 1,800 excitatory synapses from outside the network. In the first record, referred to as “random”, the strength of neuronal connections was initially set as random (Amit and Brunel, 1997) and was not modified. Thirty units were selected by chance, and their activity was recorded during 180 seconds with an accuracy of 1 ms. The activity in the “random” record corresponded to sustained spontaneous rates, in the range 0.1–11.7 spikes/s (mean=4.0 spikes/s and median=3.1 spikes/s). In the second record, referred to as “Hebbian”, the weights of specific populations of synapses were modified according to some type of learning described in detail elsewhere (Amit and Brunel, 1997). The activity of the same set of units was recorded following a stimulation correlated with the learned stimulus. Discharge rates of cells were increased after the Hebbian learning (mean and median equal to 6.5 and 4.2 spikes/s, respectively). No firing pattern was detected in the “random” record by PGA, whereas most cells (20/30 of this sample) formed patterns in the “Hebbian” record. Note that all detected patterns were formed by one cell. The firing rates (mean and median equal to 7.1 and 5.3

Time and the brain

42

spikes/s, respectively) of cells involved in significant group patterns were higher than the rates (mean and median equal to 5.3 and 3.9 spikes/s, respectively) of cells that never formed a temporal pattern. However, the firing rate or the shape of the autocorrelogram were not predictive of the presence of patterns. Most of the autocorrelograms were almost flat and indicated that the spike trains deviated only slightly from a Poisson renewal point process (Figure 20a). In this example, cell 16 was characterized by a firing rate of 17.6 spikes/s but no significant temporal pattern was detected. Cell 17 fired at a similar rate, 17.2 spikes/s, and its autocorrelogram was similar to that of cell 16 (Figure 20a), but 238 patterns of complexity 3–5 were detected. Figure 20b illustrates a significant pattern formed by 5 events repeating 10 times, i.e. <17, 17, 17, 17, 17; 31±2.0, 168±1.5, 186±3.5, 231±2.0.>. In this example the event at delay 186 ms from the pattern onset was detected only at the maximum allowed variability, whereas the other four events formed a significant subpattern at a much greater precision. The relation between spatiotemporal patterns of firing and attractors can be illustrated further by the following example. Let us consider a classical dynamical system, the 2dimensional Hénon mapping (Hénon, 1976). This map is characterized by chaotic behaviour within a certain range of input parameters. With a particular set of such initial parameters it is possible to generate a discrete time series, scaled to a base frequency of 3 spikes/s, thus simulating a possible cortical spike train recording (Tetko and Villa, 1997c). One thousand spikes were generated, corresponding approximately to 330 seconds of recording time. Figure 21a illustrates an example of a spike train generated in this way. In addition to the raster display the return map provides a useful tool to determine visually the presence of a structure. The X-Y plot of the return map is constructed by taking the value of an interval between two successive spikes as

Empirical evidence about temporal structure

43

Figure 20 . High complexity pattern elicited by an attractor mode of activity in a randomly connected network after “Hebbian” learning, (a) Autocorrelograms of three cells recorded simultaneously. The abscissa is lag (in ms) and the ordinate is scaled in rate units (spikes/s). The curves are smoothed by a 10 ms Gaussian shaped bin; the dashed lines indicate the 99% confidence level assuming a Poisson distribution. Note that the shapes of the curves are similar for the three cells, but cell #16 produced no significant patterns, (b) Ten repetitions of a pattern of complexity five, formed by repeated events of cell #17, are displayed as rasters aligned on pattern start. Note that the fourth event, at a delay of 186 ms, was detected on the limit of the maximum allowed accuracy (±3.5 ms).

X-coordinate and the immediately next interspike interval as Y-coordinate. Even if some

Time and the brain

44

regularity may be detected visually on the inspection of the raster display (Figure 21a, left panel), the geometric structure of the return map (Figure 21a, right panel) revealed deterministic dynamics more clearly. In particular, notice the values of the intervals in the range 190–260 ms. The dynamical systems can enter a chaotic regime corresponding to small perturbations of some parameters. The counterpart is that a chaotic regime can be perturbed to

Figure 21 . Raster display and return map of a spike train either generated directly following the Hénon mapping (a) or after deletion of 30% of the spikes and addition of 30% dynamical noise (b). The values of successive intervals In-1 and In are expressed in ms.

some extent, but remains chaotic because of its attractor structure. Let us consider the Hénon spike train perturbed by random deletion of some spikes, and with insertion of new spikes generated according to a uniform distribution on the actual interval. An example of such noisy Hénon spike train is illustrated on Figure 21b. The usual algorithmic methods used to determine if such noisy time series were deterministic may fail to demonstrate any significant structure in the dynamics (Tetko and Villa, 1997c) although in the present example the visual inspection of the return map suggests some regularities (Figure 21b, right panel). It is important to note that chaotic dynamics found in neurophysiological spike trains may be low-dimensional (Celletti and Villa, 1996; Celletti et al., 1998; Villa et al., 1998b), but still with attractors embedded in spaces of dimensions 3–5 whose observation is difficult on a 2-dimensional projection. The

Empirical evidence about temporal structure

45

application of pattern detection algorithms to noisy spike trains generated by attractor dynamics may indeed reveal the presence of recurrent temporal patterns. A typical example is illustrated by Figure 22a, showing a triplet characterized by spike intervals equal to 210 and 191 ms, that fall in the range of the original attractor (Figure 21a). An additional example illustrates an effect due to the noise in the pattern detection results of this type of data-set. The occurrence of 12 triplets <1, 1, 1; 60±4.0, 204±1.5>, characterized by the second event of the pattern having a much higher variability than the third, are shown in Figure 22b. In fact the delay of the third spike is equal to 204 ms, a value included in the range of the original attractor dynamics. Conversely the second event is contributed either by chance or by dynamical noise. In fact, an increase in firing rate loosely time-locked to the onset of the pattern can be observed at delays of approximately 448 and 1440 ms, thus suggesting a non-random origin. However, this example is interesting because it shows that events belonging to the same pattern may have been generated by different mechanisms, so that the temporal pattern might represent a signpost of a sort of binding between those mechanisms. Similar conclusions were discussed from experimental observations described above (Figure 11). The time series generated by chaotic attractors can produce patterns of spikes that recur significantly more often than expected by chance. This finding raises the question whether significant patterns of spikes detected in experimental data are the product of attractor behaviour of neural networks, or if they are produced solely by synfire chains. As mentioned above, the comparison between dynamics of primate motor cortical cells recorded during the period preceding the preparatory signal and during the preparatory period showed important differences of the attractors characteristics (Villa et al., 1998b). The intertrial period corresponds to an interval during which the monkey is quiet and waits for the occurrence of the preparatory signal. In such behavioural conditions, it is

Figure 22 . Raster display, aligned by displaying the first spike in the pattern at

Time and the brain

46

time 0, of the activity of a simulated neurone whose dynamics was generated according to the noisy Hénon mapping (Figure 21b). (a) The pattern repeated 5 times, and was composed of 3 spikes: a spike at time 0, the second 210 ms later, and the third spike 191 ms later. Six additional occurrences of the exact subpatterns formed by the two pair of spikes <1,1; 210> and <1,1; 191> are also displayed on this raster. The triplet was detected with a fixed accuracy of ±2 ms. (b) The pattern repeated 12 times and was composed of 3 spikes: a spike at time 0, the second 60±4.0 ms later, and the third spike 144±1.5 ms later. Notice the variable jitter. An increase in firing rate which is loosely time-locked with the pattern start is indicated by the asterisks. The abscissa full scale is 2000 ms.

possible that the primary motor cortex is not yet involved in motor preparation, because the actual salient cognitive activity is the expectancy of the preparatory signal. Then, the firing activities of motor cortical cells may be attracted in the same basin, a kind of “common attractor” as postulated theoretically for spontaneous activity (Villa, 1992; Amit, 1995; Villa and Tetko, 1995; Amit and Brunel, 1997). In the next section a model of neural dynamics is presented, based on formal definitions and on the above considerations of chaotic activity in spike trains.

Empirical evidence about temporal structure

47

12. A MODEL OF NEURAL DYNAMICS System identification rests upon a vast quantitative methodology (Nieman et al., 1971), but most of it refers to linear operations. In previous sections significant sensory, cognitive, behavioural, and simulation correlates of non-linear temporal patterns of firing have been provided. Spike trains are statistically expressed by point-like processes (Segundo et al., 1966). Thus, a crucial requirement for a theoretical framework is to identify these point process systems without any assumption as to whether or not they are linear. Specific methods for this purpose have been developed (Brillinger 1974, 1975; Rapp et al., 1994). Point process systems—systems whose input and output are point processes—are said to be identified when an acceptable model is found. The first step of the identification is to estimate certain conditional rate functions, called kernels, of the spike trains. The one of zero-order, i.e. a constant, simply measures the mean rate. The one of first-order, a function of a single time argument, relates to the average effect of a single trigger spike (presynaptic) on the spike train; the one of second-order, a function of two time arguments, relates to the interactions between pairs of spikes; and so forth for higher-order functions. Then successive models are constructed recursively and based on the kernel of zero-order, on the kernels of zero- and first-order, on the kernels of zero-, first- and second-order, and so on. The acceptance of a kernel estimate as plausible does not necessarily mean that the series up to the corresponding term is a good predictor. The predictability may be evaluated by means of coherence (Brillinger, 1975). On the basis of the above assumptions it is possible to identify the neural network system (see Appendix I) that fulfils the necessary requirement of the probability function f describing its mode of activity. With respect to the state variable µ, the system may be qualified as a dynamical system x’ where , as mentioned above (Section §9). The next step consists of assuming that the dynamical system is structurally stable. In terms of topology, that means that, for a dynamical system x’ there exists a neighbourhood N(x’) in the state space with the property that every is topologically equivalent to x’. This concept is extremely important, because a structurally stable dynamical system cannot degenerate. There is no need to know the exact equations of the dynamical system because qualitative, approximate equations—i.e. “in the neighbourhood”—show the same qualitative behaviour (Andronov and Pontryagin 1937) (Figure 23). In the case of two control parameters, the probability function ƒ is 2 defined as the points µ of IR with a structurally stable dynamics of (Peixoto, 1962). That means the qualitative dynamics x’ is defined in a neighbourhood of a pair ( x 0, µ 0) at which ƒ is in equilibrium (e.g. minima, maxima, saddle point). With these assumptions, the equilibrium surface is geometrically equivalent to the RiemannHugoniot or cusp catastrophe described by Thorn (1975). Figure 24 illustrates a possible topological interpretation of neural dynamics according to this model. The equilibrium surface represents stable modes of activity with postsynaptic potential kinetics and the membrane excitability as control parameters.

Time and the brain

48

Let us start at a point (a) on the equilibrium surface, corresponding to a high level of excitability and a relatively long decay time of the postsynaptic potentials, e.g. 12 ms. This may correspond to the tonic mode of firing described in the thalamo-cortical circuit (Steriade et al., 1990). With these “settings” an input pattern will evoke precisely timestructured activity. Different firing patterns may be evoked by the same input if the synaptic dynamics are modified with minor changes of the cellular excitability, as suggested for neuromodulatory mediators (Levitan, 1988). Furthermore, different input patterns of activity may subserve the same mode of activity, somewhat like attractors. The transitions between these states are represented graphically in Figure 24 by paths (ab-a), (a-e-a) and (a-g-a). When the excitability is decreased, several types of neurones in the cortex and in the thalamus tend to switch towards a rhythmic or bursty type of firing. This effect may be provoked by a hyperpolarization of the cell membrane, or by modifying the spike threshold level (Foote and Morrison, 1987). In the former case a smooth passage between time-structured activity and asynchronous firing is likely to occur, as suggested by path (b-c-b), especially if the synaptic decay is long. Conversely, if the synaptic decay is fast, and a modulatory input modifies the threshold potential, a sudden switch from temporal patterns of firing to desynchronized activity will occur, as indicated by paths

Figure 23 . Two-dimensional representation of dynamical systems that are topologically equivalent (see text for definition of equivalency).

Empirical evidence about temporal structure

49

Figure 24 . Topological interpretation of neural dynamics as a function of two control parameters, the cell excitability and the kinetics of the postsynaptic potentials. The equilibrium surface is represented by a cusp catastrophe, where transitions can occur either suddenly or continuously between temporally organized firing patterns and asynchronous activity. This equilibrium surface refers to the activity of a specific neural network, and it is possible that the same cell belongs to more than one cell assembly. If the cell assemblies are controlled only by one parameter in common, then temporal and rate code are not mutually exclusive.

(a-d) and (e-f). Complex spatiotemporal firing patterns may also occur with low levels of excitability, i.e. point (e) in the graph of Figure 24, as suggested by cholinergic switching within neocortical networks (Villa et al., 1996; Xiang et al., 1998). Such a point (e) on the equilibrium surface can be particularly unstable, because a further decrease in excitability [path (e-f)], but also an increase in synaptic decay [path (e-d)], may provoke a sudden change in the mode of activity, as observed in simulation studies (Villa, 1992; Hill and Villa, 1997). It is important to notice that if the excitability is low, e.g. during long lasting hyperpolarization, the kinetics of the postsynaptic potential is often irrelevant with regard to the input pattern, so that the output activity will always tend to be organized in rhythmic bursts. Conversely, if the excitability is increased from a starting point (f) and the time constant of the synaptic decay is fast (say 4–5 ms), the input patterns could switch on either stable [path (f-g)] or unstable temporally-organized modes

Time and the brain

50

of activity [path f-e] only through sudden transitions.

13. SUMMARY OF CHAOTIC DYNAMICS Point processes defined by spike trains can be analyzed from alternative conceptual frameworks. Although few reasons are generally considered for adopting these frameworks as complementary approaches to the same biological system, several observations, either from experimental or simulated data, and theoretical assumptions have lead us to raise serious questions about the relationships between Spatiotemporal firing patterns and chaotic processes in neural networks. The dual description of brain activity could be regarded as similar to the dual representation of the light, with corpuscular and wave equations representing two irreducible expressions of the same phenomenon. A careful investigation of the dual nature of highly converging/diverging neural networks could open unexpected new ways to answer the problems arising from observations at multiple scales (ranging from sets of single neurones to brain imaging) of brain activity. In mathematics, few years ago, the demonstration of the Taniyama-Shimura conjecture filled the gap between two worlds and opened the way to the demonstration of Fermat’s last theorem by Andrew Wiles. This conjecture says that every rational elliptic curve (e.g. characterized by doughnut shapes) is a modular form, and modular forms have special properties with their many symmetries. Likewise, it is appealing to suggest that the relationships between chaotic dynamics and complex patterns of firing may represent a dictionary where questions, intuitions and insights in the one world get translated to questions, intuitions and insights in the other world.

14. FUTURE PROGRAM The detection of precise Spatiotemporal firing patterns or attractors necessarily requires the stability of the generating processes over a relatively long period of time. Thus, precise spike patterns in single neurones, or across multiple neurones, may be mostly involved in long-term processes (e.g. memory traces, learned motor programs), whereas the ensemble coding based on systematic modulations of firing rate may be related to short-term operational processes (e.g. motor action, attentional or feature-binding). The research presented here is not asking the questions that most neurophysiologists usually ask: which is the most adequate stimulus for a given neurone?; how is the external world mapped in the cortex?; what precisely are the receptive fields of single units? What is being asked here is how to detect the participation of nerve cells in distributed brain information processing, e.g. sensory discrimination, preparatory activity or short-term memory. The quantitative measure of neuronal activity available to neurophysiologists is the time series of spike epochs—i.e. the spike train. Therefore, it is of utmost interest to study temporal patterns of firing and assess whether they might be carriers of information. It is true that in the nervous system the problem of learning is crucial and

Empirical evidence about temporal structure

51

can hardly be approached without taking into account synaptic modification. However, not all neural information needs to be processed through modifiable networks. Changes in strength of a particular link between two cells become meaningless, because, in a distributed system, equivalent changes at one synapse might correspond to different patterns of activity. Therefore, the information is not necessarily resident in the links among the units, but might be provided by activity organized in a highly precise temporal mode. Indeed a connectionist network with the same set of connection weights may transmit different signals (Mpitsos et al., 1988). Accurate application of complex pattern detection methods enables correlation between discrete brain states and measures of behavioural performance on a trial-by-trial basis. The fact that the timing of pattern occurrence could relate to reaction time indicates that the network phenomena underlying them reflect some state of the animal that is able to influence behavioural output. Changes in cortical network activity during the waiting period may therefore be related to the concepts of “attention” and “set”, with emphasis on processes related to motor output in the former (Wise and Kurata, 1989) and to sensory processing in the latter (Shinba et al., 1995). It is interesting to notice that in the behavioural experiments reported here we found relatively few significant patterns, in the order of few hundreds, for tens of hours of analyzed recording time. The recording of cell activity is done over short periods (10 s) per trial, but one behavioural experiment could last several hours. It is possible to postulate either that the single unit detection might be unstable over such a prolonged time, or that only those cells involved repeatedly in activity directly related to the animal behaviour can participate in the detected patterns, thus decreasing the chance of detection with a limited sample of 15 spike trains recorded simultaneously. The problem of finding several neurones from a synfire chain is analyzed by Abeles (example 7.6.2, p. 255 in Abeles, 1991). In this example, finding two or more neurones from the same chain, when recording 10 neurones simultaneously, where p 0 =0.15 is the estimated probability of finding one cell from a chain, can be performed with an approximate probability of 0.44. The probability of detecting the same number of neurones in simultaneous recordings of 15, 20 or 30 cells, increases significantly. If the assumptions of this example are correct, it becomes possible to detect 6 or more neurones belonging to the same synfire chain, when recording from 30 single units simultaneously, with a probability near to 0.30. These estimations take into account only one synfire chain, but a single neurone may be involved simultaneously in multiple synfire chains and a synfire chain can include the same neurone twice, or even a higher number of times (Abeles et al., 1994). Altogether the experimental data and theoretical assumptions presented here make it possible to outline a general framework of coding principles, in which time and rate coding do not represent mutually exclusive coding schemes. Temporal and/or noisy rate codes might operate simultaneously in a given network, and several tools of analysis should be used to test the general hypothesis that the activation of widespread cell assemblies is distributed through multiple patterns of neuronal activity. Synchronous or oscillatory activity might activate and start complex temporally-organized processes, e.g. encompassing distributed sensorimotor information over widespread brain areas or thalamo-cortical recurrent loops (Villa et al., 1996). State-of-the-art techniques have allowed several laboratories to record tens of cells simultaneously in behaving rats

Time and the brain

52

(Skaggs and McNaughton, 1996; Nicolelis et al., 1997). Simulation of large-scale neuronal networks formed by integrate-and-fire model neurones are also becoming popular to test hypotheses on the significance of temporal information processing in the brain (Amit and Brunel, 1997; Lumer et al., 1997). The possibility of inserting virtual electrodes in such networks and of recording from hundreds or thousands of virtual neurones will certainly become a routine analysis in the forthcoming years. Then, the possibility of detecting and estimating carefully the significance of high complexity spatiotemporal patterns of spikes appears to be a crucial step for the evaluation of coding schemes. The spatiotemporal firing pattern should not be considered as representing a Morse alphabet, but rather the co-representation of specific information and a certain state of “central arousal”. Future developments of temporal dynamics and topological interpretation of neuronal network activity could be an important issue in understanding the functional properties and information processing in the central nervous system. Whether this prediction is upheld or not, there is no doubt that these studies will open up new possibilities for research (new areas, new methods of analysis, new recording technique, computer-modeling assistance, etc.) and no possible outcome should be a dead end. After all, we are only 200 years from the intellectual legacy of Galvani and although we may say that the pre-history of Neuroscience is over we should not rule out the possibility that we are still in its proto-historical phase.

ACKNOWLEDGEMENTS This work was made possible particularly by the contribution of Igor V.Tetko and by the members of the Laboratory of Neuro-heuristics (www-lnh.unil.ch) and the staff personnel of the Institute of Physiology of the University of Lausanne. Partial support by the Swiss National Science Foundation (Bern) and the Istituto della Enciclopedia Italiana (Roma) is acknowledged.

APPENDIX I The following paragraphs illustrate a possible framework for system identification. Suppose that A is a stationary spike train. The overall mean rate of the A train is defined by Prob { A spike in ( t,t + h )}/ h. Suppose that B is another stationary spike train occurring simultaneously to spike train A and m B its mean rate. A first-order conditional rate function m AB (u) is defined as follows. For one cell B and close to any particular time t the AB cross-intensity function, m AB (u), measures the instantaneous rate of the B train—i.e. the probability whether there will be a B spike- u time units after an A spike, say

(1) The average change of the instantaneous rate at time u in a spike train B, when a single

Empirical evidence about temporal structure

53

spike occurs at time 0 in a spike train A is predicted at best by the kernel of first order a (u). If one defines m AA (u) as the auto-intensity function of A —i.e. the cross-intensity function of the A train with itself-then, where satisfies the following condition (Brillinger 1974),

(2)

Because of stationarity this function does not depend on t. If the spike trains are independent, then m AB (u)=m B for any u . If spike train A were Poisson noise, m AA (v) would be equal to m A for all v , and the solution of Equation (2) would be a(u)=m AB (u) −m B . In this case the kernel function would correspond to the cross-intensity function up to an additive constant and it could be estimated by the corresponding cross-correlation histogram. In order to construct a general solution a(u) of Equation (2) let us define two parameters, φ AA (λ) and φ AB (λ). The parameter

(3)

denotes the power spectrum of the A spike train (Brillinger 1974). This is proportional to the variance of the component of frequency λ of the A spike train, hence it may be interpreted as reflecting the power in each frequency component of that train. For pure Poisson noise, it is identically constant and equal to (2 p)−1 m A . The parameter (4)

denotes the cross-spectrum of the A and B spike trains. This is proportional to the covariance of the component of frequency λ of the A train with the corresponding component of the B train. Hence it may be interpreted as reflecting how a certain frequency component in the A train is associated with one in the B train. The Fourier transform of the solution a(u) is defined by

(5)

Taking the Fourier transform of Equation (2) and the definitions (3) and (4) leads to φ AB , the solution of Equation (λ)=A(λ) φAA (λ). It follows that, if (2) may be written as

Time and the brain

54

(6)

Finally, the measure of predictability of the kernel is provided by the coherence at frequency λ defined by It is never less than zero or greater than one. It is a measure of the association of the spike trains. A coherence of one would imply perfect linear prediction of the B train by the A train at frequency λ. A coherence of zero results if the two trains are statistically independent or if at frequency λ. Several studies have applied this approach to study the functional interactions between spike trains (Rosenberg et al., 1989; Brillinger and Villa, 1994, 1997).

REFERENCES Abbott, L.F., Varela, J.A., Sen, K., and Nelson, S.B. (1997) Synaptic depression and cortical gain control. Science, New York 275, 220–224. Abeles, M. (1982) Local Cortical Circuits: An Electrophysiological Study , Springer Verlag, Berlin. Abeles, M. (1983) The quantification and graphic display of correlations among three spike trains. IEEE Transactions on Biomedical Engineering 30, 235–239. Abeles, M. (1991) Corticonics: Neural Circuits of the Cerebral Cortex . Cambridge University Press, New York. Abeles, M., and Gerstein, G.L. (1988) Detecting Spatiotemporal Firing Patterns Among Simultaneously Recorded Single Neurones. Journal of Neurophysiology , 60, 909–924. Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993a) Spatiotemporal Firing Patterns in the Frontal Cortex of Behaving Monkeys. Journal of Neurophysiology , 70, 1629–1638. Abeles, M., Prut, Y., Bergman, H., Vaadia, E., and Aertsen, A. (1993b) Integration, synchronicity and periodicity. In: A.Aertsen, (ed.) Brain theory: spatio-temporal aspects of brain function , pp. 149–181. Elsevier, Amsterdam. Abeles, M., Vaadia, E., Bergman, H., Prut, Y., Haalman, I., and Slovin, H. (1993c) Dynamics of neuronal interactions in the frontal cortex of behaving monkeys. Concepts in Neuroscience , 4, 131–158. Abeles. M., Prut, Y., Bergman, H., and Vaadia, E., (1994) Synchronization in neuronal transmission and its importance for information processing, Progress in Brain Research , 102, 395–404. Addison, P.S. (1997) Fractals and Chaos: An illustrated Course . Institute of Physics Publishing, Bristol. Adrian, E.D. (1912) On the conduction of subnormal disturbances in normal nerve. Journal of Physiology (London) , 45, 389–412. Adrian, E.D. (1934) The basis of sensation: the action of the sense organs . Christophers, London. Aertsen, A.M.H.J., Gerstein, G.L., Habib, M.K., and Palm, G. (1989) Dynamics of neuronal firing correlation: modulation of effective connectivity. Journal of Neurophysiology , 61 , 900–917.

Empirical evidence about temporal structure

55

Amit, D.J. (1995) The Hebbian paradigm reintegrated: local reverberations as internal representations. Behavioral Brain Sciences , 18, 617–657. Amit, D.J., Brunel, N., and Tsodyks, M. (1994) Correlations of cortical Hebbian reverberations: theory versus experiment. Journal of Neuroscience , 14 , 6435–6445. Amit, D.J. and Brunel, N., (1997) Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex , 7, 237–252. Amit, D.J., Fusi, S., and Yakovlev, V. (1997) Paradigmatic working memory (attractor) cell in IT cortex. Neural Computation , 9, 1071–1092. Andronov, A., and Pontryagin, L. (1937) Systèmes grossiers. Doklady Akademii Nauk. SSSR , 14 , 247–251. Babloyantz, A., Salazar, J.M., and Nicolis, C. (1985) Evidence of chaotic dynamics of brain activity during the sleep cycle. Physics letters , 111A , 152–155. Bair, W., and Koch, C. (1996) Temporal precision of spike trains in extrastriate cortex of the behaving Macaque monkey. Neural Computation , 8, 1184–1202. Bialek, W., Rieke, F., de Ruyter van Steveninck, R.R., and Warland, D.S. (1991) Reading a neural code. Science, New York , 252, 1854–1857. Bialek, W., and Rieke, F. (1992) Reliability and information transmission in spiking neurons. Trends in Neuroscience , 15 , 428–434. Braitenberg, V., and Schüz, A. (1991) Anatomy of the cortex: statistics and geometry . Springer Verlag, Berlin. Brillinger, D.R. (1974) Cross-spectral analysis of processes with stationary increments including the stationary G/G/8 queue. Annals of Probability , 2, 815–827. Brillinger, D.R. (1975) The identification of point process systems. Annals of Probability , 3, 909–924. Brillinger, D.R., and Villa, A.E.P. (1994) Examples of the investigation of neural information processing by point process analysis. In: V.Z.Marmarelis (ed.) Advanced Methods of Physiological System Modeling, volume 3 , pp. 111–127, Plenum Press, New York. Brillinger, D.R., and Villa, A.E.P. (1997) Assessing connections in networks of biological neurons. In: D.R.Brillinger, L.T.Fernholz, S.Morgenthaler (eds) The Practice of Data Analysis , pp. 77–92, Princeton University Press, Princeton, NJ. Brillinger, D.R., Bryant, H.L. Jr., and Segundo, J.P. (1976) Identification of synaptic interactions. Biological Cybernetics , 22, 213–228. Bullock, T.H. (1961) The problem of recognition in an analyzer made of neurons. In: W.A. Rosemblith (ed.) Sensory communication , pp. 717–724, Wiley, New York. Castro, R., and Sauer, T. (1997) Correlation dimension of attractors through interspike intervals. Physics Reviews , A55, 287–290. Celebrini, S., and Newsome, W.T. (1994) Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. Journal of Neuroscience , 14 , 4109–4124. Celletti, A., and Villa, A.E.P. (1996) Low dimensional chaotic attractors in the rat brain. Biological Cybernetics , 74, 387–394. Celletti, A., Bajo Lorenzana, V.M., and Villa, A.E.P. (1997) Correlation dimension for paired discrete time series. Journal of Statistical Physics , 89, 877–884. Celletti, A., Tetko, I.V., and Villa, A.E.P. (1998) Analysis of the deterministic behaviour of experimental series. Journal de Physique IV 8, 209–214. Celletti, A., Froeschlé, C., Tetko, I.V., and Villa, A.E.P (1999) Deterministic behaviour of short time series. Meccanica . 34 . 145–152.

Time and the brain

56

Cutler, C.D., and Kaplan, D.T. (1997) Nonlinear dynamics and time series: building a bridge between the natural and statistical sciences. Fields Institute communications, vol. 11 , pp. 252, American Mathematical Society, Providence, R.I. Dayhoff, J., and Gerstein, G.L. (1983a) Favored patterns in spike trains. I. Detection. Journal of Neurophysiology , 49, 1334–1348. Dayhoff, J., and Gerstein, G.L. (1983b) Favored patterns in spike trains. II. Application. Journal of Neurophysiology , 49, 1349–1363. de Ruyter van Steveninck, R.R., Lewen, G.D., Strong, S.P., Koberle, R., and Bialek, W. (1997) Reproducibility and variability in neural spike trains. Science, New York , 275, 1805–1808. deCharms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds by the coordination of action-potential timing. Nature, London , 381, 610–613. Douglas, R., and Martin, K. (1991) A functional micro-circuit for cat visual cortex. Journal of Physiology (London) , 440, 735–769. Eckmann, J.P., and Ruelle, D. (1992) Fundamental limitations for estimating dimensions and Lyapunov exponents in dynamical systems. Physica D , 56, 185–187. Edmonds, B., Gibb, A., and Colquhon, D. (1995) Mechanisms of activation of glutamate receptors and the time course of excitatory synaptic currents. Annual Review of Physiology , 57, 495–519. Eckhorn, R., Grusser, O.J., Kroller, J., Pellnitz, K., and Popel, B. (1976) Efficiency of different neuronal codes: information transfer calculations for three different neuronal systems. Biological Cybernetics , 22, 49–60. Ferster, D., and Spruston, N. (1995) Cracking the neural code. Science, New York , 270, 756–757. Fetz, E.E. (1997) Neuroscience—temporal coding in neural populations. Science, New York , 278, 1901–1902. Foote, S.L., and Morrison, J.H. (1987) Extrathalamic modulation of cortical function. Annual Review of Neuroscience , 10, 67–96. Friston, K.J. (1995) Neuronal transients. Proceedings of the Royal Society of London— Series B , 261, 401–405. Frostig, R.D., Frostig, Z., and Harper, R.M. (1984) Information trains. The technique and its uses in spike train and network analysis with examples taken from the nucleus parabrachialis medialis during sleep-waking states. Brain Research , 322, 67–74. Frostig, R.D., Frostig, Z., and Harper, R.M. (1990) Recurring discharge patterns in multiple spike trains. I. Detection. Biological Cybernetics 62, 487–493. Fuster, J.M. (1990) Inferotemporal units in selective visual attention and short-term memory. Journal of Neurophysiology , 64, 681–697. Georgopoulos, A., Taira, M.L., and Lukashin, A. (1993) Cognitive neurophysiology of the motor cortex. Science, New York , 260, 47–52. Georgopoulos, A.P. (1995) Current issues in directional motor control. Trends in Neuroscience , 18, 506–510. Gerstein, G.L., and Mandelbrot, B. (1964) Random walk models for the spike activity of a single neuron. Biophysical Journal , 4, 41–68. Grassberger, P., and Procaccia, I. (1983) Characterization of strange attractors. Physics Review Letters , 50, 346–349. Hénon, M. (1976) A two-dimensional mapping with a strange attractor, Communications in Mathematical Physics , 50, 69–77. Herrmann, M., Ruppin, E., and Usher, M. (1993) A neural model of the dynamic activation of memory. Biological Cybernetics , 68, 455–463.

Empirical evidence about temporal structure

57

Hill, S.L., and Villa, A.E.P. (1997) Dynamic transitions in global network activity influenced by the balance of excitation and inhibition, Network , 8, 165–184. Klemm, W.R., and Sherry, C.J. (1981) Serial ordering in spike trains: what’s it trying to tell us? International Journal of Neuroscience , 14, 15–23. König, P., Engel, A.K., and Singer, W. (1996) Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neuroscience , 19, 130–137. Legéndy, C.R. (1975) Three principles of brain function and structure. International Journal of Neuroscience , 6, 237–254. Levitan, I.B. (1988) Modulation of ion channels in neurons and other cells. Annual Reviews of Neuroscience , 11, 119–136. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997) Neural dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cerebral Cortex , 7, 207–727. MacGregor, R.J. (1987) Neural and brain modeling . Academic Press, San Diego. Mainen, Z.F., and Sejnowski, T.J. (1995) Reliability of spike timig in neocortical neurons. Science, New York , 268, 1503–1506. Markram, H., Lubke, J., Frotscher, M., and Sakmann, B. (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APS and EPSPS. Science, New York , 275, 213–215. Martignon, L., Von Hasseln, H., Grim, S., Aertsen, A., and Palm, G. (1995) Detecting higher-order interactions among the spiking events in a group of neurons. Biological Cybernetics , 73, 69–81. McCormick, D.A., and Bal, T. (1994) Sensory gating mechanisms of the thalamus. Current Opinion in Neurobiology , 4, 550–556. McCulloch, W.S., and Pitts, W. (1943) A logical calculus of the ideas imminent in nervous activity. Bulletin of Mathematical Biophysics , 5, 115–133. Miller, R. (1996) Cortico-thalamic interplay and the security of operation of neural assemblies and temporal chains in the cerebral cortex. Biological Cybernetics , 75, 263–275. Mountcastle, V.B., Poggio, G.F., and Werner, G. (1963) The relation of thalamic cell response to peripheral stimuli varied over an intensive continuum. Journal of Neurophysiology , 26, 807–834. Mpitsos, G.J., Burton, R.M., and Creech, H.C. (1988) Connectionist networks learn to transmit chaos. Brain Research Bulletin , 21, 539–546. Mpitsos, G.J. (1989) Chaos in brain function and the problem of nonstationarity: a commentary. In: E. Başar, T.H.Bullock (eds) Dynamics of sensory and cognitive processing by the brain , pp. 521–535, Springer-Verlag, New York, N.Y. Mpitsos, G.J., and Burton, R.M.Jr. (1992) Convergence and divergence in neural networks: processing of chaos and biological analogy. Neural Networks , 5, 605–625, Mukhametov, L.M., Rizzolatti, G., and Tradardi, V. (1970) Spontaneous activity of neurones of nucleus reticularis thalami in freely moving cats. Journal of Physiology (London) , 210, 651–667. Nafe, J.P. (1968) Neural correlates of sensation. In: D.R.Kenshalo (ed.) The Skin Senses pp. 5–14, Clarence Thomas, Springfield. Nicolelis, M.A.L., Ghazanfar, A.A., Faggin, B.M., Votaw, S., and Oliveira, L.M.O. (1997) Reconstructing the engram—simultaneous, multisite, many single neuron recordings. Neuron , 18, 529–537. Nieman, R.E., Fisher, D.G., and Seborg, D.E. (1971) A review of process identification and parameter estimation techniques. International Journal of Control , 13 , 209–264.

Time and the brain

58

Payne, B.R., Lomber, S.G., Villa, A.E.P., and Bullier, J. (1996) Reversible deactivation of cerebral network components. Trends in Neuroscience , 19 , 535–542. Peixoto, M. (1962) Structural stability on 2-dimensional manifolds. Topology, 1, 101– 120. Perkel, D.H., and Bullock, T.H. (1965) Neural coding. Neurosciences Research Program Bulletin , 6 , 221–348. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967a) Neuronal spike trains and stochastic point processes. I. The single spike train. Biophysical Journal , 7, 391–418. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967b) Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophysical Journal , 7, 419– 440. Perkel, D.H., Gerstein, G.L., Smith, M.S., and Tatton, W.G. (1975) Nerve-impulse patterns: a quantitative display technique for three neurons. Brain Research , 100 , 271–296. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. (1994) Numerical Recipes in C. (2nd ed.) Cambridge University Press, New York. Prut. Y., Vaadia, E., Bergman, H., Haalman, I., Slovin, H., and Abeles, M. (1998) Spatiotemporal Structure Of Cortical Activity—Properties and Behavioral Relevance. Journal of Neurophysiology , 79, 2857–2874. Rapp, P.E., Zimmerman, I.D., Albano, A.M., Deguzman, G.C., and Greenbaum, N.N. (1985) Dynamics of spontaneous neural activity in the simian motor cortex: the dimension of chaotic neurons. Physics Letters , 110A , 335–338. Rapp, P.E., Zimmerman, I.D., Vining, E.P., Cohen, N., Albano, A.M., and JimenezMontano, M.A. (1994) The algorithmic complexity of neural spike trains increases during focal seizures. Journal of Neuroscience , 14 , 4731–4739. Rapp, P.E., Albano, A.M., Schmah, T.I., and Farwell, L.A. (1993) Filtered noise can mimic low-dimensional chaotic attractors. Physical Review , E47, 2289–2297. Riehle, A., and Requin, J. (1995) Neuronal correlates of the specification of movement direction and force in four cortical areas of the monkey. Behavioral Brain Research , 70, 1–13. Riehle, A., Grün, S., Diesmann, M., and Aertsen, A. (1997) Spike synchronization and rate modulation differentially involved in motor cortical function. Science, New York , 278, 1950–1953. Rieke, F., Bodnar, D.A., and Bialek, W. (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proceedings of the Royal Society of London—Series B , 262, 259–265. Rosenberg, J.R., Amjad, A.M., Breeze, P., Brillinger, D.R., and Halliday, D.M. (1989) The Fourier approach to the identification of functional coupling between neuronal spike trains. Progress in Biophysics and Molecular Biology , 53, 1–31. Segundo, J.P. (1986) What can neurons do to serve as integrating devices? Journal of Theoretical Neurobiology , 5, 1–59. Segundo, J.P., Moore, G.P., Stensaas, L., and Bullock, T.H. (1963) Sensitivity of neurons in Aplysia to temporal patterns of arriving impulses. Journal of Experimental Biology , 40, 643–667. Segundo, J.P., Perkel, D.H., and Moore, G.P. (1966) Spike probability in neurones: influence of temporal structure in the train of synaptic events. Kybernetik , 3, 67–82. Segundo, J.P., and Perkel, D.H. (1969) The nerve cell as an analyzer of spike trains. UCLA Forum in Medical Sciences , 11, 349–390. Shadlen, M.N., and Newsome, W.T. (1994) Noise, Neural Codes and Cortical

Empirical evidence about temporal structure

59

Organisation. Current Opinion in Neurobiology , 4, 569–579. Shadlen, M.N., and Newsome, W.T. (1998) The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. Journal of Neuro science , 18, 3870–3896. Shinba, T., Sumi, M., Iwanami, A., Ozawa, N., and Yamamoto, K. (1995) Increased neuronal firing in the rat auditory cortex associated with preparatory set. Brain Research Bulletin , 37, 199–204. Skaggs, W.E., and McNaughton, B.L. (1996) Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science, New York , 271, 1870–1873. Smale, S. (1967) Differential dynamical systems. Bulletin of the American Mathematical Society , 73, 747–817. Softky, W., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random epsps. Journal of Neuroscience , 13, 334–350. Steriade, M., Jones, E.G., and Llinás, R.R. (1990) Thalamic oscillations and signalling , Wiley & Sons, New York, NY. Stuart, G.J., and Sakmann, B. (1994) Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature, London , 367, 69–72. Tang, A.C., Bartels, A.M., and Sejnowski, T.J. (1997) Effects of cholinergic modulation on responses of neocortical neurons to fluctuating input. Cerebral Cortex , 7, 502–509. Tetko, I.V., and Villa, A.E.P. (1997a) Fast combinatorial methods to estimate the probability of complex temporal patterns of spikes. Biological Cybernetics , 76, 397– 407. Tetko, I.V., and Villa, A.E.P. (1997b) Efficient partition of learning datasets for neural network training. Neural Networks , 10, 1361–1374. Tetko, I.V., and Villa, A.E.P. (1997c) A comparative study of pattern detection algorithm and dynamical system approach using simulated spike trains. Lecture Notes in Computer Science , 1327, 37–42. Tetko, I.V., and Villa, A.E.P. (1999) Temporal interactions in multiple spike trains detected by pattern grouping algorithm. Journal of Neurophysiology (submitted). Theiler, J., and Rapp, P.E. (1996) Re-examination of the evidence for low-dimensional, nonlinear structure in the human electroencephalogram. Electroencephalography and Clinical Neurophysiology , 98, 213–222. Thorn, R. (1975) Structural stability and morphogenesis (English translation). W.A. Benjamin, Reading, MA. Thomson, A., and Deuchars, J. (1994) Temporal and spatial properties of local circuits in neocortex. Trends in Neuroscience , 17, 119–126. Thorpe, S., Fize, D., and Marlot, C. (1996) Speed of processing in the human visual system. Nature, London , 381, 520–522. Tsodyks, M.V., and Sejnowski, T. (1995) Rapid state switching in balanced cortical network models. Network , 6, 111–124. Tsodyks, M.V., and Markram, H. (1997) The neural code between neocortcal pyramid neurones depends on neurotransmitter release probability. Proceedings of the National Academy of Sciences, U.S.A. , 94, 719–723. Tsukada, M., Terasawa, M., and Hauske, G. (1982) Temporal pattern sensitive and nonsensitive responses in the cat’s retinal ganglion cells. Biological Cybernetics , 44, 197–203. Usher, M., and Stemmler, M. (1995) Dynamic pattern formation leads to 1/f noise in

Time and the brain

60

neural populations. Physical Review Letters 74, 326–329. Vaadia, E., Bergman, H., and Abeles, M. (1989) Neuronal activities related to higher brain functions—theoretical and experimental implications. IEEE Transaction in Biomedical Engineering , 36, 25–35. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London , 373, 515–518. Villa, A.E.P. (1988) Influence de I'écorce cérébrale sur I’activité spontanée et evoquée du thalamus auditifdu chat . Imprivite, Lausanne. Villa, A.E.P. (1990) Functional differentiation within the auditory part of the reticular nucleus of the cat. Brain Research Reviews . 15. 25–40. Villa, A.E.P. (1992) Les catastrophes cachées du cerveau. Le Nouveau Golem , 1, 33–63. Villa, A.E.P., and Abeles, M. (1990) Evidence for spatio-temporal firing patterns within the auditory thalamus of the cat. Brain Research , 509, 325–327. Villa, A.E.P., Rouiller, E.M., Simm, G.M., Zurita, P., de Ribaupierre, Y., and de Ribaupierre, F. (1991) Corticofugal modulation of information processing in the auditory thalamus of the cat. Experimental Brain Research , 86 , 506–517. Villa, A.E.P., and Fuster, J.M. (1992) Temporal correlates of information processing during visual short-term memory. NeuroReport , 3, 113–116. Villa, A.E.P., and Tetko, I.V. (1995) Spatio-temporal patterns of activity controlled by system parameters in a simulated thalamo-cortical neural network. In: H.Hermann, (ed.) Supercomputing in Brain Research: from tomography to neural networks , pp. 379–388, World Scientific Publishing, Singapore. Villa, A.E.P., Tetko, I.V., Riehle, A., and Requin, J. (1995) Selective spatio-temporal firing patterns in the motor cortex during a reaction-time task. Society for Neuroscience Abstracts , 21 , 516. Villa, A.E.P., Bajo, V.M., and Vantini, G. (1996) Nerve Growth Factor (NGF) modulates information processing in the auditory thalamus. Brain Research Bulletin , 39, 139– 147. Villa, A.E.P., Hyland, B., Tetko, I.V., and Najem, A. (1998a) Dynamical cell assemblies in the rat auditory cortex in a reaction-time task. Biosystems , 48, 269–278. Villa, A.E.P., Tetko, I.V., Celletti, A., and Riehle, A. (1998b) Chaotic dynamics in the primate motor cortex depend on motor preparation in a reaction-time task. Current Psychology of Cognition . 17, 763–780. Villa, A.E.P., Tetko, I.V., Hyland, B., and Najem, A. (1999a) Significance of spatiotemporal activity patterns among rat cortex neurons in performance of a conditioned task. Proceedings of the National Academy of Sciences, U.S.A. , 96, 1006– 1011. Villa, A.E.P., Tetko, I.V., Dutoit, P., de Ribaupierre, Y., and de Ribaupierre, F. (1999b) Corticofugal modulation of functional connectivity within the auditory thalamus of rat, guinea pig and cat revealed by cooling deactivation. Journal of Neuroscience Methods , 86, 161–178. Villa, A.E.P., Eriksson, J., Eriksson, C., Haeberli, C, Hyland, B., and Najem, A. (199c) Stimulus congruence affects perceptual processes in a novel Go/Nogo conflict paradigm in rats. Behavioural Processes , 48, 69–88. Villa, A.E.P., and Tetko, I.V. (1999) Spatiotemporal Activity Patterns Detected from Single Cell Measurements from Behaving Animals, Proceedings SPIE (International Society for Optical Engineering) , 3728, 20–34. Wilson, M.A., and McNaughton, B.L. (1993) Dynamics of the Hippocampal Ensemble

Empirical evidence about temporal structure

61

Code for Space. Science, New York , 261 , 1055–1058. Wilson, H.R., and Cowan, J.D. (1974) Excitatory and inhibitory interactions of localized populations of model neurons. Biophysical Journal , 12 , 1–24. Wise, S.P., and Kurata, K. (1989) Set-related activity in the premotor cortex of rhesus monkeys: effect of triggering cues and relatively long delay intervals. Somatosensory and Motor Research , 6, 455–476. Xiang, Z., Huguenard, J.R., and Prince, D.A. (1998) Cholinergic switching within neocortical inhibitory networks. Science, New York 281, 985–988. Xing, J., and Gerstein, G.L. (1996) Networks with lateral connectivity: I. Dynamic properties mediated by the balance of intrinsic excitation and inhibition. Journal of Neurophysiology , 75, 184–199. Zurita, P., Villa, A.E.P., de Ribaupierre, Y., de Ribaupierre, F., and Rouiller, E.M. (1994) Changes of single unit activities in the cat’s auditory thalamus and cortex associated to different anesthetic conditions. Neuroscience Research . 19 , 303–316.

2 Cross-Correlograms for Neuronal Spike Trains. Different Types of Temporal Correlation in Neocortex, their Origin and Significance Lionel G. Nowak 1 and Jean Bullier 2 1. Section of Neurobiology, Yale University School of Medicine, C303 Sterling Hall of Medicine, 333 Cedar Street, New Haven, CT06510, USA. e-mail: [email protected] 2. Jean Bullier: Centre de Recherche Cerveau et Cognition, CNRS-UPS UMR 5549, 133, route de Narbonne, 31062 Toulouse Cedex email: [email protected]

After a presentation of the method of computation of the cross-correlation histogram, the major interpretations of cortical synchronization are reviewed. These include indicators of different brain states (sleep, waking state), indication of cortical connectivity, participation in cell assemblies, and a different form of neural coding. It is then argued that there are several types of synchronization, reflected in the existence of CCH peaks of different widths. The function of these different forms of synchronization in cortical processing and their possible underlying generation mechanisms are discussed. KEYWORDS: Synchronization; Cell assembly; Binding; Cortex; Vision; Crosscorrelogram Synchronization of neuronal activity is a topic that has recently received a lot of interest. The major reason for this is certainly related to the publication of experimental results that supported the idea that the relative timing of action potentials between different neurones could play an essential role in perception. The enthusiasm raised by these experiments has not been shared by the whole community, and a number of criticisms have been made. Scientific controversies are often related to a misunderstanding of certain definitions. In our case, the essential question is: what does synchronization really mean and imply? This chapter is an attempt to examine synchronization in depth. First, we recall the basic principles of the cross-correlation technique, as it is the method used most frequently for tracking synchrony at the level of action potentials. Next, we present a historical perspective on synchronization, which leads to the realization that the meaning of synchronization has varied a lot at different periods. In a third section we show that one should not talk about synchronization as a

Time and the brain

64

single factor, but about several different types of synchronization. Having acknowledged this basic fact, we attempt to review the possible mechanisms that lead to the generation of these different forms of synchronization. Finally, we review recent results and the current status on the functional roles of synchronization.

1. THE CROSS-CORRELATION HISTOGRAM AS A TOOL The technique that has been used most commonly to reveal the temporal coherence in the firing of cortical neurones is to compute a cross-correlation histogram (abbreviated as CCH, also known as cross-correlogram) from their spike trains. Although the crosscorrelation technique had been used before, Perkel et al. (1967b) were the first to present formally its computation and interpretation: consider two cells that are recorded extracellularly (A and B in Figure 1). The CCH computed from the spikes emitted by neurones A and B is an estimate of the probability of firing of neurone B as a function of the time before or since a spike was fired by neurone A. This estimate is calculated by placing each spike of neurone B in a given time bin depending on its difference in time of emission from that of a spike in neurone A. It is important to note that in a CCH, a spike from neurone B usually appears in several bins, since the intervals between this spike and all the spikes of neurone A are taken into account. However, each interval between a spike in neurone A and a spike in neurone B is counted only once. Thus, a CCH is best described as a histogram of intervals between spikes-times in two neurones. Different patterns of connectivity can give rise to different forms of temporal coherence, and hence to different shapes in a CCH. We schematically illustrate these in Figure 1, considering the case of 2 neurones recorded during spontaneous activity.

Cross-correlograms for neuronal spike trains

65

Figure 1 . Typical shapes of cross-correlation histograms (CCH). Neurones A and B produce the spike trains 1 and 2. The CCH gives an estimate of the probability of a spike in neurone B at time t before or after a spike in neurone A. Uncorrelated neurones give a flat CCH. A single sharp displaced peak is the signature of a monosynaptic drive. A peak centered on the origin corresponds to synchronous firing that is usually interpreted as resulting from a common drive.

If they share no direct or indirect link, the CCH computed from the activities of the two cells (A and B in Figure 1) appears flat. This is interpreted as evidence that the cells fire independently (Figure 1a). A monosynaptic excitatory connection between 2 neurones appears in the CCH as a peak displaced from the origin of time by a latency corresponding to the sum of the conduction time in the axon and the synaptic delay (Figure 1b). In the vast majority of brain structures, not all spikes in cell A trigger a spike in cell B; conversely, not all spikes in B are triggered by a spike in cell A. As a consequence, the CCH peak emerges from a baseline probability level corresponding to those spikes in B that were not related to the spikes in A. In other words, those spikes that occurred independently in the two cells. The strength of the connection can then be estimated by different means, for example as the percentage of spikes in the presynaptic neurone that are related to spikes in the postsynaptic neurone (the “effectiveness”; Levick et al., 1972); as the percentage of spikes in the postsynaptic neurone that are related to spikes in the presynaptic neurone (the “contribution”; Levick et al., 1972); as the peak area normalized by the area under the peak (the “mean percent increase”, or “MPI”; Cope et al., 1987); as the peak height normalized by the CCH baseline (the “RMA”; Engel et al., 1990); as the number of

Time and the brain

66

spikes contained in the peak (the “asynchronous gain”; Abeles, 1991); etc. A monosynaptic inhibitory connection produces a dip in the CCH, provided the inhibitory postsynaptic potential (IPSP) is strong enough to reduce the firing probability in neurone B and that there is enough activity to be reduced by the IPSP (Aersten and Gerstein, 1985; Melssen and Epping, 1987). If the two cells are driven at the same time by a common presynaptic neurone (assuming homogeneous axonal conduction times and synaptic delays), they would tend to fire at the same time, which, in the CCH, results in a peak centered on the time origin (Figure 1b). The two cells in this case are said to synchronize their activities. Finally, a broad peak should appear in the CCH if the two neurones are simultaneously inhibited by a third neurone. This might appear counter-intuitive at first sight, but can be explained by the following reasoning: On average, the common inhibition tends to decrease their firing at the same time. When inhibition is released, the 2 cells increase their firing rate at the same time, hence the broad peak centered on the time origin. Sensory stimuli are often used to activate neurones, especially in regions of the cortex where spontaneous activity is too low to accumulate enough spikes to compute CCHs in a reasonable time. However, the use of a stimulus introduces a complication. In that case, the presence of a peak in the CCH can have two different origins: one corresponds to the stimulus-locked temporal coherence. It reflects the fact that the two neurones under study are activated with a constant temporal relationship by the sensory stimulation. This is exemplified in Figure 2 for the case of two neurones recorded, one in area V1 and the other in area V2 of the macaque visual cortex. The spike trains are shown schematically in Figure 2a. Figure 2b shows the peri-stimulus time histograms of the responses of the two cells to the onset and the offset of a flashing stimulus. The stimulus induced an increase in firing rate in both cells, at roughly the same time, and for a comparable duration. This coincident increase in firing rate contributes to the peak in the CCH (Figure 2c1, raw CCH). The delay and the size of the peak corresponding to the stimuluslocked temporal coherence depend on the

Cross-correlograms for neuronal spike trains

67

Figure 2 . Examples ofCCH and methods of the shift predictor. Recordings from units in areas V1 and V2 of the macaque monkey. A: examples of spike trains, as recorded (above) and after shifting the V2 spikes by one stimulus period (below). B: Post-stimulus histograms (PSTHS) of the responses of the two neurones to a small stimulus flashed in the receptive fields. The light is ON at time 0 and OFF at 3 s. C1: CCH computed from the normal spike trains (raw CCH) and after shifting the V2 spike train (Shift predictor), for the parts of the spikes trains corresponding to visual responses in the CCH. The deviation of the raw CCH from the Shift predictor indicates the presence of neural coupling. C2: Subtracted CCH and side peak. The subtracted CCH is the difference between the raw CCH and the shift predictor. The side peak is obtained by subtracting two shift predictors, one shifted by one and one shifted by two stimulus periods. It gives an indication of the amount of variability of the CCH trace. D1: same calculation as for C1 except that the spikes come from the period with no stimulus. D2: same as C2 for spontaneous activity (no stimulus). Note that a broad peak (H type) is observed in spontaneous activity whereas an intermediate peak (C type) is observed during visual stimulation.

latency difference in the activation of the neurones by the stimulus, as well as the duration of their responses. The other form of temporal coherence observed with sensory stimuli is the temporal

Time and the brain

68

coherence of neuronal origin. This form of coherence results from any of the neuronal interactions described above (Figure 1). To distinguish between the 2 forms of temporal coherence it is common to use a shift (or shuffle) predictor method. This method was first presented by Perkel et al. (1967b): a sensory stimulus, if repeated with a fixed periodicity, induces a periodic increase in firing rate (Figure 2a). The effect of the sensory stimulus, therefore, can be predicted, in isolation, by computing another CCH (called the shift predictor) after shifting one spike train with respect to the other by an amount of time corresponding to one (or several) periods of the stimulus (Figure 2a). Usually, the periodicity of the stimulus is several seconds. The temporal coherence of neuronal origin, on the other hand, exists only for a limited amount of time, most commonly below 10 msec in the case of fast synaptic transmission, and generally below 1 second in the case of slower (metabotropic) interactions. Therefore, by comparing the original CCH (the raw CCH) with its shift predictor it is possible to determine what is produced by the stimulation and what is due to the neuronal interactions (Figure 2c1. The subtracted CCH, corresponding to the subtraction of the shift predictor from the raw CCH, is used to characterize the temporal coherence of neuronal origin in isolation (Figure 2c2). Experimental studies were then designed to test the prediction from the theoretical study by Perkel et al. (1967b). Moore et al (1970); and Bryant et al. (1973) applied the cross-correlation technique to the study of the functional connectivity in Aplysia. The peaks observed in the CCH could be ascribed to different forms of synaptic interactions, which were directly inferred by virtue of dual intracellular recording for the same cell pairs. They showed that, indeed, a monosynaptic excitatory connection produces a displaced peak, and that a common input produces a peak centered on the origin of time. Modeling studies (Knox, 1974), and studies in the spinal cord (Kirkwood and Sears, 1978, 1982b; Fetz and Gustafsson, 1983; Gustafsson and McCrea, 1984; Cope et al., 1987) and recently in the cortex (Matsumura et al., 1996), attempted to establish a relationship between the shape of an excitatory postsynaptic potential (EPSP) and the shape of the CCH. The shape of the CCH does not reproduce the shape of the underlying EPSP. Instead, it is usually found that the shape of the peak in the CCH is similar to the temporal derivative of the EPSP; in other words, the width of the CCH peak is proportional to the rise time of the EPSP. Departure from this relationship is observed in cases of EPSPs of small size compared to the fluctuations of the membrane potential. In such cases the shape of the CCH peak can be reproduced by a linear combination of the EPSP shape and its derivative. Moore et al. (1970) and Bryant et al. (1973) showed further that the shape of the CCH peak is determined not only by the connectivity pattern, but also by the firing pattern of the neurones under study. This pattern is revealed by computing the autocorrelogram (ACH) of the neurone activity (cf. Perkel et al., 1967a). It is done in the same way as the CCH except that the ACH is a histogram of all the intervals (with or without intervening spikes) between the spikes recorded in one neurone. Moore et al. (1970) and Bryant et al. (1973) distinguished between the primary effects, which result from the synaptic interactions, and the secondary effects, which result from the firing pattern of the neurones under study. The ACH can show that some neurones fire action potentials that are not independent of each other. For example a neurone might be bursty, in which case

Cross-correlograms for neuronal spike trains

69

one action potential will usually be followed by the other action potentials of the burst. If a postsynaptic neurone shows a bursty firing, then the CCH will present a broader peak, because a spike in neurone A is followed on average by a burst in neurone B. Another case corresponds to an oscillatory common input neurone: the oscillatory firing will be replicated symmetrically in the CCH, leading to an oscillatory structure. Although the impact of the firing pattern of the cells was recognized in these early papers, many crosscorrelation studies have neglected to take it into account when interpreting the various shapes and widths shown by the peaks in the CCHs. We return to this question in Section 4.

2. SYNCHRONIZATION AND ITS MEANINGS: AN HISTORICAL PERSPECTIVE The cross-correlation technique has been used to characterize the neuronal interactions that take place within the central nervous system. In the cerebral cortex, it has shown that neurones tend to synchronize their activity in the majority of cases. However, this basic result has been open to multiple interpretations. To show how the interpretation of synchronization has evolved in parallel to the conceptual field within which it has been studied, we now trace the history of synchronization in the cortex. 2.1. Awakening and Desynchronization Synchronization of neuronal firing was first associated with sleep and anaesthesia, and was supposed to disappear when subjects or animals were awake. This notion can be traced back to the first studies on electroencephalographic activity, initiated by Berger, in which rest, sleep and anaesthesia were found to be associated with EEG traces showing large-amplitude slow waves, that appeared coherent over large distances. In contrast, awakening was associated with lower amplitude events occurring at a higher frequency, and usually independently at the different recording sites (e.g. Adrian and Matthews, 1934). This led to the hypothesis that the underlying neuronal activity should be synchronized in anaesthesia and sleep, while sensory stimulation and awakening would disrupt and suppress this synchronization. Adrian (1935), for example, stated in 1935 that “[…] if a large mass of nerve-cells is to pulsate with a co-ordinated beat it is clear that all of them must tend to pulsate at the same frequency. If afferent messages excite some to a higher frequency the co-ordinated beat must break down; and so these cortical rhythms are broken up by external stimulation, to return again when the animal is undisturbed”. Or, said in another way by the same author, “[…] groups of nerve-cells very often tend to act in unison when there is nothing to prevent them”. Hence, the first studies in which dual extracellular recording of action potentials was performed in cortex were designed to test the hypothesis that neurones synchronize their

Time and the brain

70

activity during anaesthesia and sleep, while sensory stimulation and awakening suppress this synchrony. Li (1959), recording from pairs of neurones in the somatosensory cortex of cats anaesthetized with a barbiturate, found that spontaneous activity is made up of simultaneous bursts of 5 to 10 action potentials when the animal is deeply anaesthetized. The tendency of neurones to discharge simultaneously tended to decrease as the anaesthesia level was reduced. Holmes and Houchin (1966) conducted a study in urethane anaesthetized rats, and constructed the equivalent of a CCH to detect synchronization of neuronal activity. They found it to be more prevalent in deeply anaesthetized animals, and observed it even for cells that were separated by several millimeters on the cortical surface. Synchronization was reduced, or suppressed, when the anaesthesia was lighter. They also used “strong peripheral stimuli” (which presumably were more like awakening stimuli than subtle sensory stimuli), and found that these also decreased the synchronization. Noda and Adey (1970) recorded the ongoing activity in the suprasylvian gyrus of unanaesthetized, behaving cats. They found that pairs of neurones strongly synchronize their activity during sleep, while synchronization, for the same pairs of neurones, was strongly reduced, if not completely absent, during arousal and rapid eye movement sleep. Noda and Adey (1973) completed their study by testing the action of the anaesthetic pentobarbital on neuronal activity and synchronization. The anaesthetic increased the synchrony and the burstiness of the neurones to a level comparable to that observed during sleep. With this anaesthetic, the bursts of action potentials occurred mostly in coincidence with the spindle waves appearing in the EEG. Burns and Webb (1979) added another argument in favour of a desynchronizing role for afferent sensory stimuli: They managed to deafferent pieces of cortex and to record from them. The neurones in these deafferented cortices synchronized their activities in all the cases, independently of the behavioural state of the cat, while the non-deafferented cortex only rarely showed signs of synchronization, especially when the animal was awake. As a conclusion for this section we quote Burns and Webb (1979): “[…] it is not unreasonable to state that when the brain is processing information, neighbouring cortical neurones are very unlikely to be found doing the same thing at the same time.” Hence experimental results supported the hypothesis of Adrian that neuronal activity is strongly synchronized during sleep and anaesthesia, while synchronization would be absent or at best strongly reduced in the awake brain. However, these studies can be criticized on one point, in that, in most cases, the neurones were studied during spontaneous activity; the effects of the sensory stimulation, if studied, were only poorly controlled. 2.2. Neuronal Connectivity and Common Inputs Within a different conceptual framework, inherited from the study by Perkel et al. (1967b), cross-correlation has been used as a method to provide data on the connectivity between neurones. In this context, what was studied was not the relationship between synchronization of neuronal firing and the behavioural state of the animal; instead, crosscorrelation was used as a tool to test for the presence of synaptic relationships between extracellularly recorded neurones.

Cross-correlograms for neuronal spike trains

71

Compared to other methods, calculating an index of cross-correlation between the firing of two neurones would appear as a better alternative to study their connectivity: the tracing of anatomical pathways says nothing about the physiological properties of the labelled neurones or their target cells. Electrical stimulation, as a method of identifying neurones that are targeted by a given structure, has a number of drawbacks, including the possible activation of axonal tracts issuing from other structures and passing near the stimulating electrode, or the problem of the activation of axon collaterals (e.g. Nowak and Bullier, 1998a,b). One powerful method consists of performing dual intracellular recordings, allowing the physiological and morphological identification of both the preand postsynaptic neurones involved in a synaptic connection. The use of dual intracellular recording in vitro has provided a number of important data (for recent reviews see Thomson and Deuchars, 1997; Markram, 1997), but its low yield and its requirement for long lasting, stable recordings, makes it impossible to use for in vivo mammalian studies. In the mammalian visual system, CCHs displaying narrow peaks offset from the time origin, indicating monosynaptic connections, have been observed in the subcortical visual pathways. Cleland et al. (1971a,b) and Levick et al. (1972) traced the functional connectivity between the retina and the lateral geniculate nucleus (LGN). Among other things, their results showed that a retinal ganglion cell projects on a thalamic neurone with a similar receptive field subtype. They also showed that a small number of retinal ganglion cells, sometimes only one, provides most of the excitatory drive to a thalamic neurone. After these investigations, the study of the thalamocortical pathway followed, with the studies by Tanaka (1983), who showed that ON-centre thalamic neurones provide the main drive to the ON subfield of cortical simple cells. The corresponding CCH peaks, a few msec wide, were displaced from the time origin by a latency 0.9 to 2.7 msec. Tanaka (1983) also showed that, on average, one spike in a thalamic neurone increases the discharge probability of a cortical simple cell by 10%. These results were recently confirmed by Reid and Alonso (1995), although the strength of the thalamic input appeared lower in their study, presumably due to the use of different visual stimuli (see also Swadlow, 1995, for the case of the somatosensory thalamus and cortex). The cross-correlation technique has also been used to study synaptic connectivity between cortical neurones recorded within the same area. After a pioneering work by Griffith and Horn (1963), the first cortical studies were those by Dickson and Gerstein (1974) in the auditory cortex, by Renaud and Kelly (1974) in the motor cortex, by Toyama et al. (1981a, b) and Michalski et al. (1983) in the visual cortex. The CCHs showed significant peaks, indicating temporal coherence of neuronal origin, in roughly half the cases when the neurones were close to each other (e.g. recorded from the same electrode). The incidence of significant CCHs decreased with increasing separation between the electrodes. In contrast to what was observed for the retinothalamic and thalamocortical pathways, the significant CCHs rarely displayed features indicating the presence of a monosynaptic connection: Displaced peaks and displaced troughs were observed in only 5–10% of the cases, and only when the neurones were located close to each other. Instead, most of the CCH peaks were centred on the time origin when neurones were close to each other, and only centred peaks were

Time and the brain

72

observed when they were distant. In other words, the activity of two simultaneously recorded cortical neurones appeared synchronized in the vast majority of the cases even when sensory stimuli were used to activate them. It was then inferred that such synchronized activity reflects the presence of common inputs (Figure 1), from other cortical areas or from subcortical structures. Gerstein and Dickson (1974) concluded that the connectivity between cortical neurones is made by an “arrangement that consists mostly of shared inputs.” 2.3. Functional Connectivity, Synchronization and Cell Assemblies Although cross-correlation was once thought to be a method for tracing connections in vivo, this hope had to be abandoned. The modelling study by Boven and Aertsen (1990) illustrates how a simple monosynaptic connection between two neurones can give rise to different CCHs, depending on the activity of the other neurones in the network. In their model, a neurone 1 projects to a neurone 2, with a relatively small synaptic strength α (Figure 3). For its firing to be influenced by that of neurone 1, it is necessary for neurone 2 to have its membrane potential close enough to the spike threshold, such that the relatively small synaptic events produced by neurone 1 would lead to an action potential. The membrane potential of neurone 2 is determined by the activity in the population of neurones (N) projecting onto it. If the activity in this population of neurones is low, then the membrane potential of neurone 2 would not be depolarized enough to allow the EPSPs produced by neurone 1 to trigger an action potential. As the activity in the population N increases, the probability (called effective connectivity—α’ in Figure 3) that an EPSP produced by neurone 1 leads to an action potential in neurone 2 increases, until this probability saturates. From this we can also extrapolate that if the activity in the population of neurones (N) is high, but out of phase with respect to that of neurone 1, then the signature of the monosynaptic connection will not be seen in the CCH. Furthermore, the size of the peak cannot be interpreted as an estimate of the synaptic strength α’, since it reflects the combined effects of the connection between neurones 1 and 2 and the activity of the network in which they are embedded. At this point, “one should distinguish between structural (or anatomical) connectivity on one hand and functional (or effective) connectivity on the other” (Aertsen and Preiβ1, 1990). The lack of a peak is not proof of a lack of connectivity. Conversely, the presence of a peak in a CCH computed between two neurones indicates not only the presence of a synaptic connection. It also means that the whole cortical network was in a state that allowed the expression of this connection… The appearance and disappearance of centred peaks can be envisaged simply in a situation where one common input neurone, C, projects to two neurones A and B (cf. Figure 1c). Neurones A and B may also receive a number of inputs they do not share, such that their activity with different stimuli might be of comparable strength. Neurone C could be activated by only one stimulus configuration, and only in this case

Cross-correlograms for neuronal spike trains

73

Figure 3 . Relationship between strength of effective connectivity α’ as a function of network activity Nl for different values of the synaptic weight β between the pool of neurones N and neurone 2. Neurones 1 and 2 are related by a monosynaptic connection of synaptic weight a. Nλ is the mean firing rate of the neurones in the pool N driven by the stimulus S. α’ is the ratio of the peak area in the CCH to the number of presynaptic spikes (equivalent to the “contribution” index of Levick et al., 1972). In all cases there is a strong dependence of the effective connectivity upon the activity of the network and a saturation for high activity levels. Note that the variation of the effective connectivity is much larger for small values of the network activity, which is likely to be similar to the case in the cortex, from Boveu and Aertoe (1990)

would a centred peak be visible in the CCH. The anatomical wiring is always present, but may not be expressed in all cases. Obviously the situation in the cortex is far more complex, where one neurone receives several thousands of synapses from neurones located in different cortical and subcortical structures, all with different stimulus selectivities. In support of these theoretical considerations, several experimental studies have shown that the temporal coherence of neuronal origin observed in the spike trains from the same neurone pair depends on the stimulus used to activate them, as well as on the behavioural

Time and the brain

74

context. In the auditory cortex, Frostig et al. (1983) found that 88% of the CCHs were modified by the stimulus presented: some CCHs, showing displaced peaks during spontaneous activity, became flat in the presence of a stimulus while other CCHs behaved in the opposite way. Later studies in auditory cortex confirmed this strong stimulus dependency (Vaadia and Abeles, 1987; Ahissar et al., 1992a; Eggermont, 1994; de Charms and Merzenich, 1996). Changes in correlated firing under different stimulus conditions have also been observed in subcortical structures (e.g. Epping and Eggermont, 1987; Sillito

Figure 4 . Oscillatory coupling between area 17 neurones of opposite cortical hemispheres. Decomposition of oscillatory correlation over peristimulus time. Right and left hemisphere multiunit signals are

Cross-correlograms for neuronal spike trains

75

correlated in response to a bar moving for 3 s, stationary for 5 s and moving back for 3 s. The receptive fields were partially overlapping and had optimal orientations differing by 45 degrees. A: The net peri-stimulus CCH (PSCCH) shows oscillations relative to time difference, with the position of the peak shifting slightly during the movement of the bar, and shifting in the other direction when the bar returns. Lowest panel shows PSTHs for right hemisphere (shaded) and left hemisphere (outlined). Note the high level of spontaneous activity between stimuli. Middle panel shows sections along the PST dimension for time difference windows of −3 to 3 msec (shaded) and 10 to 16 msec (outlined). Right hand panel shows CCHs obtained by summing over PST dimension. B: Array of equivalent Z-values indicating the statistical significance of the coupling. The middle panel shows profiles of the significance along the PST dimension, for time differences at the central peak (shaded) and at a negative lobe (outlined), along with the 2.5% one-sided significance level (dotted) for positive coupling (Z=1.96) and negative coupling (Z=−1.96). Coupling is highly significant during movement of the bar (Z=+/−5), while dropping below significance between movements. The panel to the right shows sections along the time-difference axis of mean Z value for PST in windows 10.3 to 11.3 seconds (shaded), and in the interstimulus period, for PST from 5 to 9 seconds (outlined) along with the one-sided 2.5% significance leves (z=+/−1.96; dotted line). The windows for the horizontal and vertical sections are indicated by brackets on their respective axes. Several lobes of the oscillatory profile during stimulation are significant, while the interstimulus profile is well below significance, in spite of high levels of spontaneous activity, from Nowak et al (1995)

et al, 1994). In frontal and prefrontal cortex, the correlated firing between pairs of neurones can also change in relation to behaviour, with a time course of a fraction of a second (Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995; Riehl et al., 1997). In the visual cortex, the first description of a change in synchronization related to a change of the stimulus-configuration can be found in the paper by Ts’o et al. (1986). They showed an example, where the CCH peak height decreased when the neurones under study were activated with 2 bars with optimal orientation, rather than with one single bar with an orientation in between the two optima. In the visual cortex, stimulus-dependent changes of synchronized activity have been investigated mostly for one form of synchronization, the synchronization of gamma range (between 20 and 80 Hz) oscillation initially described by Gray and Singer (1989) and Eckhorn et al. (1988). One example is shown in Figure 4, for the case of interhemispheric synchronization in cat. These synchronized oscillations are observed under stimulation by a moving stimulus, but not during spontaneous activity (Figure 4). Most importantly, they depend on the configuration of the stimuli in such a way that they are more prominent for one continuous stimulus activating groups of cells together and coherently, than for different stimuli activating the different neuronal populations separately in space and/or

Time and the brain

76

time (Gray et al., 1989; Engel etal., 1991c). In summary, these results show that the temporal coherence in the firing of two neurones is not invariant; instead it depends on the presence of one type of stimulus, and/or on one phase of the behaviour in which the animal is engaged. Therefore crosscorrelation is not simply an anatomical method. However, cross-correlation of spike trains can be used to show that, at a particular moment and in presence of a given stimulus, one cell can be linked to a larger group of cells, and that this link might disappear in other conditions. The acknowledgement of the specificity of this link brought the cross-correlation technique into another conceptual field, that of the cell assembly. The initial concept of the cell assembly was proposed by Hebb (1949) as a link between neuronal activity and behaviour (for other types of cell assemblies see also Gerstein et al., 1989). Hebb, as well as most contemporary neuroscientists, did not envisage a one-to-one relationship between the activity of one neurone and one perception or one behaviour. Instead perception and motor activity had to be based on the activity of pools of neurones (or “population coding”). The novelty of Hebb’s theory resided in the involvement of the connectivity between neurones, and most importantly in the functional expression of this connectivity, as the neuronal basis of perception and behaviour. He defined the cell assembly as: “a diffuse structure comprising cells in the cortex and diencephalon (and also, perhaps, in the basal ganglia of the cerebrum), capable of acting briefly as a closed system, delivering facilitation to other such systems and usually having a specific motor facilitation” (Hebb, 1949, p. XIX of introduction). Hebb (1949) also conceived that memory (in a broad sense) and perception are two aspects of the same phenomenon, supported by the same cell assemblies. Establishment of a memory trace would proceed in two steps: First, reverberation of activity within the cell assembly would provide a physiological basis for a transient memory of the stimulus. Second, this transient memory can become a long lasting one by means of structural changes. The way by which such structural changes could occur is Hebb’s famous rule, according to which the coincidence of pre- and postsynaptic activities leads to a strengthening of the synapse. Strengthening of synapses can indeed be obtained by pairing pre- and postsynaptic activities, as has been established in several studies, both in vivo and in vitro (Baranyi and Szente, 1987; Gustafsson et al., 1987; Frégnac et al., 1988; Bonhoeffer et al., 1989; Ahissar et al., 1992b; Frégnac et al., 1994b; Jester et al., 1995; Markram et al., 1997). Hebb conceived that some basic perceptual capacities are already present at birth. Some inherited neuronal mechanisms therefore would provide the basis for a primary form of segmentation and figure-ground segregation: “The primitive unity of the figure is defined here as referring to the unity and segregation from the background which seems to be a direct product of the pattern of sensory excitation and the inherited characteristics of the nervous system on which it acts” (Hebb, 1949, p. 19).

Cross-correlograms for neuronal spike trains

77

If enough repetitions of the same pattern occur, activations in these low levels would be combined and would lead to the generation of a cell assembly, supposedly taking place in association areas. Once established, the cell assemblies would provide support for the identity of a perceived figure and for the permanence of learning. One important feature of the cell assembly is the presence of convergence and divergence in the projection scheme. This provides one means by which a cell can be activated by different presynaptic elements, which in turn enables pattern completion: presentation of one part of a figure would activate the whole assembly, hence creating the internal representation of the whole. The associational properties of the cell assemblies would also enable the property of figure invariance (the fact that the recognition of an item is independent of the viewing distance, illuminating condition, viewing angle, etc.) and the property of generalization (the ability to recognize a whole class of items after familiarization with a subpopulation of it). The role of temporal coherence in the assembly was not unambiguous in Hebb’s words. On the one hand, temporal correlation of spike activities appears necessary for the strengthening of synapses. It is also clear that once the strengthening of synapses between two neurones has taken place, these will not act independently anymore: “A functional relationship of activity” should result. “The fundamental meaning of the assumption of growth at the synapse is in the effect this would have on the timing of action potentials by the efferent cells” (Hebb, 1949, p. 72). Using present day terminology and technology, this should appear as a displaced peak in a CCH. On the other hand, Hebb explicitly rejected the importance of a tight temporal coherence for the functioning of the assembly: “At each synapse there must be a considerable dispersion in the time of arrival of impulses, and in each individual fibre a constant variation of responsiveness; and one could never predicate a determinate pattern of action in any small segment of the system. In the larger system, however, a statistical constancy might be quite predictable” (Hebb, 1949, p. 76). There is one important limitation in Hebb’s theory. The paradox is that this limitation results exactly from what makes the cell assembly such an attractive theory, and can be traced back to the anatomical basis of the assembly: neighbouring neurones are strongly interconnected, and this presumably results from the application of Hebb’s rule, since neurones located close to each other are activated together by sensory stimuli. The presence of this net-like structure, with its convergence and divergence, creates a multipotentiality by virtue of which one neurone can belong to different assemblies and generalization and completion can be realized. However, when two, instead of one, figures are presented simultaneously and close to each other, it is likely that some neurones will be activated by both of them, such that the resulting increase in activity in these neurones cannot be traced back to one or the other figure: the cell assembly system is not able to encode more than one object without confounding them. This ambiguity as to what figure induced an increased activity in which assembly, was pointed out by von

Time and the brain

78

der Malsburg (1981) and is known as the “superposition catastrophe”. Hence there must be some mechanism that maintains different assemblies, segregated from one another. In terms of anatomo-functional connectivity, there must be a selection process, by which neurones will belong to one assembly or another, but not to two assemblies at the same time. Synchronization of neuronal firing appeared as one possible mechanism by which this selection procedure might be performed. A role for synchronization in this process of selection was first proposed by Milner (1974), who was concerned with the way that even elements as simple as lines can be perceived as different entities: “[…] we must attempt to explain how lines are immediately seen as units or ‘wholes’. One possibility worth considering is that primitive unity depends upon the contiguity of active cells in topographically organized parts of the projection pathway. […] If adjacent, or nearly adjacent, cells interact when excited, in such a way as to synchronize and perhaps intensify each other’s activity, this could provide the unifying characteristic that ties the elements of a figure together. At subsequent levels of the pathway, impulses from cells fired by one whole would arrive as synchronous volleys, whereas impulses from different figures would have a random temporal relationship to each other. The resulting difference in the effectiveness of temporal summation could serve to funnel the signals from different objects in the field into different paths, through the feature detection network, and thus minimize their mutual interference”. 1 A role for synchronization (or using a broader term, “correlation”) in figure-ground segregation was also proposed by von der Malsburg (1981) for whom “a figure is decomposed in parts, the strongest correlations signalling affiliation to one part of the figure, 1. Recent modeling studies by Carlos Brody (J. Neurophysiol. 80:3345–3351; Neural Comput. 11:1527–1536; Neural comput. 11:1537–1551) have shown how artefactual peaks can be obtained in cross correlation histograms, that are the consequences of 1) slow covariation of firing rates and 2) correlation of visual response latencies. Brody’s papers also provide indication as to how such correlations can be distinguished from those related to neuronal connectivity.

weaker ones affiliation to adjacent parts, and so on.” There is a “decomposition of the visual scene into a hierarchy of correlates”. Therefore, the use of a temporal coding, based on short term correlation, would solve the superposition problem—and by extension would provide a mechanism for segmentation and for figure-ground segregation. Neurones activated by one object would synchronize between each other, but not with the neurones activated by another object. Obviously the temporal segregation of neuronal activities can be more efficient if neurones have the possibility of engaging in oscillatory synchronization: then one figure would have its representation in the synchronous oscillation of one population of neurones, while the other population would have its own synchronized oscillation out of phase with those of the first. This provided the basis for a modelling study in which a fully connected set of neurones was nevertheless capable of segmentation by virtue

Cross-correlograms for neuronal spike trains

79

(among other things) of synchronized oscillatory activities (von der Malsburg and Schneider, 1986). Another important question also came into focus, which is how neuronal activity is propagated from one place to another. The basic fact is that one cortical neurone produces only small depolarizations in another cortical neurone. According to in vitro studies the average size of an EPSP elicited by one pyramidal neurone upon another is less than 1 mV in the majority of the cases (for reviews: Thomson and Deuchars, 1997; Markram, 1997). The amplitude of the depolarization required for triggering an action potential in a cortical neurone is of the order of 10–20 mV. Therefore it is required that a summation of EPSPs takes place. The number of EPSPs that have to be summated is much higher when these EPSPs occur independently from each other in time, than when they occur all together, that is, in synchrony. This led to the notion that neurones can behave as coincidence detectors, rather than as temporal summators. The role of synchronization for the propagation of activity is the starting point for the synfire chain concept proposed by Abeles (1982): “According to this hypothesis the activity of the neurons that transmit information is organized along a chain of sets of neurons. Each link in the chain is made of a set of neurons that fire in exact synchrony whenever the chain becomes active. Neurons of each set converge on neurons of the next, and therefore synchronized activity of one set elicits synchronized activity in the next set and so on. […] the relevant code is the combination of neurons that fire in synchrony.” Here synchronization serves a selection procedure, for only those neurones that are synchronized have a chance to transmit their information to the next stage. “This concept of dynamic formation and dissolution of functional chains is similar to the ‘cell assembly’ postulated by Hebb thirty years ago, except for the requirement that the transmission along the chain is secured by the synchronous firing of sets of cells.” One advantage of the synfire chain, is that it allows for the existence of “cross over” such that one neurone can belong to two different chains. Even so, the independence of the two chains is preserved, as required for the segregation of cell assemblies with common neurones, as long as they are not active in exact synchrony. Experimental supports were later given for the existence of synfire chains in the awake behaving monkey (Abeles et al., 1993). The stimulus-dependent changes in synchronization observed in visual cortex (Gray et al., 1989; Engel el al., 1991c; see also Engel et al., 1991b; Ghose et al., 1994; Sillito et al., 1995; Kreiter and Singer, 1996; Livingstone, 1996; Brosch et al., 1997) are consistent with a role for synchronization in the process of segmentation, and appeared as the experimental demonstration of the theoretical ideas developed by Milner, von der Malsburg and Abeles. Hence, this gave support to the notion that “synchrony might be the ‘glue’ that binds distributed neuronal activity into unique representations” (Engel et al., 1992).

Time and the brain

80

2.4. Synchronization and Attention We have just reviewed evidence indicating a role for synchronization in scene segmentation. There are, however, other experiments showing that, in awake behaving animals, synchronization does not always have a clear relationship with sensory stimulation or motor activity. Early studies demonstrated that high frequency oscillatory events are visible, if not prominent, in the EEG traces under behavioural conditions requiring attention and vigilance: Lopes da Silva et al. (1970) observed the presence of a “beta” rhythm (around 20 Hz) above the visual cortex of dogs that were trained to press a lever when the visual stimulation (whole field, sinusoidally modulated light) was ON. This rhythmic signal was absent if the animals were anaesthetized. Rougel et al. (1979) demonstrated the presence of a fast somatoparietal rhythm during combined focal attention and immobility in baboon and squirrel monkey. The frequencies were between 13 and 25 Hz, and appeared independently in two loci (S1 and area 5). Bouyer et al. (1981) reported the presence of 35–5 Hz oscillations, with their focus located in the frontal (area 4 or 6) and parietal cortex (area 5) in immobile cats during focused attention. In these last two studies, the oscillations appeared in cortical areas where no information had to be processed—sensory and motor activities were presumably shut down during this attentive immobile behaviour. Some of the changes in synchronization observed in relation to different phases of behaviour in the frontal cortex of awake monkeys (Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995; but see Riehle et al., 1997) and in cat visual and motor areas (Roelfsema et al., 1997) might be related to changes in attentiveness. In the somatosensory and motor cortex, Murthy and Fetz (1992) observed 25–35 Hz synchronized oscillatory activity, that did not depend on any particular movement or stimulus, but instead appeared to be more prominent during a difficult task presumably requiring attention. Sanes and Donoghue (1993), in a visuo-motor task, attributed the origin of these oscillations to preparation of movements, rather than to their execution, while Baker et al. (1997) suggested these oscillations could be associated with the motor cortex “resting” between demanding motor tasks. Finally, in monkey parietal cortex, Cardoso de Oliveira et al. (1997) showed that synchronization between pairs of neurones is stronger before the presentation of the visual stimulus that the monkey has to attend to, than during its presentation. Studies in humans also indicate several possible roles for gamma oscillations. The power of such oscillations is higher when real and illusory (Kanizsa) triangles are presented than for non-coherent stimuli (Kanizsa triangles with rotated inducers), thus suggesting a role in binding for object perception (Tallon-Baudry et al, 1996). However, using similar recording techniques, the same authors demonstrated that increased power of gamma oscillations can be recorded during a task of active visual search (TallonBaudry et al., 1997) and during the expectation period in a delayed match to sample task (Tallon-Baudry et al., 1998). These results suggest that gamma oscillations may be involved in the recollection of stored representations and in visual attention (see also Llinás et al., 1994).

Cross-correlograms for neuronal spike trains

81

2.5. Summary Four different interpretations have been advanced for the presence of synchronization in the cortex. First, it has been proposed that the synchronization of action potentials between pairs of neurones reflects the EEG waves that characterize the brain in an unconscious state—during sleep or when anaesthetized. Second, synchrony has been considered to be the manifestation of a connectivity pattern typical of the cortex, in which a pair of neurones receives common inputs from another group of cells. Third, it has been proposed that the synchronized firing of two cells indicates their participation in a cell assembly. From this it was proposed that synchronization constitutes another neuronal code, a new dimension for the neuronal representation of sensory inputs and motor outputs, to be added to the rate coding. Finally, it has been shown that synchronization of neuronal activity can occur with no relationship to sensory input or motor output, and reflects the state of the cortical network when the subject is in a state of expectancy or attention.

3. DIFFERENT TYPES OF SYNCHRONIZATION We have reviewed the accumulated evidence that synchronization is the form of temporal coherence that dominates when recording from pairs of neurones in cortex. So far we have shown that synchronization has received different and sometimes contradictory interpretations. However, different interpretations for the presence of synchronized firing may not be contradictory if there is not one, but several forms of synchronization. We will now show that, indeed, different forms of synchronization exist. 3.1. Non-oscillatory Synchronization within One Cortical Area As mentioned earlier, the large majority of CCH peaks obtained from neurones recorded in the same cortical area reflects synchronization of neuronal activity. Different types of synchronization can be distinguished on the basis of the width of the CCH peaks. One common finding among the earliest studies on cortical synchronization, when it was studied in relationship to sleep and anaesthesia, is the presence of rather large CCH peaks. The peaks presented by Holmes and Houchin (1966) were several tens or hundreds of msec wide; the peaks presented by Noda and Adey (1970, 1973) were similarly quite wide on average, although a large variability could be observed in their data, such that narrow peaks a few msec wide as well as very large peaks, several hundred msec wide, were reported. In contrast, in those reports in which neuronal interactions were studied within the context of neuronal connectivity, with sensory stimuli or motor responses, the widths of the peaks often appeared to be narrow. In cat visual cortex for example, Toyama et al. (1981a) reported the presence of very narrow centered peaks 1 msec wide, while

Time and the brain

82

Michalski et al. (1983) reported centered peaks with widths ranging from 10 to 120 msec. Several studies showed that two or three different types of peaks can be distinguished, when attempts have been made to classify them according to their widths (Murphy et al., 1985; Krüger and Aiple, 1988; Krüger and Mayer, 1990; Aertsen et al., 1991; Gochin et al., 1991; Hata et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992). Compilation of these results suggests a relatively clear segregation between three types of CCH: Some narrow peaks, with a width less than about 10 msec when measured at half width; a second class of peaks, (intermediate peaks), with widths between approximately 10 msec and a few hundred of msec, and broad peaks with widths broader than one hundred msec. These different types of peaks have been reported both in anaesthetized animals and in awake behaving monkeys. As a rule, the three different peak types are observed for neurones that are close to each other (for example, recorded from the same electrode), while mostly the intermediate and broad peaks are detected when neurones are separated by more than a few hundred microns and up to several mm (Murphy et al., 1985; Krüger and Aiple, 1988; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992; Cardoso de Oliveira et al., 1997). In primary visual cortex, orientation selectivity is another property that appears to be related to the incidence of different peak types: Narrow peaks are seen more often when neurones share a similar orientation preference, especially if the receptive fields do not overlap (Krüger and Aiple, 1988; Schwartz and Bolz, 1991; Hata et al., 1991; Livingstone, 1996; Tamura et al., 1996). Notice, however, that the probability of observing a peak in CCHs between cross-oriented or iso-oriented pairs of neurones might also depend on the type of stimulus configuration used to activate the cells (Sillito et al., 1995). In primate area V2, Tamura et al. (1996) did not observe any relationship between orientation selectivity and the probability of observing narrow peaks. The occurrence of intermediate peaks does not depend strongly on orientation selectivity (Krüger and Aiple, 1988). 3.2. Non-oscillatory Synchronization between Different Cortical Areas CCHs have been computed for neurones recorded in different visual areas. The incidence of significant peaks, indicating interactions of neuronal origin, seems to be lower than for within-area recording. The only study in which these incidences have been compared under the same conditions (Cardoso de Oliveira et al., 1997) shows a drop from 46 to 15%. In our study on inter-areal interactions in monkey visual cortex, significant peaks were observed in only 10% of the dual single unit recordings. In the other 90% of cases the subtracted CCH appeared flat, indicating the absence of neuronal interactions in the great majority of the cases (Nowak et al., 1999). The coupling strength also seems to be much weaker for between- compared to within-area CCHs (Ana Wang Roe, personal communication). Similar to reports for within-area recordings, the CCHs showed peaks centred on the time origin in the very large majority of the cases (Nelson et al., 1992; Nowak et al., 1995; Wang Roe and Ts’o, 1997; Cardoso de Oliveira et al,, 1997; Nowak et al., 1999).

Cross-correlograms for neuronal spike trains

83

Therefore, neurones belonging to different areas tend to synchronize their activity through mechanisms of neuronal origin. On the basis of the width of the peaks, synchronization between different areas can also be subdivided in three different groups. The tripartite distribution of peak width is illustrated in Figure 5 for three different studies in two animal species: Interactions between areas 17 and 18 in the cat (Nelson et al., 1992), interactions between areas 17 of opposite hemispheres in the cat (Nowak et al., 1995), and interactions between areas V1 and V2 of the same hemisphere in the monkey (Nowak et al., 1999). The tripartite distribution of peak widths becomes quite obvious when one makes histograms of the distribution of the logarithm of the peak width, as shown in Figure 5. From one study to another, there is some variability in the exact positions of the modes and of the gaps between the different groups of peaks. Nevertheless, it is quite clear that inter-areal neuronal interactions produce some narrow peaks, which we called T (“tower”) peaks, with width of a few msec up to approximately 15 msec; some intermediate peaks, which we called C (for “castle”) peaks, with widths between 10–20 msec and 100–200 msec (one example of a C peak is shown in Figure 2C2); and finally, some broad peaks (H peaks, “hills”), with width of several hundred msec up to 2 sec (one example is shown in Figure 2D2). Similar to what has been reported for intra-areal synchronization, the incidence of the different peak types observed in the inter-area studies depends on the separation of the neurones (in this case, not in terms of physical separation but in terms of receptive field separation) and on their orientation selectivities. T peaks are preferentially obtained when neurones have overlapping receptive fields and similar orientation preferences (Nelson et al., 1992; Nowak et al., 1995). C peaks can be observed even if the receptive fields do not overlap; however, the receptive field separation for which C peaks are observed appears bounded by an upper limit of 10 degrees in cat (Nelson et al., 1992; Nowak et al., 1995) and 5 degrees in macaque (Nowak et al., 1999), for the corresponding eccentricities of the recordings. In the cat, the incidence of C peaks is not related to the difference in orientation selectivity when the receptive fields are in overlap. However, C peaks show a weak dependence on orientation preference when the neurones have nonoverlapping receptive fields (Nelson et al., 1992). The broadest peaks (H peaks) occurred without any relationship with receptive field properties, except for a decreased incidence with increased receptive field separation in the monkey V1-V2 study (Nowak et al., 1999). Thus, the different types of peaks observed in the inter-area studies are not homogeneously distributed with respect to the properties of the neurones recorded. 3.3. Synchronized Oscillations Although it has received much of recent attention, gamma-range oscillatory synchronization (GROS, one example of which is shown in Figure 4) was the last one to be described in the cortex, although it was occasionally mentioned in early studies. In terms of incidence, this form of synchronization seems to be totally absent in some reports, but overwhelmingly present in others. One possible reason for this discrepancy might be

Time and the brain

84

Figure 5 . Distribution histograms of the logarithm of the width of CCH peaks. The width is measured at half height. CCHs are calculated between neurones in areas 17 and 18 of the cat (top), between areas 17/18 of both cortical hemispheres in the cat (middle) and between areas V1 and V2 in the macaque monkey (bottom). Note the tripartite distribution of peak width with narrow (T type), middle (C type) and

Cross-correlograms for neuronal spike trains

85

broad (H type) peaks for each case of cross-correlation between neurones located in different cortical areas.

the method used to reveal it: cross-correlation can reveal synchronized oscillations in the spike trains of neurones only if they are regular enough. Other methods are less strict in this regard. For instance, some methods based on signal frequency analysis, or synaptic or field potentials (like the power spectrum analysis) might reveal more or less broad “peaks”, centred close to 40 Hz, that correspond to signals which fluctuate within this frequency range (field potentials: Eckhorn et al., 1988, 1993; Munk et al., 1996; TallonBaudry et al., 1996, 1997; synaptic potentials: Jagadeesh et al., 1992; Gray and McCormick, 1996; Steriade et al., 1996a). Whether these fluctuations are large and regular enough to be termed “oscillations” or “aperiodic oscillations” (Freeman and Barrie, 1994) is not always clear. However, a membrane potential that contains a large amount of power in the gamma-range frequency does not necessarily lead to oscillatory auto- (or cross) correlation histograms (Nowak et al., 1997). The problem of the presence or absence of oscillations then becomes a methodological and terminological issue (see also Young et al., 1992; Frégnac et al., 1994a). When the presence of oscillations is tested in spike trains, another methodological problem appears, that is related to the use of averaging over long periods of recording: there is some variability in the frequency from one oscillatory event to another, if not within one period, such that averaging across multiple oscillatory episodes might result in a smoothing of the side peaks that characterise the oscillations in auto- and crosscorrelograms (Gray et al., 1990; Kreiter and Singer, 1992; Livingstone, 1996; Murthy and Fetz, 1996; Gray and Viana Di Prisco, 1997). Thus, the presence of one isolated peak in a CCH, like a T peak for instance, does not exclude the possibility that the neurones were, at one time or another, engaged in an oscillatory synchronous activity. We will concentrate on the cases where oscillations are clearly visible. Unfortunately, gamma-range oscillations do not constitute a homogeneous ensemble. Instead gammarange oscillations have been shown to occur in 3 different conditions: 1. Gamma-range oscillations have been reported in field potentials and intracellular recordings—but, to our knowledge, not in spike trains—during spontaneous activity in anaesthetized preparations. 40 Hz “brief waves” were observed by Adrian and Matthews (1934) and have been described anew with different anaesthetics, in various cortical areas and in different animal species (Steriade et al., 1996a,b; Brett and Barth, 1997). 2. GROS have also been described, that appear only during sensory stimulation, for neurones recorded within one visual cortical area in the cat (Gray and Singer, 1989; Gray et al., 1989; Eckhorn et al., 1988; Engel et al., 1990, 1991c; Roelfsema et al., 1994; König et al., 1995; Munk et al., 1996; Kruse and Eckhorn, 1996; Brosch et al., 1997; Gray and Viana Di Prisco, 1997) and the primate (area V1: Eckhorn et al., 1993; Livingstone, 1996; area V2: Frien et al., 1994; area MT: Kreiter and Singer, 1992). 3. Synchronized oscillation can also appear without any clear relationship to the presence of stimulation, but presumably in relation to the state of attentiveness (Lopes da Silva et al., 1970; Rougeul et al., 1979; Bouyer et al., 1981; Murthy and Fetz, 1992; Sanes and Donoghue, 1993; Baker et al., 1997).

Time and the brain

86

Gamma-range oscillations can be synchronized between two different cortical areas: Between areas 17 and 18 in the cat (Eckhorn et al., 1988); between areas 17 of the two hemispheres in the cat (Engel et al., 1991a, Nowak et al., 1995); between areas 17 and PMLS of the cat (Engel et al., 1991b). Oscillatory synchronization also occurs between the motor areas of the 2 hemispheres and between somatosensory and motor areas (Murthy and Fetz, 1996). The width of the central peak observed in the gamma oscillatory CCHs (3–5 msec in Engel et al., 1990) is similar to the width of the narrow peaks described above. Furthermore, there are instances in which a non-oscillatory CCH containing one narrow peak can be made to become oscillatory (Schwartz and Bolz, 1991; Nowak et al., 1995). This would suggest that non-oscillatory narrow peaks and gamma-range synchronized oscillation are somehow related. There is however one important difference: the gamma oscillations can be synchronized between neurones that are separated by large distances on the cortical surface (up to 7 mm in cat area 17; Eckhorn et al., 1988; Gray et al., 1989; Engel et al., 1990; König et al., 1995), while the non-oscillatory narrow peaks are restricted to neurones located close to each other, or with overlapping receptive fields (Murphy et al., 1985; Krüger and Aiple, 1988; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992; Nelson et al., 1992; König et al., 1995; Nowak et al., 1995). Similarly, the high frequency oscillations reported in the motor cortex of awake behaving monkeys appear coherent despite significant separation between the recording sites (Murthy and Fetz, 1996) while narrow peaks are observed only for small interneuronal separations (Murphy et al., 1985). In area 17, the stimulus-related GROS have been reported to occur between neurones displaying different orientation selectivities for overlapping receptive fields. Beyond overlap, the probability of observing this type of synchronization is higher when neurones display similar orientation preferences (Gray et al., 1989; Engel et al., 1990; Engel et al., 1991a; Livingstone, 1996). The motion of the stimuli also seems to be an important stimulus parameter to obtain GROS, as they do not appear with stationary stimuli (Engel et al., 1990; but see Ghose and Freeman, 1992). In fact, oscillations turned on by a moving stimulus can be disrupted by stationary stimuli (Kruse and Eckhorn, 1996). Summary Four different types of synchronization can be distinguished on the basis of the shape of the CCH peaks. One form corresponds to the gamma-range oscillatory synchronization. The three other forms correspond to non-oscillatory synchronization, that can be distinguished on the basis of different peak widths. These non-oscillatory forms of synchronization show different incidences depending on the properties of the recorded cells.

Cross-correlograms for neuronal spike trains

87

4. WHERE DOES SYNCHRONIZATION COME FROM? Before trying to determine the underlying mechanisms, we must first review the relationship between the different forms of synchronized activity, the waking state of the brain and the presence of a sensory stimulus. 4.1. Behavioural and Stimulus Dependency of the Different Types of Synchronization 4.1.1. Awakening and anaesthesia We have mentioned above that sleep and waking states, as well as the use of anaesthesia, affect the probability of observing synchronization between pairs of cortical neurones. Here we re-examine this issue with respect to the four different forms of synchronization defined above, in an attempt to determine their origins and functional roles. We have investigated the effects of different anaesthetic agents on the three types of non-oscillatory activities observed between areas V1 and V2 (Nowak et al., unpublished). Three anaesthetic agents have been used, which were isoflurane, halothane (both volatile anaesthetics), and the opiate fentanyl. The level of spontaneous activity was lower by a factor of 1.5 to 2 with isoflurane and halothane compared to fentanyl, but the strength of the visual responses was not significantly different between the three anaesthetics. In terms of incidence and strength of synchronized activity however, there was a major difference. The three non-oscillatory forms of synchronization were observed under all anaesthesia conditions. With isoflurane or halothane, the probability of observing a significant peak in a CCH computed from single unit activities was five time larger than with fentanyl. In parallel with this large difference in incidence, a strong difference was also observed in the proportion of action potentials contributing to the CCH peaks, which was more than twice as high with the volatile anaesthetics compared with fentanyl. This indicates a considerable reduction in the number of action potentials that were correlated by mechanisms of neuronal origin under fentanyl anesthesia. The origin of these differences is not clear, but it may be hypothesized that isoflurane and halothane have a reinforcing effect on the neuronal mechanisms generating sleepassociated rhythms, like the spindle wave and the “slow (<1 Hz) oscillation” (Adrian and Matthews, 1934; Steriade et al., 1993a). For example, the strength of GABAergic inhibition is increased by these anaesthetics (e.g., Franks and Lieb, 1994), which might lead to a strengthening of the spindling mechanisms in the thalamus (Keifer et al., 1994; for the cellular mechanisms of spindle wave generation see McCormick and Bal, 1997; Destexhe et al., 1998). Fentanyl, on the other hand, is likely to act on more specific targets related to nociception, and is less likely to influence directly the thalamocortical

Time and the brain

88

network (e.g., Pasternak, 1993). Therefore most of the C peaks observed under halothane or isoflurane might reflect the presence of spindling activity. In a similar line of evidence, barbiturate anaesthesia is known to induce spindles. At the cortical level this results in cross-correlation peaks with width very similar to that of C peaks (e.g., Contreras et al., 1997). Concerning H peaks, it is interesting to notice their similarity with the peaks observed in CCHs computed during the slow (<1 Hz) cortical oscillations associated with sleep (Amizca and Steriade, 1995). Concerning gamma-range oscillations, Livingstone (1996) found their occurrence not to be influenced by the halothane, nitrous oxide or sufentanyl. Gray and Viana di Prisco (1997) found that the incidence of gamma-range oscillations associated with visual stimulation is not different in awake behaving cats and in cats anaesthetized with halothane. This would suggest that gamma-range oscillations are neither disrupted nor reinforced by anaesthesia. On the other hand, the gamma-range oscillatory activities associated with attentive behaviour are unlikely to be preserved under anaesthesia. In keeping with this observation, Munk et al. (1996) found that stimulation of the reticular formation in anaesthetized cats, as a paradigm that would mimic awakening, increases the incidence and strength of oscillatory synchronization. Similarly, the gamma-range fluctuations that are observed transiently during the slow sleep oscillations become more sustained after stimulation of the mesopontine cholinergic nuclei (Steriade et al., 1996b). 4.1.2. Stimulus dependency The four different types of synchronized activity show different behaviour with respect to the presence of a sensory stimulus. As mentioned above, the GROS observed in the visual cortex occur only during the presentation of a visual stimulation, and are not found during periods of spontaneous activity (see Figure 4). Under fentanyl anaesthesia, we studied the stimulus dependency for two of the three classes of non-oscillatory peaks we observed between areas V1 and V2 of the monkey (Nowak et al., 1999). An example of this stimulus dependency is presented in Figure 2. It shows that the presentation of a stimulus led to the occurrence of a C peak, while during spontaneous activity an H peak was present instead (Figure 2C2 and 2D2). This behaviour summarizes the general tendency we observed. We could not establish any statistically significant trend for T peaks, due to the paucity of their occurrence. We found that C peaks were mostly associated with the presentation of a stimulus (Nowak et al., 1999). Hence, based on the effects of the presentation of a visual stimulus and on the effect of anaesthesia (see above), it might be hypothesized that there are not one, but two types of C peaks, one associated with the processing of sensory signals (see also Aertsen et al., 1991; Cochin et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995 for the occurrence of intermediate peaks in awake behaving animals), the other reflecting sleepand anaesthesia-induced rhythms. The H peaks on the other hand, showed a different behaviour with respect to the presentation of a visual stimulus, suggesting that most of them are associated with spontaneous activity rather than with visual responses. Notice that, contrary to what

Cross-correlograms for neuronal spike trains

89

Figure 2 may suggest, the C and H peaks do not necessarily replace each other depending on whether the neurones were visually driven, or were spontaneously active. For example, C peaks could be found for the visual response period while the CCH corresponding to the spontaneous activity period could be flat. This questions the relevance of H peaks as a neuronal correlate of sensory processing. However, a minority of H peaks do appear during visual stimulation only (Nowak et al., 1995, 1999), and peaks similar to H peaks have been shown to occur in awake, behaving monkeys (Aertsen et al., 1991; Gochin et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995). Here again it might be that two different neuronal mechanisms generate CCH peaks with similar widths. Besides the difference between visually driven and spontaneous activity, the configuration of the visual stimuli can strongly influence the probability of observing synchronization. This has been well documented for the GROS, which tend to occur more often when the neurones are activated by one common stimulus rather than when two stimuli activate them separately or in a conflicting way (Gray et al., 1989; Engel et al., 1991b, c; Kreiter and Singer, 1996; Livingstone, 1996; Brosch et al., 1997). Whether non-oscillatory synchronization is modified when different types of stimulus configuration are used to activate the cells has not been studied as extensively as in the case of oscillatory synchronization. In cat area 17, Engel et al. (1991c) showed that nonoscillatory CCHs display a stimulus dependency similar to that shown for the oscillatory synchronization. In cat and monkey primary visual cortex, Sillito et al. (1995) showed examples of narrow CCHs that depended on the orientation context of the stimuli used. Some H peaks observed between the two hemispheres in cat appeared to be disrupted by the use of conflicting stimuli (Nowak et al., 1995). The stimulusdependent synchronization observed in frontal and auditory cortex of awake animals seems to apply for all the different types of peaks (Frostig et al., 1983; Vaadia and Abeles, 1987; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Ahissar et al., 1992a; Vaadia et al., 1995). Altogether these data indicate that stimulus dependent changes are not restricted to gamma-range synchronous oscillations: they can also be observed for all the different types of non-oscillatory synchronization. 4.2. Neuronal Mechanisms Generating Different Types of Synchronization 4.2.1. Common inputs The peaks centred on the time origin in a CCH may result from the activation of the two recorded cells by a third cell, or by a pool of common presynaptic neurones. In the retina, two neighbouring ganglion cells can show synchronized firing within a short time scale, and Mastronade (1983a) concluded that this was due to a common input from amacrine cells. In the lateral geniculate nucleus of the cat, Sillito et al. (1994) described a form of synchronization, characterized by relatively broad peaks (whose width was in the range of tens of msec), that appears between neurones with non overlapping receptive fields,

Time and the brain

90

when they are simultaneously activated by light bars or gratings long enough to cover both receptive fields. This form of synchronization also depends on the orientation of the stimulus with respect to the alignment of the receptive fields. It is not observed, or is much smaller, when small spots of light covering only each receptive field are flashed simultaneously. This form of synchronization disappears when the visual cortex is lesioned. This suggests that this form of thalamic synchronization is due to common input fed back from cortical layer VI. As already stressed, synchronization of neuronal origin is the most often encountered type of neuronal interaction found in the cortex. Dickson and Gerstein (1974) calculated that the source of common inputs responsible for the synchronized activity they observed within the auditory cortex should be located more than 1.5 mm away from the recorded area. This obviously suggests the thalamus as one possible structure containing the neurones providing the common inputs. However, Engel et al. (1991c) and Nowak et al. (1995) showed that gamma-range oscillatory synchronization, as well as the three types of non-oscillatory synchronization, are present in the interactions between areas 17 of the two hemispheres. Since the projections of the thalamus onto the visual cortex are strictly ipsilateral, this shows that synchronized activity can be obtained without common thalamic inputs. Obviously this does not mean that the thalamic inputs do not contribute to the synchronization observed within the same hemisphere, although in this case, it is unlikely to be the source of shared inputs to neurones that are separated by a distance larger than the width of the terminal arbor of thalamic axons (Nelson et al., 1992). The corpus callosum contains only axons of cortical origin. Engel et al. (1991c) showed that lesion of the corpus callosum suppressed all instance of GROS between the two hemispheres. Munk et al. (1995) extended this result to the case of T and C peaks. This indicates that the corticocortical connections transiting through the corpus callosum are required for the intermediate and narrowest type of coupling, whether or not it is oscillatory. H peaks, on the other hand, remained quite numerous if only the posterior half of the corpus callosum was cut. They were reduced in incidence, although still not completely suppressed, only after 80% of the corpus callosum was severed (Munk et al., 1995). What types of connections sustain the different types of interhemispheric synchronization? Two types of corticocortical connections are present in the corpus callosum. The first corresponds to the fibres that link together the two areas 17. This group of connections is homologous to the horizontal connections within area 17. The other set of axons corresponds to feedback corticocortical connections: the areas 17 of both hemispheres receive common feedback inputs from several “higher” cortical areas (areas 19, 21a, PMLS and PLLS), by means of axons that bifurcate such that one branch innervates the hemisphere ipsilateral to its parent neurone; while the other innervates the hemisphere contralateral to it. Notice also that some form of “functional” bifurcation can occur if two adjacent neurones projecting one to one side, the other to the other hemisphere, are made to synchronize their activities through mechanisms of neuronal origin. Interestingly, C and H peaks are observed for separations that correspond to the known values of divergence for the feedback connections. The next step in the study of Munk et al. (1995) was therefore to establish the role of one or the other set of connections in the generation of synchronized activity. To this end, extensive lesions of

Cross-correlograms for neuronal spike trains

91

the extrastriate cortex were made, aimed at sparing only area 17 and the neighbouring strip of area 18 that represents the vertical meridian. In this way, the feedback connections were removed and the route for the polysynaptic propagation of activity through multiple cortical steps was interrupted as well. The corpus callosum was left intact in these instances (hence the “horizontal connections”). Despite these extensive lesions, the strength of the visual responses was not different from that in intact animals. The results showed that narrow peaks were not strongly affected. Similarly, gammarange oscillations were still observed, although their strength appeared to be reduced (Munk et al., unpublished). However, the incidence of C peaks was reduced, approximately by one half, in single unit paired recordings, and that of H peaks was reduced by two thirds. Together with this reduced incidence, a reduced coupling strength was observed for those H and C peaks that remained. H peaks remained unless all but the most anterior portion of the callosum was removed, but disappeared when the direct feedback inputs were removed. From this we can conclude that a majority of H peaks seem to reflect the transmission of a slow rhythm reflecting the anaesthetized or sleeping state of the animal, by way of multiple, polysynaptic pathways through the cortical mantle. Half of the C type synchronization seems to be associated with monosynaptic common inputs provided by feedback connections. It is not clear however, if higher cortical areas contain common input neurones that generate C peaks, or whether they are involved in modulating activity in a way that would make neurones more bursty, hence generating larger peaks (see below). T peaks and GROS do not seem to require a common input source from higher areas; they seem to be supported by the direct reciprocal connections between the two hemispheres. 4.2.2. The emergent properties of the cortical network At this stage we are faced with a paradox for the generation of the narrowest CCH peaks (T peaks and gamma-range oscillatory synchronization): They seem to be generated solely by the reciprocal connections linking the two synchronized neurones, but they are nonetheless centred on the time origin despite the large separation and large conduction delay between the recording sites. How is this possible? This issue has been addressed by modelling studies. A number of such studies have shown that networks composed of realistic neurones, interconnected with realistic synaptic strengths and conduction delays, show synchronized oscillatory activities when activated by an “input” that does not contain any temporal structure (some of the studies relevant to this question are: Sporns et al., 1989; Bush and Douglas, 1991; König and Schillen, 1991; Lytton and Sejnowski; 1991; Wilson and Bower, 1991; Ritz et al., 1994; Whittington et al., 1995; Bush and Sejnowski, 1996, Hansel and Sompolonski, 1996; Wang and Buzsáki, 1996; Lumer et al., 1997a,b). These studies obviously differ in their details but all suggest that synchronization can be obtained as an emergent property within the cortical network. These models provide a counter-intuitive result, namely, that the presence of a conduction delay between neurones does not prevent them from being

Time and the brain

92

engaged in synchronized activity. This is determined by the connectivity pattern, characterized by its divergence (one cell projects to hundreds of other cells), by the fact that excitatory neurones project to both excitatory and inhibitory neurones (and vice versa), by the small strength of the individual synapses, to some extent by the membrane properties of individual neurones, and, most important, to GABAergic inhibition that plays a key role in oscillatory synchronization. Model studies specifically designed to recreate the gamma-range oscillatory activity showed that neurones can be synchronized as long as the conduction delay is less than, approximately, one fourth of the oscillation period (see also Eckhorn, this volume). The presence of oscillatory forms of synchronization between the two cortical hemispheres for example, is not incompatible with the large distance between neurones and conduction delays up to about 5 msec—the conduction delay between the two hemispheres in cat cortex is below 5 msec in the majority of the cases (Harvey, 1980; Innocenti, 1980). Another issue related to the generation of gamma-range synchronous oscillation is the requirement for pacemaker neurones and their location. Some experimental results suggest that fast rhythms might be generated through synaptic interactions taking place within the cortical network, without pacemaker neurones, since oscillatory activity can be independent of the membrane potential of the recorded neurones (Bringuier et al., 1997). In some of the modelling studies mentioned above however, synchronized oscillations were obtained only after including excitatory neurones discharging in bursts of action potentials. Neurones displaying this property might be the “chattering cells”, recently described in cat cortex (Gray and McCormick, 1996; Steriade et aL, 1998), that have the property to discharge repetitive bursts of action potentials at frequencies up to 70 Hz. (However the majority of these neurones depart from “pure” pacemaker neurones in the sense that the burst frequency depends on the strength of their activation [Gray and McCormick, 1996].) In addition, some cells display subthreshold oscillations in their membrane potential when depolarized by DC injection (Llinás et al., 1991; Nuñez et al., 1992). Oscillatory activities in the gamma range do occur in subcortical structures such as the retina and the visual thalamus (Rodieck, 1967; Robson and Troy, 1987; Funke and Worgotter, 1995; Ghose and Freeman, 1992; Neuenschwander and Singer, 1996). It should be pointed out however, that the apparent “rhythmicity” observed at the level of some retinal ganglion cells during sustained activity basically reflects the regularity (low variability) of the discharges and that the frequency shown by these “rhythms” changes widely depending on the luminance conditions (Rodieck, 1967; Robson and Troy, 1987; Troy and Robson, 1992). According to Bouyer et al. (1981), the posterior thalamic group could be the pacemaker for the 40 Hz rhythms observed in the parietal cortex of attentive cats. Cells with rhythmic discharges within the gamma range have also been reported in the reticular and the intralaminar nuclei (Pinault and Deschênes, 1992a; Steriade et al., 1993c). Other modelling studies therefore, have suggested that neurones in the thalamus act as pacemakers for the production of cortical 40 Hz oscillations (Llinás et al., 1994; Funke and Worgotter, 1995; Ghose and Freeman, 1997). Nevertheless, these models cannot account for the presence of oscillations synchronized between the two hemispheres.

Cross-correlograms for neuronal spike trains

93

4.2.3. Generation of different peak widths Compared to the case of synchronized oscillations, the other forms of synchronization, despite being more commonly observed, have received less interest from modelling studies (but see: Erb and Aertsen, 1992; Koch and Schuster, 1992; Abeles et al., 1994; Arnoldi and Brauer, 1996; Hansel and Sompolinski, 1996). One aspect that has not received much attention is the variability of peak widths observed in the CCHs. Experimental data, however, can give a hint about how different peak widths (and therefore different peak classes) can be obtained. As far back as 1970, it was shown that the width of a CCH is strongly influenced by the presence of burstiness in one or both of the neurones (Moore et al., 1970). More recent studies have shown that variable amounts of burstiness generate different CCH peak widths. First, there is a similarity between the width of a CCH and the width of the ACHs of one or both of the neurones of the pair (Gochin et al., 1991; Nowak et al., 1995). Second, replacing the bursts by the first action potential of the burst, or using deconvolution methods, usually leads to a narrowing of the peak width (Mastronarde, 1983b; Eggermont et al., 1993; Eggermont and Smith, 1996). Recognizing that burstiness accounts for the width of a CCH peak just moves the problem a step farther: what neuronal mechanisms are responsible for the presence of bursty firing? Burstiness can be an intrinsic property of the neurones. Cortical neurones have been classified in different types, based on their spike response to current pulse injection in vivo and in vitro. Among these different types of neurones, two are characterized by their propensity to generate bursts of action potentials. One type corresponds to the “intrinsic bursting neurones” identified both in vitro (Connors et al., 1982; McCormick et al., 1985; Jones and Heinemann, 1988; Chagnac Amitai et al., 1990; Mason and Larkman, 1990) and in vivo (Pockberger, 1990; Baranyi et al., 1993; Gray and McCormick, 1996). The other category has been observed in vivo and corresponds to the “chattering cells” (Gray and McCormick, 1996; Steriade et al., 1998) that differ from the “intrinsic bursting” neurones by their thinner action potentials and their ability to discharge bursts in a sustained manner. Nevertheless, in both types of cells, each burst lasts for only a few msec. This would produce peaks in ACHs of a few msec width only. This would account only for the larger width shown by the narrowest class of CCHs. Intrinsic membrane properties of cortical neurones cannot therefore explain the width of the intermediate and broad CCH peaks. Neurones in the thalamus, provided they are sufficiently hyperpolarized, can discharge bursts of action potentials. The duration of a typical burst in the thalamus is of the order of ten to several tens of msec (Jahnsen and Llinás, 1984; Kim et al., 1995; Guido and Weyand, 1995). During spindle waves, these bursts are strongly synchronized such that the thalamus can be brought into a globally synchronized state (Kim et al., 1995; Contreras et al., 1997). In these conditions, the topographic organization of the thalamus, and the lack of direct interactions between relay cells, do not prevent neurones located in separate territories from behaving in unison. The corresponding ACH and CCH peaks display widths of several tens of msec (Kim et al., 1995). This synchronized activity,

Time and the brain

94

transmitted to cortex, would produce bursts of action potentials there, that would not be of intrinsic but of synaptic origin. This would generate CCH peaks with a width compatible with that of intermediate (C) peaks between spatially remote loci in cortex. Therefore, some intermediate peaks (those that are associated with sleep and anaesthesia) could be generated through the thalamocortical network, with a prominent role played by the membrane properties of thalamic neurons. For those C peaks that appear without a relationship to spindle activity (for example during sensory stimulation only, or in awake animals), it can be expected that the synaptic burstiness originates from within the cortex itself. If it is the same C peaks that appear to depend on feedback connections, then it could be that feedback connections are able to control the duration of cortical neuronal bursts. Since bursts are more likely to produce the temporal summation that is required to trigger action potentials in a post synaptic neurone, it could be hypothesized that feedback inputs from higher cortical areas would ultimately control the transmission of activity through the cortex. We have proposed that the majority of H peaks are correlates in the spike synchronization domain of the slow (<1 Hz) oscillations associated with sleep (Adrian and Matthews, 1934; Steriade et al., 1993a). Steriade et al. (1993b) showed that the slow oscillations survive extensive lesion of the thalamus, which indicates that it is generated within the cortex. The neuronal mechanisms that lead to the slow build up of the associated depolarizing synaptic potential, and to its resetting, remain to be elucidated. 4.3. A Global Picture As we entered into a more detailed examination of cortical synchronization, we found that there are not one, but four different forms of synchronization in cortex. Then, examining the stimulus and sleep/waking dependency of these different forms of synchronization, as well as the possible mechanisms that could account for their generation, we are forced to recognize that there might also be some subclasses within the different forms of synchronization. 1. The gamma-range synchronized oscillations might not be a single form of synchronization. One form appears in the membrane potential and in the field potential of neurones when animals are asleep or anaesthetized. Another form of gamma-range synchronized activity, that displays a strong stimulus dependency, depends apparently on the direct reciprocal connections between the recorded cells, although subcortical structures might contribute as well. There is a third form of gamma-range synchronized oscillation, that cannot be associated with the presentation of a stimulus, but which seems to be associated with attentiveness and expectancy. Whether these three forms of gamma-range synchronization share similar mechanisms for their generation, or whether their similarity is just coincidental, remains to be determined. 2. Non-oscillatory, narrow peaks. There are two types of narrow peaks. Those that are neatly separated from the time origin can be considered as representing a serial synaptic connection between the two recorded cells; this form of temporal coherence nonetheless appears extremely rare in the cortex. The majority of narrow peaks are centred on the time origin. They could be produced by common inputs from other

Cross-correlograms for neuronal spike trains

95

structures, especially when they are observed between neurones located close to each other. When the participation of common inputs can be dismissed, they appear to be generated by the direct reciprocal connections between the two recorded cells. 3. Intermediate and C peaks. Because they are observed between neurones that are spatially separated, it is necessary to postulate that they are generated through mechanisms that do not respect a strict topographic organization. In the visual cortex, feedback connections do not respect the retinotopic organization of the lower cortical areas, and it has been shown that these connections contribute to their generation. The feedback connections could further sustain the stimulus dependency exhibited by this type of synchrony. Another form of C peak could be related to spindle wave generation in the thalamus during sleep and anaesthesia. In that case their width could be related to the interaction taking place within the thalamocortical network. 4. Broad and H peaks. A majority of H peaks seem to reflect the transmission of a slow rhythm reflecting the anaesthetized or sleeping state of the animal, through multiple, eventually polysynaptic pathways. Since this form of coupling does not really show any relation to sensory processing, it is not surprising to observe it between neurones that have nothing in common in terms of receptive field properties. Nevertheless, there is another form of broad peak, that is observed in awake behaving animals. In these conditions too, they are observed between neurones that are spatially separated from each other. Since this type of synchrony can change with respect to the behavioural state of the animal, it is tempting to propose at it reflects slow, global changes, associated perhaps with the generation of sustained attention.

5. FUNCTIONAL ROLES FOR SYNCHRONIZATION We have shown that synchronization, in its different forms, can be explained as the consequence of the anatomical organization of corticocortical and thalamocortical connections and of their functional expression. The question to be solved now is whether synchrony is only the consequence of the network at work, or whether synchronization is also used for some purposes. In this attempt to examine the functional significance of synchronization, we will not examine the significance of the synchronization that occurs in the states of sleep and anaesthesia. Likewise, we will not discuss the importance of synchronization for plastic changes of neuronal connectivity. 5.1. Multiplexing of Information Synchronization of neuronal origin can be explained by the presence of common input to the two recorded cells. In two studies, one in retina (Meister et al., 1995) and the other in cortex (Ghose et al., 1994b), attempts have been made to map the receptive field of the synchronized action potentials. One result from the two studies is that the receptive field corresponding to the synchronized action potentials can differ, in size and properties,

Time and the brain

96

from those of the two recorded units. This means that the synchronized action potentials from two cells contain information not available from the response of each cell taken individually. By this way there would be a multiplexing of information, allowing for an increased amount of information transmission in the axons. However attractive this hypothesis is, it requires that neurones beyond the synchronized ones be able to demultiplex the inputs they receive. Whether such a subtle information processing can be performed in the cortex remains to be determined. 5.2. Binding by Synchronization 5.2.1. Recent results, challenges, and constraints Synchronization of neuronal activity has been proposed as a solution to the binding problem. In short, neurones that respond to the same object would synchronize their activity, while remaining desynchronized from neurones responding to different objects. Synchronization would provide a means of selecting specific neuronal populations, by coincident arrival of EPSPs. Electrophysiological evidence has been provided, showing that neurones synchronize their activities when activated by a common stimulus, but remain desynchronized when activated by conflicting stimuli (see above). Additional support for a role of synchronization in perception was given in two studies. In the first one, Roelfsema et al. (1994) studied amblyopia (poor vision in one eye) that was induced by strabismus in cats: cats are not able to perceive gratings of high spatial frequency through their amblyopic eye, but the strength of the neuronal responses to these high frequency gratings is similar for the two eyes. What differed instead between the two eyes was the ability for neurones to synchronize, which was lower for neurones activated by the amblyopic eye, compared to those activated by the normal eye. A deficit in perception appears correlated with a deficit in synchrony. In a second study, Fries et al. (1997) studied the effects of interocular rivalry in awake strabismic (but not amblyopic) cats. They found that presenting two competing stimuli to both eyes did not affect the strengths of visual responses. However, in the condition of rivalry, synchronization was stronger between neurones activated by the dominant eye, and lower in neurones activated by the non-dominant eye. This suggested that “at early stages of visual processing the degree of synchronicity rather than the amplitude of the responses determines which signals are perceived and control behavioural responses.” Additional support for a role of synchrony in binding comes from large scale modelling studies (Neven and Aertsen, 1992; Tononi et al., 1992). Based on Hebbian cell assembly organization, these artificial neural networks are able to learn and to recognize stimuli. In doing so they can solve the in variance and the occlusion problems. However, as already stressed, segmentation cannot easily be performed in a pure Hebbian cell assembly system. In these models therefore, it is synchronization that has been used to tag which parts belong to the same object. Hence these modelling studies demonstrate the plausibility of synchronization as a means of solving the binding problem. Nevertheless, there are a number of recent experimental studies that have challenged

Cross-correlograms for neuronal spike trains

97

the binding-by-synchrony hypothesis. First, Gochin et al. (1991), recording in inferotemporal cortex of awake behaving monkeys, found that different visual stimuli did not produce different CCH peaks, despite the differences of neuronal responses. Likewise, Gawne and Richmond (1993) in inferotemporal cortex, and Gawne et al. (1996) in primary visual cortex of awake behaving monkeys, found that correlation in the millisecond range does not correlate with the similarity of the responses of the neurones to a battery of different stimuli. This does not prove that synchrony does not play a role, but it suggests that a rate coding, eventually combined with a slow temporal coding, would do a better job in discriminating between the different stimuli used in these studies, than would a code based solely on synchronized activity in the millisecond range. Second, a group of studies in the motor cortex has failed to find a clear relationship between oscillatory synchronization and movement execution (Murthy and Fetz, 1992; Sanes and Donoghue, 1993; Baker et al., 1997). These results do not argue against a role for synchronization in binding in sensory cortex. However, the recent study by Cardoso de Oliveira et al. (1997), in parietal visual cortex of monkey, showed the intriguing result of a suppression of synchronization by visual stimulation, rather than an enhancement. In awake behaving animals therefore, synchronization might have a stronger relationship to attention than to segmentation. It has been suggested that 40 Hz oscillatory activities are instrumental for the generation of long range synchronization (König et al., 1995). Therefore perturbing 40 Hz oscillations should prevent synchronization, and ultimately prevent accurate perception. In two psychophysical studies however (Fahle and Koch, 1995; Kiper et al., 1996), it was found that perceptual performances are neither improved not impaired by the use of stimuli flickering in time near 40 Hz. If synchronization plays a role in binding, then it could be argued that either oscillations are not required for the generation of synchronization, or that non-oscillatory forms of synchronization play a role. Thorpe et al. (1996) demonstrated that it takes only 150 msec of processing for recognition of a visual image. The images used in this study were complex colour photographs, therefore requiring binding and figure-ground segregation to allow recognition. This does not argue against a role of synchronization in segmentation, but this imposes serious constraints on the onset of synchronous activity with respect to the onset of the neuronal responses in different cortical areas. Therefore, one important issue that remains to be investigated is that of the onset of synchronized activity in cortex.

Time and the brain

98

5.2.2. Coincidence detection The cornerstone of the binding-by-synchronization hypothesis is the requirement for neurones to behave as coincidence detectors: an action potential is triggered only after enough EPSPs have been summated (see above). The time it takes for this summation to occur corresponds to the integration time. The largest time window within which the summation of two EPSPs can take place is approximately equivalent to the time constant of the neurone. Beyond this time interval, the depolarization elicited by the first EPSP has vanished, since the membrane potential has returned to its resting level. Theoretical and modelling studies show that, for the same mean presynaptic firing rate, the number of randomly occurring EPSPs required to fire an action potential is much larger than the number of EPSPs required if they occur in synchrony (Abeles, 1982; Bernander et al., 1991; Reyes and Fetz, 1993b). This makes the neurone extremely sensitive to the occurrence of synchronized EPSPs. For this reason it has been proposed that cortical neurones may behave as coincidence detectors. All neurones perform temporal summation. The distinction between the temporal integrator and the coincidence detector resides in the time window during which this summation is accomplished. König et al. (1996) proposed a frontier between the two types, such that a neurone is considered as a coincidence detector if the integration time is smaller than the mean interspike interval, while the integration time of a temporal integrator would be equal or larger than the mean interspike interval. The critical value would be around 5 msec. Experimental measurements of the integration time of cortical neurones during sensory stimulation are still lacking. However, the variability of action potential timing has provided one entry to study the way neurones integrate synaptic activity: For a constant mean firing rate, the interspike intervals (ISI) can be highly variable, such that short and long ISIs can be observed adjacent to each other. The distribution of ISIs is then characterized by its skewed nature, giving a Poisson-like shape to the ISI histogram. The coefficient of variation (the standard deviation divided by the mean ISI) is often close to unity, indicating apparent randomness of spike timing (e.g. Softky and Koch, 1993). The variability of ISIs has been noticed since the earliest electrophysiological investigations of the central nervous system. Softky and Koch (1993) attempted to determine its origin by confronting experimental data obtained from awake monkey visual cortex with different computer models of synaptic integration. One important conclusion they reached is that the variability observed in vivo is incompatible with the integration of random EPSPs that are simply summated: if EPSPs of small size have to be summated in order to yield a constant mean firing rate, then the total synaptic current is almost constant in time, and action potential timing would not vary. One way, however, to obtain the high variability in the membrane potential and therefore in spike timing, is to have the neurone depolarized not by small random EPSPs, but by large compound EPSPs. In other words, it is possible to explain the presence of a high variability of action potential timing if the action potentials are triggered by synchronous EPSPs. This provided evidence favouring the role of synchronization in determining action potential triggering. In the light of these results, it was proposed that, indeed, cortical neurones do

Cross-correlograms for neuronal spike trains

99

behave as coincidence detectors. Unfortunately, later studies have indicated that variability of action potential timing might be explained by other mechanisms. Shadlen and Newsome (1994) showed that variability of action potential timing can be obtained if neurones integrate not only random EPSPs, but also random IPSPs. For this neuronal model to work in accordance with the in vivo data, the EPSPs and the IPSPs have to be balanced (that is to say, the total inhibitory current should be equivalent or close to the total excitatory current a neurone receives). The role of inhibition in generating spike timing variability has been confirmed in other modelling studies (Hansel and Sompolinski, 1996; van Vreeswijk and Sompolinski, 1996; Schindler et al., 1997; Troyer and Miller, 1997). Large scale modelling studies might provide an indication about the usefulness of synchronization in eliciting activity in cortical neurones. In their model, Lumer et al. (1997b) could modify the amount of synchrony between neurones without changing other parameters (a procedure impossible to achieve experimentally). They found that reducing synchronization decreases the firing rate of the neurones in the network. However, in the worst case, the firing rate was not reduced to zero, but to a value approximately half the original one. This suggests that synchronization can help the transmission of signal and maintenance of activity, but also that synchronization is not absolutely necessary for spike generation. Altogether it could be concluded that neurones might behave as coincidence detector, in which case synchronization would help transmitting information. It does not mean, however, that neurones cannot work as temporal summators. The experimental demonstration that cortical neurones are activated by synchronized EPSPs remains to be done. 5.2.5. Alternatives to the binding-by-synchronization The central issue in the question of segmentation within a Hebbian cell assembly is that of the selection, among those anatomical connections that are present, of those that have any relevance to the stimulus. Synchronization has been proposed as one possible mechanism, but it is important to ask whether there are other neuronal mechanisms of segmentation. Another possible solution to the segmentation problem, which at the same time provides the correct selection of neuronal pathway, is to suppress the possibilities of cross talk: this is falling back into the labelled line and cardinal neurone hypotheses, that have been largely criticized (for example for the lack of flexibility they imply), and will not be discussed further here. Another possibility to be considered is that the selection among a set of anatomical connections does rely on the inhibition of the activity of neurones that are not relevant to the situation. Centre-surround interactions, that have been well documented in various visual cortical areas (Nelson and Frost, 1975; Allman et al., 1985; Schein and Desimone, 1990; Knierim and Van Essen, 1992; DeAngelis et al., 1994; Lamme, 1995; Sillito et al., 1995), might play this role. In short, centre-surround interactions are such that, as long as no discontinuity is present in a stimulus pattern, the activity of neurones is actively

Time and the brain

100

maintained at a relatively low level. It is only when there are discontinuities that neuronal activities are higher, and therefore are more likely to transmit information to the next processing stage. In this case the transmission of information does not have to depend on the presence of synchronization, whether or not it is oscillatory, and the postsynaptic neurones do not have to behave as coincidence detectors. In the future it will be important to determine, for the same neurone-pairs, which of the two mechanisms (selection by synchronization, selection by inhibition) have the largest impact on neuronal processing. In conclusion, we have shown that there are multiple types of synchronization in cortex. These different forms of synchronization can be related, to some extent, to different states of the cortical network related to sleep and anaesthesia, to information processing, or to attentiveness and expectancies. Synchronization associated with information processing has been proposed as a solution to the binding problem. However, not all experimental results seem to agree with this notion, and alternative mechanisms besides synchronization exist, that could also potentially play a role in segmentation and figure-ground segregation. Whatever the outcome of this question, synchronization of neuronal activity will remain an important phenomenon, for it reflects, in one way or another, the functioning of the cortex.

ACKNOWLEDGEMENTS Thanks to Joshua Brumberg for comments and help with the English. Thanks to Naura Chounlamountri for help with the Figures.

REFERENCES Abeles, M. (1982) Local cortical circuits: An electrophysiological study . Springer Verlag. Abeles, M. (1991) Corticonics: Neuronal circuits of the cerebral cortex . Cambridge University Press. Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993) Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology , 70, 1629–1638. Abeles, M., Prut, Y., Bergman, H., and Vaadia, E. (1994) Synchronization in neuronal transmission and its importance for information processing. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal Coding in the Brain , Springer Verlag, Heidelberg, pp. 39–50. Adrian, E.D. (1935) The electrical activity of the brain. Proceedings of the Royal Society of Medicine , 29 , 197–200. Adrian, E.D., and Matthews, B.H.C. (1934) The interpretation of potential waves in the cortex. Journal of Physiology (London) , 81 , 440–471. Aertsen, A., and Gerstein, G.B. (1985) Evaluation of neuronal connectivity: sensitivity of cross-correlation. Brain Research , 340 , 341–354. Aertsen, A., and Preißl, H. (1990) Dynamics of activity and connectivity in physiological neuronal networks. In: H.G.Schuster (ed.) Nonlinear Dynamics and Neuronal Network , VCH, pp. 281–301.

Cross-correlograms for neuronal spike trains

101

Aertsen, A., Vaadia, E., Abeles, M., Ahissar, E., Bergman, H., Karmon, B., Lavner, Y., Margalit, E., Nelken, I., and Rotter, S. (1991) Neural interactions in the frontal cortex of a behaving monkey: signs of dependence on stimulus context and behavioral state. Journal für Hirnforschung , 32, 735–743. Ahissar, M., Ahissar, E., Bergman, H., and Vaadia, E. (1992a) Encoding of sound-source location and movement: Activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. Journal of Physiology (London), 67, 203–215 Ahissar, E., Vaadia, E., Ahissar, M., Bergman, H., Arieli, A., and Abeles, M. (1992b) Dependence of cortical plasticity on correlated activity of single neurons and on behavioral context. Science, New York , 257, 1412–1415. Allman, J., Miezin, F., and McGuinness, E. (1985) Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception , 14 , 105–126. Amzica, F., and Steriade, M. (1995) Short- and long-range neuronal synchronization of the slow (<1 Hz) cortical oscillation. Journal of Neurophysiology , 73, 20–38. Arnoldi, H.M., and Brauer, W. (1996) Synchronization without oscillatory neurons. Biological Cybernetics , 74, 209–223. Baker, S.N., Olivier, E., and Lemon, R.N. (1997) Coherent oscillations in monkey motor cortex and hand muscle EMG show task-dependent modulation. Journal of Physiology (London) , 501 , 225–241. Baranyi, A., and Szente, M.B. (1987) Long lasting potentiation of synaptic transmission requires postsynaptic modification in the cortex. Brain Research , 423, 378–384. Baranyi, A., Szente, M.B., and Woody, C.D. (1993) Electrophysiological characterization of different types of neurons recorded in vivo in the motor cortex of the cat. I. Patterns of firing activity and synaptic responses. Journal of Neurophysiology , 69, 1850–1864. Bernander, O, Douglas, R.J., Martin, K.A.C., and Koch, C. (1991) Synaptic background activity influences spatiotemporal integration in single pyramidal cells. Proceedings of the National Academy of Sciences, U.S.A. , 88, 11569–11573. Bonhoeffer, T., Staiger, V., and Aertsen, A. (1989) Synaptic plasticity in rat hippocampal slice cultures: Local “hebbian” conjunction of pre- and postsynaptic stimulation leads to distributed synaptic enhancement. Proceedings of the National Academy of Sciences, U.S.A. , 86, 8113–8117. Bouyer, J.J., Montaron, M.F., and Rougeul, A. (1981) Fast fronto-parietal rhythms during combined focused attentive behavior and immobility in cat: cortical and thalamic localizations. Electroencephalograhy and Clinical Neurophysiology , 51 , 244–252. Boven, K.-H., and Aertsen, A. (1990) Dynamics of activity in neuronal networks give rise to fast modulations of functional connectivity. In: R.Eckmiller, G.Hartmann and G.Hauske (eds) Parallel Processing in Neural System and Computers , Elsevier Science Publishers B.V. (North Holland), pp. 53–56. Bringuier, V., Frégnac, Y., Baranyi, A., Debanne, D., and Shulz, D. (1997) Synaptic origin and stimulus dependency of neuronal oscillatory activity in the primary visual cortex of the cat. Journal of Physiology (London) , 500 , 751–774. Brett, B., and Barth, D.S. (1997) Subcortical modulation of high-frequency (gamma range) oscillating potentials in auditory cortex. Journal of Neurophysiology , 78, 573– 581. Brosch, M., Bauer, R., and Eckhorn, R. (1997) Stimulus-dependent modulations of correlated high-frequency oscillations in cat visual cortex. Cerebral Cortex , 7, 70–76. Bryant, H.L., Ruiz Marcos, A., and Segundo, J.P. (1973) Correlation of neuronal spike discharges produced by monosynaptic connections and by common inputs. Journal of

Time and the brain

102

Neurophysiology , 36, 205–225. Burns, B., and Webb, A.C. (1979) The correlation between discharge times of neighbouring neurons in isolated cerebral cortex. Proceedings of the Royal Society of London, series B. , 203, 347–360. Bush, P.C., and Douglas, R.J. (1991) Synchronisation of bursting action potentials discharge in a model network of neocortical neurons. Neural Computation , 3, 19–30. Bush, P., and Sejnowski, T. (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network model. Journal of Computational Neuroscience . 3 , 91–110. Cardoso de Oliveira, S., Thiele, A., and Hoffman, K.-P. (1997) Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. Journal of Neuro science , 17, 9248–9260. Chagnac-Amitai, Y., Luhmann, H.J., and Prince, D.A. (1990) Burst generating and regular spiking layer 5 pyramidal neurons of rat neocortex have different morphological features. Journal of Comparative Neurology , 296, 598–613. Cleland, B.C., Dubin, M.W., and Levick, W.R. (1971a) Simultaneous recording of input and output of lateral geniculate neurones. Nature (New Biology) , 231, 191–192. Cleland, B.G., Dubin, M.W., and Levick, W.R. (1971b) Sustained and transient neurones in the cat’s retina and lateral geniculate nucleus. Journal of Physiology (London) , 217, 473–496. Connors, B.W., Gutnick, M.J., and Prince, D.A. (1982) Electrophysiological properties of neocortical neurons in vitro. Journal of Neurophysiology, 48, 1302–1320. Contreras, D., Destexhe, A., Sejnowski, T.J., and Steriade, M. (1997) Spatiotemporal patterns of spindle oscillations in cortex and thalamus. Journal of Neuroscience , 17, 1179–1196. Cope, T.C., Fetz, E.E., and Matsumura, M. (1987) Cross-correlation assessment of synaptic strength of single Ia fibre connections with triceps surae motoneurones in cats. Journal of Physiology (London) , 390, 161–188. DeAngelis, G.C., Freeman, R.D., and Ohzawa, I. (1994) Length and width tuning of neurons in the cat’s primary visual cortex. Journal of Neurophysiology , 71, 347–374. de Charms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds by the coordination of action potential timing. Nature, London. , 381, 610–613. Destexhe, A., Contreras, D., and Steriade, M. (1998) Mechanisms underlying the synchronizing action of corticothalamic feedback through inhibition of thalamic relay cells. Journal of Neurophysiology , 79, 999–1016. Dickson, J.W., and Gerstein, G.L. (1974) Interactions between neurons in auditory cortex of the cat. Journal of Neurophysiology , 37, 1239–1261. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboek, H.J. (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Biological Cybernetics , 60, 121–130. Eckhorn, R., Frien, A., Bauer, R., Woelbern, T., and Kehr, H. (1993) High frequency (60–90 Hz) oscillations in primary visual cortex of awake monkey. NeuroReport , 5, 2273–2277. Eggermont, J.J. (1992) Neural interaction in cat primary auditory cortex. Dependence on recording depth, electrode separation, and age. Journal of Neurophysiology , 68, 1216– 1228. Eggermont, J.J., Smith, G.M., and Bowman, D. (1993) Spontaneous burst firing in cat primary auditory cortex: age and depth dependence and its effect on neural interaction measures. Journal of Neurophysiology , 69, 1292–1313.

Cross-correlograms for neuronal spike trains

103

Eggermont, J.J. (1994) Neural interaction in cat primary auditory cortex. II. Effects of sound stimulation. Journal of Neurophysiology , 71, 246–270. Eggermont, J.J., and Smith, G.M. (1996) Neuronal connectivity only accounts for a small part of neural correlation in auditory cortex. Experimental Brain Research , 110, 379– 391. Engel, A.K., König, P., Gray, C.M., and Singer W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: inter-columnar interaction as determined by crosscorrelation analysis. European Journal of Neuroscience , 2, 588–606. Engel, A.K., König, P., Kreiter, A.K., and Singer, W. (1991a) Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, New York , 252, 1177–1179. Engel, A.K., Kreiter, A.K., König, P., and Singer, W. (1991b) Synchronization of oscillatory neuronal reponses between striate and extrastriate visual areas of the cat. Proceedings of the National Academy of Science, U.S.A., 88, 6048–6052. Engel, A.K., König, P., and Singer, W. (1991c) Direct physiological evidence for scene segmentation by temporal coding. Proceedings of the National Academy of Science, U.S.A. , 88, 9136–9140. Engel, A.K., König, P., Kreiter, A.K., Schillen, T.B., and Singer, W. (1992) Temporal coding in the visual cortex: new vistas on integration in the nervous system. Trends in Neuroscience , 15, 218–226. Epping, W.J.M., and Eggermont, J.J. (1987) Coherent activity in the auditory midbrain of the grass-frog. Journal of Neurophysiology , 57, 1464–1483. Erb, M., and Aertsen, A. (1992) Dynamics of activity in biology-oriented neural network models: Stability at low firing rates. In: A.Aertsen and V.Braitenberg (eds.) Information processing in the cortex: Experiments and theory , Springer, Heidelberg, pp. 201–223. Fahle, M., and Koch, C. (1995) Spatial displacement, but not temporal asynchrony, destroys figural binding. Vision Research , 35, 491–494. Fetz, E.E., and Gustafsson, B. (1983) Relation between shapes of post-synaptic potentials and changes in firing probability of cat motoneurones. Journal of Physiology (London) , 341, 387–410. Franks, N.P., and Lieb, W.R. (1994) Molecular and cellular mechanisms of general anesthesia. Nature, London , 367, 607–614. Freeman, W.J., and Barrie, J.M. (1994) Chaotic oscillations and the genesis of meaning in cerebral cortex. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 13–37. Frégnac, Y., Shulz, D., Thorpe, S., and Bienenstock, E. (1988) A cellular analog of visual cortical plasticity. Nature, London , 333, 367–370. Frégnac, Y., Bringuier, V., and Baranyi, A. (1994a) Oscillatory neuronal activity in visual cortex: a critical re-evaluation. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 81–102. Frégnac, Y., Burke, J.P., Smith, D., and Friedlander, M.J. (1994b) Temporal covariance of pre- and postsynaptic activity regulates functional connectivity in the visual cortex. Journal of Neurophysiology , 71, 1403–1421. Frien, A., Eckhorn, R., Bauer, R., Woelbren, T., and Kehr, H. (1994) Stimulus-specific fast oscillations at zero phase between areas V1 and V2 of awake monkey. NeuroReport , 5, 2273–2277. Fries, P., Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997)

Time and the brain

104

Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proceedings of the National Academy of Science, U.S.A. , 94, 12699–12704. Frostig, R.D., Gottlieb, Y., Vaadia, E., and Abeles, M. (1983) The effects of stimuli on the activity and functional connectivity of local neuronal groups in the cat auditory cortex. Brain Research , 272, 211–221. Funke, K., and Wörgötter, F. (1995) Temporal structure in the light response of relay cells in the dorsal lateral geniculate nucleus of the cat. Journal of Physiology (London) , 485, 715–737. Gawne, T.J., and Richmond, B.J. (1993) How independent are the messages carried by adjacent inferior temporal cortical neurons? Journal of Neuro science , 13, 2758–2771. Gawne, T.J., Kjaer, T.W., Hertz, J.A., and Richmon, B.J. (1996) Adjacent visual cortical complex cells share about 20% of their stimulus-related information. Cerebral Cortex , 6, 482–489. Gerstein, G.L., Bedenbaugh, P., and Aertsen, A.M.H.J. (1989) Neuronal assemblies. IEEE Transactions on Biomedical Engineering , 36, 4–14. Ghose, G.M., and Freeman, R.D. (1992) Oscillatory discharge in the visual system: Does it have a functional role? Journal of Neurophysiology , 68 , 1558–1574. Ghose, G.M., Ohzawa, I., and Freeman, R.D. (1994) Receptive-field maps of correlated discharge between pairs of neurons in the cat’s visual cortex. Journal of Neurophysiology , 71, 330–346. Ghose, G., and Freeman, R.D. (1997) Intracortical connections are not required for oscillatory activity in the visual cortex. Visual Neuroscience , 14, 963–979. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of awake macaque. Experimental Brain Research , 84 , 505–516. Gray, C.M., and Singer, W. (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 86, 1698–1702. Gray, C.M., König, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-colomnar synchronisation which reflects global stimulus properties. Nature, London . 338, 334–337. Gray, C.M., Engel, A.K., König, P., and Singer, W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: receptive field properties and feature dependence. European Journal of Neuroscience , 2, 607–619. Gray, C.M., and McCormick, D.A. (1996) Chattering cells: Superficial pyramidal neurons contributing to the generation of synchronous oscillations in the visual cortex. Science, New York , 274, 109–113. Gray, C.M., and Viana Di Prisco, G. (1997) Stimulus dependent neuronal oscillations and local synchronization in striate cortex of the alert cat. Journal of Neuroscience , 17, 3239–3253. Griffith, J.S., and Horn, G. (1963) Functional coupling between cells in the visual cortex of the unrestrained cat. Nature, London , 199, p. 876 and 893–895. Guido, W., and Weyand, T. (1995) Burst responses in thalamic relay cells of the awake behaving cat. Journal of Neurophysiology , 74, 1782–1786. Gustafsson, B., and McCrea, D. (1984) Influence of stretch-evoked synaptic potentials on firing probability of cat spinal motoneurones. Journal of Physiology (London) , 347, 431–451. Gustafsson, B., Wigström, H., Abraham, W.C., and Huang, Y.-Y. (1987) Long-term

Cross-correlograms for neuronal spike trains

105

potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials. Journal of Neuroscience , 7, 774–780. Hansel, D., and Sompolinsky, H. (1996) Chaos and synchrony in a model of a hypercolumn in visual cortex. Journal of Computational Neuroscience , 3, 7–37. Harvey, A.R. (1980) A physiological analysis of subcortical and commissural projections of area 17 and 18 of the cat. Journal of Physiology (London) , 302, 507–535. Hata, Y., Tsumoto, T., Sato, H., and Tamura, H. (1991) Horizontal interactions between cortical neurones studied by cross-correlation analysis in the cat. Journal of Physiology (London) , 441, 593–614. Hebb, D.O. (1949) The organization of behavior: A neuropsychological theory . John Wiley & Sons, New York. Holmes, O., and Houchin, J. (1966) Units in the cerebral cortex of the anesthetized rat and the correlations between their discharges. Journal of Physiology (London) , 187, 651–671. Innocenti, G.M. (1980) The primary visual pathway through the corpus callosum: Morphological and functional aspects in the cat. Archives Italiennes de Biologie , 118, 124–188. Jagadeesh, B., Gray, C.M., and Ferster, D. (1992) Visually evoked oscillations of membrane potential in cells of cat visual cortex. Science, New York , 257, 552–554. Jahnsen, H., and Llinás, R. (1984) Electrophysiological properties of guinea-pig thalamic neurons, an in vitro study. Journal of Physiology (London) , 349, 205–226. Jester, J.M., Campbell, L.W., and Sejnowski, T.J. (1995) Associative EPSP-spike potentiation induced by pairing orthodromic and antidromic stimulation in rat hippocampal slices. Journal of Physiology (London) , 484, 689–705. Jones, R.S.G., and Heinemann, U. (1988) Synaptic and intrinsic responses of medial enthorinal cortical cells in normal and magnesium-free medium in vitro. Journal of Neurophysiology , 59 , 1476–1496. Keifer, J.C., Baghdoyan, H.A., Becker, L., and Lydic, R. (1994) Halothane decreases pontine acetylcholine release and increases EEG spindles. NeuroReport , 5, 577–580. Kim, U., Bal, T., and McCormick, D.A. (1995) Spindle waves are propagating synchronized oscillations in the ferret LGNd in vitro. Journal of Neurophysiology , 74, 1301–1323. Kiper, D.C., Gegenfurtner, K.R., and Movshon, J.A. (1996) Cortical oscillatory responses do not affect visual segmentation. Vision Research , 36, 539–544. Kirkwood, P.A., and Sears, T.A. (1978) The synaptic connexions to intercostal motoneurones as revealed by the average common excitation potential. Journal of Physiology (London) , 275, 103–134. Kirkwood, P.A., and Sears, T.A. (1982) The effect of single afferent impulses on the probability of firing of external intercostal motoneurones in the cat. Journal of Physiology (London) , 322, 315–336. Knierim, J.J., and Van Essen, D.C. (1992) Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. Journal of Neurophysiology , 67, 961–980. Knox, C.K. (1974) Cross-correlation functions for a neuronal model. Biophysical Journal , 14 , 567–582. Koch, C., and Schuster, H. (1992) A simple network showing burst synchronization without frequency locking. Neural Computation , 4, 211–223. König, P., and Schillen, T.B. (1991) Stimulus-dependent assembly formation of oscillatory responses: I.Synchronization. Neural Computation , 3, 155–166. König, P., Engel, A.K., and Singer, W. (1995) Relationship between oscillatory activity

Time and the brain

106

and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 92, 290–294. König, P., Engel, A.K., and Singer, W. (1996) Integrator or coincidence detector? the role of cortical neurons revisited. Trends in Neuroscience , 19 , 130–137. Kreiter, A.K., and Singer, W. (1992) Oscillatory neuronal responses in the visual cortex of the awake monkey. European Journal of Neuroscience , 4, 369–375. Kreiter, A.K., and Singer, W. (1996) Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. Journal of Neuroscience , 16 , 2381–2396. Krüger, J., and Aiple, F. (1988) Multimicroelectrode investiguation of monkey striate cortex: Spike train correlations in the infragranular layers. Journal of Neurophysiology , 60 , 798–828. Krüger, J., and Mayer, M. (1990) Two types of neuronal synchrony in monkey striate cortex. Biological Cybernetics , 64 , 135–140. Kruse, W., and Eckhorn, R. (1996) Inhibition of sustained gamma oscillations (35–80 Hz) by fast transient responses in cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 93, 6112–6117. Lamme, V.A.F. (1995) The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience , 15, 1605–1615. Levick, W.R., Cleland, E.G., and Dubin, M.W. (1972) Lateral geniculate neurons of cats: Retinal inputs and physiology. Investigative Ophthalmology , 11 , 302–311. Li, C.-L. (1959) Synchronization of unit activity in the cerebral cortex. Science, New York , 129, 783–784. Livingstone, M.S. (1996) Oscillatory firing and interneuronal correlations in squirrel monkey striate cortex. Journal of Neurophysiology , 75, 2467–2485. Llinás, R.R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10- to 50-Hz frequency range. Proceedings of the National Academy of Science, U.S.A. , 88, 897–901. Llinás, R., Ribary, U., Joliot, M., and X.-J. Wang (1994) Content and context in temporal thalamocortical binding. In: G. Buzsáki, R. Llinás, W. Singer, A. Berthoz and Y. Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 251– 272. Lopes da Silva, F.H., van Rotterdam, A., Storm van Leeuwen, W., and Tielen, A.M. (1970) Dynamic characteristics of visual evoked potentials in the dog. I. Beta frequency selectivity in evoked potentials and background activity. Electroencephalography and Clinical Neurophysiology , 29 , 260–268. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997a) Neuronal dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cerebral Cortex , 7, 207–227. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997b) Neuronal dynamics in a model of the thalamocortical system. II. The role of neuronal synchrony tested through perturbations of spike timing. Cerebral Cortex , 7, 228–236. Lytton, W.W., and Sejnowski, T.J. (1991) Simulation of cortical pyramidal neurons synchronized by inhibitory interneurons. Journal of Neurophysiology , 66 , 1059–1079. Markram, H. (1997) A network of tufted layer 5 pyramidal neurons. Cerebral Cortex , 7, 523–533. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, New York , 275, 213–215.

Cross-correlograms for neuronal spike trains

107

Mason, A., and Larkman, A. (1990) Correlations between morphology and electrophysiology of pyramidal neurons in slices of rat visual cortex. II. Electrophysiology. Journal of Neuroscience , 10, 1415–1428. Mastronarde, D.N. (1983a) Correlated firing of cat retinal ganglion cells. I. Spontaneously active inputs to X- and Y-cells. Journal of Neurophysiology , 49 , 303– 324. Mastronarde, D.N. (1983b) Correlated firing of cat retinal ganglion cells. II. Responses of X-and Y-cells to single quantal events. Journal of Neurophysiology , 49, 325–349. Matsumura, M., Chen, D.-F., Sawaguchi, T., Kubota, K., and Fetz, E.E. (1996) Synaptic interactions between primate precentral cortex neurons revealed by spike-triggered averaging of intracellular membrane potentials in vivo. Journal of Neuroscience , 16 , 7757–7767. McCormick, D.A., Connors, B.W., Lighthall, J.W., and Prince, D.A. (1985) Comparative electro-physiology of pyramidal and sparsely spiny stelate neurons of the neocortex. Journal of Neurophysiology , 54, 782–806. McCormick, D.A., and Bal, T. (1997) Sleep and arousal: thalamocortical mechanisms. Annual Review of Neuroscience , 20 , 185–215. Meister, M., Lagnado, L., and Baylor, D.A. (1995) Concerted signaling by retinal ganglion cells. Science, New York , 270, 1207–1210. Melssen, W.J., and Epping, W.J.M. (1987) Detection and estimation of neuronal connectivity based on crosscorrelation analysis. Biological Cybernetics , 57, 403–414. Michalski, A., Gerstein, G.L., Czarkowska, J., and Tarnecki, R. (1983) Interactions between cat striate cortex neurons. Experimental Brain Research , 51 , 97–107. Milner, P.M. (1974) A model for visual shape recognition. Psychological Review , 81 , 521–535. Moore, G.P., Segundo, J.P., Perkel, D.H., and Levitan, H. (1970) Statistical signs of synaptic interaction in neurons. Biophysical Journal , 10, 876–900. Munk, M.H.J., Nowak, L.G., Nelson, J.I., and Bullier, J. (1995) The structural basis of cortical synchronization. II. Localization of the common input. Journal of Neurophysiology , 74, 2401–2414. Munk, M.H.J., Roelfsema, P.R., König, P., Engel, A.K., and Singer, W. (1996) Role of reticular activation in the modulation of intracortical synchronization. Science, New York , 272, 271–274. Murphy, J.T., Kwan, H.C., and Wong, Y.C. (1985) cross-correlation studies in primate motor cortex: synaptic interaction and shared input. Canadian Journal of Neurological Science , 12 , 11–23. Murthy, V., and Fetz, E.E. (1992) Coherent 25- to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys. Proceedings of the National Academy of Science, U.S.A. , 89, 5670–5674. Murthy, V.N., and Fetz, E.E. (1996) Synchronization of neurons during local field potential oscillations in sensorimotor cortex of awake monkeys. Journal of Neurophysiology , 76, 3968–3982. Nelson, J.I., and Frost, B.J. (1978) Orientation-selective inhibition from beyond the classical receptive field. Brain Research , 139 , 359–365. Nelson, J.I., Salin, P.A., Munk, M.H.-J., Arzi, M., and Bullier, J. (1992) Spatial and temporal coherence in cortico-cortical connections: A cross-correlation study in area 17 and 18 in the cat. Visual Neuroscience , 9, 21–37. Neuenschwander, S., and Singer, W. (1996) Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature, London , 379 ,

Time and the brain

108

728–733. Neven, H., and Aertsen, A. (1992) Rate coherence and event coherence in the visual cortex: a neuronal model of object recognition. Biological Cybernetics , 67 , 309–322. Noda, H., and Adey, W.R. (1970) Firing of neuron pairs in cat association cortex during sleep and wakefulness. Journal of Neurophysiology , 33, 672–684. Noda, H., and Adey, W.R. (1973) Neuronal activity in the association cortex of the cat during slepp, wakefulness and anesthesia. Brain Research , 54, 243–259. Nowak, L.G., Munk, M.H.J., James, A.C., Nelson, J.I., and Bullier, J. (1995) The structural basis of cortical synchronization. I. 3 types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–2400. Nowak, L.G., Sanchez-Vives, M.V., and McCormick, D.A. (1997) Influence of low and high frequency fluctuations on spike timing in visual cortical neurons. Cerebral Cortex , 7, 487–501. Nowak, L.G., and Bullier, J. (1998a) Axons, but not cell bodies, are activated by electrical stimulation in the cortical gray matter. I. Evidence from chronaxie measurements. Experimental Brain Research , 118, 477–488. Nowak, L.G., and Bullier, J. (1998b) Axons, but not cell bodies, are activated by electrical stimulation in the cortical gray matter. II. Evidence from selective inactivation of cell bodies and axon initial segment.Experimental Brain Research , 118, 489–500. Nowak, L.G., Munk, M.H.J., James, A.C., Girard, P., and Bullier, J (1999) Crosscorrelation study of the temporal interactions between areas V1 and V2 of the macaque monkey. Journal of Neurophysiology . 81:1057–1074. Nuñez, A., Amzica, F., and Steriade, M. (1992) Voltage-dependent fast (20–40 Hz) oscillations in long-axoned neocortical neurons. Neuroscience , 51, 7–10. Pasternak, G. (1993) Pharmacological mechanisms of opioid analgesics. Clinical Neuropharmacology , 16 , 1–18. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967a) Neuronal spike trains and stochastic point processes. I. The single spike train. Biophysical Journal , 7, 391–418. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967b) Neuronal spike trains and stochastic point processes. II. simultaneous spike trains. Biophysical Journal , 7, 419– 440. Pinault, D., and Deschênes, M (1992) Voltage-dependent 40 Hz oscillations in rat reticular thalamic neurons in vivo. Neuroscience , 51, 245–258. Pockberger H. (1991) Electrophysiological and morphological properties of rat motor cortex neurons in vivo. Brain Research. 539, 181–190. Reid, R.C., and Alonso, J.-M. (1995) Specificity of monosynaptic connections from thalamus to visual cortex. Nature, London , 378, 281–284. Renaud, L.P., and Kelly, J.S. (1974) Identification of possible inhibitory neurons in the pericruciate cortex of the cat. Brain Research , 79, 9–28. Reyes, A.D., and Fetz, E.E. (1993) Effects of transient depolarizing potentials on the firing rate of cat neocortical neurons. Journal of Neurophysiology , 69 , 1673–1683. Riehle, A., Grün, S., Diesmann, M., and Aertsen, A. (1997) Spike synchronization and rate modulation differentially involved in motor cortical function. Science, New York , 278, 1950–1953. Ritz, R., Gerstner, W., Fuentes, U., and van Hemmen, J. (1994) A biologically motivated and analytically soluble model of collective oscillations in the cortex. II. Application to binding and pattern segmentation. Biological Cybernetics , 71, 349–358. Robson, J.G., and Troy, J.B. (1987) Nature of the maintained discharge of Q, X, and Y

Cross-correlograms for neuronal spike trains

109

retinal ganglion cells of the cat. Journal of the Optic Society of America , Series A, 4, 2301–2307. Rodieck, R.W. (1967) Maintained activity of cat retinal ganglion cells. Journal of Neurophysiology , 30, 1043–1071. Roelfsema, P.R., König, P., Engel, A.K., Sireteanu, R., and Singer, W. (1994) Reduced synchronization in the visual cortex of cats with strabismic amblyopia. European Journal of Neuroscience , 6, 1645–1655. Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997) Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature, London , 385, 157–161. Rougeul, A., Bouyer, J.J., Dedet, L., and Debray, O. (1979) Fast somato-parietal rhythms during combined focal attention and immobility in baboon and squirrel monkey. Electroencephalography and Clinical Neurophysiology , 46, 310–319. Sanes, J.N., and Donoghue, J.P. (1993) Oscillations in local field potentials of the primate motor cortex during voluntary movement. Proceedings of the National Academy of Science, U.S.A. , 90 , 4470–4474. Schein, S.J., and Desimone, R. (1990) Spectral properties of V4 neurons in the macaque. Journal of Neuroscience , 10, 3369–3389. Schindler, K.A., Bernasconi, C.A., Stoop, R., Goodman, P.H., and Douglas, R.J. (1997) Chaotic spike patterns evoked by periodic inhibition in rat cortical neurons. Zeitschrift für Naturforschung . 52a, 509–512. Schwarz, C, and Bolz, J. (1991) Functional specificity of a long-range horizontal connection in cat visual cortex: A cross-correlation study. Journal of Neuroscience , 11, 2995–3007. Shadlen, M.N., and Newsome, W.T. (1994) Noise, Neural codes and cortical organization. Current Opinion in Neurobiology , 4, 569–579. Sillito, A.M., Jones, H.E., Gerstein, G.L., and West, D.C. (1994) Feature linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London , 369, 479–82. Sillito, A.M., Grieve, K.L., Jones, H.E., Cudeiro, J., and Davies, J. (1995) Visual cortical mechanisms detecting focal orientation discontinuities. Nature, London , 378, 492– 496. Softky, W., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. Journal of Neuroscience , 13, 334–350. Sporns, O., Gally, J.A., Reeke, G.N., and Edelman G.M. (1989) Reentrant signaling among simulated neuronal groups leads to coherency in their oscillatory activity. Proceedings of the National Academy of Science, U.S.A. , 86, 7265–7269. Steriade, M., Nuñez, A., and Amzica, F. (1993a) A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. Journal of Neuroscience , 13, 3252–3265. Steriade, M., Contreras, D., Curro Dossi, R., and Nuñez, A. (1993b) The slow (<1 Hz) oscillation in reticular thalamic and thalamocortical neurons: scenario of sleep rhythm generation in interacting thalamic and neocortical networks. Journal of Neuroscience , 13, 3284–3299. Steriade, M., Curró Dossi, R., and Contreras, D. (1993c) Electrophysiological properties of intralaminar thalamocortical cells discharging rhythmic (~ 40 Hz) spike-bursts at ~ 1000 Hz during waking and rapid eye movement sleep. Neuroscience , 56, 1–9. Steriade, M., Nuñez, A., and Amzica, F. (1996a) Synchroniszation of fast (30–40 Hz)

Time and the brain

110

spontaneous cortical rhythms during brain activation. Journal of Neuroscience , 16, 392– 417. Steriade, A., Contreras, D., Amzica, F., and Timofeev, I. (1996b) Synchronization of fast (30–40 Hz) spontaneous oscillations in intrathalamic and thalamocortical networks. Journal of Neuroscience , 16, 2788–2808. Steriade, M., Timofeev, I., Dürmüller, N., and Grenier, F. (1998) Dynamic properties of corticothalamic neurons and local cortical interneurons generating fast rhythms (30–40 Hz) spike bursts. Journal of Neurophysiology , 79, 483–90. Swadlow, H.A. (1995) Influence of VPM afferents on putative interneurons in S1 of the awake rabbit: evidence from cross-correlation, microstimulation, and latencies to peripheral sensory stimulation. Journal of Neurophysiology , 73, 1584–1599. Tallon-Baudry, C., Bertrand O., Delpuech C., and Pernier, J. (1996) Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. Journal of Neuroscience , 16, 4240–4249. Tallon-Baudry, C., Bertrand, O., Delpuech, C., and Pernier, J. (1997) Oscillatory gammaband (30–70 Hz) activity induced by a visual search task in human. Journal of Neuroscience , 17, 722–734. Tallon-Baudry, C., Bertrand, O., Peronnet, F., and Pernier, J. (1998) Induced gammaband activity during the delay of a visual short-term memory task in humans. Journal of Neuroscience , 18, 4244–4254. Tamura, H., Sato, H., Katsuyama, N., Hata, Y., and Tsumoto, T. (1996) Less segregated processing of visual information in V2 than in V1 of the monkey visual cortex. European Journal of Neuroscience , 8, 300–309. Tanaka, K. (1983) Cross-correlation analysis of geniculostriate neuronal relationship in cats. Journal of Neurophysiology , 49 , 1303–1318. Thomson, A.M., and Deuchars, J. (1997) Synaptic interactions in neocortical local circuits: Dual intracellular recordings in vitro. Cerebral Cortex , 7, 510–522. Thorpe, S., Fize, D., and Marlot, C. (1996) Speed of processing in the human visual system. Nature, London , 381, 520–522. Tononi, G., Sporns, O., and Edelman, G.M. (1992) Reentry and the problem of integrating multiple cortical areas: simulation of dynamic integration in the visual system. Cerebral Cortex , 2, 310–335. Toyama, K., Kimura, M., and Tanaka, K. (1981a) Cross-correlation analysis of interneuronal connectivity in cat visual cortex. Journal ofNeurophysiology , 46, 191– 201. Toyama, K., Kimura, M., and Tanaka, K. (1981b) Organization of cat visual cortex as investigated by cross-correlation technique. Journal of Neurophysiology , 46, 202–214. Troy, J.B., and Robson, J.G. (1992) Steady discharges of X and Y retinal ganglion cells of cat under photopic illuminance. Visual Neuro science , 9, 535–553. Troyer, T.W., and Miller, K.D. (1997) Physiological gain leads to high ISI variability in a simple model of a cortical regular spiking cell. Neural Computation , 9, 971–983. Ts’o, D.Y., Gilbert, C.D., and Wiesel, T.N. (1986) Relationship between horizontal interactions and functinal architecture in cat striate cortex as revealed by crosscorrelation analysis. Journal of Neuroscience , 6, 1160–1170. Vaadia, E., and Abeles, M. (1987) Temporal firing patterns of single units, pairs and triplets of units in the auditory cortex. Israel Journal of Medical Science , 23, 75–83. Vaadia, E., Ahissar, E., Bergman, H., and Lavner, Y. (1991) Correlated activity of neurons: A neural code for higher brain functions? In: Krüger, J. (ed) Neuronal Cooperativity , Springer-Verlag Berlin Heidelberg, 1991, 205–279.

Cross-correlograms for neuronal spike trains

111

Vaadia, E., Haalman, I, Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London , 373, 515–518. van Vreeswijk, C., and Sompolinski, H. (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, New York , 274, 1724–1726. von der Malsburg, C. (1981) The correlation theory of brain function. Internal Report 81–2, Dept. of Neurobiology, Max-Plank-Institute for Biophysical Chemistry, 3400 Göttingen, W-Germany. von der Malsburg, C., and Schneider, W. (1986) A neural cocktail-party processor. Biological Cybernetics , 54, 29–40. Wang, X.-J., and Buzsáki, G. (1996) Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. Journal of Neuroscience , 16, 6402–6413. Wang Roe, A., and Ts’o, D.Y. (1997) The functional architecture of area V2 in the macaque monkey. In: K.E.Rockland, J.H.Kaas and A.Peters (eds), Cerebral Cortex, Volume 12, Extrastriate Cortex in primate , Plenum Press, New York and London, pp. 295–257. Wilson, M.A., and Bower, J.M. (1991) A computer simulation of oscillatory behavior in primary visual cortex. Neural Computation , 3, 498–509. Whittington, M.A., Traub, R.D., and Jeffreys, J.G.R. (1995) Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, London , 373, 612–615. Young, M.P., Tanaka, K., and Yamane, S. (1992) On oscillating neuronal responses in the visual cortex of the monkey. Journal of Neurophysiology , 67, 1464–1474.

3 The Space-Time Continuum in Mammalian Sensory Pathways Asif A.Ghazanfar and Miguel A.L. Nicolelis Department of Neurobiology, Box 3209 Duke University Medical Center, Durham, NC 27710 correspondence to: Miguel A.L.Nicolelis, M.D., Ph.D, Department Neurobiology, Box 3209, Duke University Medical Center, 101 Research Drive, Bryan Research Building, Durham, NC, 27710 USA Phone: 919–684–4580; FAX: 919–684–5435; e-mail: [email protected]

Animals in their natural environments actively process spatiotemporally complex sensory signals in order to guide adaptive behaviour. It therefore seems likely that the properties of both single neurones and neural ensembles should reflect the dynamic nature of such interactions. We review several recent studies that demonstrate the existence of timedependent receptive fields and distributed coding of sensory signals by neurones in the auditory, visual and somatosensory thalamocortical pathways—features that seem to be constrained by the need to interpret actively time-varying stimuli. Based on these results, the central nervous system does not appear to operate as a static decoder, composed of highly-specialized feature-detector neurones, which sits idly until inputs from the environment are conveyed through it. On the contrary, the evidence indicates that perception emerges as the outcome of highly distributed interactions between populations of broadly-tuned neurones throughout the multiple and reciprocally-connected subcortical and cortical areas that define a sensory system. KEYWORDS: Neural ensembles; Coding; Spatiotemporal; Thalamocortical; Feedback “The trace of any activity is not an isolated connexion between sensory and motor elements. It is tied in with the whole complex of spatial and temporal axes of nervous activity which forms a constant substratum of behaviour”—Lashley, 1950.

1. INTRODUCTION It is no longer tenable to ignore the fact that the temporal structure of both natural signals

The space-time continuum in mammalian sensory pathways

113

and exploratory behaviours used by animals in their ecological niche constrain the mechanisms employed by the central nervous system to acquire, process, and decode sensory information. Until recently, however, very few neurophysiological theories of perception incorporated these dynamic aspects of sensory information processing. Instead, the emphasis of mainstream neural coding theories was placed on the existence of static and topographic mapping functions, which had as their central elements highly specialized “feature extracting” single neurones (Barlow, 1972; Hubel and Wiesel, 1962). The fact that sensory cues in the natural environment typically have intricate timevarying properties, and that animals can perceive such sensory stimuli very rapidly, suggests that the mammalian central nervous system is capable of processing complex time-dependent signals very efficiently. For example, speech and other vocalization signals have unique spectro-temporal signatures in which the timing and order of basic components (e.g. syllables, phonemes, motifs, etc.) play a critical role in distinguishing the meaning of different sounds. Thus, neural circuits have to decode the temporal structure of these signals in order to allow the subject to perceive and respond adaptively to them. The necessity of having time-dependent neural processes underlying perception is also indicated by the fact that, in natural settings, animals gather information about the surrounding environment by actively moving their sensory receptor arrays as they engage in exploratory behaviours. For example, during active exploration of their haptic environments, primates employ stereotypical movements of their hands to perceive minute tactile differences in object size, texture, and shape. When primates voluntarily move their fingers across an object, their ability to discriminate a variety of tactile features is improved considerably when compared to the passive reception of the same features (Darian-Smith, 1984). Analogous dependencies between active exploration and optimal sensory discrimination have been observed in other sensory modalities, such as saccadic eye movements during visual search and object recognition or head orientation movements for sound localization and recognition. Thus, most of the sensory information gleaned from the environment is sampled through actions, and leads to the genesis of percepts that seem to unfold continuously over time rather than in a series of discrete events. In this review, we address one of the central questions in modern neuroscience: how does the central nervous system rely on time-dependent computations to process behaviourally relevant sensory signals? In the last decade or so, many laboratories have dealt with a variety of questions that revolve around this central theme. Here, an attempt is made to summarize some of these new results. In the first half of this chapter, we describe how the discovery of spatiotemporal receptive fields across the thalamocortical loop has revealed a new dimension of sensory information-coding, which emphasizes the ability of the CNS to process time-varying signals during exploratory behaviours. In the second half of this chapter, we discuss how time-dependent neural coding schemes can be exploited by populations of neurones in the visual, auditory, and somatosensory thalamocortical pathways. The data reviewed here suggest that the presence of stimulus-dependent changes in the temporal structure of neuronal spike trains can be used to encode different types of sensory information. The temporal precision required in most of these schemes is of the

Time and the brain

114

order of 5–50 milliseconds, a time-scale very similar to those used to describe the spatiotemporal organization of thalamic and cortical neuronal receptive fields.

2. DYNAMIC RECEPTIVE FIELDS 2.1. Beyond the Classical Receptive Field Since the early work in the visual system (Hartline, 1950), the analysis of the receptive field (RF) properties of single neurones has occupied centre stage in sensory neurophysiology and, as a consequence, has guided the definition of a variety of cardinal principles in the investigation of sensory systems. Classically, the RF of a neurone is defined in terms of spatial co-ordinates (for the visual and somatosensory system), spectral co-ordinates (in the case of the auditory system), or by an arbitrary parameter space that has behavioural relevance (such as a set of odours or tastes for the olfactory and gustatory systems). Simply put, the RF of a neurone has been defined as the area of the particular “parameter space” within which a stimulus can elicit a neuronal response (measured in terms of the number of action potential) which is significantly above the spontaneous firing rate. Although the response properties of neurones are inherently both spatial and temporal, most studies have focused only on the spatial aspects of the RF of single neurones. This is evident from the voluminous amounts of work that have focused on “mapping” the organization of topographic representations of sensory space found in cortical and subcortical areas of the brain. As a result of the limited scope of these studies, neurones with particular spatial or spectral RF properties have been characterized simply as static filters of sensory inputs—filters that describe a neurone’s selectivity for features such as orientation or frequency tuning. The development of new RF mapping techniques has facilitated the characterization of RFs in both the spatial and temporal domains. Application of these methods in different sensory systems has demonstrated the existence of time-dependent RFs in multiple processing levels of the neuraxis. The central results of all these studies was the demonstration that it is inadequate to describe stimulus selectivity or representation in terms of the maximal or average spike counts averaged over a large time epoch, because the structure of RFs varies as a function of time. 2.2. Dynamic Frequency Tuning of Auditory Cortical Neurones: Adaptations to Complex Sound Processing In no other system has the role of time-dependent variables been emphasized more than in the auditory system. Over the years, information embedded in the form of time duration, interval and temporal order of stimulus features has been demonstrated to convey important information-bearing parameters for auditory signal processing in different species. Elegant studies (Hauser et al., 1998; Knudsen and Konishi, 1979; Margoliash and Fortune, 1992; Suga et al., 1983) have established that different nervous

The space-time continuum in mammalian sensory pathways

115

systems use a variety of temporal cues (i.e. interaural time difference, echolocation, sequences of sounds, etc.) to interpret auditory signals which are used to guide adaptive behaviours. In this context, even a small change in the temporal structure of an auditory signal can have a dramatic effect on its behavioural significance. Despite this wealth of information, very few neurophysiological studies have addressed how the spectrotemporal information embedded in complex sounds is reflected in the receptive fields of single neurones in the mammalian auditory system. In general, studies of the auditory cortex have focused on neuronal responses to simple stimuli, such as pure tunes and broad-band noise at different intensities. Only a handful of studies have been carried out to examine the time-dependence of frequency tuning in the mammalian auditory cortex. These studies have revealed that auditory cortical neurones can exhibit “spectrotemporal” RF, i.e. their frequency tuning may vary as a function of post-stimulus time. In other words, these neurones have “multi-peaked” tuning curves, such that they respond best to a particular frequency interval at one post-stimulus time, and to another frequency range at a later post-stimulus epoch. Thus, whereas most neurones in the peripheral auditory pathways have RFs that respond to a clearly definable frequency range, there is a small proportion of cortical neurones whose RFs contain two or more frequencies that can drive the neurones at the same intensity level (Abeles and Goldstein, 1972; Sutter and Schreiner, 1991). In a systematic study of these neurones, Sutter and Schreiner (1991) presented 675 different frequency-level combinations to each primary auditory cortical neurone in a pseudo-random order. The approach was very similar to the “response plane” technique used in some visual RF mapping studies (see below). These authors found that the two frequencies that elicit robust responses in multi-peaked neurones are often separated by frequencies to which the neurones do not respond at the same intensity levels. Moreover, the bandwidth of each peak in a multi-peaked RF was not significantly different from that of single-peaked neurones. The authors found that there were latency differences between peaks, and that these differences were highly variable; most often, however, the lowest frequency stimulus induced the shortest latency neuronal response, while the highest frequency stimulus led to the longest latency responses. Thus, these auditory cortical neurones seem to exhibit what one would consider a spectrotemporal RF. This means that the frequency tuning of the neurone changes as a function of post-stimulus time, or that the frequency response and time are coupled. Interestingly, the spectral and temporal ranges of these multi-peaked RFs corresponded to the ranges of behaviourally-relevant auditory signals for the species used for these studies. This raised the question of how these spectrotemporal RFs relate to how auditory neurones respond to natural auditory signals. Studies of the auditory responses to vocalizations in squirrel monkeys have demonstrated that neurones in the auditory cortex are sensitive to species-specific calls and have unique firing patterns for different call types (Winter and Funkenstein, 1973; Wollberg and Newman, 1972). Auditory cortical neurones responded with temporally-complex patterns of discharges, and for a given call type, a variety of temporal firing patterns were observed among different cortical neurones. Likewise, a given cortical neurone could respond to different call types. Editing out parts of a particular vocalization pattern indicated that single cortical neurones could dynamically integrate information from different call components into

Time and the brain

116

their firing patterns (Wollberg and Newman, 1972). Of the cortical neurones that responded to vocalizations, the majority of them responded to steady pure tones as well. Winter and Funkenstein (1973) reported that the responses to vocalizations could be predicted qualitatively by the neurone’s response to simpler stimuli, such as clicks and pure tones. More recent experiments on call-selective neurones in the auditory cortex of anaesthetized rhesus (Rauschecker et al., 1995) and marmoset monkeys (Callithrix jacchus) (Wang et al., 1995) have largely supported the latter results from squirrel monkeys. In the marmoset, primary auditory neurones phaselocked to the envelope portion of the vocalization, which was centred around the neurone’s frequency tuning (Wang et al., 1995). In the rhesus monkey, when different segments of a particular vocalization were used as stimuli for a call-selective neurone, the response could also be predicted by the frequency tuning curve of the neurone (Rauschecker et al., 1995). Thus, the spectral and temporal nature of a particular auditory neurone’s RF can be used, at least to a first approximation, to predict the firing patterns of a neurone to behaviourally-relevant vocal signals. Further testing of this assumption was performed by mapping the RF of primate auditory cortical neurones with reverse-correlation techniques, and by generating corresponding synthetic sounds (deCharms et al., 1998; Tramo et al., 1996). In these cases, the auditory stimuli used for the reverse-correlation consisted of rapid sequences of tones or chords. As mentioned above, for cats, in the primate primary auditory cortex, neurones have RFs whose best-frequency varies as a function of time. In addition, these spectro-temporal RFs can be used to a certain extent to predict the responses of these neurones to synthetic versions of species-specific vocalizations (Tramo et al., 1996). In the primary auditory cortex of the owl monkey, deCharms et al. (1998) showed that neurones have complex patterns of spectrotemporal RFs that indicate preferences for stimulus edges, stimulus transitions in frequency or intensity, and conjunctions of different stimulus features. To test the integrative properties of these neurones, different classes of auditory stimuli were generated based on the structure of their spectro-temporal RFs. Once these synthetic stimuli were presented they were able to drive the corresponding auditory cortical neurone at much higher rates than simple stimuli, the when the optimal stimulus was expanded or contracted in either the spectral or temporal domain (deCharms et al., 1998). These results support the notion that spectrotemporal RF properties reflect the solution found by the central nervous system for processing and interpreting natural, time-varying, and behaviourally-relevant auditory signals. In this sense there is a reciprocal relationship between the dynamic structure of single neurone RFs and the time-varying signature of natural sounds. Indeed, the responses to natural vocal signals could be used to predict, to some extent, the spectrotemporal structure of a neurone’s RF, and, conversely, the spectrotemporal RF could be used to predict a given neurone’s response to complex synthetic sounds. 2.3. Receptive Field Dynamics in the Lateral Geniculate Nucleus Traditionally, the RFs of lateral geniculate nucleus (LGN) neurones have been described

The space-time continuum in mammalian sensory pathways

117

in purely spatial terms as circular with an “on-centre”, which responds to the onset of a bright stimulus, and an “off-surround”, which responds to the onset of a dark stimulus (or the off-set of a bright stimulus). LGN neurones of the opposite polarity exist as well. Perhaps, the earliest study suggesting that time was an important dimension in the structure of LGN RFs was conducted by Stevens and Gerstein (1976). To characterize the RF of LGN neurones, these authors used the “response plane” technique, a method in which the visually-responsive area of a neurone is divided into a matrix and the spiking of the neurone elicited by a brief flash of light (a spot, or any other stimulus feature) is measured at each position of the matrix. A number of post-stimulus time histograms (equal to the size of the stimulus matrix) are generated and then used to compile the temporal sequences of responses. Using this approach, Stevens and Gerstein (1976) found that the simple, spatial description of cat LGN RFs, such as “on-off” and “centresurround”, was insufficient to describe completely even the simplest of LGN RFs. Overall, these authors reported two different types of LGN RFs. One type displayed spatially homogeneous distributions of excitatory and inhibitory domains, and another type exhibited spatially heterogeneous excitatory and inhibitory domains, which shifted as a function of post-stimulus time. These different dynamic behaviours were hypothesized to originate from two different functional streams, X and Y, within the cat visual system. Thus, these authors state that, “time is at least as important an analytical variable as space to visual neurones” (Stevens and Gerstein, 1976) emphasizing that the coupling between space and time is an integral component of LGN RFs. More recent experiments have supported the original results of Stevens and Gerstein by demonstrating the existence of spatiotemporally coupled visual RFs in both the cat (Dan et al., 1996) and primate LGN (Golomb et al., 1994). These studies employed “reverse-correlation” techniques to map the spatiotemporal response function of a given neurone. In this approach, the structure of the receptive field of a single neurone can be described fully by the measured response to any complete set of stimuli presented in rapid, pseudo-random sequences. One complete set of stimuli is defined by localized flashes of light presented in several random sequences. The average spike count of the neurone is then correlated with the past location and time of each flash, i.e. “reversecorrelated”. In the primate LGN, Golomb et al. (1994) found that both parvocellular and magnocellular neurones had spatiotemporal RFs. Thus, although these neurones still exhibited centre-surround organization, the response to the surround was delayed relative to the response of the centre. Thus, the RFs of these neurones could only be described as space-time inseparable, since at a given moment in time the overall spatial distribution of the RF was different from those observed in a previous or a future time epoch. These RFs could also be used to predict the neurone’s response to complex pattern stimuli (Golomb et al., 1994; see below). Do the spatiotemporal response properties of LGN neurones elicited by simple stimuli relate to their function in coding and representing natural visual signals? This is an important issue, because the demonstration that the spatiotemporal structure of a single neurone RF could be used to predict the neurone’s temporal coding of different visual stimuli would support the notion that the coupling of spatial and temporal dimensions are relevant for sensory information processing. This question was addressed by experiments done in the cat LGN, where the RF structure of single neurones was used to predict the

Time and the brain

118

responses to natural, time-vary ing visual stimuli (Dan et al., 1996). Natural scenes are highly redundant, having both spatial and temporal correlations. Because coding efficiency could be advantageous, both evolutionarily and computationally, it was hypothesized that LGN neurones would decorrelate natural scene stimuli in order to transform this information into a less redundant signal (Atick and Redlich, 1990). This was tested by recording the activity of LGN neurones in response to natural, time-varying scenes and measuring the power spectra of the neural signals (Dan et al., 1996). Indeed, the responses of LGN neurones to movies of natural scenes revealed that their individual spike trains were temporally decorrelated, that is their power spectra were essentially flat from 3Hz to 15Hz in response to natural scenes. Thus, although the natural scenes have intrinsic correlations in their structure, the LGN neuronal responses do not reflect these redundancies and, hence, code the information more efficiently. Furthermore, these authors reported that this decorrelation of LGN responses to natural scenes could only be predicted by the filtering properties of their spatiotemporal RFs (Dan et al., 1996). Thus, in both cat and primate the time-dependent RF structures of LGN neurones have been used to predict the responses and temporal coding of stimuli, for both artificial (Golomb et al., 1994) and natural stimuli (Dan et al., 1996). In both of these studies, feedforward circuitry was hypothesized to account for the dynamic behaviour LGN RFs. However, no experimental test of this “feedforward” hypothesis has been carried out thus far. 2.4. Dynamic Organization of the Primary Visual Cortex Traditionally, visual cortical neurones are classified according to their response selectivity and the shapes of their RFs. “Simple” cells of the primary visual cortex receive most of the LGN inputs, and have RFs that are classically described as being oriented spatially. These RFs exhibit subregions that respond primarily to bright or dark elongated stimuli. “Complex” cells respond to both bright and dark stimuli anywhere within their RFs. Recently, approaches similar to those used for mapping the spatial and temporal properties of LGN neurones have been used to characterize the RFs of visual cortical neurones, in both the cat and the monkey. Both response-plane and reverse-correlation approaches have demonstrated that the RF structure of visual cortical neurones is not static, but varies as function of post-stimulus time. These studies revealed that primary visual cortical neurones also have spatiotemporal RFs whose time-dependent structure can be measured by using a variety of stimulus parameters (overall RF size, location of on-off regions, orientation tuning, directional tuning, and length tuning). Receptive field mapping, in both space and time, has shown that V1 neurones respond to flashes of light in different subregion of their RFs at different times. Using the response plane technique, Dinse et al. (1991) reported that each subfield within the RF of visual cortical neurones, for both on- and off-responses, exhibits its own temporal characteristics. When each response-plane is plotted in sequence (from 20 to 200 milliseconds), a wave of excitation seems to travel and spread across the entire RF. Using the reverse-correlation analysis, where the stimuli consisted of rapidly-flashed patterns of

The space-time continuum in mammalian sensory pathways

119

spots or bars, DeAngelis et al. (1993) found that simple cells of the cat visual cortex have bright and dark excitatory elongated subregions that move as a function of time. The elongated on-off subregions of these spatiotemporal RFs could be used to predict the responses of these simple cells’ responses to bars of light (DeAngelis et al., 1995). In addition to the time-dependent organization of on-off responses, other properties of visual cortical neurones, such as orientation tuning, end-stopping, and directional selectivity, also show a robust time-dependence on a millisecond scale. The first evidence for this time-dependence was provided by Dinse et al. (1991), who employed flickering bars of light of different lengths to investigate the time course of orientation tuning and end-stopping in cat visual cortical neurones. Classically, orientation tuning is defined simply as the selectivity of neuronal responses to bars of a particular orientation, while end-stopping is the selectivity to bars of a particular length. Compiling the sequences of tuning curves for each of these stimulus features in post-stimulus time-steps of 20 milliseconds, these authors demonstrated that the selectivity of single neurones to these features evolved over post-stimulus time. For instance, in the case of orientation tuning, some neurones first exhibited a state of excitation that did not show selectivity for any particular orientation, followed by the gradual sharpening of the selectivity over time. Other neurones showed selectivity for a particular orientation that remained stable over time, while for yet another group of neurones, the selectivity changed over time (i.e. responding best to one orientation at one time epoch, and another orientation at a later time epoch). Nearly identical results were found for length-tuning (Dinse et al., 1991). A laminar comparison of dynamic orientation tuning in the macaque primary visual cortex was performed by applying the reverse-correlation technique in the orientation domain. Ringach et al. (1997) found that orientation selectivity developed after approximately 30–45 milliseconds, and persisted for up to 85 milliseconds over poststimulus time. Comparing the nature of orientation selectivity across different layers of macaque V1, these authors observed that the degree of dynamic orientation tuning depended on the laminar position of the neurones. The cortical layers which received direct input from the LGN (layers 4Ca and 4Cb) contained neurones which showed a single, broadly-tuned orientation preference, that did not change over time, while neurones in the output layers of V1 (layers 2, 3, 4B, 5 and 6) showed a range of dynamic behaviours. In these “output” layers, orientation tuning was narrower than neurones of the input layers, and their preferred orientations changed over post-stimulus time. Furthermore, as reported by Dinse et al. (1991) for cat visual cortex, a subset of neurones in V1 showed orientation tuning that increased in sharpness over time (Ringach et al., 1997). Taken together, the results from these experiments indicate that the neurones in the primary visual cortex are not simply a collection of static spatial filters or feature detectors; instead, their RF size and feature selectivity vary as function of post-stimulus time. 2.4.1. Potential circuit mechanisms underlying visual cortical dynamics An important and unresolved question concerns the neural circuitry underlying the

Time and the brain

120

genesis of visual spatiotemporal receptive fields. Both feedforward and feedback mechanisms have been proposed to account for the type of cortical and subcortical dynamics described above. Feedforward mechanisms could include the convergence of multiple cortical neurones, which do not exhibit spatiotemporal RFs, upon neurones which do. Alternatively, feed-forward inputs from two different classes of LGN neurones, each with different temporal dynamics (lagged versus non-lagged LGN neurones [Saul and Humphrey, 1992]) could account for some of the time-dependency of RFs in V1. As proposed by DeAngelis et al. (1995), simultaneous, multi-site recordings of LGN and V1 neurones (Reid and Alonso, 1995) are needed to test these hypotheses. Feedback cortical circuitry, on the other hand, could account for the dynamic feature tuning and the increases in selectivity of V1 neurones observed over poststimulus time (Dinse et al., 1991; Ringach et al., 1997). For orientation tuning and end-stopping, many neurones displayed orientation tuning which became narrower over time. It is conceivable that this increase in selectivity results from a “sculpturing” process, carried out by the asynchronous convergence of multiple feedback projections to the same set of V1 neurones (Dinse et al., 1991). According to this hypothesis, the initial state of widespread excitation of the cortex caused by the onset of the stimulus would be followed by reverberating activity from nearby local circuitry that could increase the selectivity of cortical neurones. In addition to local circuitry, other cortical areas could contribute, via feedback projections, to the definition of the RF structure of V1 neurones at longer latencies (Mignard and Malpeli, 1991). Overall, these results from both the LGN and visual cortex indicate that spatiotemporal RFs properties are likely to exist at most, if not all, processing levels of the visual system, and that this time-dependency in single neurone firing likely results from the asynchronous convergence of many inputs from different feedforward and feedback projections to individual neurones. To test such a hypothesis, one would need to carry out simultaneous multi-site recordings of neuronal responses (i.e. from thalamus and cortex) that are obtained before, during, and after selective inactivation of feedforward and/or feedback pathways. Experiments of this type are more feasible using the rat somatosensory pathway as a model system. 2.5. Spatiotemporal Receptive Fields Across the Somatosensory Thalamocortical Loop 2.5.1. Spatiotemporal receptive fields in the somatosensory thalamus Ensembles of well-isolated single neurones from the ventral posterior medial (VPM) nucleus of the thalamus, and the posteromedial region of the primary somatosensory (SI) cortex, of awake and anaesthetized rats have been simultaneously recorded using a technique based upon chronically-implanted arrays of microwires (Nicolelis and Chapin, 1994; Nicolelis et al., 1997a). These brain regions are known to process tactile information coming from the large facial whiskers on the rat snout. As in traditional studies, the RFs of thalamic and cortical neurones were reconstructed in these studies,

The space-time continuum in mammalian sensory pathways

121

following repetitive mechanical deflections of individual whiskers. For this analysis, a single neurone’s response to the stimulation of ~5x5 matrix of whiskers (deflected one at a time) was divided into 5 millisecond post-stimulus epochs. For each epoch, the whisker that elicited the greatest response (in terms of spike counts) was defined as the “RF centre” for that time epoch. As the “centre” whisker often varied as a function of poststimulus time, we were able to map spatiotemporal RFs for most neurones in both the VPM and SI cortex (Nicolelis and Chapin, 1994; Ghazanfar et al., 1995). This approach is identical to the response plane techniques used to map spatiotemporal RFs in the visual system (Dinse et al., 1991; Stevens and Gerstein, 1976). The chronic and simultaneous neural ensemble recording approach is uniquely suited to carrying out a quantitative analysis of the response properties of neurones across the thalamocortical pathway. First, the stable recordings obtained using the chronic placement of microwires allows one to sample from populations of well-isolated single neurones for several hours at a time, permitting the use of a large stimulus set (5 whiskers) and a large number of trials (360) per site. Second, since many neurones are recorded in parallel, several non-stationary effects (e.g. anaesthesia, placement of stimulus, etc.) can be reduced or eliminated. Finally, this approach permits one to record from the exact same set of neurones before, during and after experimental manipulations, such as deafferentation and/or pharmacological inactivation of different pathways. Traditionally, the RF of a somatosensory neurone is defined as the area of the body surface that, when stimulated, triggers a significant increase in this neurone’s firing rate. As in other species, most classical electrophysiological studies of the “whisker region” of rodent somatosensory system have focused on the characterization of the spatial attributes of the RF of neurones located in the subcortical and cortical processing centres of the trigeminal pathway. Recently, Nicolelis, Chapin and their colleagues (Nicolelis et al., 1993a; Nicolelis and Chapin, 1994) employed simultaneous neural ensemble recordings to sample the sensory responses of populations of neurones distributed across the whisker representation area of the main thalamic nucleus of the trigeminal somatosensory system, the VPM. These authors carried out a detailed analysis of the RFs of individual VPM neurones in both awake and anaesthetized animals, and revealed that single VPM neurones have large, multiwhisker RFs whose centre is defined by one whisker whose stimulation elicits the strongest sensory response at the shortest latency. This whisker is classically known as the principal whisker (PW) of the RF, and is commonly used to identify the location of a given neurone in the topographic maps of the whisker pad observed across the trigeminal system. Further analysis, however, revealed that the location of the PW (or RF centre) of a given VPM neurone could vary as a function of poststimulus time. Indeed, VPM neurones could be divided into two functional classes, according to the time-dependency of their RF centres. In about 41% of the recorded VPM neurones, the spatial position of the RF centre shifted over poststimulus time (Figure 1, bottom panel). The remaining neurones exhibited RF centres that remained in the same whisker over time (Nicolelis and Chapin, 1994).

Time and the brain

122

Figure 1 . Top panel. Spatiotemporal receptive field of a cortical neurone located in layer V of the rat primary somatosensory cortex. Bottom panel Spatiotemporal receptive field of neurone located in the ventral posterior medial nucleus of the rat thalamus. Notice that the spatial domain of these receptive fields varies as a function of post-stimulus time.

The first class of neurones exhibited a spatiotemporal coupling of their RFs, because the position of the RF centre, and its surround, varied as a function of post-stimulus time. In other words, the PW in the short latency component (PWSL) was different from the PW in the long latency component (PWLL) of the response. Interestingly, all spatiotemporal VPM RFs exhibited time-dependent RF shifts of centres in a caudal-to-rostral direction, and, consequently, were termed C->R cells. For example, one particular C->R neurone responded best to its PWSL, whisker C1, at 5–10 milliseconds post-stimulus time, while still responding significantly to many other caudal and rostral whiskers which defined its RF surround. By 15–20 milliseconds, the strongest response for this neurone was no longer elicited by whisker C1, but by whisker B3, which became the PW of the RF at long-latency (or PWLL). Thus, the RF shift observed in the VPM was clearly defined by a smooth transition in the spatial distribution of sensory responses observed in the first 30 ms of post-stimulus time. In general, C->R neurones had the largest RFs (average of 17.2 whiskers). The time-dependent shifting of the centres of these cell’s RFs seemed to evolve gradually, and appears to involve two factors: 1) an early component composed of short-latency responses from the caudal whiskers; and 2) a time-dependent

The space-time continuum in mammalian sensory pathways

123

enhancement of responses to more rostral whiskers. Conversely, a second class of VPM neurone was observed, which did not exhibit any evident time-dependent shift in their RFs. Invariably, this group of neurones had their PWs located in the rostral part of the whisker pad, which contains small and immobile vibrissae. These neurones were labelled “rostral position (RP) cells”. The RFs of these neurones was characterized by the existence of a single whisker (the PW) which elicited the largest response at all post-stimulus times, and by the presence of surrounding whiskers which constituted the RF surround. 2.5.2. Dynamic receptive fields in the primary somatosensory cortex Classically, neurones located within a given cortical barrel column are believed to respond preferentially to the deflection of one “principal” whisker (PW), and to fire less vigorously to stimulation of several adjacent whiskers that define the surround region of the neurone receptive field (SRF) (Armstrong-James and Fox, 1987). As in the VPM, responses to the PW occur at shorter latencies than responses to SRFs; the distribution of response latencies to centre and surround whiskers is clearly bimodal. In a series of experiments, Ghazanfar and Nicolelis, (1999) investigated whether the differences in latency observed between a stimulus applied to a PW or the SRFs could define spatiotemporal RF in the rat SI cortex. This was an interesting question to ask, because until recently most studies of rat SI cortex had focused primarily on the spatial aspects of cortical RF organization. Although multi-whisker RFs had been described in the rat SI before (Chapin, 1986; Simons, 1978), and the existence of a time-dependent spread of cortical recruitment had been proposed (Armstrong-James et al., 1992), the spatiotemporal organization of SI cortical neurone RFs was never characterized. Ghazanfar and Nicolelis, (1999) analyzed in detail the time-dependent RF dynamics of 197 cortical neurones located in layer V of the rat SI cortex. Overall, these authors observed that, for the vast majority of these neurones, the spatial structure of their RFs changed as a function of post-stimulus time (Figure 1, top panel). However, the patterns of these SI cortical spatiotemporal RFs varied widely, unlike those reported for the VPM, which shifted primarily in a caudal to rostral direction, or not at all (Nicolelis and Chapin, 1994). We identified four directions of SI cortical RF shifts: 1) rostral-to-caudal (RC); 2) caudal-to-rostral (CR); 3) dorsal-to-ventral (DV); and 4) ventral-to-dorsal (VD). Neurones whose RFs traversed an equal distance in two or more directions were defined as unclassifiable (UC). Out of the 197 neurones in our sample, 175 (88.8%) exhibited time-dependent shifts in their RF centres, while 22 (11.2%) did not. Of those with spatiotemporal RFs, 17.7% were RC shifters, 16.0% CR shifters, 23.4% DV shifters, and 6.3% VD shifters. The remaining 36.6% (64 out of 175 neurones) could not be classified as having a directional bias. In this study, Ghazanfar and Nicolelis, (1999) also demonstrated that the direction of the spatiotemporal RF shift was dependent upon the location of the PWSL. If a neurone’s PWSL was located in the dorsal part of the whisker pad, then the neurone was more likely to exhibit a DV spatiotemporal RF. If a neurone’s PWSL was in the more rostral part of the whisker pad, then it was more likely to exhibit a RC spatiotemporal RF. In summary,

Time and the brain

124

the authors observed that the large, asymmetrical RFs of layer V SI cortical neurones, and the variations in minimal latencies for surround whiskers, define complex spatiotemporal RFs in which the centre (or the PW) varies as function of post-stimulus time. Four classes of spatiotemporal RF types could be characterized based on directional biases of the RF movement, and these biases were, in part, dependent upon the location of the PWSL. Therefore, the existence of such time-dependent changes in RF structure suggests that populations of SI neurones located in layer V provide rather dynamic representations of the whisker pad, which incorporates both spatial and temporal domains of tactile sensory responses. 2.5.3. Hypothesis concerning the origins of spatiotemporal RFs in the somatosensory thalamocortical pathway The organization of VPM and SI cortical layer V RFs indicated that both thalamic and cortical neurones exhibit a variety of directional biases and complex spatiotemporal coupling. In their original studies (Nicolelis et al., 1993a; Nicolelis and Chapin, 1994), Nicolelis and Chapin postulated that the dynamic nature of these RFs resulted from the asynchronous convergence of feedforward and feedback pathways on to VPM neurones. The same hypothesis was put forth later by Ghazanfar and Nicolelis, (1999) to account for the genesis of spatiotemporal RFs in the SI cortex. For both VPM and SI cortical neurones, asynchronous inputs could arise from parallel, feedforward inputs from the ascending lemniscal and paralemniscal pathways. These pathways arise from the trigeminal brain-stem nuclei, and are known to have different temporal lags. Such a mechanism would be similar to that proposed for the genesis of spatiotemporal RFs in the visual system (DeAngelis et al., 1995). Moreover, feedback connections from a variety of cortical somatosensory areas also converge upon VPM and SI neurones (Fabri and Burton, 1991; Beck et al., 1998). These cortical feedback projections could also contribute to the time-dependent changes in RF centre, observed in both the VPM and the SI cortex. For instance, examination of the VPM C->R RF shifts suggested that these RFs could be broken down into early and late temporal components. These different temporal components of the RFs could emerge as the result of the temporally-asynchronous convergence of multiple excitatory and inhibitory afferents to VPM neurones. In this model, the early component of VPM RFs would result from inputs from the fast ascending somatosensory lemniscal pathways that include afferents from the principal and spinal trigeminal complex of the brainstem (which terminate on the same neurones in the VPM). The late component is a little more difficult to account for, but potential sources of input would include slower-conducting trigeminal pathways, and the very dense excitatory corticothalamic inputs that converge on the distal dendrites of VPM neurones. Although the spatioternporal properties of cortical neurones are likely to vary according to their laminar location, the dynamic RFs of SI cortical neurones described above are also likely to arise from the asynchronous convergence of multiple afferents on the same population of cortical neurones. Such inputs could arise from parallel, feedforward inputs from the lemniscal and paralemniscal pathways, which converge onto

The space-time continuum in mammalian sensory pathways

125

layer V neurones at different timelags, from intracortical feedback connections within SI (Gottlieb and Keller, 1997; Hoeflinger et al., 1995), or from other sensory and motor cortical areas (Fabri and Burton, 1991; Koralek et al., 1990). 2.5.3.1. Testing the asynchronous hypothesis for generation of thalamic spatiotemporal RFs Two types of experiments were employed to test whether an “asynchronous” convergence of inputs could account for the generation of spatiotemporal RFs in the rat VPM nucleus. First, the contribution of feedforward somatosensory pathways to the genesis of their RFs was tested by inducing reversible deafferentation of the whisker pad, through subcutaneous infusions of a local anaesthetic, lidocaine. This local anaesthetic allowed measurements of changes in RF organization which emerge as a consequence of altering the flow of tactile information through ascending feedforward trigeminal somatosensory pathways to the VPM and SI cortex (Nicolelis et al., 1993b; Faggin et al., 1997; Nicolelis, 1997). As predicted, this reversible peripheral deafferentation induced immediate reorganization in both the spatial and temporal properties of VPM and SI RFs. The two major effects were 1) a temporal shift resulting in the elimination or reduction of early, short-latency component of RFs, located within the “anaesthetized zone”, with the enhancement of the later components, and 2) a spatial shift where RFs that were originally centred in the “anaesthetized region” of the whisker pad shifted, immediately following the induction of the deafferentation, to neighbouring territories of the whisker pad. These experiments suggest that peripheral deafferentation largely effects the early, “feedforward” components of the VPM and SI cortical RFs. In a second series of experiments, the potential role of corticofugal projections in the genesis of spatiotemporal structure of VPM RFs was investigated (Ghazanfar et al., 1997; Krupa et al., 1999). The results obtained in these studies were complimentary to those reported for peripheral deafferentation (Nicolelis et al., 1993b; Faggin et al., 1997). Using a combination of multi-site chronic microwire arrays and a stereotaxically-placed cannula, Ghazanfar et al. (1997) and Krupa et al. (1999) were able to inactivate the SI cortex pharmacologically, using the GABA agonist, muscimol. Since the cannula was placed adjacent to a chronically implanted microwire array in the SI cortex, these authors were able to monitor the onset, duration, and tangential spread of muscimol. These experiments revealed that corticofugal projections contribute very importantly to the genesis of longer latency neuronal responses in the VPM nucleus, since, in most cases, blockade of cortical activity attenuates these later response components. As mentioned before, these long-latency components are critical to the definition of spatiotemporal RFs in VPM (Nicolelis and Chapin, 1994), and for the C->R RF shifts observed in this nucleus, since the PWLL (principal whisker or RF centre at long-latency) differs from the one defined at short-latency (the PWSL). Furthermore, it was also observed that cortical feedback projections can account for up to 50% of the immediate plastic effects observed in the VPM following peripheral deafferentation (Krupa et al., 1999). As described above, the results obtained with both peripheral deafferentation and inactivation of corticofugal projections support the asynchronicity hypothesis, and

Time and the brain

126

demonstrate that spatiotemporal RFs, at least in the rat VPM thalamus, are defined by large-scale interactions between multiple brain structures. 2.5.4. Active tactile exploration is required for the genesis of spatiotemporal RFs How do the spatiotemporal structures of VPM and SI RFs emerge in the first place? Upon the discovery of directional RF shifting in the rat VPM, a series of studies tested the possibility that experience-dependent mechanisms, via active whisker movements occurring from the time of early postnatal life, could account for the “sculpturing” of directional bias into the spatiotemporal structure of VPM and cortical RFs. Rats use rhythmic protractions of their long caudal whiskers to explore objects that are placed in front of them (Welker, 1964), while the small, immobile rostral whiskers are used primarily to establish and maintain contact with objects. Because active tactile exploration plays a critical role in the normal development of haptic perception in mammals (Bushnell and Boudreau, 1991), Nicolelis et al. (1996) and Oliveira et al. (1997) postulated that the spatiotemporal RFs of layer V SI cortical and VPM thalamic neurones in the rat probably reflect the pattern of spatiotemporal whisker inputs that occurs during the active “whisking” of object surfaces during natural behaviours. Patterned tactile stimulation generated by active exploration during postnatal development may be required for the normal functional organization of the somatosensory system. To test this hypothesis, the effects of neonatal disruption of active whisker movements on the development of thalamic and cortical spatiotemporal RFs was investigated (Nicolelis et al., 1996; Oliveira et al., 1997). In rats, the facial muscles used during whisker movements are innervated solely by branches of the facial nerve, which is clearly separated from the infraorbital nerve, a branch of the trigeminal nerve, which carries low-threshold tactile information from the facial whiskers. Because of this anatomical peculiarity, it was possible to eliminate the production of active whisker movements in neonatal rats by sectioning the facial nerve before these animals experienced any sort of active whisker movements. Complete peripheral nerve section prevented rats from making active whisker movements on one side of the face throughout their development and adulthood, without affecting the sensory innervation—via the infraorbital nerve—of their facial vibrissae. After the rats reached adulthood, the spatiotemporal structure of somatosensory RFs was examined and it was observed that preventing the expression of active tactile exploration during development dramatically alters the spatiotemporal RF organization of VPM and SI neurones. First, the size of neuronal RFs in both structures was significantly reduced. In the VPM, for instance, the reduction was on average from 13 whiskers (normal animals) to 6 whiskers (in animals with facial nerve section). The spatiotemporal organization of RFs was also disrupted in the VPM and SI cortex. For example, in normal adult rats 100% of VPM neurones whose PWSL were located in the caudal part of the whisker pad exhibited a caudal-to-rostral shift in their RF centres over post-stimulus time. In rats with neonatal facial nerve sections, on the other hand, only 19% of VPM neurones with PWSL in the caudal whisker pad exhibited time-dependent shifts in their RF centres (Nicolelis et al., 1996). In other words, the dynamic properties of VPM neurones were dramatically reduced by

The space-time continuum in mammalian sensory pathways

127

preventing these animals from actively exploring their environment during development. Therefore, these results underscore the importance of active movements for the functional maturation of normal spatiotemporal RFs, in both cortical and subcortical relays of the rat somatosensory system, and implicate experience-dependent mechanisms for the establishment of spatiotemporal RFs in the somatosensory system.

3. SPATIOTEMPORAL CODING BY SINGLE NEURONES AND NEURAL ENSEMBLES 3.1. Rate Versus Temporal Coding? Although the analysis of single neurone RFs continues to play an important role in classical neurophysiology, more contemporary views recognize that the RF of a neurone offers only a probabilistic description of the “potential” responses that can be exhibited by that neurone within the parameter space of a particular type of sensory stimulus (e.g. all the facial whiskers, or all the orientations of a bar of light). Thus, the investigation of how neurones encode information derived from a subset of the parameter space of a sensory stimulus does not rely exclusively on the analysis of single neurone RFs. The investigations of neural codes revolves around identifying a measurement or metric of neuronal activity to describe a particular stimulus parameter, behaviour, or cognitive skill. Neural codes are also studied in the context of how different areas of the brain exchange information and relate to each other. Many neural metrics have been proposed to operate as candidate neural codes in different neural structures and in different species. In general, most of these algorithms have been dichotomized as variations of two main themes: “rate” or “temporal” codes (Shadlen and Newsome, 1994). The prevailing view that firing rate is the universal code was first advocated by Adrian (1928), who suggested that a neural code is simply the mean firing rate of the action potentials evoked from a neurone, and rate variability is “noise”. Because this noise can be filtered out, or reduced, by averaging across time or by recording the activity of large populations of neurones, a rate code performs very well in the presence of noise, but has limited information-carrying capacity. In contrast to the classical rate code hypothesis, multiple schemes have been proposed in which temporal aspects of neuronal firing play a significant role in conveying information. Temporal coding schemes include measurements based on: 1) the precise timing of individual spikes (in a 1–3 millisecond range); 2) the precise temporal sequence of firing across a population of neurones (in a range of <5 millisecond); 3) the occurrence of widely distributed synchronous neuronal firing activity at frequency ranges varying from 1–100 Hz (periods between >10 ms and <1000 ms); 4) the time-dependent modulation of firing rate exhibited by single neurones; and, 5) the occurrence of stimulus or behaviour-related spatiotemporal patterns of firing across populations of neurones (ranging from 10 to 100 ms). Although the jury is still out and not likely to return an early verdict, it seems fair to say that the discussion between rate versus temporal coding is an artificial one. First,

Time and the brain

128

most of the schemes of temporal coding actually represent a particular case of a rate code, which is limited to a particular time epoch. On the other hand, as discussed above, the high level of stimulus selectivity of neurones seems to vary over time, and thus, the identity of a stimulus cannot be ascertained simply by time-averaging each neurone’s response function. This suggests that a strict rate code does not seem to do justice to the richness of neuronal dynamics observed throughout the central nervous system. Instead, the coding of stimuli seems to be best represented when the temporal modulation of the sensory response is taken into account. 3.2. Temporal Coding of Space by Single Neurones 3.2.1. The presence of stimulus-related information in the response waveform ofLGN neurones For many decades, the dominant assumption in visual physiology has been that the strength of a visual neurone’s response is the only variable which contains relevant information about the stimulus (Barlow, 1972). Consequently, the description of temporal structures in the sensory responses of neurones was used simply to classify different LGN cell types (Marrocco, 1976) or to investigate how these neurones integrate their inputs (Dawis et al., 1984). More recently, however, studies using complex patterns of visual stimulation have demonstrated that neurones in the visual pathway respond to a variety of different stimulus attributes. These studies have also revealed that, by including the temporal structure of their sensory responses as an additional dimension of coding, one can increase the information carrying capacity of these neurones. Using behaving primates performing a fixation task, McClurkin et al. (1991a,b) studied the visual responses of single neurones in the parvocellular layers of LGN to a set of two-dimensional black-and-white patterns, called Walsh patterns, that could be used for the linear reconstruction of any picture of the same resolution (in this case, 8 x 8 pixels). The neuronal responses to these different patterns were characterized by examining the strength and temporal pattern of the responses using principle component analysis (PCA). These authors found that parvocellular LGN neurones displayed a variety of temporal firing patterns, and that these firing patterns varied in a reproducible way as a function of the patterns presented. The differences in firing patterns included changes in the magnitude, width, and number of peaks seen in the initial response evoked by the stimulus, as well as changes in the magnitude of the sustained, longer-latency component of the responses. These results were interpreted as an indication that the time-dependent modulation of neuronal firing, i.e. the temporal distribution of LGN spikes, might also encode information in addition to the total number of spikes fired in a given epoch of time (McClurkin et al., 1991a). McClurkin et al. (1991b) then employed PC A to quantify the differences in temporal response patterns elicited by the set of Walsh patterns. By definition, PCs are linearly weighted polynomia which are orthogonal to each other. By correlating the derived PCs with different attributes of neuronal responses, these authors showed that the first

The space-time continuum in mammalian sensory pathways

129

principal component was highly correlated with the overall strength of the response. Higher principal components either were related to the shape of the response (i.e. the temporal distribution of spikes) or identified the particular stimulus eliciting the response. To assess how much stimulus-related information was carried simply in the firing rate versus both the magnitude and shape of the response, information theory analysis was applied to the response strength alone versus the first three principal components, which carry both response strength and temporal patterning information. This analysis revealed that the inclusion of both the response strength and temporal distribution of spikes elicited by a given stimulus increased the information carrying capacity of LGN neurones 1.5 times when compared to the response strength alone. Based on these results, the authors hypothesized that this form of temporal coding cannot be explained by traditional models of LGN neurone properties, and proposed that these neurones and other neurones along the visual pathway should be considered as spatiotemporal filters. To test this hypothesis directly, Golomb et al. (1994) analyzed the spatiotemporal RF properties of primate LGN neurones and used this information to predict their responses to the Walsh patterns used by McClurkin et al. (1991a,b). They found that the spatiotemporal RF properties could indeed be used to predict the temporal response patterns seen by McClurkin et al. (1991a,b). Similarly, information theory analysis revealed that these spatiotemporal firing patterns could be used to distinguish between different Walsh stimuli on the basis of a single response. Therefore, these two studies (McClurkin et al., 1991a,b; Golomb et al., 1994) provided a direct link between the spatiotemporal RF properties and temporal coding capacity of LGN neurones. 3.2.2. Stimulus-dependent nature of temporal coding in the visual cortex Using an experimental approach identical to that used to study LGN neurones McClurkin et al. (1991a,b), Richmond et al. (1990) and Richmond and Optican (1990) studied the potential coding dimensions of primate primary visual cortical neurones. Overall, the results were very similar to those found for LGN neurones. Responses of both simple and complex V1 neurones showed robust differences in their response strength, and temporal modulation to different Walsh patterns (Richmond et al., 1990). Once again, principal component and information theory analyses revealed that the nature of the stimulus could best be predicted by taking into account both the response magnitude and the temporal pattern of spiking (Richmond and Optican, 1990). These authors also reported that V1 complex neurones exhibited substantially different temporal response profiles when Walsh patterns (or oriented bars) and their contrast-reversed mates were employed as stimuli. Along similar lines, McClurkin and Optican (1996) reported that the number of spikes elicited by V1, V2 and V4 primate visual cortical neurones varied only for one or the other of the stimulus types (colour vs. pattern). However, the temporal distribution of spikes was affected by both stimulus types, suggesting that neurones in multiple visual cortical areas can encode multiple stimulus features simultaneously, both through changes in number of spikes and by time-dependent modulation of firing rates (McClurkin and Optican, 1996; Richmond et al., 1990). Put in another way, these results indicate that different features of a stimulus do not need to be encoded by distinct

Time and the brain

130

populations of neurones, each devoted to a particular stimulus attribute. Instead, the same population of neurones could encode multiple stimulus attributes simultaneously. Using an entirely different analytical approach, Victor and Purpura (1996) have proposed that temporal coding of stimuli in V1 and V2 visual cortical areas of primates performing a fixation task is dependent upon the nature of the stimulus. These authors developed a metric to compare the similarities between two spike trains, based on three basic parameters: the total number of spikes, the intervals between spikes, and the absolute spike time. Each of these parameters was then used to calculate the distance between all pairs of spikes within each data-set. By varying the width of their analysis window, these authors were able to characterize parametrically the precision of spike timing across different stimulus conditions. Their analyses revealed that five different stimulus attributes (contrast, check size, orientation, spatial frequency, and texture) could elicit responses from V1 and V2 neurones that varied systematically in both the number of spikes and their temporal patterns (Victor and Purpura, 1996). Interestingly, there was evidence for temporal coding for all the stimulus attributes tested, but the precision of the temporal code varied. For instance, contrast stimuli elicited spike trains with a ~ 10–30 millisecond precision, while texture stimuli elicited responses with a ~ 100 millisecond precision. Like the results of Richmond et al. (1990), these findings suggest that multiple stimulus attributes could be represented simultaneously in the spike trains of visual cortical neurones. For example, at a high temporal resolution, stimulus contrast could be encoded, while at a lower resolution, the same spike trains could encode texture (Victor and Purpura, 1996). 3.2.3. Coding of sound location by the temporal discharge patterns of single auditory cortical neurones In addition to identifying different types of sounds such as vocalization, the auditory system must also determine the source of sounds. In the visual and somatosensory systems, it is presumed that stimulus location, at least at early stages of these pathways (i.e. up to the level of the primary sensory cortices) is somehow represented by the orderly retinotopic or somatotopic maps found in these brain regions. Although neuropsychological studies have demonstrated that lesions of the auditory cortex result in sound localization deficits (Wortis and Pfeiffer, 1948), no analogous topographic maps, formed by neurones sharply-tuned to sound-source location, have been found in the auditory cortex (Middlebrooks et al., 1998). For instance, auditory cortical neurones in the anterior ectosylvian sulcus of the cat respond to noise bursts, and are broadly-tuned for sound source location (Middlebrooks et al., 1994, 1998). Thus, by using only their average firing rates, one cannot get a very precise estimation of the location of a sound. Middlebrooks et al. (1994, 1998) tested whether the patterns of spiking of single neurones could more reliably encode the location of a sound. An artificial neural network (ANN) was trained to classify the temporal spike patterns of single auditory cortical neurones to sounds coming from different locations in azimuth. Surprisingly, it was found that the spiking pattern of single neurones could encode the location of a sound across 360 degrees of azimuth at more than twice the level of chance. In other words, the

The space-time continuum in mammalian sensory pathways

131

neurones were not tuned to a single “best area” of sound localization. When the resolution of the spike trains was decreased beyond 1 millisecond, the ability of the ANN to discriminate different patterns degraded. Based on these results, Middlebrooks et al. (1998) argued that the organization of the auditory system is uniquely suited to preserving the temporal fidelity of sounds. Nevertheless, as they pointed out, since the performance of single neurones was still below the psychophysical ability of the animal to locate sounds (Jenkins and Masterton, 1982), it is likely that a distributed population of neurones is necessary to identify the precise location of a sound in a real world scenario. 3.3. Dynamic and Distributed Representations of Sensory Inputs by Neural Ensembles The presence of neurones with large receptive fields, whose tuning changes as a function of time, across different sensory pathways, makes it apparent that even the simplest of stimuli is likely to elicit temporally-complex responses from large populations of cortical and sub-cortical neurones. The distinction between temporal and rate coding, in this context, can be seen only as a heuristic dichotomy, since both measures supply one with important information about different stimuli. It, therefore, seemed reasonable to focus our experimental efforts to measure the dynamics of populations of neurones distributed across a given brain region, or multiple brain regions simultaneously, in order to address the question of how time is used by the central nervous system to encode information. With the recent advent of electrophysiological and analytical techniques for multielectrode recordings (Nicolelis, 1998 in press), multi-site neural ensemble recordings have sparked renewed interest in the properties of neural assemblies and their potential roles in brain function. Much of this has focused on pair-wise analysis of neurones, but in some instances larger-scale approaches have been successful in recording and analyzing small ensembles across different brain structures (Ghazanfar and Nicolelis, 1997; Nicolelis et al., 1997a). Therefore, it is not surprising that many laboratories have begun applying neural ensemble recordings to investigate how neuronal populations encode sensory and motor information. In the next sections, some of this recent work carried out in the mammalian visual, auditory, and somatosensory systems is reviewed. 3.3.1. Ensemble coding of visual features Due to the extensive interconnections between neurones both within and across multiple brain structures, even the simplest stimuli will elicit a response from a large population of neurones. This gives rise to the problem of how multiple stimuli can be represented by overlapping populations of neurones. To address this problem, a relational neural code based on synchronous neuronal firing has been suggested as a solution. According to this hypothesis, the joint activity of neurones could allow the many different features of a particular stimulus to be “bound” together; i.e. “feature extractor” neurones should fire synchronously in order to represent the multitude of stimulus features that define a visual scene (Singer and Gray, 1995). In support of this contention, multi-electrode experiments

Time and the brain

132

have been carried out in a variety of visual structures, and these studies have shown that, under certain conditions, neurones can synchronize their firing in response to global stimulus features. In the visual cortex of the anaesthetized cat, Gray and Singer (1989) showed that neurones in a cortical column fire synchronously during the presentation of a moving bar of light. From this result, they hypothesized that synchronous firing may occur between neurones in two spatially distinct cortical columns. To test this idea, they recorded multiunit responses to oriented bars simultaneously from several spatially separate sites in cat primary visual cortex (Gray et al., 1989). Using cross-correlations, they demonstrated that neurones with non-overlapping RFs from spatially distinct locations (up to 7 mm apart), but with similar orientation preferences, could robustly synchronize their firings if the stimulus extended across both neurones’ RFs. When two separate, smaller stimuli were moved together, at the same time and in the same direction, to drive each neurone, weak correlations occurred. However, moving the two stimuli in opposite directions completely abolished synchronization. These results suggest that this neuronal synchronization depended on global features of the stimuli, such as coherent motion (movement in the same direction vs. opposing directions), and continuity (one long bar vs. two short bars) (Gray et al., 1989). Similar results have been reported for neurones located in different cortical areas and between hemispheres (Nelson et al., 1992; Nowak et al., 1995). Nonetheless, it is important to emphasize that, implicitly, this theory still supports the notion that single neurones behave as feature extractors and that explicit mechanisms for “binding information” are required. The presence and potential relevance of neuronal synchronization in the visual system has also been investigated by studying the correlated activity of LGN neurones in the cat. Sillito et al. (1994) have shown that neurones in the LGN of the cat may show significant correlated activity under the influence of stimulus continuity and coherent motion. These authors reported that LGN neurones with non-overlapping receptive fields did not show any significant temporal correlations when activated by a pair of stationary spots. However, when a single bar or moving gratings that covered both neurones’ receptive fields were presented, the same cells would synchronize their firing. By inactivating the overlying visual cortex, these authors also demonstrated that cortical feedback to the LGN was required for the occurrence of this synchronization. Similarly, Alonso et al. (1996) recorded from pairs of cat LGN neurones and found that neurones that had overlapping RFs and monosynaptic connections with primary visual cortical neurones would correlate their firing during visual stimulation. These authors reported that this correlated firing of LGN neurones enhances the efficacy of their inputs upon visual cortical neurones. Whether synchronization under these circumstances was due to feedforward or feedback influences remains to be investigated. In a departure from pairwise cross-correlation studies, Radons et al. (1994) investigated visual neuronal population responses to moving bars, both light and dark. The data-set, containing both multiple- and single-units, was obtained from a 30microelectrode array implanted in the primary visual cortex of an anaesthetized monkey (Kruger and Aiple, 1988). These authors used hidden Markov models, a patternrecognition technique, to determine how well the spatiotemporal firing patterns of neural ensembles could be used to recognize the different visual stimuli. Using this approach,

The space-time continuum in mammalian sensory pathways

133

they found that the identity of the stimulus types could be classified with 90% accuracy, several times above chance levels, when the time window of the analyses was between 25 and 100 milliseconds; increasing or decreasing the time window resulted in a drop in performance. Because of this relatively coarse time scale, the recognition of stimulus types was based simply on the spatiotemporal patterns of mean firing rates and not on the covariance of firing among neurones within the ensemble. Thus, these results argue that the representation of a stimulus can be achieved using the graded responses of a population of neurones without the need for high levels of neuronal synchrony. Another solution to “feature binding” is the convergence of many neurones coding for particular features upon a small numbers of neurones which code for the conjunction of those features (Hubel and Wiesel, 1962). Higher order visual areas are the places where one would expect to find such “grandmother” neurones. Gochin et al. (1994) tested whether neural ensemble coding was necessary for the complex feature detection in a higher order visual area. This was accomplished by focusing on the inferior temporal (IT) cortex, a region located along the ventral pathway of the macaque visual cortex (the “what” pathway, encoding object identities). Because of its position in the visual pathway, this cortical region is ideally-suited for investigating the existence of local versus distributed coding schemes. Although some of these IT neurones are highly selective, and can respond selectively to complex visual patterns such as faces and hands (Desimone et al., 1984), most of them do not respond only to one particular visual stimulus. It follows, therefore, that the stimulus discrimination capability of the IT area would best be assessed by studying groups of neurones. Thus, Gochin et al. (1994) tested the ability of ensembles of IT neurones, in a paradigm in which each neurone was recorded sequentially from a monkey performing a paired-association task, to encode different complex visual patterns. Data were analyzed by constructing firing rate vectors (time-scale =100 milliseconds) from serially recorded neurones, and applying linear discriminant analysis techniques to assess the ability of these reconstructed ensembles to distinguish between five different visual images. The main finding was that ensembles of IT neurones were able to discriminate between different stimuli much better than single IT neurones. Furthermore, the discrimination capacity varied as a function of ensemble size: the bigger the ensemble, the better the performance. Systematically dropping neurones from the ensemble resulted in “graceful” degradation of discrimination capacity. Consistent with this finding, the application of information theory analysis revealed that the distribution of information across the IT neural ensemble was unimodal, and thus, not segregated to particular neurones within the ensemble. Interestingly, the amount of information carried by the ensemble was less than the arithmetic sum of the information carried by individual neurones, suggesting that there is considerable redundancy across the population. In an important control, data from simultaneously recorded IT neurones were analyzed in the same way, and there was no significant increase in the amount of information contained in the ensembles. Based on this result, the authors suggest that, at least in IT cortex, little information is contained in the covariance of neural activity. Nevertheless, in a previous study Gochin et al. (1991) demonstrated, by recording small ensembles of IT neurones simultaneously from a 3-microwire bundle, that nearby neurones can synchronize their firing patterns in response to a visual stimulus, while neurones that

Time and the brain

134

were far apart are weakly correlated. The role of such synchronous firing, if any, in information processing of complex visual images remains to be investigated. 3.3.2. Encoding of sounds by neural ensembles in the auditory cortex In the auditory cortex of the rat, Maldonado and Gerstein (1996) were interested in determining the changes in neuronal ensemble dynamics that follow sensory reorganization induced by intracortical microstimulation (ICMS). ICMS is known to produce a broadening of the receptive fields of cells located at the stimulation site. In addition, neurones recorded from adjacent electrodes have been shown to increase their responsiveness to the best frequency of the cells recorded from the stimulating electrode. The functional relationships between neurones distributed within the primary auditory cortex were assessed by using the gravity analysis (Gerstein et al., 1985), a method in which the temporal relationships between neuronal spike trains are represented as a series of clusters in a multidimensional space. In this multidimensional space, neurones attract or repel each other depending on the coincidence of their neuronal firing. In this study, the responses of up to eight neurones were recorded simultaneously following auditory stimuli and ICMS. The experiments demonstrated that the functional clustering of a subset of the simultaneously recorded neurones, obtained during the delivery of the auditory stimuli, could be strengthened following ICMS. Interestingly, the formation of these functional clusters of neurones did not necessarily relate to the anatomical distance between the cells. In other words, neurones that were anatomically close did not necessarily have a stronger interaction, and neurones that were far apart did not necessarily have weaker interactions. Maldonado and Gerstein’s results indicate that neural ensembles can be established transiently in time, and that they are not necessarily composed of neurones within a highly circumscribed location (e.g. a cortical column). Moreover, their data suggest that the membership of a neural ensemble is mutable as a function of reorganization induced by ICMS. In a direct test of Middlebrooks et al.’s (1994, 1998) distributed coding of sound location hypthesis, Ahissar et al. (1992) demonstrated that broadly-tuned auditory neurones could encode sound location and movement by their pair-wise interactions. By recording the simultaneous extracellular activity of up to nine auditory cortex neurones in awake monkeys trained to sit still, Ahissar et al., studied the sensitivity of primary auditory cortical activity to azimuth and angular movements of acoustic stimuli. They found that two neurones, each broadly-tuned for spatial location, could synchronize their firing to predict spatial location more accurately. Analysis of cross-correlation histogram shapes revealed that different stimulus conditions could modulate the “effective connectivity” of the two neurones; that is, the magnitude and time course of synchronization varied according to stimulus parameters. These authors also noted that correlation of firing between neurones could occur in the absence of changes in the firing rates of each individual neurone. This suggests that temporal interactions between pairs of broadly-tuned auditory neurones, whose firing rates alone were unable to predict spatial location, could be used to discriminate between different spatial locations and angular movements of sound.

The space-time continuum in mammalian sensory pathways

135

The results of Ahissar et al. (1992), suggesting a role for relational coding in auditory processing, are supported by another data set obtained in the primary auditory cortex of anaesthetized marmosets. In this study, cross-correlation analyses revealed that pairs of neurones, or multiunit clusters, may also synchronize their firing according to another stimulus parameter, sound duration (deCharms and Merzenich, 1996). Using pure tone stimuli appropriate to the tuning of the recorded neurones, deCharms and Merzenich showed that short duration stimuli could change the firing rates of auditory cortical neurones. However, the same stimuli delivered for a longer duration could coordinate the firing patterns of neurones recorded simultaneously from two nearby locations. Importantly, this synchronization occurred without changes in the overall firing rates of each of the neurones. The coordination of activity between neurones could last for the duration of the stimulus, while the firing rate remained unchanged. Thus, only pair-wise corticocortical interactions were able to represent duration of a sound. Similar to the results reported by Ahissar et al., (1992), the degree of synchronization varied according to stimulus parameters. The correlation of firing between neurones was strongest when the stimuli were adjusted to fit their frequency tuning profiles, and tended to weaken when the distance between neurones increased. As mentioned in the introduction to this chapter, both the behavioural relevance and the past experience of an animal are believed to play an important role in the function of sensory systems. The recognition of species-specific vocalizations is particularly relevant for many animals since an important part of their everyday life depends on these sensory experiences. In the anaesthetized marmoset, Wang et al. (1995) have shown that subpopulations of neurones in the primary auditory cortex respond selectivity to a speciesspecific vocalization, the “twitter” call. Although most of the neurones were recorded serially, the time-varying and spatially distributed population responses were reconstructed according to their frequency tuning and post-stimulus time. This study revealed that the twitter call could elicit activity from a population of neurones that were spatially distributed across the primary auditory cortex, and that the pattern of firing was correlated with the spectro-temporal characteristics of the vocalization, but on a coarser time scale. In other words, the firing of these neurones phase-locked to large portions of the stimulus, and not to the finer-grained changes in frequency modulation. Two different vocalizations elicited two different cortical population response profiles, with considerable overlap in terms of the members of the population. Temporally-manipulated synthetic versions of the call, which retained the appropriate spectral content, resulted in a degradation of neuronal responses and decreased the amount of synchronization across the population of neurones. This suggests that the temporal characteristics of the individual neurones’ RFs play a role as important as the spectral features. 3.3.3. Encoding of tactile stimuli by thalamocortical somatosensory ensembles Temporal interactions between neuronal populations located in different brain regions played a fundamental role in the concept of cell assemblies first put forward by Donald Hebb (1949). Testing Hebb’s assumptions requires one to record from populations of neurones distributed across multiple cortical areas and subcortical nuclei. Using a chronic

Time and the brain

136

multiple-electrode recording preparation, Nicolelis et al. (1995, 1997a) were able to record from populations of neurones distributed throughout the brainstem, thalamic and cortical processing levels of the trigeminal somatosensory pathway in both anaesthetized and awake rats. The results of these studies have emphasized that the co-ordinated activity of large ensembles of neurones, distributed across multiple cortical and subcortical structures, may provide the basis for the encoding of tactile information in the mammalian somatosensory system. Ensemble recordings of neurones in both VPM and SI cortex have revealed that, even in structures classically thought to employ a labelled line coding scheme, punctate tactile stimuli can elicit distributed responses across each of these brain regions (Figure 2). During normal exploratory behaviours, rats make many simultaneous whisker contacts with objects in the environment. It is important, therefore, to investigate how the somatosensory system integrates information that are more complex than punctate stimuli such as

The space-time continuum in mammalian sensory pathways

137

Figure 2 . Population histograms reveal the distributed nature of responses to punctate tactile stimuli in both the SI cortex (top panel) and VPM thalamus (bottom panel). Notice that individual neurones, all recorded simultaneously, within each population vary in their response magnitudes and latencies.

single whiskers. Studies in other sensory systems have uncovered non-linear neuronal properties when naturalistic stimuli are used to elicit sensory responses (e.g., Rauschecker et al., 1995; Wang et al., 1995). Towards this goal, Ghazanfar and Nicolelis

Time and the brain

138

(1997) recorded concurrently the responses of several tens of well-isolated single neurones from the VPM and primary somatosensory (SI) cortex following single- and multi-whisker deflections. These multi-whisker deflections consisted of three coincidentally-displaced whiskers oriented in rows or columns within the matrix of facial whiskers. Examination of ensemble responses revealed that both single and multiwhisker stimuli could elicit widespread and concurrent activation of neurones across the thalamocortical loop for several milliseconds. Multi-whisker stimuli oriented along columns of whiskers elicited non-linear cortical ensemble responses by activating significantly SI cortex more vigorously, and faster than the arithmetic sum of singlewhisker ensemble responses. This was not the case for multi-whisker stimuli oriented along rows of whiskers. Interestingly, this bias towards whisker columns was not apparent in the single neurone data for SI cortex. Therefore, these results suggest that ensembles of cortical neurones can generate complex spatial transformations that are not seen at the single neurone level. 3.3.4. Large-scale neural ensemble interactions across the somatosensory system In another set of studies, simultaneous multi-site neural ensemble recordings were used to investigate the large-scale interactions between populations of neurones distributed across multiple processing levels of the rat trigeminal somatosensory system (Nicolelis et al., 1995). These recordings included neurones located in the trigeminal ganglion, the principal and spinal subnuclei of the trigeminal brainstem complex, the VPM thalamus, and the infragranular layers of the SI cortex in freely behaving animals. The first important observation derived from this study was the discovery that cortical and subcortical somatosensory neurones can exhibit widespread 7–12 Hz synchronous oscillations (peak frequency at 10Hz), which tend to precede the initiation of tactile exploratory behaviours (whisker twitching movements). By correlating the electrophysiological data with behavioural measurements, obtained through quantitative analysis of facial electromyograms and frame-by-frame video records, Nicolelis et al. (1995) observed that these oscillations tended to appear during periods in which rats assume an immobile and alert posture (Figure 3). Interestingly, these oscillations preceded, by an average of 581 ms, the occurrence of small amplitude rhythmic whisker movements (whisker twitches) in the same frequency range (7–12 Hz). In about 80% of these episodes, the oscillations were first observed in the SI cortex where, on average, 92% of the neurones exhibit oscillatory activity as measured by autocorrelation functions. About 20 ms after the oscillations appeared in cortex, 98% of the VPM neurones, located in both hemispheres, were entrained into these oscillations (Figure 3). Just prior to the initiation of whisker twitches, about 49% of the neurones located in the spinal nucleus of the trigeminal brainstem complex began to exhibit the same oscillations. Neurones located in the principal nucleus of the trigeminal complex or in the trigeminal ganglion did not exhibit these oscillations. By stimulating individual whiskers during the occurrence of these oscillations, we observed that only stimuli delivered during the peak of the oscillations were capable of eliciting sensory-evoked responses. Stimuli delivered outside this small time window

The space-time continuum in mammalian sensory pathways

139

would either induce a phase shift in the underlying oscillatory activity or have no effect. Interestingly, stimuli delivered in a particular phase of these oscillations could block the oscillatory activity for several seconds. In some instances, tactile stimuli could also trigger the occurrence of such oscillations. Altogether, these results raised the hypothesis that top-down influences, generated primarily in the neocortex, are capable of entraining most of the somatosensory system into a pattern of firing (oscillatory mode), which may be ideal for processing sensory information obtained during rhythmic active exploration (through whisker movements) of the surrounding environment.

Figure 3 . Local field potentials recorded simultaneously from VPM and SI cortex before and during whisker twitching behaviour. Oscillations begin first in SI cortex, then in VPM, just prior to whisker twitching movements. Arrows denote the beginning of SI cortical oscillations. First six channels were recorded from electrodes in SI cortex, and last two were recorded in VPM (figure courtesy of Erika E.Fanselow).

3.3.5. Effects of chronic peripheral deafferentation on somatosensory ensembles Many studies have investigated the effects of peripheral sensory deafferentation on the properties of topographic maps in different brain regions (Kaas, 1991; Nicolelis, 1997). In the somatosensory system, peripheral deafferentation is known to result in plastic changes across multiple cortical and subcortical relays of the pathway (Florence and Kaas, 1995). For example, neonatal rats that underwent a complete unilateral removal of their facial whiskers had an extensive reorganization of their ventral posterior medial (VPM) nucleus (Nicolelis et al., 1991). Specifically, this reorganization resulted in the

Time and the brain

140

expansion of RFs to include abnormal regions of the body, such as the shoulder and forepaw, an increase in the response latencies, and an increase in the reverberatory bursting activity of VPM neurones. The effects of this reorganization of the dynamic properties of VPM neurones on the ability of VPM neural ensembles to represent the location of a tactile stimulus were investigated by Nicolelis et al. (1997b). Until recently, no study had examined how such peripheral manipulations affect the ability of the nervous system to encode various stimulus attributes. Ensembles of 13–20 single neurones were recorded simultaneously, and their spatiotemporal firing patterns were analyzed using discriminant analysis (DA). DA was used to quantify the effects of neonatal sensory deprivation on the ability of VPM neural ensembles to “discriminate”, on a trial-by-trial basis, the location of a tactile stimulus. Using this approach, it was determined that when the spatiotemporal patterns of VPM neural ensemble responses in normal rats were used, the neural ensembles were able to classify correctly the location of a tactile stimulus, among three different sites, on 88.2% of the trials. In contrast, ensembles of VPM neurones from whisker-deprived animals performed significantly worse, classifying correctly the location of the stimulus on 73.5% of the trials, even though the three different stimulus locations chosen were much farther apart (making it easier to discriminate) than in the normal animals. Furthermore, removing, one-by-one, the neurones that contributed the most weight in the population response, revealed a graceful degradation of ensemble performance, demonstrating the distributed nature of normal VPM function. Overall, these results suggest that peripheral deafferentation, by altering both the spatial and temporal aspects of single VPM neuronal properties, can decrease the ability of somatosensory neural ensembles to discriminate the location of a tactile stimulus and provides an unique experimental/neurophysiological model for studying chronic peripheral sensory deprivation that is similar to clinical disorders such as phantom limbs. 3.3.6. Simultaneous coding of tactile stimulus location across multiple primate somatosensory cortical areas One of the most basic functions of the nervous system is to identify the location of sensory stimuli. Information about stimulus location is transduced first by activation of the peripheral receptor array. From there, this information may travel along several parallel pathways in the central nervous system until reaching the neocortex. In the complex somatosensory system of primates, which includes many reciprocally-connected cortical areas, a fundamental question is whether information about tactile stimulus localization is preserved throughout these many cortical areas, or whether such information is conserved within distinct cortical areas that then serve as a general reference for other cortical areas. It has been suggested that the degradation of topographic maps of the body surface, due to increases in receptive field sizes as one proceeds from early to later stages of cortical processing, should result in a corresponding degradation in the ability of higher order areas to localize a tactile stimulus. For example, the secondary somatosensory cortex and the different fields of the posterior parietal cortex have long been considered to be uninvolved in stimulus location coding, because

The space-time continuum in mammalian sensory pathways

141

the large receptive fields of neurones in these regions would seem to preclude the representation of this information (Kaas and Pons, 1988). It is seemingly easier to locate a stimulus using a population of neurones with very small RFs, and such a population would be, functionally, a labelled line system. Thus, a fundamental question is whether information about tactile stimulus location is lost in cortical areas where neurones have large receptive fields. It is possible that information about stimulus location could be transformed from a spatial code at the periphery, easily recognized at a single neurone level, to a population code, best recognized in the simultaneous activities of large populations of cortical neurones. Nicolelis et al. (1998) tested these two potential coding schemes by chronically and simultaneously recording populations of neurones from the hand, arm and face regions of three different somatosensory cortical areas in owl monkeys (Aotus trivirgatus). Quantitative analyses of single neurone RF sizes in the infragranular layers of primary somatosensory (area 3b), secondary somatosensory (SII), and parietal area 2, revealed that most neurones had broadly-tuned RFs. As such, single neurones were not reliable indicators of stimulus location on a single-trial basis. As reported previously for the rat somatosensory system (Ghazanfar et al., 1995; Ghazanfar and Nicolelis, 1997; Nicolelis and Chapin, 1994), punctate stimuli elicited unique spatiotemporal patterns of population responses from each of these primate somatosensory cortical areas. In order to test whether these distributed population responses could identify the location of a stimulus on a single-trial basis, neural ensemble analyses were carried out using multivariate statistical pattern recognition approaches (Nicolelis et al., 1998 in press). These analyses, which included an artificial neural network based on learning vector quantization (LVQ), revealed that small ensembles of neurones (~20–40 neurones) from area 3b, SII, or area 2 could encode the location of a stimulus reliably, on a single trial basis (Figure 4a). For example, an ensemble of 24 neurones from area 3b could correctly identify the stimulus location among three different skin sites on 72.5% of the trials (chance is 33.3%). Similarly sized neural ensembles in SII and area 2 also demonstrated above-chance discrimination on a single trial basis. Decreasing ensemble sizes, by removing sequentially the “best predictor” neurone, revealed a graceful degradation of ensemble performance in all three cortical areas, a hallmark of distributed coding (Figure 4a). In order to investigate which parameters of the neural ensemble response were important for localizing the tactile stimulus, two data manipulations were carried out: 1) decreasing the temporal resolution of the response by increasing bin sizes; and 2) shuffling the trials, to eliminate the covariance structure between elements of the population. Increasing the bin size, from 3 milliseconds to 45 milliseconds, significantly reduced the performance of SII ensembles to discriminate the location of a stimulus, suggesting that the temporal modulation of ensemble firing plays an important role for encoding stimulus location in this cortical area (Figure 4b). Surprisingly, ensembles in areas 3b and 2 did not show a deficit in performance, demonstrating that simple firing rate changes were sufficient (Figure 4b). This finding, which is reminiscent of the results in the visual system (Richmond and Optican, 1990; Victor and Purpura, 1996), suggests that different somatonsensory cortical areas may employ different encoding strategies to represent the same type of tactile information (i.e. stimulus location).

Time and the brain

142

Finally, independent shuffling of trial order for each single neurone had little effect on the ability of neural ensembles in any of the three areas to correctly classify stimulus location. If anything, ensemble performance sometimes improved following trial shuffling. These results indicate that, for the set of analytical tools used in this study, neither intra-trial precise timing of ensemble spike patterns, neural synchronization, nor linear covariance across the population of neurones were necessary for effective single trial discrimination of tactile stimulus location. Instead, the location of a stimulus could be predicted by the unique spatiotemporal patterns of sensory responses across the neural population. In consideration of these results, it is important to realize that the use of multi-site neural ensemble recordings from areas 3b, SII, and 2 revealed that neural responses occur in these regions concurrently. As such, it does not appear likely that information is passed only sequentially from one cortical area to another; rather, the different encoding strategies used by each area may occur almost simultaneously. Indeed, analysis of the

The space-time continuum in mammalian sensory pathways

143

Figure 4 . a. For two cortical areas analyzed (SII and 3b) the single trial discrimination capability varied as a function of the ensemble size. This figure illustrates the effects of sequential removal of the best predictor neurons from the population on single trail discrimination of the stimulus location. These curves also revealed a graceful decay in discrimination performance as a function of ensemble size. b. Effect of bin size on the discrimination capability of two different cortical areas. While a significant reduction in discrimination capability was observed when bin clumping (i.e. increasing the bin describing neuronal firing from 3 to 45 ms) was applied to SII neural ensembles, no effect was observed in area 3b neural ensembles. These results suggest that the temporal structure of population responses could play a more fundamental role in area SII than in the other two somatosensory regions for encoding stimulus location.

time epoch in which the maximum amount of information is contained revealed that there was a 15 millisecond overlap between area 2 and areas 3b and SII sensory responses.

Time and the brain

144

4. CONCLUSION The findings reviewed in this chapter underscore the fundamental relevance of time in sensory information processing by the auditory, visual, and somatosensory systems of mammals. Altogether, these data reveal that the existence of spatiotemporal coupling in both single neurone responses and neural ensemble firing is pervasive throughout these sensory pathways. Furthermore, dynamic RFs and the spatiotemporal coding of stimulus attributes, which have been now identified in all three sensory systems, seem to emerge and to be constrained by both the need to interpret time-varying stimuli and the fact that active exploratory behaviours are used by animals to scan their surrounding environment. Regarding the mechanisms used to generate the spatiotemporal properties of single neural RFs and neural ensemble coding, the evidence suggests that the asynchronous convergence of multiple afferent pathways gives rise to the dynamic neural properties observed in all three sensory systems. In this context, descending corticofugal projections play as crucial a role in sensory information processing as the feedforward pathways, since they provide the anatomical substrate for propagating top-down signals from the neocortex to subcortical relays and, consequently, act upon the ascending flow of sensory information. It is also likely that the recurrent connectivity of sensory systems contributes to the finding that slight alterations at either peripheral or central levels can trigger system-wide alterations in the functional organization of a sensory pathway. Therefore, embedded in this model is the notion that adaptive neural phenomena (i.e. immediate and long-term sensory plasticity, learning, etc.), occurring at different time scales (i.e. seconds, minutes, hours, days, etc.) both influence and are constrained by the dynamics of sensory information processing at a millisecond scale. Thus, in addition to contributing to an extra dimension for information coding, the dynamic properties of neural ensembles provide the substrates for triggering immediate system-wide reorganization that follow changes in sensory experience over the course of the organism’s life (Nicolelis, 1997). Fundamentally, mammalian sensory systems can no longer be considered as a collection of passive filters or feature extractors that are specialized in detecting the presence of particular stimulus attributes (i.e. orientation, colour, tones, etc.). The experimental evidence reviewed in this chapter indicates that perception emerges as the outcome of highly distributed interactions between populations of broadly tuned neurones dispersed throughout the multiple and reciprocally-connected subcortical and cortical areas that define a sensory system. Put in another way, the central nervous system does not operate as a static decoder, which sits idle until inputs from the environment are conveyed through it, by feedforward neural pathways that connect the sensory periphery to the cerebral cortex. Instead, a more parsimonious hypothesis is that during adulthood, the spatiotemporal organization of sensory systems reflect their active attempt, during exploratory behaviours, to compare the meaning encoded into a time-varying stimulus with the accumulated representation of the sensory experiences acquired during a lifetime.

The space-time continuum in mammalian sensory pathways

145

REFERENCES Abeles, M., and Goldstein, M.H. (1972) Responses of single units in the primary auditory cortex of the cat to tones and to tone pairs. Brain Research , 42, 337–352. Adrian, E.D. (1928) The basis of sensation . Norton, New York. Ahissar, M., Ahissar, E., Bergman, H., and Vaadia, E. (1992) Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. Journal of Neurophysiology , 67, 203–215. Alonso, J.-M., Usrey, W.M., and Reid, R.C. (1996) Precisely correlated firing in cells of the lateral geniculate nucleus. Nature, London , 383, 815–819. Armstrong-James, M., and Fox, K. (1987) Spatiotemporal convergence and divergence in the rat SI “barrel” cortex. Journal of Comparative Neurology , 263, 265–281. Armstrong-James, M., Fox, K., and Das-Gupta, A. (1992) Flow of excitation within rat barrel cortex on striking a single vibrissa. Journal of Neurophysiology , 68, 1345– 1358. Atick, J.J., and Redlich, A.N. (1990) Towards a theory of early visual processing. Neural Computation , 2, 308–320. Barlow, H.B. (1972) Single units and sensation: a neuron doctrine for perceptual psychology? Perception , 1, 371–394. Beck, P.D., Fitzpatrick, L., and Nicolelis, M.A.L. (1998) Corticothalamic projections of somatosensory and motor cortical areas in the rat. Society for Neuroscience Abstracts , p. 131. Bushnell, E.W., and Boudreau, J.P. (1991) The development of haptic perception during infancy. In: M.A.Heller and W.Schiff (eds.) The psychology of touch . Erlbaum, Hillsdale, NJ, pp. 139–161. Carvell, G.E., and Simons, D.J. (1990) Biometric analyses of vibrissal tactile discrimination in the rat. Journal of Neuroscience , 10, 2638–2648. Chapin, J.K. (1986) Laminar differences in sizes, shapes, and response profiles of cutaneous receptive fields in the rat SI cortex. Experimental Brain Research , 62, 549– 559. Dan, Y., Atick, J.J., and Reid, R.C. (1996) Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. Journal of Neuroscience , 16, 3351–3362. Darian-Smith, I. (1984) The sense of touch: performance and peripheral neural processes. In: I. Darian-Smith (ed.) Handbook of physiology. I. the nervous system. Sensory processes. Vol. 3 , American Physiological Society, pp. 739–787. Dawis, S., Shapley, R., Kaplan, E., and Tranchina, D. (1984) The receptive field organization of X-cells in the cat: spatiotemporal coupling and asymmetry. Vision Research , 24, 549–564. DeAngelis, G.C., Ohzawa, I., and Freeman, R.D. (1993) Spatiotemporal organization of simple-cell receptive fields in the cat’s striate cortex. I. General characteristics and postnatal development. Journal of Neurophysiology , 69, 1118–1135. DeAngelis, G.C., Ohzawa, I., and Freeman, R.D. (1995) Receptive field dynamics in the central visual pathways. Trends in Neurosciences , 18, 451–458. deCharms, R.C., Blake, D.T., and Merzenich, M.M. (1998) Optimizing sound features for cortical neurons. Science, New York , 280, 1439–1443. deCharms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds

Time and the brain

146

by the coordination of action-potential timing. Nature, London , 381, 610–613. Desimone, R., Albright, T.D., Gross, C.G., and Bruce, C.J. (1984) Stimulus selective properties of inferior temporal neurons in the macaque. Journal of Neuroscience , 4, 2051–2062. Dinse, H.R., Kruger, K., Mallot, H.A., and Best, J. (1991) Temporal structure of cortical information processing: cortical architecture, oscillations, and non-separability of spatio-temporal receptive field organization. In: J.Kruger (ed.) Neuronal Cooperativity Springer-Verlag, Berlin, pp. 68–104. Fabri, M., and Burton, H. (1991) Ipsilateral cortical connections of primary somatic sensory cortex in rats. Journal of Comparative Neurology , 311, 405–424. Faggin, B.M., Nguyen, K.T., and Nicolelis, M.A. (1997) Immediate and simultaneous sensory reorganization at cortical and subcortical levels of the somatosensory system. Proceedings of the National Academy of Sciences, U.S.A. , 94, 9428–33. Florence, S.L., and Kaas, J.H. (1995) Large-scale reorganization at multiple levels of the somato-sensory pathway follows therapeutic amputation of the hand in monkeys. Journal of Neuroscience , 15, 8083–8095. Gerstein, G.L., Perkel, D.H., and Dayhoff, J.E. (1985) Cooperative firing activity in simultaneously recorded populations of neurons: detection and measurement. Journal of Neuroscience , 5, 881–9. Ghazanfar, A.A., and Nicolelis, M.A.L. (1999) Spatiotemporal properties of layer V neurons of the rat primary somatosensory cortex. Cerebral Cortex , 9, 348–361. Ghazanfar, A.A., Krupa, D.J., and Nicolelis, M.A.L. (1997) Tactile processing by thalamic neural ensembles: the role of cortical feedback. Society for Neuroscience Abstracts . Ghazanfar, A.A., and Nicolelis, M.A.L. (1997) Nonlinear processing of tactile information in the thalamocortical loop. Journal of Neurophysiology , 78, 506–510. Gochin, P.M., Colombo, M., Dorfman, G.A., Gerstein, G.L., and Gross, C.G. (1994) Neural ensemble coding in inferior temporal cortex. Journal of Neurophysiology , 71 , 2325–2337. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of the awake monkey. Experimental Brain Research , 84, 505–516. Golomb, D., Kleinfeld, D., Reid, R.C., Shapley, R.M., and Shraiman, B.I. (1994) On temporal codes and the spatiotemporal response of neurons in the lateral geniculate nucleus. Journal of Neurophysiology , 72 , 2990–3003. Gottlieb, J.P., and Keller, A. (1997) Intrinsic circuitry and physiological properties of pyramidal neurons in rat barrel cortex. Experimental Brain Research , 115 , 47–60. Gray, C.M., Konig, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, London , 338 , 334–337. Gray, C.M., and Singer, W. (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A. , 86 , 1698–1702. Hartline, H.K. (1950) The receptive fields of optic nerve fibres. American Journal of Physiology , 130 , 690–699. Mauser, M.D., Agnetta, B., and Perez, C. (1998) Orienting asymmetries in rhesus monkeys: effect of time-domain changes on acoustic perception. Animal Behaviour , 56, 41–47. Hebb, D.O. (1949) The organization of behavior. A neuropsychological theory . John Wiley and Sons.

The space-time continuum in mammalian sensory pathways

147

Hoeflinger, B.F., Bennett-Clarke, C.A., Chiaia, N.L., Killackey, H.P., and Rhoades, R.W. (1995) Patterning of local intracortical projections within the vibrissae representation of rat primary somatosensory cortex. Journal of Comparative Neurology , 354, 551– 563. Hubel, D.H., and Wiesel, T.N. (1962) Receptive fields, binocular interactions, functional architecture in the cat’s visual cortex. Journal of Physiology (London) , 195 , 215–243. Jenkins, W.M., and Masterton, R.B. (1982) Sound localization: effects of unilateral lesions in central auditory system. Journal of Neurophysiology , 47 , 987–1016. Kaas, J.H. (1991) Plasticity of sensory and motor maps in adult mammals. Annual Review of Neuroscience , 14 , 137–168. Kaas, J.H., and Pons, T.P. (1988) The somatosensory system of primates. In: H.D. Steklis and J. Erwin (eds) Comparative Primate Biology, Vol. 4 , Alan R.Liss, New York, pp. 421–468. Knudsen, E.I., and Konishi, M. (1979) Mechanisms of sound location in the barn owl (Tyto alba). Journal of Comparative Physiology , 133 , 13–21. Koralek, K.-A., Olavarria, J., and Killackey, H.P. (1990) Areal and laminar organization of corticocortical projections in the rat somatosensory cortex. Journal of Comparative Neurology , 299 , 133–150. Kruger, J., and Aiple, F. (1988) Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. Journal of Neurophysiology , 60, 798–828. Krupa, D.J., Ghazanfar, A.A., and Nicolelis, M.A.L. (1999) Immediate thalamic sensory plasticity depends on corticothalamic feedback. Proc. Natl. Acad. Sci . USA, 96, 8200– 8205. Lashley, K.S. (1950) In search of the engram. Symposium of the Society of Experimental Biology , 4, 454–482. Maldonado, P.E., and Gerstein, G.L. (1996) Neuronal assembly dynamics in the rat auditory cortex during reorganization induced by intracortical microstimulation. Experimental Brain Research , 112 , 431–41. Margoliash, D., and Fortune, E.S. (1992) Temporal and harmonic combination-sensitive neurons in the zebra finch’s HVc. Journal of Neuroscience , 12 , 4309–4326. Marrocco, R.T. (1976) Sustained and transient cells in monkey lateral geniculate nucleus: conduction velocities and response properties. Journal of Neurophysiology , 39 , 340– 353. McClurkin, J.W., Gawne, T.J., Optican, L.M., and Richmond, B.J. (1991 a) Lateral geniculate neurons in behaving primates. II. Encoding of visual information in the temporal shape of the response. Journal of Neurophysiology , 66 , 794–808. McClurkin, J.W., Gawne, T.J., Richmond, B.J., Optican, L.M., and Robinson, D.L. (1991b) Lateral geniculate neurons in behaving primates. I. Responses to twodimensional stimuli. Journal of Neurophysiology , 66 , 777–93. McClurkin, J.W., and Optican, L.M. (1996) Primate striate and prestriate cortical neurons during discrimination. I. simultaneous temporal encoding of information about color and pattern. Journal of Neurophysiology , 75, 481–495. Middlebrooks, J.C., Clock, A.E., Xu, L., and Green, D.M. (1994) A panoramic code for sound location by cortical neurons. Science, New York , 264 , 842–844. Middlebrooks, J.C., Xu, L., Eddins, A.C., and Green, D.M. (1998) Codes for soundsource location in nontonotopic auditory cortex. Journal of Neurophysiology , 80 , 863–881. Mignard, M., and Malpeli, J.G. (1991) Paths of information flow through visual cortex.

Time and the brain

148

Science, New York , 251, 1249–1251. Nelson, J.I., Salin, P.A., Munk, M.H., Arzi, M., and Bullier, J. (1992) Spatial and temporal coherence in cortico-cortical connections: a cross-correlation study in areas 17 and 18 in the cat. Visual Neuroscience , 9, 21–37. Nicolelis, M.A.L. (1997) Dynamic and distributed somatosensory representations as the substrate for cortical and subcortical plasticity. Seminars in Neuroscience , 9, 24–33. Nicolelis, M.A.L. (1998, in press) Methods in Neural Ensemble Recordings . CRC Press, New York. Nicolelis, M.A., Chapin, J.K., and Lin, R.C. (1991) Thalamic plasticity induced by early whisker removal in rats. Brain Research , 561, 344–9. Nicolelis, M.A., Lin, R.C., Woodward, D.J., and Chapin, J.K. (1993a) Dynamic and distributed properties of many-neuron ensembles in the ventral posterior medial thalamus of awake rats. Proceedings of the National Academy of Sciences, U.S.A. , 90, 2212–6. Nicolelis, M.A.L., Lin, R.C.S., Woodward, D.J., and Chapin, J.K. (1993b) Induction of immediate spatio-temporal changes in thalamic networks by peripheral block of ascending cutaneous information. Nature, London , 361, 533–536. Nicolelis, M.A.L., and Chapin, J.K. (1994) Spatiotemporal structure of somatosensory responses of many-neuron ensembles in the rat ventral posterior medial nucleus of the thalamus. Journal of Neuroscience , 14, 3511–3532. Nicolelis, M.A., Baccala, L.A., Lin, R.C., and Chapin, J.K. (1995) Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system. Science, New York , 268, 1353–8. Nicolelis, M.A.L., Oliveira, L.M.O., Lin, R.C.S., and Chapin, J.K. (1996) Active tactile exploration influences the functional maturation of the somatosensory system. Journal of Neurophysiology , 75, 2192–2196. Nicolelis, M.A.L., Ghazanfar, A.A., Faggin, B.M., Votaw, S., and Oliveira, L.M.O. (1997a) Reconstructing the engram: simultaneous, multisite, many single neuron recordings. Neuron , 18, 529–537. Nicolelis, M.A.L., Lin, R.C.S., and Chapin, J.K. (1997b) Neonatal whisker removal reduces the discrimination of tactile stimuli by thalamic ensembles in adult rats. Journal of Neurophysiology , 78, 1691–1706. Nicolelis, M.A.L., Ghazanfar, A.A., Stambaugh, C.R., Oliveira, L.M.O., Laubach, M., Chapin, J.K., Nelson, R.J., and Kaas, J.H. (1998) Simultaneous encoding of tactile information by three primate cortical areas. Nature, Neuroscience , 1 (in press). Nicolelis, M.A.L., Stambaugh, C.R., Brisben, A., and Laubach, M. (1998 in press) Methods for simultaneous multisite neural ensemble recordings in behaving primates. In: M.A.L.Nicolelis (ed.) Methods in Neural Ensemble Recordings . CRC Press, New York. Nowak, L.G., Munk, M.H., Nelson, J.I., James, A.C., and Bullier, J. (1995) Structural basis of cortical synchronization. I. Three types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–400. Oliveira, L.M.O., Shuler, M.G., Casal, S.G., and Nicolelis, M.A.L. (1997) Effects of neonatal unilateral facial nerve cuts on tactile responses in adult rat SI cortex. Society for Neuroscienc Abstracts. , p. 1798. Radons, G., Becker, J.D., Dulfer, B., and Kruger, J. (1994) Analysis, classification, and coding of multi-electrode spike trains with hidden Markov models. Biological Cybernetics , 71, 359–373. Rauschecker, J.P., Tian, B., and Hauser, M. (1995) Processing of complex sounds in the

The space-time continuum in mammalian sensory pathways

149

macaque nonprimary auditory cortex. Science, New York , 268, 111–114. Reid, R.C., and Alonso, J.-M. (1995) Specificity of monosynaptic connections from thalamus to visual cortex. Nature, London , 378, 281–284. Richmond, B.J., and Optican, L.M. (1990) Temporal encoding of two-dimensional patterns by single units in primate primary visual cortex. II. Information transmission. Journal of Neurophysiology , 64, 370–80. Richmond, B.J., Optican, L.M., and Spitzer, H. (1990) Temporal encoding of twodimensional patterns by single units in primate primary visual cortex. I. Stimulusresponse relations. Journal of Neurophysiology , 64, 351–69. Ringach, D.L., Hawken, M.J., and Shapley, R.M. (1997) Dynamics of orientation tuning in macaque primary visual cortex. Nature, London , 387, 281–284. Saul, A.B., and Humphrey, A.L. (1992) Evidence of input from lagged cells in the lateral geniculate nucleus to simple cells in cortical area 17 of the cat. Journal of Neurophysiology , 68, 1190–1207. Shadlen, M.N., and Newsome, W.T. (1994) Noise, neural codes and cortical organization. Current Opinion in Neurobiology , 4, 569–579. Sillito, A.M., Jones, H.E., Gerstein, G.L., and West, D.C. (1994) Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London , 369, 479–482. Simons, D.J. (1978) Response properties of vibrissa units in rat SI somatosensory neocortex. Journal of Neurophysiology , 41, 798–820. Singer, W., and Gray, C.M. (1995) Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience , 18, 555–586. Stevens, J.K., and Gerstein, G.L. (1976) Spatiotemporal organization of cat lateral geniculate receptive fields. Journal of Neurophysiology , 39 , 213–238. Suga, N., O’Neill, W.E., Kujirai, K., and Manabe, T. (1983) Specificity of combinationsensitive neurons for processing biosonar signals in auditory cortex of the mustached bat. Journal of Neurophysiology , 49 , 1573–1626. Sutter, M.L., and Schreiner, C.E. (1991) Physiology and topography of neurons with multipeaked tuning curves in cat primary auditory cortex. Journal of Neurophysiology , 65 , 1207–1226. Tramo, M.J., Bellew, B.F., and Hauser, M.D. (1996) Discharge patterns of auditory cortical neurons evoked by species-specific vocalizations and synthetic complex signals in alert Macaca mulatta. Society for Neuroscience Abstracts , p. 1623. Victor, J.D., and Purpura, K.P. (1996) Nature and precision of temporal coding in visual cortex: a metric-space analysis. Journal of Neurophysiology , 76 , 1310–1326. Wang, X., Merzenich, M.M., Beitel, R., and Schreiner, C.E. (1995) Representation of a species-specific vocalization in the primary auditory cortex of the common marmoset: temporal and spectral characteristics. Journal of Neurophysiology , 74, 2685–2706. Welker, W.I. (1964) Analysis of sniffing of the white albino rat. Behaviour , 12, 223– 244. Winter, P., and Funkenstein, H.H. (1973) The effect of species-specific vocalizations on the discharge of auditory cortical cells in the awake squirrel monkey (Saimiri sciureus) Experimental Brain Research , 18, 489–504. Wollberg, Z., and Newman, J.D. (1972) Auditory cortex of squirrel monkeys: response properties of single cells to species-specific vocalizations. Science, New York , 175, 212–214. Wortis, S.B., and Pfeiffer, A.Z. (1948) Unilateral auditory-spatial agnosia. Journal of Nervous and Mental Disease , 108, 181–186.

4 Information Flow along Neocortical Axons Harvey A.Swadlow Department of Psychology (U-20), The University of Connecticut, Storrs, Connecticut 06269 Tel: (860) 486–2252; FAX: (860) 486–2760; e-mail: [email protected]

Neocortical axonal systems convey the results of cortical processing to cortical target structures. This chapter is concerned with the timing of this information flow and with delays introduced by different types of cortical axon. Neocortical white matter is composed of axons that range from <0.1 µm to >10 µm in diameter. Such axons generate a similarly wide range of conduction velocities, and conduction times between soma and axon terminals. Work from the author’s laboratory is reviewed in which conduction properties of corticocortical and descending corticofugal efferent systems of rabbit neocortex were examined in each of four sensory and motor cortical regions. These data are compared with corresponding axonal systems of primates, cats and rodents. Whereas some cortical efferent systems are composed of exclusively fast-conducting axons, others have a large number of very slowly conducting elements, and impulse conduction times along these latter elements may take several tens of milliseconds. Properties of specific thalamocortical axonal systems are also reviewed, and the role of axonal conduction time in the generation of sharply and broadly synchronous cortical activity is explored. KEYWORDS: Axonal conduction; Cerebral cortex; Cortico-cortical axons; Conduction velocity; Conduction delay; Synchrony

1. INTRODUCTION Neocortical axonal systems convey the results of cortical processing to a host of destinations. This chapter is concerned with the timing of this information flow, and with delays introduced by different types of cortical axons. Considerable information is available concerning the laminar and regional origins of neocortical efferent pathways and the locus of terminal synapses. Less attention has been paid to the axon lying between soma and synaptic terminals. General surveys of cortical white matter (Bishop and Smith, 1964; Waxman and Swadlow, 1977) have shown that neocortical axons exhibit a wide spectrum of diameters. Unmyelinated axons range from <0.1–0.6 µm, and often have a median diameter of ~0.2 µm (e.g. Waxman and Swadlow, 1976b; Swadlow et al., 1980). Most such axons cannot be visualized without the aid of electron

Information flow along neocortical axons

151

microscopy. Myelinated axons may be as fine as 0.2 µm in diameter (e.g. Harding and Towe, 1985; Waxman and Bennett, 1972). The diameter of the largest myelinated axons found in a given cortical system is species dependent (see below), and may be >10 µm (e.g. Biedenbach et al., 1986). Thus, neocortical axons span a range of >1:10,000 in cross-sectional area, and a range in expected conduction velocities of >1:100 (Waxman and Bennett, 1972, see below). This chapter reviews work from the author’s laboratory in which conduction properties of corticocortical and descending corticofugal efferent systems of rabbit neocortex were examined in each of four sensory and motor cortical regions. These data are compared with results from corresponding axonal systems of primates, cats, and rodents. This is not intended to be an all-inclusive review of neocortical axonal systems, and many axonal systems are not included. The cortical regions examined in rabbit were primary visual cortex (V1), primary and secondary somatosensory cortices (S1, S2) and motor cortex (Swadlow, 1988, 1990, 1991, 1994). Efferent systems studied in each of these regions include: (1) the contralateral callosal projection (CC axons); (2) ipsilateral corticocortical projections (C-IC axons) linking primary and secondary sensory cortical areas, or motor and sensory cortical areas; (3) descending corticofugal projections of layer 5 (CF-5 axons); and (4) descending corticofugal projections of layer 6 (CF-6 axons). In addition, sharply synchronous (±1 ms), non-oscillatory activity between putative GABAergic neocortical interneurones is described (Swadlow et al., 1998), and the relationship of such activity to properties of thalamocortical axons is explored.

2. METHODOLOGICAL CONSIDERATIONS Information about axonal conduction delays can be derived from several sources, and each has characteristic limitations. 2.1 Conduction Delays Inferred from Fibre-diameter Spectra The theoretical basis for the relationship between axonal diameter and conduction velocity has been well-explored, and conduction time can be inferred from a knowledge of axon diameter and conduction distance. The conduction velocity of myelinated axons is expected to be linearly related to axon diameter (Rushton, 1951; Goldman and Albus, 1968) and that of non-myelinated axons should be proportional to the square root of diameter (Bennett and Waxman, 1972; Goldman and Albus, 1968; Matsumoto and Tasaki, 1977; Rushton, 1951). These relations assume that axons are “dimensionally similar”, and are dependent upon on factors such as specific membrane properties, internodal distances and myelin thickness. The scaling factor for myelinated axons has been derived largely from peripheral nerve, and considerable variability among axonal systems has been observed (cf., Boyd and Kalu, 1979). Assuming a scaling factor of 5.5 (5.5 m/s for a one micron fibre, including the myelin) and a minimal diameter of 0.2 µm (Waxman and Bennett, 1972),

Time and the brain

152

the smallest myelinated central axons should conduct at velocities of 1.1 m/s, and a 10 µm axon should conduct at about 55 m/s. Central unmyelinated axons are generally from 0.08–0.6 µm in diameter (e.g. Swadlow et al., 1980; Waxman and Swadlow, 1976b). In the periphery they are considerably larger, and axons of 1.1 µm in diameter have conduction velocities of ~2.1 m/s (e.g. Rushton, 1951). However, no systematic studies have examined the scaling factor of central unmyelinated axons, which often conduct at velocities as low as 0.3 m/s (e.g. Bester et al., 1995; Kaba et al, 1986; Manis, 1989; Swadlow, 1994). Assuming that this velocity value represents the finest unmyeli-nated axons, the largest unmyelinated central axons should conduct at a velocity of ~0.8 m/s (given the square root relationship described above). Assuming that axon diameter-conduction velocity relationships are known, two practical considerations seriously limit inferences of conduction time based on axonal diameter and conduction distance: (1) Only a few cortical efferent systems (e.g. the corpus callosum, pyramidal tract) are clearly packaged as fibre bundles accessible to the electron microscope. The axons of most cortical efferent systems mingle in both grey and white matter, and specific systems cannot be readily disentangled (however, see Woodward and Coull, 1984). In classical studies of axonal degeneration following cortical lesions, efferent axons seen coursing towards their targets were often characterized as “course” or “fine” (e.g. Guillery, 1966, 1967). However, such methods, using light microscopy, are generally biased towards the detection of myelinated axons because most unmyelinated axons fall below the level of resolution of the light microscope. The same limitation applies to most studies of cortical axons using retrograde or orthograde tracers (e.g. Boyapati and Henry, 1984; Rockland, 1992). (2) Electron microscopic procedures generally result in each axon being measured at a single point along it’s trajectory. Since axons may vary considerably in diameter and myelination along their length, inferences regarding conduction delays are thereby limited. 2.2. Conduction Delays Inferred from Antidromic Latencies Electrical stimulation of the axonal pathway will elicit antidromic responses in the soma, which can be differentiated from synaptic responses using collision and other procedures (e.g. Bishop et al, 1962; Fuller and Schlag, 1976). A major advantage of this procedure, when compared with inferring conduction time from single measurements of axon diameter, is that the antidromic latency reflects the conduction time along the entire length of axon between stimulation site and soma. However, this method is subject to numerous sampling biases, most of which converge to select against the detection of neurones with slowly conducting axons (see Swadlow, 1998, for a recent review). Factors that contribute to this bias include: (1) Neurones with fine-diameter axons often have small somata which are more difficult to sample than are neurones with larger somata (Humphrey and Weller, 1988; Stone, 1973; Towe and Harding, 1970).

Information flow along neocortical axons

153

(2) Cortical neurones with very fine diameter axons often show no spontaneous impulse activity, and lack sensory receptive fields (Swadlow, 1988, 1990, 1991, 1994). Such neurones are difficult to detect, and are often passed or killed by the microelectrode before they can be studied. (3) Fine diameter axons have higher thresholds to electrical stimulation than do larger diameter axons (Nowak and Bullier, 1996; Ranck, 1975). Thus, whereas a stimulating electrode in close proximity (e.g. <0.5 mm) to a small group of fibres may activate both large and small diameter axons, a somewhat more distant stimulating electrode will selectively activate the larger diameter axons. Increasing stimulus intensity will activate larger axons yet further from the stimulation site, but few unmyelinated axons will be activated at distances of >0.5 mm, even when stimulus intensity is very high (e.g. Swadlow, 1998 [Figure 3]). This results in a strong bias against fine-diameter axons unless stimulating electrodes are in very close proximity to the axons under study. This bias will be heightened when it is deemed necessary to keep stimulus intensity to very low levels in order to limit stimulus spread to a small cortical region. (4) The conduction time along fine preterminal branches may easily be excluded from the measured antidromic conduction time. Although great care may be taken in placing stimulating electrodes near the expected site of axon termination, the impulse may be first initiated some distance away, in a myelinated segment of the axon (where threshold is much lower). Because unmyelinated axons may conduct at velocities of <0.3 m/s (below), one or two millimeters of unmyelinated terminals can add several milliseconds to axonal conduction. (5) Fine-diameter axons often exhibit significant variations in axonal conduction velocity, that result from prior impulse activity (e.g. Ferreyra Moyano and Molina, 1980; Segal et al., 1983; review by Swadlow et al., 1980; see below). Antidromic latency may, therefore, vary considerably from trial-to-trial (in some cases by >15%), depending on the timing of stimulus pulses to antecedent “spontaneous” impulse activity (Swadlow et al., 1978a, b). Although the antidromic nature of the response in such cells can be verified by collision (e.g. Bishop et al., 1962; Fuller and Schlag, 1976) and other procedures (Swadlow et al., 1978b), many experimenters employ a “constant latency” (often defined as <0.1 or 0.2 ms variability) as a defining requirement for antidromic identification; cells showing greater latency variability are thereby rejected from analysis. This creates a strong bias towards the fast-conducting axons, which generally do have a relatively constant antidromic latency. 2.3. Conduction Delays Inferred from Synaptic Latencies Stimulation of one region of cortex will elicit synaptic effects in other regions, and the latencies of these effects depend on conduction times along intervening axons. There are a number of limitations to such procedures: (1) The number of intervening synapses mediating a specific effect is often unknown; (2) In reciprocal pathways, orthodromic synaptic effects cannot be readily differentiated from synaptic effects that are mediated by antidromically-activated recurrent collaterals; and

Time and the brain

154

(3) Synaptic effects mediated by large diameter axons with short conduction times are usually easily observed (because impulses from many such axons arrive nearly simultaneously, and result in considerable temporal summation of EPSPs). However, long latency effects are often temporally dispersed and below threshold for generating spikes or evoked potentials. These problems place limits on the relevance of measured synaptic latencies to understanding the normal conduction delays occurring between cortical areas (cf., Swadlow et al., 1979). However, such procedures may be very useful in determining the fastest possible conduction times between cortical regions.

3. RESULTS FROM THE RABBIT Extracellular recordings were obtained from antidromically identified efferent neurones in rabbit V1, S1, S2 and motor cortex (Swadlow, 1988, 1990, 1991 and 1994). These studies, conducted under virtually identical experimental conditions, were aimed at understanding the common features of parallel processing within diverse neocortical regions, and axonal conduction times were one of the variables under study. Estimated axonal conduction velocities and antidromic latencies of the efferent populations studied are presented in Figures 1 and 2. Also shown are conduction velocities and antidromic latencies of specific thalamocortical axons projecting to V1 and S1 (Swadlow, 1985, 1995). Conduction velocities presented in Figure 1 reflect mean axonal conduction velocities between stimulation sites and recording sites near the soma. 3.1. Callosal Axons Figure 1a shows the estimated axonal conduction velocities for rabbit callosal axons originating in V1, S1, S2 and motor cortex. Note that the axonal stimulation sites in these studies were just contralateral of the midline, so slowing of impulses as they invade preterminal branches in not reflected here. In each of the cortical systems of the rabbit, the mean conduction velocities ranged from 1.47–3.5 m/s, and the slowest callosal axons conducted at velocities of 0.29–0.6 m/s. In each system, a significant population was seen that conducted at velocities of <1 m/s, and a small number of axons conducted at velocities of >5 m/sec. Thus, these axon conduction velocities are indicative of a mixed population of unmyelinated and finediameter myelinated axons. In fact, ultrastructural studies (Waxman and Swadlow, 1976b) found 45% of rabbit splenial axons to be unmyelinated (diameters of 0.08–0.6 µm, mean=0.2 µm) and the remaining to be myelinated with diameters of 0.3–1.85 µm (mean=0.74 µm). Figure 2a shows the antidromic conduction times for the same callosal axons shown in Figure 1a. Note that these conduction times represent somewhat less than 1/2 of the interhemispheric conduction time (this is because the terminal 1/2 of the axonal length is not included, and the excluded portion contains the highly branched and more slowly

Information flow along neocortical axons

155

conducting terminal region). In each of the cortical regions studied in rabbit, a few fast conducting callosal axons result in a small number of very short conduction times (1–2 ms). However, in each system, the conduction times along most fibres are quite long. In S2, for example, conduction times of up to 29.9 ms were seen, implying an interhemispheric conduction time of >60 ms for this neurone. In the above studies of V1, S1, S2 and motor cortex, some rabbit callosal axons projected a collateral to some ipsilateral cortical areas (to V2, S2, S1 and S1 for the studies of V1, S1, S2 and motor cortex, respectively). The proportion of identified callosal axons that projected such branches was smaller in V1 (8%) and S1 (12%) than in S2 or motor cortex (24% in each: Swadlow, 1988, 1990, 1991, 1994). In nearly all cases the site of branching was shown to be temporally near the soma (<1 ms), and conduction velocities were considerably higher in the callosal branch than in the ipsilateral

Time and the brain

156

Figure 1 . Histograms of axonal conduction velocities of various axonal populations. From above downwards: CC, C-IC, CF-5 and CF-6 axons of V1, S1, S2 and motor cortex. In lowest histogram, conduction velocities of thalamocortical axons projecting from LGNd and VB thalamus to V1 and SI, respectively, are shown. These data, and those presented in Figure 2 are derived from Swadlow, 1988–

Information flow along neocortical axons

157

1995. Asterisks shown above uppermost histogram show comparable results from callosal axons of monkey prelunate gyrus (Swadlow et al., 1978a)

Figure 2 . Antidromic latencies of the same populations shown in Figure 1

corticocortical branch (Swadlow, 1991, 1994). A similar ipsilateral projection of callosal

Time and the brain

158

axons has been found in primate (Schwartz and Goldman-Rakic, 1982). 3.2. Ipsilateral Corticocortical Axons Figure 1b presents results from ipsilateral cortical systems studied in each of the cortical regions of the rabbit. In V1 and SI, the projections to V2 and S2, respectively, were studied. In S2 the “feedback” projection to SI was studied, and in motor cortex the projection to SI was studied. In each of these systems, stimulating electrodes were placed, under physiological guidance, in the topographically aligned region of the target structure under study. The diverse nature of these corticocortical projections might lead one to expect considerable differences in their axonal conduction properties. However, this was not found to be the case. In each of the cortical systems, the mean conduction velocities ranged from 0.86–1.4 m/s. In each system, a very significant population was seen to conduct at velocities of <1m/s (lowest velocities of 0.21–0.4 m/s in the four cortical regions), and a small number of axons conducted at velocities of >4 m/sec. Thus, these axon conduction velocities are appropriate for a mixed population of unmyelinated and fine-diameter myelinated axons. Figure 2b shows the antidromic conduction time for the C-IC axons shown in Figure 1b. In each of these systems studied in rabbit, the few fastest conducting axons resulted in a small number of very short conduction times (1–2 ms). However, in each system, the conduction times along most fibres were quite long. In the motor cortical projection to S1, for example, conduction times of up to 32.5 ms are seen. 3.3. Descending Corticofugal Axons of Layer-5 Figure 1c presents results for CF-5 axons studied in the four cortical regions of the rabbit. In V1, results are limited to corticotectal neurones that were antidromically activated via stimulating electrodes placed in topographically aligned regions of the superior colliculus. In SI and in S2, stimulating electrodes were placed within and above the topographically aligned region of ventrobasal thalamus, and in the motor cortex stimulating electrodes were placed within and above ventrolateral thalamus. In the studies of S1, S2 and motor cortex, the thalamic stimulating electrodes were in close proximity to the internal capsule, and high-intensity stimulation was used to activate axons terminating within the thalamus as well as those descending to regions beyond the thalamus. Thus, the CF-5 axons studied in each of these cortical regions undoubtedly represent a heterogeneous mixture of descending corticofugal neurones originating in layer 5. It is, therefore, somewhat surprising that their conduction properties were so homogeneous, and distinct from those of the callosal and ipsilateral corticocortical axons described above, and also distinct from the CF-6 axons described below. Figure 1c shows that mean axonal conduction velocities of each of the CF-5 axonal systems studied ranged from 10.9–12.9 m/s. In none of the systems were axons observed which conducted at velocities of <2 m/s (lowest velocities for the four regions were 3.7–5.7 m/s). Thus, these axon conduction velocities are suggestive of a population of myelinated axons, and no

Information flow along neocortical axons

159

evidence was seen for the presence of any unmyelinated axons. Figure 2C shows the antidromic conduction time for the CF-5 axons. In each of the systems studied, many axons had very short conduction times (1–2 ms), and very few axons with conduction times of >3 ms were seen. 3.4. Descending Corticofugal Axons of Layer-6 Figures 1d and 2d present results for CF-6 axons. In V1, stimulating electrodes were placed within and above the topographically aligned region of the dorsal lateral geniculate nucleus. In SI and in S2, stimulating electrodes were placed within and above the topographically aligned region of ventrobasal thalamus, and in motor cortex stimulating electrodes were placed within and above ventrolateral thalamus. Highintensity stimulation was used to maximize the probability of activating the axons of all of the layer-6 neurones projecting to the thalamus. The mean conduction velocities of CF-6 axons ranged from 1.73–2.4 m/s in the four cortical regions studied. In each region, a large population of CF-6 axons conducted at velocities of <1m/s (lowest velocities of 0.26–0.52 in the four cortical regions), and a small number of axons conducted at velocities of >6 m/sec. Thus, as was the case for the CC and C-IC populations, these axon conduction velocities are consistent with a mixed population of unmyelinated and finediameter myelinated axons. Figure 2d shows the antidromic conduction time for the above CF-6 axons. In each of the regions studied in rabbit, the few fastest conducting axons result in a small number of very short antidromic latencies (1–2 ms). However, in each region, the conduction times along most CF-6 axons were quite long, and antidromic latencies of >25 ms were not uncommon. 3.5. Specific Thalamocortical Axons Figures le and 2e show the high axonal conduction velocities and short conduction times, respectively, of “specific” thalamocortical axons projecting to V1 (from LGNd; Swadlow and Weyand, 1985) and S1 (from VB; Swadlow, 1995). 3.6. Axonal Conduction Velocities and Neuronal Response Properties In each of the cortical regions studied in rabbit, CF-5 axons were more rapidly conducting than CC, C-IC or CF-6 axons. CF-5 neurones also had much higher rates of spontaneous impulse activity, and were much more responsive to peripheral sensory stimulation than were CC, C-IC or CF-6 neurones. Many CC, C-IC and CF-6 neurones lack both spontaneous impulse activity and a demonstrable suprathreshold peripheral receptive field, and in each of the cortical regions studied these characteristics were observed most frequently in neurones with axonal conduction velocities of <1 m/s (Swadlow, 1988, 1990, 1991,1994). It has been shown that many such “silent”, slowly conducting cortical elements do have subthreshold receptive fields (Swadlow and Hicks,

Time and the brain

160

1996, 1997), and the conditions under which such fields reach threshold is currently under study. These observations indicate that cortical neurones with very slowly conducting axons form functionally distinct subclasses within the CC, C-IC and CF-6 populations.

4. RESULTS FROM CATS, MONKEYS AND RODENTS 4.1. Callosal Axons 4.1.1. Anatomy The corpus callosum is one of the most thoroughly studied axonal systems of the neocortex. A large body of information is available concerning the regional and laminar distribution of both the cells of origin and the sites of synaptic contact in the contralateral hemisphere (e.g. Innocenti, 1986 for review). Because callosal axons remain segregated from other white matter systems for considerable distances on either side of the midline they are readily studied using anatomical procedures. Callosal axons have been examined in mouse (Tomasch, 1957), rat (Seggie and Berry, 1972), rabbit (Waxman and Swadlow, 1976), cat (Fleischhauer and Wartenburg, 1967), monkey (LaMantia and Rakic, 1990; Swadlow et al., 1978) and man (Aboitiz et al., 1989, 1992; Tomasch, 1954). In each of these species, a considerable proportion of unmyelinated axons (19–45%) is reported, and the mean diameter of myelinated axons is ~1 µm. Although the mean axonal diameters of all species studies are very similar, a very few large-diameter axons (5–9 µm) axons are present in species with larger brains (i.e., cat, monkey and man). In monkey, regional differences in axonal composition have been observed in the anterior-posterior axis of the corpus callosum (LaMantia and Rakic, 1990). The most rostral sector, for example, contains ~28% unmyelinated axons, and the mean diameter is 0.7 µm (not including the myelin sheath). The most posterior section (splenium), in contrast, was found to contain only 3.5% unmyelinated axons and the mean axonal diameter was 0.9 µm. (However, Swadlow et al., 1980, reported 31% of splenium axons of a single macaque monkey to be unmyelinated). 4.1.2. Conduction velocities and conduction delays Conduction velocities similar to those of rabbit have been reported in sensorimotor cortex of rat (mean axonal velocity=2.9 m/s, range=0.4–8.3 m/s, Catsman-Berrevoets et al., 1980) and visual cortex of the mouse (mean conduction velocities of <1 m/s Simmons and Pearlman, 1983). However, conduction velocities in carnivores and primates may be considerably higher than those seen in rodents and rabbit. Stimulation and recording methods virtually identical to those used in rabbit were applied to callosal axons of the

Information flow along neocortical axons

161

macaque prelunate gyrus (Swadlow et al., 1978a). Figures 1a and 2a (asterisks) show, respectively, the conduction velocities and conduction times (following stimulation near the midline) for these axons. Conduction velocities were considerably higher than those seen in rabbit (median=9 m/s, range=5–27 m/s), and no conduction velocities consistent with unmyelinated fibres were seen. In cat visual cortex, several authors have reported mean or median conduction velocities of 10–15 m/sec (Harvey, 1980; Innocenti, 1980; McCourt et al., 1990; Toyama et al., 1974), although a few neurones with very slowly conducting axons were reported (up to 15 ms; Innocenti, 1980). A higher proportion of very slowly conducting axons has been found in sensorimotor cortex of the cat. Miller (1975) reported antidromic latencies of <2 ms to >30 ms to stimulation near the callosal midline, and estimated axonal conduction velocities to be from <1 m/s to 10 m/s. Zarzecki (1989) reported a mean antidromic latency of 6.7 ms (range=1.9–15.2 ms) between motor cortices of the cat. Assuming a conduction distance of 30 mm, this implies a mean conduction velocity of 4–5 m/s, considerably lower than that reported in visual cortex. The question of whether interhemispheric conduction times are invariant over a large range of species and brain sizes has been recently addressed by Ringo et al. (1994). These authors argue, both from measurements of callosal axon diameter in a broad range of species and from allometric considerations, that conduction times cannot be held constant across this axonal system by compensatory variations in mean axonal diameter. They point out that to equalize the interhemispheric delay in mouse and man would require human callosal axons to be 15 times the diameter of those of the mouse, but that a diameter difference of only 26% is actually observed. They note, however, that extremely rare “giant” axons are seen in the corpus callosum of large brains. For example, Lamantia and Rakic (1990 [Figure 6]) presented an electron micrograph of one such axon in the monkey corpus callosum, with a diameter of ~9 µm. This axon was believed to originate in somatosensory cortex. Given an interhemispheric conduction distance of about 54 mm in this species (Swadlow et al., 1978a), the conduction time along this axon would be ~1 ms (Waxman and Bennett, 1972). Thus, although the mean axonal conduction time of the callosal system of different species is not scale-invariant, the high-velocity end of the frequency distribution may well be. 4.2. Ipsilateral Corticocortical Efferent Systems 4.2.1. Anatomy Each region of the neocortex projects to numerous cortical areas within the ipsilateral hemisphere, and hundreds of retrograde and anterograde tracing studies have characterized the laminar distribution and morphology of the cells of origin of these systems, their terminal arborization patterns and their sites of synaptic contact. There is considerably less information available regarding the axons of specific ipsilateral corticocortical systems. Some degeneration and/or labelling studies have characterized the apparent diameter of axons coursing towards a particular target (e.g. Rockland, 1989,

Time and the brain

162

1992, 1997; Zeki, 1978). In a recent review, Rockland (1997) concluded that most feedback as well as feedforward corticocortical axons of monkey V1 were about 1 µm in diameter. The one exception to this were the axons projecting from V1 to area MT/V5, which were commonly 2–3 µm in diameter. Such observations are very useful in characterizing the medium and large-diameter myelinated axons of a particular system. However, most central unmyelinated axons as well as the finest myelinated axons will be excluded from detection when using light microscopy. 4.2.2. Conduction velocities and conduction delays Visual cortical projections. Bullier et al. (1988) used collision and other methods to identify neurones projecting from area 17 to areas 18 and 19. They found both rapidly conducting (antidromic latencies of <2 ms) and slowly conducting elements (antidromic latencies of >8 ms), with median antidromic latencies of 2–4 ms and 6–8 ms for the projections to area 18 and 19, respectively. Assuming conduction distances of 5 mm for area 18, and 10 mm for area 19, the authors concluded that “the conduction velocities in fibres in each pathway and its reciprocal are of the order of 1 m/s.” Thus, the conduction properties of C-IC axons in cat V1 and rabbit V1 (above, Swadlow, 1988) appear quite similar. Considerably higher axon conduction velocities are seen in the projection from V1 of monkey to area MT (all latencies were <2 ms, Movshon and Newsome, 1996), and from the 17/18 border to area 19 of the cat (conduction velocities of 9–21 m/s; Toyama et al., 1974). However, this latter study employed intracellular recordings from C-IC somata, and also required a “fixed” latency (total range of 0.1 ms) for antidromic identification. Together, these methods would create a strong bias towards recording from neurones with rapidly conducting axons (see above). Somatosensory cortical projections. Neurones projecting from SI to S2 have been studied in cat. Whereas Miller (1975) reported both short (<2 ms) and very long (>30 ms) antidromic latencies, Manzoni et al (1979) found all antidromic latencies to be <2 ms. However, the methods employed in this latter study guaranteed that only rapidly conducting elements would be obtained because a “short latency” was a defining requirement for antidromic identification. A reluctance to consider long latencies as being antidromically mediated is not uncommon in studies of cortical projection neurones. In a pioneering study of corticotectal and corticogeniculate neurons of the cat (Gilbert, 1977) a “short and fixed latency…” was required for antidromic identification and no slowly conducting elements were reported. Later studies of the corticogeniculate system did not require a short and constant latency, and concluded that many corticogeniculate axons were very slowly conducting (Ferster and Lindstrom, 1983; Harvey, 1980; Tsumoto and Suda, 1980). The projection from SI to motor cortex has also been studied in the cat (Waters et al., 1982; Zarzecki et al., 1983; Zarzecki and Wiggin, 1982). Although most latencies were short (means=1.6–2.5 ms), some long antidromic latencies were reported in each of these studies. Waters et al. (1982), for example, found 87% of antidromic latencies to lie between 1–2.2 ms, corresponding to conduction velocities of 3–7 m/s. However, the longest antidromic latency was 6.8 ms (corresponding to a conduction velocity of ~1

Information flow along neocortical axons

163

m/s). A larger number of very slowly conducting elements may have been undetected in the above studies of cat S1 neurones, because these authors took great care to ensure that stimulus current did not spread beyond the motor cortex, using stimulus currents that were limited to <40 µA. Such low currents would favour the activation of large diameter axons, which have lower thresholds to electrical stimulation than do fine diameter axons (Nowak and Bullier, 1996; Ranck, 1975; see Figure 3 of Swadlow, 1998). Motor cortical projections. A mixed population of fast and very slowly conducting axons has also been reported in the motor cortical projection to SI in the cat (Deschênes, 1977). Although most such neurones had rapidly conducting axons (antidromic latencies of <2 ms), ~10% had antidromic latencies of 7–16 ms. 4.3. Descending Corticofugal Axons of Layer 5 4.3.1. Anatomy Layer 5 is the origin of virtually all cortical efferent systems that descend beyond the thalamus (Jones, 1984), as well as many that terminate within the thalamus (e.g. Deschênes et al., 1994; Ojima, 1994; Raczkowski and Diamond, 1978). For some of these systems, single axons branch to terminate in multiple destinations (e.g. Baker et al, 1983; Deschênes et al., 1994). Although the axons of many of these systems intermingle with those of other cortical efferent and ascending axons, some stay segregated for considerable distances. The pyramidal tract is a prime example of this, and this pathway has been subject to detailed morphological analysis in a wide range of species. In each, myelinated fibres show a characteristic skewed diameter distribution, with a peak at ~1 µm and a few fibres as large as 5–10 µm (e.g. Beidenbach et al., 1986; Doetsch and Towe, 1981; von Keyserlingk and Schramm, 1984; Harding and Towe, 1985; Leenen et al., 1985). Unmyelinated pyramidal tract fibres are rare in monkey, forming <1% of the population (Ralston et al., 1987), but are common in both cat (12%, Biedenbach et al., 1986) and rat (30–60%; Harding and Towe, 1985; Leenen et al., 1985). Tracing studies using electron microscopy have shown that the unmyelinated pyramidal tract axons of the rat are of cortical origin (Joosten and Gribnau, 1988). 4.3.2. Conduction velocities and conduction delays The high axonal conduction velocities seen in CF-5 axons of the rabbit are consistent with values reported for two classes of CF-5 axon that have been carefully studied in a number of species: (1) Corticotectal axons have been studied in many cortical areas and in several species (e.g. Finlay et al., 1976; Lemmon and Pearlman, 1981; Weyand et al., 1986, 1991). Medium-high conduction velocities (mean or median values of 8–10 m/s) were reported in all species, and no conduction velocities indicative of unmyelinated axons

Time and the brain

164

were reported in any of these studies. (2) Pyramidal tract neurones (PTNs) have been differentiated into fast and slowly conducting groups, with different membrane and other characteristics (e.g. Baranyi et al, 1993; Sakai and Woody, 1988; Takahashi, 1965). However, both “fast” and “slow” PTNs have conduction velocities indicative of myelinated axons (11–18 m/s and >21 m/s for slow and fast, respectively; Takahashi, 1965). In monkey, PTNs display a similar wide range of antidromic latencies, but all conduction velocities are characteristic of myelinated axons (e.g. Evarts, 1965). Similarly, no conduction velocities indicative of unmyelinated axons were seen amongst antidromically activated PTNs of rat (conduction velocities 5–19 m/s: Mediratta and Nicoll, 1983). This latter finding is puzzling in light of the high numbers of unmyelinated axons in the pyramidal tract of this species (above), and their demonstrated cortical origin (Joosten and Gribnau, 1988). 4.4. Descending Corticofugal Axons of Layer 6 4.4.1. Anatomy Throughout the neocortex, layer 6 is the origin of an extensive descending corticothalamic pathway (e.g. Jones, 1984). Early Golgi studies described fine fibre systems within the LGNd of cat (Guillery, 1966; Szentagothai, 1963) and monkey (Guillery, 1967) and suggested that these were the terminals of corticogeniculate neurones. Although most such fibres were “thin”, some “thick” axons were also observed (Guillery, 1966). Fine-diameter corticothalamic preterminal axons have also been reported using both retrograde and orthograde tracing methods (Boyapati and Henry, 1984; Hoogland et al., 1987; Robson, 1984). Recent single-cell labelling studies (Zhang and Deschenes, 1997) traced rat layer-6 corticothalamic axons from cortex to thalamus, and reported axon diameters of 0.5–0.8 µm. However, most unmyelinated fibres would not have been detected in this light-microscopic analysis and nearly 1/2 the axons were too faintly labelled to follow to thalamic targets. 4.4.2. Conduction velocities and conduction delays The wide spectrum of antidromic latencies seen in CF-6 neurones of rabbit is in agreement with similar values obtained for corticogeniculate neurones of the cat V1 (Ferster and Lindstrom, 1983; Tsumoto and Suda, 1980) and corticothalamic neurones of cat SI (Landry and Dykes, 1985). For example, Ferster and Lindstrom reported antidromic latencies of 2.5–45 ms for corticogeniculate neurones of cat, with >25% of this sample having antidromic latencies of >20 ms.

Information flow along neocortical axons

165

4.5. Specific Thalamocortical Axons Virtually all specific thalamic afferents to the neocortex have axonal conduction velocities characteristic of myelinated axons. In the cat, geniculo-cortical neurones may be separated into numerous sub-groups “x”, “y”, “w”, “lagged” and “non-lagged”) with different conduction times and velocities (e.g. Cleland et al., 1976; Humphrey and Weller, 1988; Wilson et al., 1976). Whereas thalamocortical conduction times for most LGNd neurones are <2 ms, a very small number show latencies of 5–10 ms. Thus, assuming a conduction distance of 20 mm (Humphrey and Weller, 1988), even the slowest of these axons conduct at velocities of >2 m/s, a value consistent with very fine diameter myelinated axons (Waxman and Bennett, 1972). Similarly, geniculo-cortical conduction times of monkey (Marrocco, 1976; Schiller and Malpeli, 1978), tree shrew (Sherman et al., 1975 and rat (Fukuda, 1973) are indicative of a myelinated axonal population.

5. CONDUCTION VELOCITIES OF MANY NEOCORTICAL AXONS ARE NOT STABLE Discussions of cortical axonal conduction time often make the implicit assumption that conduction time between soma and axon terminals is constant, at least over a time scale of minutes. As noted above, some experiments have imposed a constant latency (usually ±0.1 or 0.2 ms) as a defining requirement of antidromic invasion. However, many fine diameter axonal systems demonstrate significant variations in both axonal conduction velocity and excitability, that are dependent on prior impulse activity. Thus, a “supernormal” period of increased axonal conduction velocity and excitability (lasting from 4→100 ms) follows a single prior impulse, and this is followed by a “subnormal” period of decreased conduction velocity and excitability. The depth of the subnormal period increases with number of prior impulses, and can last for several minutes (Ferreyra Moyano and Molina, 1980; Raymond et al., 1990; Segal et al., 1983; Swadlow et al., 1978a). Such behaviour is seen in many central axonal systems composed of small diameter myelinated axons and/or unmyelinated axons (see Swadlow et al., 1980 for review). A pronounced supernormal period is seen in most CC, C-IC and CF-6 axons of rabbit V1, S1, S2 and motor cortex (Swadlow, 1985, 1988, 1990, 1994). However, minimal or no supernormal conduction is seen in CF-5 axons of these regions. Supernormal impulse conduction is also seen in visual callosal axons of the monkey (Swadlow et al., 1978a) and in some callosal (Innocenti, 1980) and ipsilateral corticocortical axons of the cat (Zarzecki and Wiggin, 1982). Interestingly, it appears that no critical fibre diameter or axonal conduction velocity determines the presence or absence of supernormal conduction (Swadlow, 1985), and this phenomenon is seen in myelinated as well as unmyelinated cortical axons (Waxman and Swadlow, 1976a). The variations in axonal conduction time that result from supernormal and subnormal effects may, in some cases,

Time and the brain

166

exceed 15% of the baseline conduction time (taken in the absence of prior impulse activity). The effects of such variations in conduction velocity on impulse conduction time will depend on the baseline activity pattern of the neurone in question. If the firing rate is very low (as is often the case in very slowly conducting axons: Swadlow, 1988, 1990, 1991, 1994), a single impulse occurring every few seconds will have a relatively constant conduction time. However, when the firing rate is high and encompasses a wide range of interspike intervals, axonal conduction velocity of different impulses will vary considerably depending on the temporal relationship of each impulse to prior impulse activity. Thus, an interspike interval distribution measured near the axon terminals of a cortical neurone may differ considerably from the distribution as measured at the site of impulse generation near the soma.

6. AXONAL PROPERTIES AND SHARPLY SYNCHRONOUS ACTIVITY 6.1. Thalamocortical Axons and “Sharp” Synchrony (±1 ms) between Neighbouring Cortical Neurones Synchronous activity between cortical neurones occurs over a wide range of time scales with differing putative underlying mechanisms (Kruger and Aiple, 1988; Kirkwood et al., 1982a; Moore et al., 1970; Nowak et al., 1995; Sears and Stagg, 1976; Swadlow et al., 1998; see chapters by Villa, by Nowak and Bullier, by Eckhorn and by Wennekers and Palm in this volume). Sharply synchronous activity at near “0” delays is sometimes observed in cross-correlograms (CCGs) obtained from closely spaced cortical (e.g. Gochin et al., 1991; Kruger and Aiple, 1988) or subcortical (Alonso et al., 1996; Sears and Stagg, 1976) neurones. One example of such sharply synchronous activity was recently demonstrated between all members of a class of putative GABAergic inhibitory interneurones (suspected inhibitory interneurones, SINs) within individual S1 “barrels” of the rabbit (Swadlow et al., 1998). Sharp synchrony (±1 ms) was seen between all of the SINs within a barrel that received monosynaptic input from ventrobasal thalamus. The above “sharp” synchrony was predicted based on the properties of a richly divergent/convergent network proposed to link nearly all of the SINs of an S1 barrel with neurones of the topographically aligned ventrobasal thalamic “barreloid” (Swadlow, 1995; Swadlow et al., 1998). Figure 3a presents a schematic illustration of the proposed network, and figure 3b shows a CCG obtained from two SINs within an SI barrel recorded via independent microelectrodes separated by ~200 µm. Such synchrony was not seen when SINs were paired with neurones of the same barrel column that did not receive monosynaptic thalamic input or when SINs were paired with other SINs located in neighbouring barrel columns. Control procedures showed that the observed sharp synchrony was not dependent upon peripheral stimulation, was not oscillatory, and survived general anaesthesia (see chapter by Nowak and Bullier, 1999, this volume, for interpretation of cross-correlation histograms). The above thalamocortical network linking SINs of an S1 barrel with ventrobasal

Information flow along neocortical axons

167

thalamic afferents of the aligned thalamic barreloid approaches the description of a “complete transmission line” (Griffith, 1963; Abeles, 1991), where each element in one node of a network makes synaptic contact with all of the members of the successive node. Sharp synchrony among postsynaptic elements of such networks is expected because of near-simultaneous EPSPs generated by the common inputs (cf. Moore et al., 1970). For this reason, and based on findings (Mendell and Henneman, 1971) showing that each muscle spindle 1A afferent fibre branches profusely before contacting nearly all homonymous motoneurones, Sears and Stagg (1976) predicted that a “short-term” synchrony should be observable among the recipient motorneurones, and that the duration of this synchrony should be comparable to the rise time of an EPSP. A very brief synchrony was, in fact, found between intercostal motoneurones, and was subject to considerable subsequent analysis (Kirkwood and Sears, 1978; Kirkwood et al., 1982a, b; Sears and Stagg, 1976).

Figure 3 . a. Highly divergent and convergent network proposed to link a population of VB thalamic barreloid neurones with SINs of the topographically aligned SI barrel (from Swadlow, 1995). b. Crosscorrelogram of spike trains of two SINs located within a single barrel. Each SIN was shown to receive monosynaptic input from VB thalamus. Note the sharply synchronous, near-coincident (±1 ms) responses in the CCG. Responses were generated from “spontaneous” activity, in the absence of peridpheral stimulation (from Swadlow et al., 1998).

The presence and degree of sharp synchrony between two post-synaptic neurones of such divergent and convergent networks will depend on the number of inputs shared by the two neurones and on the “contribution” and “efficacy” (cf., Levick et al., 1972) of these inputs (see Nowak and Bullier, this volume, for definitions of “contribution” and “efficacy”). Figure 4a illustrates a simple schematic illustration, where two neurones (“a”

Time and the brain

168

and “b”) are each presynaptic to neurones “x” and “y” and receive most of their input from these neurones. The hypothetical CCG between the spike trains of neurones “x” and “y” is illustrated in Figure 4b. The central peak occurs at near zero delay. This represents the near-simultaneous spikes in “x” and “y” that result from near-simultaneous EPSPs. This CCG will not be influenced by differences between cell “a” and “b” in conduction velocities along the main axonal trunks. So long as impulses from the two branches of each presynaptic axon reach their targets at the same time, a central peak will result. However, the CCG will be strongly influenced by differences in conduction times along the two branches of neurone “a” or “b”. Figure 4c-d illustrates this point. Here, the preterminal branches of neurone “a” that innervates neurones “x” and “y” are rapidly, and slowly conducting ones, respectively. Similarly, the branches of neurone “b” that

Figure 4 . a. Schematic illustration of a fully divergent/convergent network linking two presynaptic neurones (“a” and “b”) with two postsynaptic neurones (“x” and “y”). Each presynaptic neurone makes a potent “contribution” to the two postsynaptic neurones. Terminal branches of neurone “a” have identical conduction times, as do the terminal branches of neurone “b”. b. Schematic of CCG between spike trains of neurones “x” and “y” shown in Figure 4a c. Situation is identical to that seen in A except one of the branches of each of the presynaptic neurones (dashed lines) is slowly conducting, d. Schematic of CCG between spike trains of neurones “x” and “y” shown in Figure 4c.

innervate neurones “x” and “y” are slowly, and rapidly conducting, respectively. Figure

Information flow along neocortical axons

169

4d presents the schematic CCG between neurones “x and “y” that will result. Here, two peaks are seen. The displaced peaks reflect the different arrival times of impulses generated by each of the presynaptic neurones. When additional presynaptic branches are added to the circuit, the temporal dispersion of the central peak resulting from differing preterminal conduction times in the individual presynaptic neurones will dilute any sharply synchronous responses. 1 When sharp synchrony (±1 ms) does result from a highly divergent and convergent presynaptic network, the above considerations suggest that the presynaptic branches are likely to be rapidly conducting and myelinated until very near the region of synaptic contact. This will ensure near-simultaneous arrival of impulses from each presynaptic neurone at their post-synaptic targets. Since branching of thalamocortical afferents may occur at distances of >2 mm from sites of synaptic termination, unmyelinated terminal branches (which can conduct at velocities as low as 0.2–0.3 m/s) would result in a temporal dispersion of EPSPs in the postsynaptic elements and a resultant broadening of any central peak in the CCGs. In fact, thalamocortical afferents do retain their myelin following considerable branching within the cortex (Deschênes and Landry, 1980; Ferster and LeVay, 1978; Freund et al., 1989), as do retinal ganglion cells (e.g. Famiglietti and Peters; 1972) and 1A afferents to spinal motor neurones (Brown and Fyffe, 1978). Divergence of preterminal branches of each of these three axonal systems is believed to mediate a potent sharp synchrony of postsynaptic neurones (Alonso et al., 1996; Sears and Stagg, 1978; Swadlow et al., 1998). If rapidly conducting, myelinated terminal branches are necessary to achieve sharp synchrony between post-synaptic elements, then it is likely that the main axon trunk must also be myelinated. This conclusion follows from both empirical observations (axons generally decrease in diameter distal to branch points and few, if any, instances of increased diameter have been reported), and from theoretical considerations indicating that axons must decrease in diameter distal to branch points if impulses are to faithfully invade terminal arbors (e.g. Khodorov and Timin, 1975; Parnas et al., 1979 Swadlow, el al., 1980; Waxman, 1975 for reviews). 6.2. Corticocortical Axons and Synchrony between Distant Cortical Neurones The “sharp” synchrony described above, is very brief and non-oscillatory (Swadlow et al., 1998). In contrast, synchrony between distant cortical neurones generally has a longer time course and is often oscillatory (e.g. Eckhorn et al., 1988; Engel et al., 1991; Eckhorn, this volume). However, some types of distant synchrony may be mediated by mechanisms similar to those proposed above for sharp synchrony. For example, “T coupling” (Munk et al., 1995; Nowak et al., 1995; Nowak and Bullier, this volume) occurs between visual cortical neurones in opposite hemispheres, and has a time course of a few ms. In the simplest case, such mterhemispheric synchrony between two distant neurones could result if each received divergent inputs from a set of axons originating in only a single hemisphere. The 1. This analysis assumes that more slowly-conducting branches are randomly distributed to

Time and the brain

170

postsynaptic elements. A sharp peak at various delays can, occur if slowly conducting branches are selectively distributed to one of the postsynaptic neurons under study.

origin of these diverging axons cannot be extrastriate cortex because T coupling survives destruction of these regions (Munk et al., 1995; Nowak and Bullier, this volume). However, rich sources of such divergence are provided by the local recurrent collateral branches of callosal efferent neurones (e.g. White and Czeiger, 1991). In this scheme, synchrony between neurones (located in separate hemispheres) could be mediated by a population of callosal projection neurones. The main interhemispheric axon is envisaged to contact one element, while the recurrent collaterals contact the other. When conduction time along the main axon is very rapid (i.e. 1–2 ms), arrival times of impulses in the two hemispheres may be very similar. Synchrony between the spike trains of these distant neurones would result if they shared a significant number of such common inputs (above). According to this notion, slowly conducting recurrent collaterals may be acting as delay lines (cf., Waxman, 1975), synchronized with main axonal trunks and resulting in near-coincident arrival of impulses at post-synaptic targets in separate hemispheres. If conduction along the main axon was somewhat longer than that along the recurrent collaterals, a commensurate delay in the CCG peak would result. However, the corpus callosum is a reciprocal pathway, and if the two recipient neurones received divergent input from axons originating in both hemispheres, a symmetry about time 0 would result. Thus, the time course of synchrony resulting from such a mechanism would depend on the difference between interhemispheric and intrahemispheric conduction times, and on the proportion of inputs to each of the correlated neurones that originated from callosal efferent neurones in each of the two hemispheres. A predictable consequence of this proposal is that each of the neurones that demonstrate “T coupling” (in opposite hemispheres) should be a member of a relatively small subpopulation of visual cortical neurones that respond robustly to electrical stimulation of the corpus callosum (cf., Harvey, 1980; Innocenti, 1980). In addition, differences between the two neurones in latency to midline stimulation should be correlated with differences in the timing of the peak in the CCG (cf., Swadlow et al., 1998 [Figure 7c]).

Information flow along neocortical axons

171

7. CONDUCTION DELAYS AND CORTICAL PROCESSING Neocortical white matter is composed of axons that span a vast spectrum of diameters and the conduction times along such axons are consequently diverse. The proportion of the very finest, and the very largest axons may differ among both efferent systems and animal species. The finest unmyelinated cortical axons conduct at velocities of <0.5 m/s and conduction times for these axons may be measured in tens of milliseconds. As noted above, these elements sometimes form functionally distinct subpopulations within cortical efferent systems, showing little or no spontaneous impulse activity and only subthreshold or no responses to peripheral stimulation (e.g. Swadlow and Hicks, 1996, 1997; Tsumoto and Suda, 1980). As yet we know little about the role of such slowly conducting axons in cortical processing. Each of the neocortical axonal systems reviewed above had a subpopulation of rapidly conducting axons that mediate conduction times of a few milliseconds or less. The cost, in brain volume, of such rapid communication is quite high (cf., Ringo et al., 1994). A myelinated axon of 1 µm in diameter occupies >100 times the volume of the finest unmyelinated axon of equal length, and this ratio increases to >1:10,000 for the largest cortical axons. It would seem likely, therefore, that the uses of such biologically expensive processes are limited to special situations. Fast conducting axons will, of course, be found in systems that mediate tasks such as sensation and action that require speed, per se. Such axons may also be required when information from widely separated neurone pools must converge to produce coincident input on a post-synaptic target. Thus, for example, binocular responses of some neurones with receptive fields lying near the vertical meridian are mediated by a convergence of inputs from LGNd and the corpus callosum (Berlucchi and Rizzolatti, 1968; Payne et al., 1984), and binocular input to some neurones in the superior colliculus is mediated by a convergence of cortical (fastconducting corticotectal neurones) and retinal input (Palmer and Rosenquist, 1974). Considerations discussed in the previous two sections suggest that large diameter axons may be required to mediate sharply synchronous activity between the spike trains of either closely spaced or distant cortical neurones. Rapidly conducting axons may also be required to achieve synchronous, gamma-band oscillatory activity between neurones in distant cortical areas. Models of interacting non-linear oscillators (Schuster and Wagner, 1989) or neural networks coupled via excitatory (Sompolinsky et al., 1990) or inhibitory (Bush and Sejnowski, 1996) connections have shown that zero phase-lag synchrony between distant areas is dependent upon relatively short delays (i.e. <5 ms or <1/4 the period of oscillation). Thus, whereas unmyelinated axons could mediate such synchrony between closely spaced neural ensembles, myelinated axons would be required when greater distances are involved. In this regard, it is important to determine whether callosal and ipsilateral cortico-cortical projection neurones do take part in such stimulus-dependent oscillatory behaviour and whether fast-or slowly conducting elements are differentially involved. ACKNOWLEDGMENTS: Supported by grants from the National Institute of Neurological Diorders and Stroke (NS-32021) and the National Science Foundation

Time and the brain

172

(IBN-9723967)

REFERENCES Abeles, M. (1991) Corticonics: Neural circuits of the cerebral cortex . Cambridge University Press, Cambridge. Aboitiz, F., Zaidel, E., and Scheibel, A.B. (1989) Variability in fiber composition in different regions of the posterior corpus callosum in humans. Anatomical Record , 223:6A. Aboitiz, F., Scheibel, A.B., Fisher, R.S., and Zaidel, E. Fiber composition of the human corpus callosum. Brain Behavior and Evolution , 143–153. Alonso, J.-M., Usrey, W.M., and Reid, R.C. (1996) Precisely correlated firing in cells of the lateral geniculate nucleus. Nature, London , 383, 815–819. Baker, J., Gibson, A., Mower, G., Robinson, F., and Glickstein, M. (1983) Cat visual corticopontine cells project to the superior colliculus. Brain Research , 265 , 227–232. Baranyi, A., Szente, M.B., and Woody, C.D. (1993) Electrophysiological: characterization of different types of neurons recorded in vivo in the motor cortex of the cat. II. Membrane parameters, action potentials, current induced voltage responses and electrotonic structures. Journal of Neurophysiology , 69 , 1865–1879. Berlucchi, G., and Rizzolatti, G. (1968) Binocularly driven neurons in visual cortex of split-chiasm cats. Science, New York , 159 , 308–310. Bester, H., Menendez, L., Besson, J.M., and Bernard, J.F. (1995) Spino (trigemino) parabrachiohypothalamic pathway: electrophysiological evidence for an involvement in pain processes. Journal of Neurophysiology , 73, 568–585, 1995. Biedenbach, M.A., De Vito, J.L., and Brown, A.C. (1986) Pyramidal tract of the cat: axon size and morphology. Experimental Brain Research , 61 , 303–310. Bishop, G.H., and Smith, J.M. (1964) The sizes of nerve fibers supplying cerebral cortex. Experimental Neurology , 9, 483–501. Bishop, P.O., Burke, W., and Davis, R. (1962) The identification of single units in central visual pathways. Journal of Physiology (London) , 162, 409–431. Boyapati, J., and Henry, G. Corticofugal axons in the lateral geniculate nucleus of the cat. Experimental Brain Research , 53, 335–340. Boyd, I.A., and Kalu, K.U. (1979) Scaling factor relating conduction velocity and diameter for myelinated afferent nerve fibres in the cat hind limb. Journal of Physiology (London) , 289, 277–291. Brown, A.G., and Fyffe, R.E.W. (1978) The morphology of group la afferent fibre collaterals in the spinal cord of the cat. Journal of Physiology (London) , 274, 111– 127. Bullier, J., McCourt, M.E., and Henry, G.H. (1988) Physiological studies on the feedback connection to the striate cortex from cortical areas 1B and 19 of the cat. Experimental Brain Research , 70, 90–98. Bush, P., and Sejnowski, T. (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. Journal of Computational Neuroscience , 3, 91–110. Catsman-Befrevoets, C.E., Lemon, R.N., Verburgh, C.A., Bentivoglio, M., and Kuypers, H.G.J.M. (1980) Absence of callosal collaterals derived from rat corticospinal neurons: a study using fluorescent retrograde tracing and electrophysiological techniques. Experimental Brain Research , 39, 433–440.

Information flow along neocortical axons

173

Cleland, E.G., Levick, R.W., Morstyn, R., and Wagner, H.G. (1976) Lateral geniculate relay of slowly conducting retinal afferents to cat visual cortex. Journal of Physiology, (London) , 255, 299–320. Deschênes, M. (1977) Dual origin of fibers projecting from motor cortex to S1 in cat. Brain Research , 132, 159–162. Deschênes, M., Bourassa, J., and Pinault, D. (1994) Corticothalamic projections from layer V cells in rat are collaterals of long-range corticofugal axons. Brain Research , 664 , 215–219. Deschênes, M., and Landry, P. (1980) Axonal branch diameter and spacing of nodes in the terminal arborization of identified thalamic and cortical neurons. Brain Research , 191, 538–544. Doetsch, G.S., and Towe, A.L. (1981) The pyramidal system of the woodchuck. Brain Behavior and Evolution , 19, 37–55. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H.J. (1988) Coherent oscillations: a mechanism of feature linking iri the visual cortex? Biological Cybernetics , 60 :121–130. Engel, A.K., Konig, P., Kreiter, A.K., and Singer, W. Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, New York , 252, 1177– 1179. Evarts, E.V. (1965) Relation of discharge frequency to conduction velocity in pyramidal tract neurons. Journal ofNeurophysiology , 28, 216–228. Famiglietti, E.V.Jr., and Peters, A. (1972) The synaptic glornerulus and the intrinsic neuron in the dorsal lateral geniculate nucleus of the cat. Journal of Comparative Neurology , 144, 285–334. Ferreyra Moyano, H., and Molina, J.C. (1980) Axonal projections and conduction properties of olfactory peduncle neurons in the rat. Experimental Brain Research , 39, 241–248. Ferster, D., and LeVay, S. (1978) The axonal arborizations of lateral geniculate neurons in the striate cortex of the cat. Journal of Comparative Neurology , 182, 923–944. Ferster, D., and Lindstrom, S. (1983) An intracellular analysis of geniculo-cortical connectivity in area 17 of the cat. Journal of Physiology, (London) , 342, 181–215, 1983. Finlay, B.L., Schiller, P.H., and Volman, S.F. (1976) Quantitative studies of single cell properties in monkey striate cortex. IV. Corticotectal cells. Journal of Neurophysjology , 39, 1352–1361. Fleischhauer, K., and Wartenburg, H. (1967) Elektronenmikroskopische untersuchungen uber das wachstum der nervenfasern and Uber das auftreten von markscheiden im corpus callosum der katze. Zeitschrift fur Zellforschung und Mikroskopische Anatomie , 83, 568–581. Freund, T.F., Martin, K.A.C., Soltesz, I., Somogyi, P., and Whitteridge, D. (1989) Arborization pattern and postsynaptic targets of physiologically identified thalamocortical afferents in striate cortex of the macaque monkey. Journal of Comparative Neurology , 289, 315–336. Fukuda, Y. (1973) Differentiation of principal cells of the rat lateral geniculate nucleus into two groups: fast and slow cells. Experimental Brain Research , 17, 242–260. Fuller, J.H., and Schlag, J. (1976) Determination of antidromic excitation by the collision test: Problems of interpretation. Brain Research , 122, 283–298. Gibson, A., Baker, J. Mower, G., and Glickstein, M. Corticopontine cells in area 18 of the cat. Journal of Neurophysiology , 41, 494–495.

Time and the brain

174

Gilbert, C.D. (1977) Laminar differences in receptive field properties of cells in cat primary visual cortex. Journal of Physiology (London) , 268, 391–421, 1977. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of the awake macaque. Experimental Brain Research , 84 , 505–516. Goldman, L., and Albus, S. (1968) Computation of impulse conduction in myelinated fibres: theoretical basis of the velocity-diameter relation. Biophysical Journal , 8, 596– 607. Griffith, J.S. (1963) On the stability of brain-like structures. Biophysical Journal , 3, 299–308. Guillery, R.W. (1967) A study of Golgi preparations from the dorsal lateral geniculate nucleus of the adult cat. Journal of Comparative Neurology , 128, 21–50, 1966. Guillery, R.W. (1967) Patterns of fiber degeneration in the dorsal lateral geniculate nucleus of the cat following lesions in the visual cortex. Journal of Comparative Neurology , 130, 197–222. Harding, G.W., and Towe, A.L. (1985) Fiber analysis of the pyramidal tract of the laboratory rat. Experimental Neurology , 87, 503–518, 1985. Harvey, A.R. (1980) A physiological analysis of subcortical and commissural projections of areas 17 and 18 of the cat. Journal of Physiology, (London) , 302, 507–534. Hoogland, P.V., Welker, E., and Van der Loos, H. (1987) Organization of the projections from barrel cortex to thalamus in mice studied with Phaseolus vulgaris-leucoagglutinin and HRP. Experimental Brain Research , 68, 73–87. Humphrey, A.L., and Weller, R.E. (1988a) Functionally distinct groups of X-cells in the lateral geniculate nucleus of the cat. Journal of Comparative Neurology , 268, 429– 441. Innocenti, G.M. (1980) The primary visual pathway through the corpus callosum: morphological and functional aspects in the cat. Archives Italiennes de Biologie , 118, 124–188. Innocenti, G.M. (1986) General organization of callosal connections in the cerebral cortex. In: E.G.Jones and A.Peters (eds.) Cerebral Cortex, Vol. 5 , Plenum Press, New York. Jones, E.G. (1984) Laminar distribution of cortical efferent cells. In: A.Peters and E.G.Jones (eds) Cerebral cortex. Vol. 1. Cellular components of the cerebral cortex , Plenum Press, New York. Joosten, E.A., and Gribnau, A.A. (1988) Unmyelinated corticospinal axons in adult rat pyramidal tract. An electron microscopic tracer study. Brain Research , 459 , 173–177. Kaba, H., Saito, H., Otsuka, K., and Seto, K. (1986) Ventrolateral medullary neurons projecting to the medial preoptic/anterior hypothalamic area through the medial forebrain bundle: an electrophysiological study in the rat. Experimental Brain Research , 63, 369–374. Khodorov, B.I., and Timin, E.N. (1975) Nerve impulse propagation along nonuniform fibres. Progress in Biophysics and Molecular Biology , 30, 145–184. Kirkwood, P.A., and Sears, T.A. (1978) The synaptic connexions to intercostal motoneurones as revealed by the average common excitation potential. Journal of Physiology (London) , 275, 103–134. Kirkwood, P.A., Sears, T.A., Tuck, D.L., and Westgaard, R.H. (1982a) Variations in the time course of the synchronization of intercostal motoneurones in the cat. Journal of Physiology (London) , 327, 105–135. Kirkwood, P.A., Sears, T.A., Stagg, D., and Westgaard, R.H. (1982b) The spatial

Information flow along neocortical axons

175

distribution of synchronization of intercostal motoneurones in the cat. Journal of Physiology (London) , 327, 137–155. Kruger, J., and Aiple, F. (1988) Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. Journal of Neurophysiology , 60, 798–828. LaMantia, A.S., and Rakic, P. (1990) Cytological and Quantitative characteristics of four cerebral commissures in the rhesus monkey. Journal of Comparative Neurology , 291, 520–537. Landry, P., and Dykes, R.W. (1985) Identification of two populations of corticothalamic neurons in cat primary somatosensory cortex. Experimental Brain Research , 60, 289– 298. Lesnen, L.P., Masd, J., Posthuma, P.R., and Nieuwenhuys, R. (1985) A detailed morphometrical analysis of the pyramidal tract of the rat. Brain Research , 359, 65–80. Lemmon, V., and Pearlman, A.L. (1981) Does laminar position determine the receptive field properties of cortical neurons? A study of corticotectal cells in area 17 of the normal mouse and the reeler mutant. Journal of Neuro science , 1, 83–93. Levick, W.R., Cleland, G.G., and Dubin, M.W. (1972) Lateral geniculate neurons of the cat: retinal inputs and physiology. Investigative Ophthalmology , 11 , 302–311. Manis, P.B. (1989) Responses to parallel fiber stimulation in the guinea pig dorsal cochlaar nucleus in vitro. Journal of Neurophysiology , 61 , 149–161. Marrocco, R.T. (1976) Sustained and transient cells in monkey lateral geniculate nucleus: conduction velocities and response properties. Journal of Neurophysiology , 39, 340– 353. Manzoni, T., Caminiti, R., Spidalieri, G., and Morelli, E. (1979) Anatomical and functional aspects of the associative projections from somatic areas S1 to S11. Experimental Brain Research , 34, 453–470. Matsumoto, G., and Tasaki, I. (1977) A study of conduction velocity in nonmyelinated nerve fibers. Biophysical Journal , 20, 1–13. McCourt, M.E., Thalluri, J., and Henry, G.H. (1990) Properties of area 17/18 border neurons contributing to the visual transcallosal pathway in the cat. Visual Nsuroscience , 5, 83–98. Mediratta, N.K., and Nicoll, J.A. (1983) Conduction velocities of corticospinal axons in the rat studied by recording cortical antidromic responses. Journal of Physiology (London) , 336, 545–561. Mendell, L.M., and Henneman, E. (1971) Terminals of single la fibres: location, density and distribution within a pool of 300 homonymous motoneurons. Journal of Neurophysiology , 34, 171–187. Miller, R. (1975) Distribution and properties of commissural and other neurons in cat sensorimotor cortex. Journal of Comparative Neurology , 164, 361–374. Moore, G.P., Segundo, J.P., Perkel, D.H., and Levitan, H. (1970) Statistical signs of synaptic interaction in neurons. Biophysical Journal , 10 , 876–900. Movshon, J.A., and Newsome, W.T. (1996) Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. Journal of Neuroscience , 16, 7733–7741. Munk, M.H.J., Nowak, L.G., Nelson, J.H.I., and Bullier, J. (1995) Structural basis of cortical synchronization 11. Effects of cortical lesions. Journal of Neurophysiology , 74, 2401–2414. Nowak, L.G., and Bullier, J. (1996) Spread of stimulating current in the cortical gray matter of rat visual cortex studied on a new in vitro slice preparation. Journal of

Time and the brain

176

Neuroscience Methods , 67 , 237–248. Nowak, L.G., Munk, M.H.J., Nelson, J.I., James, A.C., and Bullier, J. (1995) Structural basis of cortical synchronization. 1. Three types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–2400. Ojima, H. (1994) Terminal morphology and distribution of corticothalamic fibers originating from layers 5 and 6 of cat primary auditory cortex. Cerebral Cortex , 4, 646–663. Palmer, L.A., and Rosenquist, A.L. (1974) Visual receptive fields of single striate cortical units projecting to the superior colliculus of the cat. Brain Research , 67, 27–42. Parnas, I., and Segev, I.A. (1979) Mathematical model for conduction of action potentials along bifurcating axons. Journal of Physiology (London) , 295, 323–343, 1979. Payne, B.R., Pearson, H.E., and Berman, N. (1984) Role of corpus callosum in functional organization of cat striate cortex. Journal of Neurophysiology , 52, 570–593. Raczkowski, D., and Diamond, I.T. (1978) Connections of striate cortex of Galago senegalensis. Brain Research , 144, 383–388. Ralston, D.D., Milroy, A.M., and Ralston, H.J. 3d. (1987) Non-myelinated axons are rare in the medullary pyramids of the macaque monkey. Neuroscience Letters , 73, 215– 219. Ranck, James B. (1975) Which elements are excited in electrical stimulation of mammalian central nervous system: A review. Brain Research , 98, 417–440. Raymond, S.A., Thalhammer, J.G., Popitz-Bergez, F., and Strichartz, G.R. (1990) Changes in axonal impulse conduction correlate with sensory modality in primary afferent fibers in the rat. Brain Research , 526, 318–321. Ringo, J.L., Doty, R.W., Demeter, S., and Simard, P.Y. (1994) Time is of the essence: a conjecture that hemispheric specialization arises from interhemispheric conduction delay. Cerebral Cortex , 4, 331–343. Robson, J.A. (1984) Reconstructions of corticogeniculate axons in the cat. Journal of Comparative Neurology , 225, 193–200. Rockland, K.S. (1989) Bistratified distribution of terminal arbors of individual axons projecting from area V1 to middle temporal area (MT) in the macaque monkey. Visual Neuroscience , 3, 155–170. Rockland, K.S. (1992) Configuration, in serial reconstruction, of individual axons projecting from area V2 to V4 in the macaque monkey. Cerebral Cortex , 2, 353–374. Rockland, K.S. (1997) Elements of cortical architecture: Hierarchy Revisited. Cerebral Cortex , 12, 243–293. Rushton, W.A.H. (1951) A theory of the effects of fibre size in medullated nerve. Journal of Physiology (London) , 115, 101–122. Sakai, H., and Woody, C.D. (1988) Relationships between axonal diameter, soma size, and axonal conduction velocity of HRP-filled, pyramidal tract cells of awake cats. Brain Research , 460, 1–7. Schiller, Ph.H., and Malpeli, J.G. (1978) Functional specificity of lateral geniculate nucleus laminae of the rhesus monkey. Journal of Neurophysiology , 41, 788–797. Schuster, H.G., and Wagner, P. (1989) Mutual entrainment of two limit cycle oscillators with time delayed coupling. Progress in Theoretical Physics , 81 , 939–945. Schwartz, M.L., and Goldman-Rakic, P.S. (1982) Single cortical neurons have axon collaterals to ipsilateral and contralateral cortex in fetal and adult primates. Nature, London , 299 , 154–156, 1982. Sears, T.A., and Stagg, D. (1976) Short-term synchronization of intercostal motoneurone activity. Journal of Physiology (London) , 263, 357–387.

Information flow along neocortical axons

177

Segal, M., Foote, S.L., and Aston-Jones, G. (1983) Physiological properties of ascending locus coeruleus axons in the squirrel monkey. Brain Research , 274, 381–387. Seggie, J., and Berry, M. (1972) Ontogeny of interhemispheric evoked potentials in the rat: significance of myelination of the corpus callosum. Experimental Neurology , 35, 215–232. Sherman, S.M., Norton, T.T., and Casagrande, V.A. (1975) X- and Y-cells in the dorsal lateral geniculate nucleus of the tree shrew (Tupaia glis). Brain Research , 93 , 152– 157. Simmons, P.A., and Pearlman, A.L. (1983) Receptive field properties of transcallosal visual cortical neurons in the normal and reeler mouse. Journal of Neurophysiology , 50, 838–848. Sompolinsky, H., Golomb, D., and Kleinfeld, D. (1990) Global processing of visual stimuli in a neural network of coupled oscillators. Proceedings of the National Academy of Sciences, U.S.A. , 87, 7200–7204. Stone, J. (1973) Sampling properties of microelectrodes assessed in the cat’s retina. Journal of Neurophysiology , 36, 1071–1079. Swadlow, H.A. (1985) Physiological properties of individual cerebral axons studied in vivo for as long as one year. Journal of Neurophysiology , 54, 1346–1362. Swadlow, H.A. (1988) Efferent neurons and suspected interneurons in binocular visual cortex of the awake rabbit: receptive fields and binocular properties. Journal of Neurophysiology , 59, 1162–1187. Swadlow, H.A. (1990) Efferent neurons and suspected interneurons in S-1 forelirnb representation of the awake rabbit: receptive fields and axonal properties. Journal of Neurophysiology , 63, 1477–1498. Swadlow, H.A. (1991) Efferent neurons and suspected interneurons in second somatosensory cortex of the awake rabbit: receptive fields and axonal properties. Journal of Neurophysiology , 66 , 1392–1409. Swadlow, H.A. (1994) Efferent neurons and suspected interneurons in motor cortex of the awake rabbit: axonal properties, sensory receptive fields and subthreshold synaptic inputs. Journal of Neurophysiology , 71 , 437–453. Swadlow, H.A. (1995) Influence of VPM thalamic afferents on putative inhibitory interneurons in SI of the awake rabbit: Evidence from: cross-correlation, microstimulation and latencies to peripheral sensory stimulation. Journal of Neurophysiology , 73, 1584–1599. Swadlow, H.A. (1998) Neocortical efferent neurons with very slowly conducting axons: strategies for reliable antidromic identification. Journal of Neuro science Methods , 79, 131–141. Swadlow, H.A., Beloozerova, I., and Sirota, M. (1998) Sharp, local synchrony among putative feed-forward inhibitory interneurons of rabbit somatosensory cortex. Journal of Neurophysiology , 79, 567–582. Swadlow, H.A., Geschwind, N., and Waxman, S.G. (1979) Commissural transmission in humans. Science, New York , 204 , 530–531. Swadlow, H.A., and Hicks, T.P. (1996) Somatosensory cortical efferent neurons of the awake rabbit: latencies to activation via supra- and subthreshold receptive fields. Journal of Neurophysiology , 75, 1753–1759. Swadlow, H.A., and Hicks, T.P. (1997) Subthreshold receptive fields and baseline excitability of “silent” S1 callosal neurons in awake rabbits: contributions of AMPA/kainate and NMDA receptors. Experimental Brain Research , 115, 403–409. Swadlow, W.A., Kocsis, J.D., and Waxman, S.G. (1980) Modulation of impulse

Time and the brain

178

conduction along the axonal tree. Annual Reviews of Biophysics and Bioengineering , 9, 143–179. Swadlow, H.A., Rosene, D.L., and Waxman, S.G. (1978a) Characteristics of interhemispheric impulse conduction between prelunate gyri of the rhesus monkey. Experimental Brain Research , 33, 455–467. Swadlow, H.A., Waxman, S.G., and Geschwind, N. (1980) Small diameter nonmyelinated axons in the primate corpus callosum. Archives of Neurology , 37, 114– 115. 1980. Swadlow, H.A., Waxman, S.G., and Rosene, D.L. (1978b) Latency variability and the identification of antidromically activated units in mammalian brain. Experimental Brain Research , 32, 439–443. Swadlow, H.A., and Weyand, T.G. (1985) Receptive-field and axonal properties of neurons in the dorsal lateral geniculate nucleus of awake unparalyzed rabbits. Journal of Neurophysiology , 54, 168–183. Szentagothai, J. (1963) The structure of the synapse in the lateral geniculate body. Acta Anatomica , 55, 166–185. Takahashi, K. (1965) Slow and fast pyramidal tract cells and their respective membrane properties. Journal of Neurophysiology , 28 , 908–924. Tomasch, J. (1954) Size, distribution, and number of fibers in the human corpus callosum. Anatomical Record , 119, 119–135, 1954. Tomasch, J. (1957) The number of fibers in the corpus callosum of the white mouse. Journal of Comparative Neurology , 107, 165–168. Towe, A.L., and Harding, G.W. (1970) Extracellular microelectrode sampling bias. Experimental Neurology , 29 , 366–38. Toyama, K., Matsunami, K., Ohno, T., and Tokashiki, S. (1974) An intracallular study of neuronal organization in the visual cortex. Experimental Brain Research , 21, 46–66. Tsumoto, T., and Suda, K. (1980) Three groups of cortico-geniculate neurons and their distribution in binocular and monocular segments of cat striate cortex. Journal of Comparative Neurology , 193, 223–236. von Keyserlingk, D.G., and Schramm, U. (1984) Diameter of axons and thickness of myelin sheaths of the pyramidal tract fibres in the adult human medullary pyramid. Anatomischer Anzeiger , 2, 97–111. Waters, R.S., Favorov, O., and Asanuma, H. (1982) Physiological properties and patterns of projection of cortico-cortical connections from the anterior bank of the ansate sulcus to the motor cortex, area 4 (gamma), in the cat. Experimental Brain Research , 46 , 403–412. Waxman, S.G. (1975) Integrative properties and design principles of axons. International Reviews of Neurobiology , 18, 1–40. Waxman, S.G., and Bennett, M.V.L. (1972) Relative conduction velocities of small myelinated and non-myelinated fibres in the central nervous system. Nature, New Biology , 238, 217–219. Waxman, S.G., and Swadlow, H.A. (1976a) Morphology and physiology of visual callosal axons: Evidence for a supernormal period in central myelinated axons. Brain Research , 113, 179–187. Waxman, S.G., and Swadlow, H.A. (1976b) Ultrastructure of visual callosal axons in the rabbit. Experimental Neurology , 53, 115–128. Waxman, S.G., and Swadlow, H.A. (1977) The conduction properties of axons in central white matter. Progress in Neurobiology , 8, 297–324, 1977. Weyand, T.G., Malpeli, J.G., Lee, C., and Schwark, H.D. (1986) Cat area 17. III

Information flow along neocortical axons

179

Response properties and orientation anisotropies of corticotectal cells. Journal of Neurophysiology , 56, 1088–1101. Weyand, T.G., Malpeli, J.G., and Lee, C. (1991) Area 18 corticotectal cells: response properties and identification of sustaining geniculate inputs. Journal of Neurophysiology , 65, 1078–1088. White, E. (1979) Thalamocortical synaptic relations: a review with emphasis on the projections of specific thalamic nuclei to the primary sensory areas of the neocortex. Brain Research Reviews , 1, 275–311, 1979. White, E.L., and Czeiger, D. (1991) Synapses made by axons of callosal projection neurons in mouse somatosensory cortex: emphasis on intrinsic connections. Journal of Comparative Neurology , 303, 233–244, 1991. Woodward, W.R., and Coull, B.M. (1984) Localization and organization of geniculocortical and corticofugal fiber tracts within the subcortical white matter. Neuroscience , 12 , 1089–1099. Wilson, P.D., Rowe, M.H., and Stone, J. (1976) Properties of relay cells in cat’s lateral geniculate nucleus: a comparison of W-cells with X- and Y-cells Journal of Neurophysiology , 39, 1193–1209. Zhang, Z.-W., and Deschenes, M. (1997) Intracortical axonal projections of lamina V1 cells of the primary somatosensory cortex in the rat: a single- cell labeling study. Journal of Neuroscience , 17 , 6365–6379. Zarzecki, P. (1989) Influence of somatosensory cortex on different classes of cat motor cortex output neuron. Journal of Neurophysiology , 62 , 487–94, 1989. Zarzecki, P., Blum, P.S., Bakker, D.A., and Herman, D. (1983) Convergence of sensory inputs upon projection neurons of somatosensory cortex: vestibular, neck, head, and forelimb inputs. Experimental Brain Research , 50, 408–14. Zarzecki, P., and Wiggin, D.M. (1982) Convergence of sensory inputs upon projection neurons of somato-sensory cortex. Experimental Brain Research , 48, 28–42. Zeki, S.M. (1978) The cortical projections of foveal striate cortex in the rhesus monkey. Journal of Physiology (London) , 277, 227–244, 1978.

5 Psychophysics of Human Timing Thomas H.Rammsayer 1 and Simon Grondin 2 1 Georg

Elias Mueller Institute of Psychology, University of Goettingen, Gosslerstrasse 14, D-37073, Goettingen, Germany Tel: (0049) 551–393611; FAX: (0049) 551–393662 e-mail: [email protected] 2 Ecole de Psychologie, Universite Laval, Quebec, Qc, Canada G1K 7P4 Tel: (001) 418–656–2131; FAX: (001) 418–656–3646 e-mail: [email protected]

This chapter introduces the main methods and findings in the field of psychophysics of human timing. Furthermore, two theoretical views of human timing are proposed. It has been shown that, for temporal intervals below 1 s, the Weber fraction generally decreases with longer interval durations. A generalized form of Weber’s law is often reported to hold for the temporal domain and, thus, supports the notion of a single timing mechanism underlying temporal processing. On the other hand, a series of behavioural and pharmacopsychological studies is presented that suggests the presence of two different timing mechanisms for temporal processing. One mechanism, involved in processing of very brief durations, appears to be located at a subcortical level, while the other mechanism, involved in processing of longer intervals (>1 s), is based on memory processes. KEYWORDS: Time estimation; Time perception; Psychophysical methods; Timing mechanisms This chapter reviews major recent findings in the field of human temporal information processing. After providing the reader with an overview of the methods for addressing the problem of temporal processing, several studies are reviewed, in order to compare the relative accuracy for estimating short and long time intervals.

1. PSYCHOPHYSICAL METHODS FOR ASSESSING PERFORMANCE OF TEMPORAL PROCESSING Traditionally, there are four major methods for assessing accuracy of temporal processing (Bindra and Waksberg, 1956; Clausen, 1950; Doob, 1971; Hicks et al., 1976; Wallace and Rabin, 1960; Zakay, 1990). The main characteristics of each method can be briefly summarized as follows. In a verbal estimation task, the duration of the target interval is

Time and the brain

182

estimated verbally in terms of temporal units. A production task involves producing an interval equal to a duration that is verbally indicated. With the method of reproduction, after presentation of a target interval, an attempt is made to reproduce its duration by means of some operation. Finally, the method of comparison involves the decision, after the presentation of two intervals, which one appeared to be longer. Although none of the methods has shown general superiority to the others (Allan, 1979; Carlson and Feinberg, 1970; McConchie and Rutschmann, 1971; Zakay, 1993), several studies suggest that verbal estimation and production are less reliable than operative reproduction (e.g. Block, 1989; Zakay, 1990). In verbal estimation, subjects give a verbal estimate in conventional temporal units, i.e. clock time such as seconds or minutes, whereas in production subjects produce a duration stated by the experimenter in clock time. Obviously, both methods use a translation of duration into socially learnt time units and, thus, depend on the relation of subjective time to clock time (Block, 1989; Clausen, 1950). In contrast, the method of reproduction as well as the method of comparison do not require such a translation (Block, 1989) and, therefore, results obtained by both these latter methods appear to be freer from such a possible confound (Eisler and Eisler, 1994; Rammsayer, 1997a). With the method of reproduction, subjects activate and stop some measurement device. While this appears to be an appropriate procedure for estimation of longer durations, motor response latencies are much too long for reliable assessment of timing of extremely brief durations. Therefore, for quantification of performance on temporal processing of brief durations in the range of milliseconds, the method of comparison has to be used. Another important factor in evaluating the existing literature on human timing performance is the methodological distinction between prospective and retrospective experimental paradigms (Block, 1989; Block and Zakay, 1997; Boltz, 1995; Brown, 1985; Brown and Stubbs, 1988; Gilliland et al., 1946; James, 1890; Zakay, 1990). In the prospective paradigm, subjects know in advance that they will be required to make a time judgement and, will therefore, be able to activate specific timing processes and attend to any available temporal information (Doob, 1971; Zakay, 1990). In the retrospective paradigm, on the contrary, subjects are unexpectedly asked to judge the duration of a temporal interval after it has already passed by. Therefore, in the retrospective paradigm, subjects incidentally encode temporal information which may be retrieved from memory some time later (Block and Zakay, 1997). Hence, it is plausible to assume that retrospective time judgements are more incidental and more variable than are prospective ones (Block and Zakay, 1997; Brown, 1985; Zakay, 1990). For this reason, the present review of psychophysical data on temporal processing in humans will focus exclusively on studies applying a prospective paradigm. On the strict psychophysical ground, one can distinguish two main methods for studying time (Allan, 1979): duration scaling and duration discrimination. With the scaling method, one can use verbal estimation, interval production, or interval reproduction. The purpose of this method is to relate the magnitude of a psychological or subjective impression ( Ψ ) to the magnitude of a physical stimulus ( S ) under investigation. This relationship between Ψ and S is exponential, linear, or logarithmic. Stevens (1975) reports that this relationship obeys a power law:

Psychophysics of human timing

183

(1) where k is a constant whose value depends on physical units used, and n is an exponent specific to the given continuum. When plotted on log-log co-ordinates, the functions are straight lines with n being equal to 1 if the relation is linear (constant steepness), greater than 1 if the relation is one of increasing steepness, and lower than 1 if the relation is one of decreasing steepness. Thus, n is the main characteristic of each sensory continuum and is sometimes referred to as the signature of a continuum. For the temporal continuum, the value of n is often reported to range around .9 (Eisler, 1976), with some authors arguing that psychological time tends to correspond to real time and, hence, n is assigned a value of 1 (see Allan, 1979). Even more important for the sections below is the understanding of results observed with duration discrimination tasks. In such tasks, the method of comparison is employed, and the sensitivity of subjects can be measured by determining the difference threshold as a psychophysical indicator of performance. Traditionally, there are three methods for threshold estimation in psychophysics: a) method of constant stimuli; b) method of limits; and c) method of adjustment (Gescheider, 1985). For investigating temporal processing in humans, the first two methods are employed most frequently. With the method of constant stimuli, the difference threshold is estimated on the basis of a psychometric function where the probability of judging a comparison value as longer than a standard is plotted as a function of this comparison value. The difference threshold, or just noticeable difference, is most often defined as half the interquartile range (Luce and Galanter, 1963). In this case, the interquartile range is represented by the difference between the x values corresponding to 75% and 25% of long responses. In other cases, difference threshold has been defined as one standard deviation of the psychometric function (Getty, 1975; Killeen and Weiss, 1987). With the method of limits, there are series of trials where a comparison value is gradually increased or decreased in fixed steps. Difference thresholds are estimated on the basis of transition points where a standard and a comparison value start or stop to be perceived as equal. A variation of the method of limits for estimation of difference thresholds represents adaptive psychophysical procedures (Kaernbach, 1991; Pentland, 1980; Rammsayer, 1992a). “Adaptive” means that the duration of the comparison interval is adjusted from trial to trial in order to reach a given probability of responding correctly. Depending on the rule chosen to change the duration of the comparison interval, adaptive procedures converge to a specific duration of the comparison interval required for a specific level of performance.

Time and the brain

184

2. ACCURACY OF TEMPORAL PROCESSING AS A FUNCTION OF BASE DURATION For more than a century, psychophysicists have been concerned with the question of whether accuracy of timing processes in humans is affected by the duration of a target interval (e.g. Vierordt, 1968). The answer to this question depends largely on the psychophysical measure chosen for quantifying timing accuracy. If, using the method of comparison, difference threshold is the criterion, then smaller threshold values, indicating better performance, will generally be observed for shorter base durations. Similar results will be obtained with difference scores, i.e. actual duration of the target interval minus estimated duration, when the methods of verbal estimation, production, or reproduction are applied. However, both difference thresholds and difference scores cannot be considered appropriate psychophysical indicators of performance since they are highly misleading when used for a wide range of different base durations (Hornstein and Rotter, 1969). For example, a 100 ms error in time judgement would be treated as being equally inaccurate regardless of whether the target duration was 500 ms or 5 s. The so-called Weber fraction represents a much more valid, and widely accepted, approach for quantification of timing accuracy as a function of base duration (e.g. Killeen and Weiss, 1987). In the case of verbal estimation, production, or reproduction, Weber fractions can be obtained by dividing the standard deviation of the time estimates by the base duration. However, typically, Weber fractions are obtained with duration discrimination tasks and are calculated by dividing the difference threshold (∆S) by the standard value of the stimulus (S) under investigation. This relation guides thinking in psychophysics. An ubiquitous finding in psychophysics is that difference threshold increases with the magnitude of the stimuli to be discriminated. When this relation is a linear function, it is referred to as Weber’s law:

(2) What is frequently observed in psychophysics is that Weber’s law holds for a given range of a sensory continuum, but, with relatively small values on this continuum, the difference threshold is larger than would be expected from a linear relationship. To account for this increase with smaller values, a constant value, c, is added to the above relationship:

(3) This relationship is referred to as the generalized form of Weber’s law (Getty, 1975). A large amount of empirical work has been devoted to test the robustness of some form of Weber’s law for duration (e.g. Killeen and Weiss, 1987; Getty, 1975, 1976; Treisman,

Psychophysics of human timing

185

1963). Getty (1975) offered a systematic analysis of this question for a duration discrimination task. Basically, the total variance in a timing task is assumed to be composed of a duration-dependent source, and of a duration-independent source. Depending on the assumptions made for the timing component, the form of the relationship between total variance and the duration of the target interval varies. Thus, total variance grows linearly in relation to base duration if timing depends on a Poisson process:

(4) where T is the duration of the target interval, and k and c are constants representing the rate of growth of the duration-dependent variability and the variability due to the duration-independent component, respectively. Note, that what was S in Equations 2 and 3 is now referred to as T, for physical time, in Equations 4 and 5. If variability of the timing process conforms to the generalized form of Weber’s law, then total variance of timing could be described as follows:

(5) Getty’s (1975) study was designed to provide a direct test of these two models. As could be shown, the generalized form of Weber’s law provides the best description of the relationship between total variability and the duration of the target interval. Interestingly, this demonstration of Getty was shown to hold for durations ranging from 200 ms to 2 s (see Table 1). With durations below or above this range of base durations, Weber fractions increased rapidly. Indeed, most data collected with human subjects support this generalized form of Weber’s law, at least for a restricted range of base durations (see Table 1). This conclusion was shown to hold with an interval production task (Grondin, 1992), with both time production and duration discrimination tasks (Ivry and Hazeltine, 1995), and with several other psychophysical methods applied to both human and animal subjects (Fetterman and Killeen, 1992). Literature on animal behaviour also provides support for the view that timing obeys some form of Weber’s law. One of the most popular theories in animal timing in the past 20 years, the scalar timing theory (Gibbon, 1991), also predicts Weber’s law. In addition to the empirical support received from experiments with animals, properties of scalar timing, which include Weber’s law, were shown to apply to human subjects as well (Allan and Gibbon, 1991; Wearden, 1991). Table 1 shows the changes in Weber fraction for different base durations (standard intervals) from various studies. Although the outcome of most studies suggests that accuracy of temporal processing increases with increasing base durations, as indicated by decreasing Weber fractions, this seems to be true only for a limited range of base durations. Rather, there is some evidence that decreasing Weber fractions are most likely to occur with increasing base durations. This effect becomes evident especially at the lower end of the series of base durations applied in an experiment. Many illustrations of

Time and the brain

186

this relationship between Weber fraction and base duration are also reported in Killeen and Weiss (1987). This relationship is indeed exactly what is predicted by the generalized form of Weber’s law. Increased values of the Weber fraction for shorter base durations are associated with nontemporal sources of variability. These sources can, for instance, be used to mark sensory mode (auditory or visual) or the structure (filled or empty)

Table 1 Changes in Weber fractions for different base durations from various psychophysical studies on temporal processing. Psychophysical method: C=comparison, P=production, R=reproduction, VE=verbal estimation. Change of Weber fractions: dec=decrease, inc=increase, ( )=range of base durations for dec/inc.

Authors(s)

Psychophysical Method

Range of Base Durations

Change of Weber Fractions

Abel (1972a)

C

0.1 6–960 ms

dec (0.1 6–10 ms)

Elbert et al., (1991)

R

1–8 s

dec (1–4 s)

Fetterman & Killeen (1992)

C

25 ms–5 s

dec (25–100 ms)

Getty (1975)

C

50 ms–3.2 s

dec (50–200 ms) inc (2.4–3.2 s)

Grondin (1993)

C

125 ms–4 s

dec (125 ms–1 s)

Grondin (1996b)

C

125 ms–4 s

dec (125 ms-2 s)

Hancock et al., (1994) P

1–20 s

dec (1–7 s)

Henry (1948)

C

32–480 ms

dec (32–480 ms)

Kinchla (1972)

C

1–8 s

dec (1–4 s)

Rammsayer (1992b)

C

50 ms vs. 1 s

dec

Rammsayer (1996)

R

5–40 s

dec (15–40 s)

Rammsayer & Vogel (1992)

C

50 ms vs. 1 s

dec

Rousseau et al., (1983) C

600ms–1.8 s

dec (600ms–1.2 s)

Smith (1969)

VE

15–1 20 s

dec (60–1 20 s)

Thomas & Brown (1974)

R

750 ms–5.5 s

dec (750ms–1.5 s)

Treisman (1963)

R, P

500 ms–9 s

dec (500 ms–2 s)

5–1 20s

dec (30–60 s)

Williams et al., (1989) VE, R

temporal intervals (Grondin, 1993; Grondin et al, 1996; Rammsayer, 1994c; Rammsayer and Lima, 1991; Skrandies and Rammsayer, 1995). On the other hand, whether or not Weber fractions increase with base durations longer

Psychophysics of human timing

187

than approximately 2 seconds, is closely related to the strategy adopted by the subject. When subjects refrain from counting, Weber fractions consistently increase with increasing base duration (Getty, 1975), whereas explicit counting improves performance on discrimination of intervals with base durations as short as 1.5 seconds (Grondin, 1996a). It should be noted, however, that there are some instances where neither Weber’s law, nor its generalized form, seem to apply to time judgements. One classical report of violation of Weber’s law is that of Kristofferson (1980). In his study, the subject received extensive training. When only the last 3 of 20 consecutive sessions with given base durations ranging from 100 to 1,480 msec were analysed, the function relating difference threshold to base duration was not continuous, but showed a series of steps. Threshold remained the same for a range of base durations until some point at which it doubled and then stayed constant for another range of base durations. There are also some studies of rhythm perception that call into question the general validity of Weber’s law for timing processes in humans. In a recent experiment, ten Hoopen et al. (1995) presented their subjects with isochronic intervals marked by a series of sounds, with the last sound creating anisochrony. With target intervals ranging from 50 to 200 ms, difference thresholds rather than Weber fractions remained constant indicating better performance with increasing base durations. Concurrently, these authors found a marked decrease in perceived duration of the last interval of the sequence, a phenomenon they called “time shrinking” (see also Nakajima et al., 1991; ten Hoopen et al., 1993); Drake and Botte (1993) also showed that Weber fractions are not constant for a wide range of base durations in temporal discrimination tasks. Their subjects had to discriminate tempi of two auditory sequences presented with inter-onset intervals (IOIs) ranging from 100 to 1,500 ms. Weber fractions were smallest for IOIs ranging from 300 to 800 ms and markedly increased for IOIs beyond this range. In conclusion, psychophysical data do not support the notion that temporal processing is more accurate for short than for longer intervals. Furthermore, there is no definitive answer to the question of whether there is a certain point on the time continuum where timing is most accurate. The answer to this question depends on the criterion adopted for quantifying timing performance. In general, variability is lower for shorter base durations, a finding that may be interpreted in terms of better temporal accuracy for brief intervals. However, within the framework of Weber-fraction analysis, shorter base durations do not provide better accuracy.

3. IS THERE EVIDENCE FOR DIFFERENT TIMING MECHANISMS? The finding that Weber’s law holds, at least for a given range of base durations, suggests that there is a single central mechanism underlying temporal processing for this range. This timing mechanism is often referred to as the “internal clock”, and most psychophysical models assume that such a clock is based on a neural counting process (e.g. Allan and Kristofferson, 1974; Church, 1984; Creelman, 1962; Treisman, 1963; Treisman et al., 1990). More specifically, the internal-clock mechanism is basically

Time and the brain

188

characterized by a neural pacemaker and an accumulator. The pacemaker generates pulses and the number of pulses relating to a physical time interval is recorded by the accumulator. Thus, the number of pulses counted during a given time interval is the internal representation of this interval. Hence, the higher the clock rate, the finer the temporal resolution of the internal clock will be, which is equivalent to more accuracy and better performance on time perception tasks. However, some authors propose that temporal processing involves more than one timing mechanism. Many of these views are related to the idea that time judgements may depend directly on memory processes. Indeed, in general, the role of cognitive mediation on time estimation has been amply documented (for reviews see Block, 1990; Levin and Zakay, 1989; Macar et al., 1992; Nichelli, 1993). Within the framework of prospective temporal processing, a distinction is sometimes made between estimation of time and perception of time (Fraisse, 1984; Rammsayer, 1994a). According to Fraisse (1984), perception of time is assumed to be mediated by experiential processes such as the psychological present (Fraisse, 1963; James, 1890; Michon, 1978). Estimation of time, on the other hand, is based on memory processes and refers to temporal processing of longer intervals in the range of seconds and above. From an information-processing point of view, Michon (1985) argued that, while temporal processing of shorter intervals is highly perceptual in nature and not accessible to cognitive control, temporal processing of intervals above 500 ms involves working memory. Indeed, the critical role of working memory on time estimation was recently reported by Fortin et al. (1993) who showed that concurrent non-temporal processing in working memory systematically lengthens temporal production of a 2-second interval. Likewise, Rammsayer and Lima (1991) found that temporal discrimination of 1-second auditory intervals is markedly impaired by an increased working-memory load while temporal discrimination of 50-millisecond intervals is unaffected by the same cognitive load. The notion of different mechanisms underlying temporal information processing is also supported by the outcome of pharmacopsychological studies. While a moderate dose of 0.65 g/kg ethanol does not interfere with temporal processing of one-second time intervals, the same dose of ethanol adversely affects processing of intervals in the range of milliseconds (Rammsayer, 1995; Rammsayer and Vogel, 1992). Additional evidence converging on this conclusion is provided by Mitrani et al. (1977). These authors showed that LSD and mescaline, both being substances that strongly affect time estimation of longer intervals (Cohen, 1966; Fischer, 1966), did not alter temporal discrimination of intervals shorter than one second. Both LSD and mescaline are well known for their cognitively mediated psychoactive effects, e.g. hallucinations (Barr et al., 1966; Longo, 1972). Therefore, Mitrani et al. (1977) concluded that long time intervals are processed at higher levels of the central nervous system, whereas brief intervals are processed almost automatically at some lower, subcortical, brain level. Finally, in a pharmacopsychological study on temporal information processing and memory (Rammsayer, 1994b), the benzodiazepine midazolam significantly reduced accuracy of temporal discrimination of intervals with a base duration of 1 s. Since midazolam produces marked deterioration of memory functions (Curran and Birch, 1991; File et al., 1992; Hennessy et al., 1991) and, furthermore, a highly significant correlation was found between performance on free recall and performance on time estimation under midazolam, decreasing task accuracy

Psychophysics of human timing

189

was attributed to midazolam-induced impairment of memory functions. Most interestingly, however, temporal processing of intervals in the range of milliseconds with a base duration of 50 ms has been shown to be resistant to the memory impairing effects of midazolam. The data presented so far, suggest that temporal processing of extremely brief intervals may depend on neural counting mechanisms rather than on cognitive processes. Most interestingly, Abel (1972a, b) showed that temporal discrimination of brief intervals below approximately 100 ms can be explained best by the assumption of neural counting mechanisms. Although the concept of an internal clock underlying temporal processing has been a central feature of many theoretical accounts of time perception (Allan, 1992), the neuroanatomical substrate of the assumed neural timing mechanism is still unknown. There is some evidence, however, that two subcortical structures, the cerebellum (Braitenberg, 1967; Ivry, 1993; Ivry and Keele, 1989; Keele and Ivry, 1991) and the basal ganglia (Artieda et al., 1992; Obeso et al., 1987; O’Boyle et al., 1996; Rammsayer, 1993, 1994a, 1997b; Rammsayer and Classen, 1997) may primarily be involved in timing functions. Finally, a more recent approach also provides converging evidence for the existence of neural networks in the brain mediating temporal information processing in the range of tens to hundreds of milliseconds. Simulation studies (Buonomano and Merzenich, 1995) applying a neural network composed of integrate-and-fire elements with realistic properties showed that these networks can represent any given interval between 30 and 300 ms. Taken together, there is convincing evidence for two distinct timing mechanisms involved in temporal information processing: One mechanism is based on workingmemory processes for timing of intervals in the range of seconds or above, and another one, which appears to be beyond cognitive control and, most likely, located at the subcortical level, is used for timing of extremely brief intervals in the range of milliseconds.

REFERENCES Abel, S.M. (1972a) Duration discrimination of noise and tone bursts. Journal of the Acoustical Society of America, 52, 1219–1223. Abel, S.M. (1972b) Discrimination of temporal gaps. Journal of the Acoustical Society of America, 52, 519–524. Allan, L.G. (1979) The perception of time. Perception and Psychophysics, 26, 340–354. Allan, L.G. (1992) News on the internal clock. International Journal of Psychology, 27, 21. Allan, L.G., and Kristofferson, A.B. (1974) Psychophysical theories of duration discrimination. Perception and Psychophysics, 16, 26–34. Allan, L.G., and Gibbon, J. (1991) Human bisection at the geometric mean. Learning and motivation, 22, 39–58. Artieda, J., Pastor, A., Lacruz, F., and Obeso, J.A. (1992) Temporal discrimination is abnormal in Parkinson’s disease. Brain, 115, 199–210. Barr, H.L., Langs, R.J., Holt, R.R., Goldberger, L., and Klein, G.S. (1972) Personality and Experience. New York: Wiley.

Time and the brain

190

Bindra, D., and Waksberg, H. (1956) Methods and terminology in the studies of time estimation. Psychological Bulletin, 53, 155–159. Block, R.A. (1989) Experiencing and remembering time: affordances, context, and cognition. In: I.Levin and D.Zakay (eds), Time and Human Cognition: A Life Span Perspective, Amsterdam: North-Holland, pp. 333–363. Block, R.A. (1990) Cognitive Models of Psychological Time. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Block, R.A., and Zakay, D. (1997) Prospective and retrospective duration judgments: a meta-analytic review. Psychonomic Bulletin and Review, 4, 184–197. Boltz, M.G. (1995) Effects of event structure on retrospective duration judgments. Perception and Psychophysics , 57, 1080–1096. Braitenberg, V. (1967) Is the cerebellar cortex a biological clock in the millisecond range? In: C.A. Fox and R.S.Snider (eds) The Cerebellum (Progress in Brain Research, vol. 25), Elsevier, Amsterdam, pp. 334–346. Brown, S.W. (1985) Time perception and attention: the effects of prospective versus retrospective paradigms and task demands on perceived duration. Perception and Psychophysics, 38, 115–124. Brown, S.W., and Stubbs, D.A. (1988) The psychophysics of retrospective and prospective timing. Perception, 17, 297–310. Buonomano, D.V., and Merzenich, M. (1995) Temporal information transformed into a spatial code by a neural network with realistic properties. Science, New York, 267, 1028–1030. Carlson, V.R., and Feinberg, Z. (1970) Time judgement as a function of method, practice, and sex. Journal of Experimental Psychology, 85, 171–180. Church, R.M. (1984) Properties of the internal clock. In: J.Gibbon and L.G.Allan (eds) Timing and Time Perception, New York: New York Academy of Sciences, pp. 566– 582. Clausen, J. (1950) An evaluation of experimental methods of time judgment. Journal of Experimental Psychology, 40, 756–762. Cohen, J. (1966) Subjective time. In: J.T.Fraser (ed), The Voices of Time. New York: Braziller, pp. 257–275. Creelman, C.D. (1962) Human discrimination of auditory duration. Journal of the Acoustical Society of America, 34, 582–593. Curran, H.V., and Birch, B. (1991) Differentiating the sedative, psychomotor and amnesic effects of benzodiazepines: a study with midazolam and the benzodiazepine antagonist, flumazenil. Psychopharmacology, 103, 519–523. Doob, L.W. (1971) Patterning of Time. New Haven, CT: Yale University Press. Drake, C., and Botte, M.-C. (1993) Tempo sensitivity in auditory sequences: Evidence for a multiple-look model. Perception and Psychophysics, 54, 277–286. Eisler, A.D., and Eisler, H. (1994) Subjective time scaling: influence of age, gender, and Type A and Type B behavior. Chronobiologia, 21, 185–200. Eisler, H. (1976) Experiments on subjective duration 1868–1975: A collection of power function exponents. Psychological Bulletin, 83, 1154–1171. Elbert, T., Ulrich, R., Rockstroh, B., and Lutzenberger, W. (1991) The processing of temporal intervals reflected by CNV-like brain potentials. Psychophysiology, 28, 648– 665. Fetterman, J.G., and Killeen, P.R. (1992) Time discrimination in Columbia livia and Homo sapiens. Journal of Experimental Psychology: Animal Behavior Processes, 18, 80–94.

Psychophysics of human timing

191

File, S.E., Skelly, A.M., and Girdler, N.M. (1992) Midazolam-induced retrieval impairments revealed by the use of flumazenil: a study in surgical dental patients. Journal of Psychopharmacology, 61, 81–87. Fischer, R. (1966) Biological time. In: J.T.Fraser (ed) The Voices of Time. New York: George Braziller, pp. 357–382. Fortin, C., Rousseau, R., Bourque, P., and Kirouac, E. (1993) Time estimation and concurrent nontemporal processing: specific interference from short-term-memory demands. Perception and Psychophysics, 53, 536–548. Fraisse, P. (1963) The Psychology of Time. New York: Harper and Row. Fraisse, P. (1984) Perception and estimation of time. Annual Review of Psychology, 35, 1–36. Gescheider, G.A. (1985) Psychophysics: Method, Theory, and Application. Hillsdale, NJ: Lawrence Erlbaum Associates. Getty, D.J. (1975) Discrimination of short temporal intervals: a comparison of two models. Perception and Psychophysics, 18, 1–8. Getty, D.J. (1976) Counting processes in human timing. Perception and Psychophysics, 20, 191–197. Gibbon, J. (1991) Origins of scalar timing. Learning and Motivation, 22, 3–38. Gilliland, A.R., Hofeld, J., and Eckstrand, G. (1946) Studies in time perception. Psychological Bulletin, 43, 162–176. Grondin, S. (1992) Production of time intervals from segmented and nonsegmented inputs. Perception and Psychophysics, 52, 345–350. Grondin, S. (1993) Duration discrimination of empty and filled intervals marked by auditory and visual signals. Perception and Psychophysics, 54, 383–394. Grondin, S. (1996a) When should one start explicit counting in time estimation? Paper presented at the 37th Annual Meeting of the Psychonomic Society. Chicago, Illinois. Grondin, S. (1996b) Discriminating intermodal time intervals. In: S.Masin (ed.) Fechner Day 96: Proceedings of the Twelth Annual Meeting of the International Society for Psychophysics. Padova, Italy: The International Society for Psychophysics, pp. 275– 280. Grondin, S., Ivry, R.B., Franz, E., Perreault, L., and Metthé, L. (1996) Markers’ influence on the duration discrimination of intermodal intervals. Perception and Psychophysics, 58, 424–433. Hancock, P.A., Arthur, E.J., Chrysler, S.T., and Lee, J. (1994) The effect of sex, target duration, and illumination on the production of time intervals. Acta Psychologica, 86, 57–67. Hennessy, M.J., Kirkby, K.C., and Montgomery, I.M. (1991) Comparison of the amnesic effects of midazolam and diazepam. Psychopharmacology, 103, 545–550. Henry, P.M. (1948) Discrimination of the duration of sound. Journal of Experimental Psychology, 38, 734–743. Hicks, R.E., Miller, G.W., and Kinsbourne, M. (1976) Prospective and retrospective judgements of time as a function of the amount of information processed. American Journal of Psychology, 89, 719–730. Hornstein, A.D., and Rotter, G.S. (1969) Research methodology in temporal perception. Journal of Experimental Psychology, 79, 561–564. Ivry, R.B. (1993) Cerebellar involvement in the explicit representation of temporal information. In: P.Tallal, A.M.Galaburda, R.R.Llinás and C. von Euler (eds) Temporal Information Processing in the Nervous System, New York: Annals of the New York Academy of Sciences, pp. 214–230.

Time and the brain

192

Ivry, R.B., and Hazeltine, R.E. (1995) The perception and production of temporal intervals across a range of durations: Evidence for a common timing mechanism. Journal of Experimental Psychology: Human Perception and Performance, 21, 3–18. Ivry, R.B., and Keele, S.W. (1989) Timing functions of the cerebellum. Journal of Cognitive Neuroscience, 1, 136–152. James, W. (1890) The Principles of Psychology (Vol. 1). New York: Henry Holt. Kaernbach, C. (1991) Simple adaptive testing with the weighted up-down method. Perception and Psychophysics, 49, 227–229. Keele, S.W., and Ivry, R.B. (1991) Does the cerebellum provide a common computation for diverse tasks: a timing hypothesis. In: A.Diamond (ed.) Development and Neural Basis of Higher Cognitive Function. New York: Annals of the New York Academy of Sciences, pp. 179–211. Killeen, P.R., and Weiss, N.A. (1987) Optimal timing and the Weber function. Psychological Review, 94, 455–468. Kinchla, J. (1972) Duration discrimination of acoustically defined intervals in the 1- to 8sec range. Perception and Psychophysics, 12, 318–320. Kristofferson, A.B. (1980) A quantal step function in duration discrimination. Perception and Psychophysics, 27, 300–306. Levin, I., and Zakay, D. (1989) Time and Human Cognition: A Life Span Perspective. Amsterdam: North Holland. Longo, V.G. (1972) Neuropharmacology and Behavior. San Francisco: Freeman. Luce, R.D., and Galanter, E. (1963) Discrimination. In: R.D.Luce, R.R.Bush and E.Galanter (ed.) Handbook of Mathematical Psychology. Vol. I. New York: Wiley, pp. 191–243. Macar, F., Pouthas, V., and Friedman, W.J. (1992) Time, Action and Cognition. Towards Bridging the Gap. Dordrecht: Kluwer Academic Publishers. McConchie, R.D., and Rutschmann, J. (1971) Human time estimation: on differences between methods. Perceptual and Motor Skills, 32, 319–336. Michon, J.A. (1978) The making of the present. In: J.Requin (ed.) Attention and Performance, Vol. 7. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 90–111. Michon, J.A. (1985) The compleat time experiencer. In: J.A. Michon and J.L. Jackson (eds) Time, Mind, and Behavior. Berlin: Springer, pp. 21–52. Mitrani, L., Shekerdjiiski, S., Gourevitch, A., and Yanev, S. (1977) Identification of short time intervals under LSD25 and mescaline. Activitas Nervosa Superior, 19, 103–104. Nakajima, Y., ten Hoopen, G.H., and van der Wilk, R.G.H. (1991) A new illusion of time perception. Music Perception, 8, 431–448. Nichelli, P. (1993) The neuropsychology of human temporal information processing. In: F.Boller and J.Grafman (eds), Handbook of Neuropsychology, Vol. 8. Amsterdam: Elsevier Science Publishers, pp. 339–371. O’Boyle, D.J., Freeman, J.S., and Cody, F.W. (1996) The accuracy and precision of selfpaced, repetitive movements in subjects with Parkinson’s disease. Brain, 119, 51–70. Obeso, J.A., Lacruz, F., Artieda, J., Pastor, A., and Martinez-Lage, J.M. (1987) Temporal discrimination of paired electrical stimuli is abnormal in Parkinson’s disease. Neurology, 37, (Suppl. 1), 281–282. Pentland, A. (1980) Maximum likelihood estimation: The best PEST. Perception and Psychophysics, 28, 377–379. Rammsayer, T.H. (1992a) An experimental comparison of the weighted up-down method and the transformed up-down method. Bulletin of the Psychonomic Society, 30, 425– 427.

Psychophysics of human timing

193

Rammsayer, T. (1992b) Effects of benzodiazepine-induced sedation on temporal processing. Human Psychopharmacology, 7, 311–318. Rammsayer, T.H. (1993) On dopaminergic modulation of temporal information processing. Biological Psychology, 36, 209–222. Rammsayer, T. (1994a) A cognitive-neuroscience approach for elucidation of mechanisms underlying temporal information processing. International Journal of Neuro science, 77, 61–76. Rammsayer, T.H. (1994b) Temporal information processing and memory. In: L.M. Ward, Fechner Day 94. Proceedings of the Tenth Annual Meeting of the International Society for Psychophysics, Vancouver, Canada: The International Society for Psychophysics, pp. 48–53. Rammsayer, T. (1994c) Effects of practice and signal energy on duration discrimination of brief auditory intervals. Perception and Psychophysics, 55, 454–464. Rammsayer, T. (1995) Extraversion and alcohol: Eysenck’s drug postulate revisited. Neuropsychobiology, 32, 197–207. Rammsayer, T.H. (1996) Why does time pass more quickly for women than men? Sexrelated differences in temporal information processing. In: I.Mervielde (ed.) 8th European Conference on Personality-Abstracts, p. 158. Ghent, Belgium: Academia Press. Rammsayer, T.H. (1997a) On the relationship between personality and time estimation. Personality and Individual Differences, 23, 739–744. Rammsayer, T.H. (1997b) Are there dissociable roles of the mesostriatal and mesolimbocortical dopamine systems on temporal information processing in humans. Neuropsychobiology, 35, 36–45. Rammsayer, T.H., and Classen, W. (1997). Impaired temporal discrimination in Parkinson’s disease: temporal processing of brief durations as an indicator of degeneration of dopaminergic neurons in the basal ganglia. International Journal of Neuroscience, 91, 45–55. Rammsayer, T.H., and Lima, S.D. (1991) Duration discrimination of filled and empty auditory intervals: Cognitive and perceptual factors. Perception and Psychophysics, 50, 565–574. Rammsayer, T., and Vogel, W.H. (1992) Pharmacologic properties of the internal clock underlying time perception in humans. Neuropsychobiology, 26, 71–80. Rousseau, R., Poirier, J., and Lemyre, L. (1983) Duration discrimination of empty time intervals marked by intermodal pulses. Perception and Psychophysics, 34, 541–548. Skrandies, W., and Rammsayer, T. (1995) The perception of temporal structure and auditory evoked brain activity. Biological Psychology, 40, 267–280. Smith, N.C. (1969) The effect on time estimation of increasing the complexity of a cognitive task. Journal of General Psychology, 81, 231–235. Stevens, S.S. (1975) Psychophysics: Introduction to Its Perceptual, Neural and Social Prospects. New York: Wiley. ten Hoopen, G.H., Hilkhuysen, G., Vis, G.Nakajima, Y., Yamauchi, F., and Sasaki, T. (1993) A new illusion of time perception—II. Music Perception, 11, 15–38. ten Hoopen, G., Hartsuiker, R., Sasaki, T., Nakajima, Y., Tanaka, M., and Tsumura, T. (1995) Auditory isochrony: time shrinking and temporal patterns. Perception, 24, 577– 593. Thomas, E.A.C., and Brown, I. (1974) Time perception and the filled-duration illusion. Perception and Psychophysics, 16, 449–458. Treisman, M. (1963) Temporal discrimination and the indifference interval: implications

Time and the brain

194

for a model of the “internal clock”. Psychological Monographs, 77, 1–31. Treisman, M., Faulkner, A., Naish, P.L.N., and Brogan, D. (1990) The internal clock: evidence for a temporal oscillator underlying time perception with some estimates of its characteristic frequency. Perception, 19, 705–743. Vierordt, K. (1868) Der Zeitsinn nach Versuchen. Tübingen: Laupp. Wallace, M., and Rabin, A. (1960) Temporal experience. Psychological Bulletin, 57, 213–235. Wearden, J.H. (1991) Do humans possess an internal clock with scalar timing properties? Learning and Motivation, 22, 59–83. Williams, J.M., Medwedeff, C.H., and Haban, G. (1989) Memory disorder and subjective time estimation. Journal of Clinical and Experimental Neuropsychology, 11, 713–723. Zakay, D. (1990) The evasive art of subjective time measurement: some methodological dilemmas. In: R.A.Block (ed.) Cognitive Models of Psychological Time, Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 59–84. Zakay, D. (1993) Time estimation methods—do they influence prospective duration estimates? Perception, 22, 91–101.

6 Cortical Processing by Fast Synchronization: High Frequency Rhythmic and Non-rhythmic Signals in the Visual Cortex Point to General Principles of Spatiotemporal Coding Reinhard Eckhorn Department of Physics, NeuroPhysics Group, Philipps-University, Renthof 7. D35032 Marburg, Germany Tel: (0049) 6421–2824–164 FAX: (0049) 6421–2827–034 e-mail: [email protected]

Synchronized signals are functionally important because they can activate a neurone more effectively than an uncorrelated input, especially if the input’s correlation is over a time span not broader than the temporal window of the neurone’s integration. In active cortices, integration times can be in the range of milliseconds, so that precisely synchronized signals seem important for cortical processing. We analysed such high frequency activities in visual cortex of cats and monkeys by multiple microelectrode recordings. Transiently changing scene segments were represented by synchronized population activities that were often non-rhythmic and phase-coupled to visual stimuli. During sustained activation, as in periods of stable retinal input, activities were more oscillatory (30–100 Hz) and dominated by cortical dynamics. Such oscillations were synchronized within a vertical column across cortical layers, among different columns of the same cortical area within some millimeters (“synchronization field”), and between different cortical areas amongst neurones with neighbouring receptive fields. Oscillation frequencies were highly variable, while average phase-delays, even in separate locations, were narrowly distributed around zero. Experimental data and related computer simulations support a comparably simple explanation: Fast oscillations are generated during states of sustained activation in local populations via local feedback inhibition, while distributed populations are synchronized via mutual facilitatory coupling. Synchronization at near-zero phase can be explained by common input from shared cortical and subcortical sources. The width of cortical synchronization fields with oscillations at zero-phase delay can be explained by temporal properties of cortical circuits shaped by Hebbian learning. Since the synchronization of fast cortical oscillations depends on specific grouping of visual features, its role in scene segmentation, object definition, and other more general association processes are discussed. Finally, it is argued how basic visual processing operations might be carried

Cortical processing by fast synchronization

197

out in a stepwise fashion by a modular circuit derived from interactions among simple and complex cells in primary visual cortex. KEYWORDS: Cortex; Gamma oscillations; Cortico-cortical interactions; Population coding; Perceptual grouping; Association field

1. INTRODUCTION 1.1. Synchronized Input Excites Neurones Strongly and Precisely 1.1.1. Definition of “Integration time” and “Synchronization” in the present context The term synchronization will be used with respect to the duration of a single neurone’s integration time, which is mainly given by the time course of postsynaptic influences in response to an input spike. To a first approximation we might think of the integration time as the half-height duration of an excitatory postsynaptic potential (EPSP) at the neurone’s spike encoder. In cortical neurones these durations span a broad range from about 2 up to 100 milliseconds. Integration times depend on the site and type of synapses, on the different dendro-somatic membranes in different types of neurones, but also on dynamic changes in membrane time constants of single neurones (Häusser and Roth, 1997). One example of variability in integration time is the prolongation of EPSPs from NMDA synapses with increasing levels of membrane depolarization (Fox and Daw, 1992), while another example is the shortening of PSPs with increase in a neurone’s average synaptic activation (Agmon-Snir and Segev, 1993; Nelson, 1994). In the activated awake cortex in which we are interested in this paper dominant integration times are estimated in the range 2 to 10 ms (e.g. König et al., 1995a), where the shorter time constants will be found in “low impedance locations” with several neighbouring synapses activated in parallel, while longer ones will occur at “high impedance locations” when only a single synapse is activated. Neural signal components covarying, with the same sign, during such integration windows will be called here “synchronized”. Synchronized events may occur singly, or repetitively with a more stochastic or rhythmic character. With respect to the processing of behavioural output, this synchronization range of 2 to 20 ms is rather short and we will therefore use the term “fast” for it. Accordingly, neural signals which are correlated positively in this range will be called “fast synchronized”. In this sense, rhythmic signals with half cycle durations of 5 to 10 ms (100 to 20 Hz) will be termed synchronized fast cortical oscillations (FCOs).

Time and the brain

198

1.1.2. Influence of synchronized input on the spike encoder The spike encoder of a neurone, including its threshold and refractory dynamics, favours synchronized compared to unsynchronized inputs, because the former produce higher and more steeply depolarized membrane excursions at equal numbers of input spikes. Output spikes will be generated at higher probability if a neurone’s excitatory inputs are activated in concert within its preferred integration window (e.g. Abeles 1991; Murthy and Fetz 1994; Bernander et al., 1995). If input spike patterns are even precisely coinciding, a spike encoder with short refractory period and appropriate threshold can extract this pattern and send it to other targets. In contrast, more dispersed input spike patterns, including statistically independent ones with the same number of spikes, will result in lower momentary excursions of membrane potentials at the spike encoder and hence, in a lower influence on spike firing (Abeles, 1991). 1.2. Potential and Effective Connectivity The brain is a highly connected structure, and it is generally agreed that its rich connectivity and special topography are the main reasons for its astonishing capabilities in sensory, motor, and cognitive processing (e.g. Braitenberg and Schüz, 1991). In recent decades, systems neuroscientists have therefore concentrated on the connectionistic properties of the brain. However, the brain’s wiring provides only a potential network of processing, while an actual task dynamically recruits a rather low percentage of neurones and connections, characterized by their effective connectivity. This is explained by the fact that any connection is functionally present only while a presynaptic spike influences the postsynaptic cell. This period may be about 10 to 100 ms at low activation levels but may be even shorter with stronger inputs (Agmon-Snir and Segev, 1993; Nelson, 1994). In addition, spike rates of cortical neurones are low without special activations, ranging from silence in many neurones to 1–3 per second in spontaneously active cells. These low rates are often suppressed by nearby activated neurones. Hence, it is argued that cortical processing relies, at each moment, on a small percentage of active neurones, and on the fact that the effective network configuration can change dynamically within milliseconds, due to specific activations. Indeed, rapidly-changing effective connectivity has been shown repeatedly in the past on the basis of multiple microelectrode recordings combined with cross-correlation techniques (e.g. Aertsen et al., 1989; Schneider et al., 1983; Eckhorn and Obermueller, 1993). In the present chapter we focus on the causes, mechanisms, and possible functional relevance of cortical synchronization dynamics. We will report on our observations of temporally precise correlations of cortical population signals, in particular of synchronized fast oscillations (FCOs) in the gamma range (30 to 100 Hz) and on high frequency stimulus-locked non-rhythmic synchronization. This includes discussion about neural codes—from precise spike pattern coding to spike rate coding—and the formation of neural assemblies defined by simultaneous activation of interacting neurones. As our experience is based on multiple microelectrode recordings from the visual cortex of

Cortical processing by fast synchronization

199

anaesthetized cats and awake monkeys, we will discuss the potential relevance of fast synchronized signals for visual perception, but generalizations to other systems will also be developed. Finally, we will try to explain the observed cortical dynamics with simulations including simple neural mechanisms and circuits.

2. FAST SYNCHRONIZED SIGNALS IN THE VISUAL CORTEX– RECORDED BY MULTIPLE MICROELECTRODES 2.1 Synchronized Components in Population Activities Our empirical evidence is based on multiple microelectrode recordings from the visual cortex of anaesthetized cats and awake monkeys. We have used local population activities (multiple unit spike activity (MUA) and local slow wave field potentials (1 to 150 Hz) (LFP)) for the analysis of cortical synchrony, because they comprise the synchronized components of local populations (e.g. Eckhorn, 1992). In particular, LFPs are a local weighted average of the dendro-somatic postsynaptic signals, reflecting mainly the synchronized components at the inputs of the population within (approximately) 0.5 mm of the electrode tip (Mitzdorf, 1987). MUAs, on the other hand, comprise, in their amplitudes, the simultaneity of spikes occurring at the outputs of a local population within (approximately) 0.05 mm of an electrode tip (Legatt et al., 1980; Gray et al., 1995). Both population signals are more suitable for detecting correlations among dispersed cortical recording locations than are single unit spike trains (Eckhorn et al., 1988, 1990; Eckhorn, 1992; Gray et al., 1989, 1990; Engel et al., 1990, 1991). Higher numbers of neurones contribute to LFP than to MUA, due to their shallower spatial decay (Legatt et al., 1980; Mitzdorf, 1987). In addition, the cortical ranges where FCOs occur with near zero phase delay is some millimeters, so that LFP data consist of superimposed signals from large numbers of neurones participating in the same oscillatory state. In the visual cortex, the main contribution to LFPs is from neurones involved in the same functional task, because neurones with functional similarity, activated by the same stimulus feature, are grouped in clusters (e.g. Gilbert, 1993). In conclusion, this superposition results in lower variability of LFP-FCOs for identical stimulus repetitions, and a higher number of significantly correlated LFP pair recordings from separate positions, compared to MUA. 2.2. Sustained Activation is Required for the Generation of FCOs A general condition for the occurrence of FCOs was sustained activation, with average spike rates typically larger than 10 impulses per second, and the absence of any fast excitatory or inhibitory response transients (Kruse and Eckhorn, 1996). This condition was particularly given with features fitting the classical receptive field (cRF) properties of the respective neurones. For example, stimulation with an oriented, one-dimensional, luminance grating, activated in a sustained manner those neurones preferring

Time and the brain

200

approximately the spatial frequency, orientation and contrast of that grating. Slow temporal modulations (below 5 Hz) also led to FCOs, which were modulated in their probability of occurrence and amplitude, by the slow temporal modulation. Such states, with sustained activation, are typical for situations in which gaze is nearly stabilized on a visual object (for more details see section 3.). Model investigations (Juergens and Eckhorn, 1997; and below) supported the experimental observation that the generation of FCOs depends on a sustained activation of a local network of coupled neurones. 2.3. Single Neurones are Differently Involved in Synchronized FCOs Three types of coupling dynamics were observed between single cell spikes and oscillatory population activity (LFP and MUA) in cat (Eckhorn and Obermueller, 1993) and monkey visual cortex (Frien et al., 1994) as well as in related neural network simulations (Juergens and Eckhorn, 1997): 1. In rhythmic states, single cell spike patterns had significant rhythmic modulation, and spikes were significantly correlated with oscillatory population activity; 2. In lock-in states, rhythmic modulation was not present in single cell spike patterns, while spikes were significantly correlated with oscillatory population activity; and 3. In non-participation states, rhythmic modulation was absent in spike trains, and in addition the spikes were not correlated with the actually present oscillatory population activity (Figure 1). State transitions of single cell coupling with oscillatory population activities depended on stimulation and receptive field properties. For example, a weak lock-in state changed into a strong synchronized rhythmic state in a cortical (directionally sensitive) neurone when reversing the movement direction of an oriented contour stimulus (Eckhorn and

Cortical processing by fast synchronization

201

Figure 1. Three different states of single cell couplings with oscillatory population activities in the primary visual cortex, a: rhythmic; b: lock-in, and c: non-participation states of three different neurones. AC: auto-correlation histograms of single cell spikes (SUA), of multiple unit activity (MUA), and of local field potential (LFP). STA denotes spike-triggered averages of multiple unit activity or local field potentials. According to the classification, STAs have oscillatory modulations in the rhythmic and lock-in states, and lack such modulation in the non-participation state. Note that in the rhythmic state (a) the single cell correlogram (top) is clearly modulated at 44 Hz, while in the lock-in (b) and the non-participation states (c) rhythmic modulations in the range 35–80 Hz are not visible (by definition). Lowest row of panels: power spectra for the above row of correlograms. (Figure modified from Eckhorn and Obermueller, 1993).

Obermueller, 1993). Stimulation in the preferred direction caused higher levels of average sustained activation (also in other neurones of the same directional preference), and this resulted in rhythmic synchronized states. 2.4. Frequency and Amplitude of FCOs are Highly Variable The oscillation frequencies and amplitudes of FCOs are probably not coding sensory features, because they are highly variable. Variability of the dominant frequency of FCOs is typically varying in the range over 20 Hz with constant stable stimuli. Such variations were found in several visual cortical areas of anaesthetized cats, as well as in awake monkeys, during the execution of perceptual tasks (Eckhorn et al., 1988; Gray et al., 1989; Eckhorn and Schanze, 1991; Kreiter and Singer, 1996; Frien et al., 1994; review in Gray, 1994). A general observation was that the frequency within a single oscillation spindle (typical duration being 80 to 200 ms) was more constant than in successive

Time and the brain

202

spindles. However, we observed some systematic changes in the average oscillation frequency with stimulation, ranging from 20 to 100 Hz, which is described in the next paragraph. 2.5. Average Oscillation Frequency and Amplitude of FCOs Depend on Visual Stimulation Even though the variability of FCO frequencies and amplitudes was high during stationary visual stimuli, they showed some systematic dependency of their average values on stimulation. In general, with increasing amplitudes of cortical activation, we found increasing FCO amplitudes accompanied by decreasing frequencies. When neurones were slightly activated above their “spontaneous” rate, incoherent broad band stochastic signals predominated, with some short and low amplitude FCOs of high frequency (<70 Hz). These oscillatory epochs seldom occurred coherently across distances of more than 1 mm cortex. With intermediate activation levels, domains of synchronized FCOs were on average broader, their amplitudes higher and their frequencies lower (50 to 60 Hz). Strong sustained activation resulted in high amplitudes of population activity at low frequencies (30 to 40 Hz) and the synchronized domains became larger, spanning 4 to 6 mm in V1 and V2 cortex. Figure 2c shows the continuous transition between the different signal types in a recording from monkey visual cortex, in which the contrast of a visual stimulus was continuously increased. The velocity and spatial extent of a visual stimulus also influenced the average oscillation frequency of FCOs (Eckhorn et al., 1989; Eckhorn and Frien, 1995). When visual stimuli moved slower, or were larger in extent, FCOs in the visual cortex of awake monkeys had on average lower oscillation frequencies and higher amplitudes (Figure 2a,b). We can explain the lower frequencies observed with larger stimuli by the longer average activation delays within the larger populations (for details see section 3.8). However, lower frequencies may also be due to the increased activation levels, with more extended stimuli leading to increased inhibition periods in the local populations. The influence of stimulus velocity on FCOs might have corresponding reasons. Slow velocities allow the formation of larger assemblies engaged in lower frequency activity than for faster movements. 2.6. Zero Average Phase Delay Among Synchronized FCOs Average phase differences between FCOs at separate locations of the same visual cortical area (e.g. V1 or V2), and among different areas, were found to be narrowly distributed around zero (Eckhorn et al., 1988; Gray et al., 1989; Engel et al., 1991b). Zero phase shift was commonly found, even among V1 and V2 of monkeys (Frien et al., 1994) where we expected this even less, because these areas are known as serially arranged in the visual processing stream, and thus suggesting a delay of V2 against V1 oscillations. This result seemed at first counterintuitive because the average conduction velocity of cortical axons is relatively slow (Swadlow, 1983; Nelson et al., 1992; Miller, 1994; Nowak et al. 1995b). In V1 and V2 of cat and monkey the velocity in horizontally

Cortical processing by fast synchronization

203

projecting axons can be estimated roughly at 1 m/s which is a value that can lead to clearly measurable phase shifts within the cortical distances measured by us (1 to 6 mm; Jordan, 1989; Brosch et al., 1995; Juergens et al., 1996). To find a plausible explanation we want to mention that non-rhythmic signals, occurring in epochs between FCOs, and before visual activation at the same electrode positions, can also show peaks in their cross-correlograms at zero-delay (Frien et al., 1994). Zero delay non-rhythmic correlation was also observed in V1 and V2 of anaesthetized cats (Nelson et al., 1992; Nowak et al., 1995a,b). We argue that single central peaks in cross-correlograms are indicative of inputs from a common source, with the same activation delay between source and cortical recording positions, because such common inputs guarantee zero delay independent of the type of signal (stochastic or rhythmic) and its frequency range (low or high)(see also Nowak and Bullier, this volume).

Figure 2. a-d. Dependencies of average oscillation frequencies on stimulus size, velocity and contrast in the primary visual cortex of an awake monkey, a: The average oscillation frequencies of LFPs shifted to lower values accompanied by higher amplitudes if the length of a moving stimulus (light bar) was elongated or if its width was broadened, or b: if its velocity was decreased. Frequency changes with size were restricted to the spatial range in which significant values of coherence were observed, c: Average oscillation frequency of FCOs decreased with stimulus contrast (20 consecutive MUA recordings from the same experiment), d: Linear regression fits to oscillation frequencies of LFP-data from the same experiment (c) during increase in stimulus contrast. (Figure 2a ,b and d modified from Woelbern et al., 1994; Figure 2c: unpublished from a.)

Time and the brain

204

2.7. Cortical Synchronization Range of FCOs The average range in which synchronized FCOs occurred in primary and secondary visual cortex areas V1 and V2 of cat and monkey was 4 to 6 millimeters (Jordan, 1989; Brosch et al., 1995; Frien et al., 1996; Juergens et al., 1996; König et al., 1995b). This was established by recordings with linearly arranged arrays of equally spaced microelectrodes and the calculation of coherence among FCOs at different separations. Spatial decline of coherence was less steep in positions where neurones represented similar visual features (for example for similar local orientation of a contrast border) compared to those with dissimilar features. Within the synchronization range, phase delays of local FCOs were narrowly distributed around zero (with a standard deviation of typically 1 ms). Beyond that range, covariance declined to noise levels (Figure 3). 2.8. Are Adjacent Visual Objects Represented by Phase-Shifted or by Uncorrelated FCOs? A visual scene generally consists of many segments that can be discriminated perceptually. If we follow the above mentioned “synchronization hypothesis”, adjacent or overlapping segments of different visual objects should be coded by temporally separable FCOs (or, more generally, by any other type of uncorrelated signals). Discrimination would be possible if, for example, the neurones representing adjacent objects are

Cortical processing by fast synchronization

205

activated at different relative phase shifts at the same frequency or at different uncorrelated frequencies. To our knowledge, there are only few experimental examples of uncorrelated oscillations mat were induced by two adjacent stimulus objects (Gray et al., 1990; Kreiter and Singer,

Time and the brain

206

Figure 3. Coherence of FCOs declines with cortical distance. Local field potentials (LFP, 35–80 Hz) recorded in primary visual cortex of awake monkey with linear multiple µ-electrode array from upper cortical layers. Horizontal bars are proportional to relative frequency of occurrence. Continuous line indicates mean, dotted standard deviation, of coherence. High values of coherence were obtained with similar, low with different, receptive field properties in the recording locations. (Figure modified from Juergens et al., 1996).

1996; Brosch et al., 1997) and not a single example of a phase difference at the same oscillation frequency. We have obtained preliminary experimental evidence that adjacent objects (or an object against a background) do not lead to phase-shifted FCOs (at the same frequency) in their cortical representations, but to statistically independent FCOs (Figure 4). We could show that the same local populations became engaged in synchronized FCOs if they were activated by a single coherent stimulus, but decoupled their FCOs when an object’s boundary separated the populations. In this case, the frequencies became different (typically higher in the neurones representing the smaller of two stationary objects; for influences on average FCO frequency see section 2.5). 2.9. Suppression of FCOs by Fast Stimulus-Locked Activations FCOs of high amplitude were generally induced in visual cortex during epochs of slow drift rates of retinal images, when cortical cells were activated by sustained thalamic inputs. In contrast, fast transient components were primarily evoked by stimulus epochs of changing high velocity and direction. This was extensively studied in the visual cortical areas V1 and V2 of anaesthetized cats (Kruse and Eckhorn, 1996). With

Cortical processing by fast synchronization

207

increasing amplitudes of fast transient stimulus movements, FCOs were gradually reduced in amplitude, and were finally fully suppressed (Figure 5). Related observations were recently made in awake monkeys (Guettler et al., 1996). Rapid changes of the retinal image due to (micro) saccades or sudden changes of an object in its contrast or position could immediately disturb ongoing oscillations. The latter were, as in the cat, generated only during states of stable or slowly moving retinal images.

Figure 4. Figure-ground segregation defined by the reduction of the coherence of FCOs across object contours. LFP recordings (35–80 Hz) in 3 mm lateral separation from upper layers of V1, while the monkey kept fixation within 0.5°. The measure of signal coupling utilized was similar to spectral coherence, which was calculated with the dominant oscillations in the range 35–80 Hz. The stimulus, a sinusoidal grating (the “background”: spatial period 0.7° visual angle), was presented stationary for 0.8 s and then moved perpendicular to its orientation at a velocity of 0.65°/s while a patch of it (the “object”: 1° x 4° visual angle) remained stationary. RF1 and RF2 (left upper display) indicate the cRF positions of the recording pairs in- and outside the “object”. Since the contour of the object was only visible when the gratings were out of phase, we were able to examine a dynamical figure-ground separation. (Modified from Guettler et al., 1997).

The occurrence of strong stimulus-locked components in cortical responses per se, is not a sufficient explanation for the observed partial or full suppression of fast oscillations. In principle, stimulus-locked responses and oscillations might superimpose and coexist independently without major interactions. Instead, we found strong suppression of oscillations by stimulus-locked responses (Figure 5). What are probable explanations of

Time and the brain

208

this suppression? (1) Inhibition of oscillation circuits by transient afferent activities. This possibility is supported by the finding that the fast stimulus-locked components, being preferentially transmitted via the magnocellular (transient type) afferents, are particularly effective in evoking fast inhibitory responses in the visual cortex (Douglas and Martin, 1991). (2) Disturbance of oscillations by pushing the phase of oscillatory circuits. From theories (Kuramoto, 1991), and simulations of loosely coupled oscillators (Eckhorn et al., 1990; Schuster and Wagner, 1990; Schillen and König, 1994; Sompolinsky et al., 1990; Gerstner et al., 1993) it is known that oscillations can only be maintained, if strong out-of-phase inputs are prevented. However, such out-of-phase disturbances were delivered to the cortex in the investigation of Kruse and

Figure 5. Perturbation of synchronized fast cortical oscillations by stimuluslocked synchronized signals. Note that stimulus-locked responses inhibit synchronized oscillations. Local field potential recordings from cat visual cortex. Stimulus: grating, moving with a random component superimposed on a constant slow velocity. Open symbols: average normalized response power of the high-frequency oscillatory components; filled symbols: normalized power of the stimulus-locked components. Amplitudes (standard deviation) of the random stimulus movement are indicated at the abscissa. Each value is the average of N=104 single curves (modified from Kruse and Eckhorn, 1996).

Eckhorn (1996) by the stimulus-locked afferent input, because its transient response components generally occurred at random times relative to the phases of the oscillations.

Cortical processing by fast synchronization

209

With intermediate levels of stimulus-locked activity, oscillations were only partially suppressed. This can also be explained by deteriorations of oscillatory processes by random out-of-phase inputs. Theoretically one would expect a broadening and reduction of the dominant frequency peak of oscillations in the power spectrum due to an increased phase jitter and a reduced amplitude of local oscillatory processes. This has, indeed, been observed (Eckhorn and Schanze, 1991; Kruse and Eckhorn, 1996). (3) Suppression of oscillations by transient reductions of membrane resistances due to strong stimulus locked activations. If strong phase-locked input activates a considerable number of excitatory and inhibitory synapses of “oscillating” neurones nearly simultaneously, this may transiently lead to massive reductions in membrane resistances of dendrites and somata (Agmon-Snir and Segev, 1993; Nelson, 1994). Consequently, the decay of postsynaptic potentials becomes faster, leading to a lower level of temporal superposition so that the membrane potentials are reduced in amplitude. In conclusion, our observations support the assumption that FCOs are generated as long as the afferent visual activation is sustained and lacks fast transients. Suppression of oscillations by random stimulus-locked responses may well be due to transient activations of inhibitory interneurones, or to out-of-phase disturbances or reductions of membrane potentials in neurones participating in the oscillations. However, the presently available knowledge does not provide enough information to decide whether these or other still unknown mechanisms can explain the observed effects. 2.10. Synchronized FCOs Occur Primarily During States of “Desynchronized” EEG FCOs were extensively recorded by several researchers with intracortical microelectrodes in the visual system (cited above). In addition, much work has recently been published about recordings of FCOs in other sensory modalities, as well as in the motor system (Adrian, 1950; Barth and McDonald, 1996; Baker et al., 1997; Bragin et al., 1995; McKay and Mendoca, 1995; Murthy et al., 1994, 1997a,b; Rougeul et al., 1979; Sanes and Donoghue, 1993; Steriade et al., 1996; Traub et al., 1996). This growing body of experimental data supports the hypothesis that the occurrence of synchronized FCOs is associated with cognitive processing requiring awake attentive states. In particular, if the reticular formation was stimulated electrically, mimicking arousal, FCOs occurred at an increased probability even in lightly anaesthetized cat (Munk et al., 1996). However, EEG recordings resembled generally “desynchronized” activity in such states characterized by broad band random signals of low amplitudes (compared to the high amplitude low frequency activity during sleeping stages). Hence, it seems as if FCOs either do not show up in EEGs, or that humans do not generate synchronized FCOs (Juergens et al., 1999; but see Tallon Baudry et al., 1997). We believe that the former is the case, mainly because the cortical field over which synchronized FCOs occur in anaesthetized cats and awake monkeys was restricted to less than 6 mm. However, EEG signals recorded by standard electrodes on the skull are superpositions of cortical activity

Time and the brain

210

over a range of at least 40 mm. In this range several pools of uncorrelated FCOs superimpose their activities. It seems probable that they largely average out because of their statistical independence.

3. POTENTIAL RELEVANCE OF SYNCHRONIZED FCOs FOR VISUAL PROCESSING 3.1. Preattentive Scene Segmentation is a Prerequisite for Object Recognition Stimuli composed of coherent features are integrated by our sensory systems into perceptual entities, even if the features are dispersed among different sensory modalities. We can perceive a sensory object as a perceptual whole even if various aspects of the object are occluded, obscured by the background, or are not present at all. The visual system can easily detect coherencies in an object’s local stimulus features, and is able to link, intensify and isolate them. These capabilities of grouping, mutual facilitation, and figure-ground separation require neural mechanisms for self-organization that are able to construct reliable and unique percepts out of ambiguous sensory signals. Early processing of these relations is highly important, because the infinite number of possibilities in which even simple “toy scenes” can be arranged has to be reduced substantially before visual associative memories can solve tasks of recognition. 3.2. Binding of Spatially Dispersed Visual Features by Sychronized FCOs? It has been proposed that binding of spatially distributed features is based on synchronized fast excitations of those neurones belonging to an object’s neural representation. More precisely, neurones participating in the processing and representation of a visual object engage in a common synchronized state in response to stimulation by that object (Milner, 1974; Reitboeck, 1983). This hypothesis attracted attention when stimulusspecific synchronized oscillations of 35–90 Hz were found in the visual cortex of anaesthetized cats (Eckhorn et al., 1988; Gray et al., 1989) and awake monkeys (Kreiter and Singer, 1992; Eckhorn et al., 1993). In addition, synchronized non-oscillatory signals were also proposed to support feature binding, including synchronization due to phase-locking to transient stimulus events. The latter effects were investigated in cortical recordings (Frien et al., 1994; Kruse and Eckhorn, 1996) and have been modelled (Eckhorn et al., 1990). Finally, the first experimental support for the validity of the feature linking hypothesis was recently obtained from the occurrence of precisely synchronized signals correlated with perceptual processes of monkeys and cats, respectively (spike coherence: Vaadia et al., 1995;—FCO coherence: Kottmann and Eckhorn, 1996; Fries et al., 1997).

Cortical processing by fast synchronization

211

3.3. Sharper Orientation Tuning with Amplitudes of Synchronized FCOs Compared to Spike Rates In recent years the debate about neural codes has intensified with the finding of FCOs and the hypothesis that fine temporal structure might code important information, in addition to rate coding. However, quantitative comparisons between temporal and rate coding have seldom been made for the same set of data. We have recently obtained recordings in which the sharpness of orientation tuning was measured in V1 of awake monkeys, using the common method of spike rates that were evoked by stimulus contours of different orientations, this being compared with FCO amplitudes of the same responses. We found that the width of orientation tuning with FCOs was considerably sharper compared to the spike count measure (Figure 6). Perception of contour orientation is much more sensitive than the orientation characteristics with rate codes would predict. In this case, at least, the use of FCOs for orientation discrimination seems therefore to be more effective than the use of a rate code. In addition, we have studied the coupling strength among FCOs generated in pairs of spatially separate groups of neurones, according to their cortical distance and the angle between their preferred stimulus orientations (Figure 7a; Frien et al., 1996). We found that the degree of coherence (synchronization) of spike activities (MUA) depended strongly on stimulus orientation relative to the preferred orientations at the recording sites. If stimulus orientation and orientation preference at the two sites were similar, the strongest and most far-reaching coupling was found. Synchronization was less strong if the preferred orientations differed. In these cases the strongest coupling values were obtained with stimulation at half the angle between the preferred ones at the two sites. It is worth mentioning that the low frequency signals simultaneously present in these recordings did not show the orientation specific synchronization effects (Figure 7b). In conclusion, neurones coding the local visual feature “contour orientation” mutually synchronize by a fast mechanism, which probably means by using fast axonal links.

Time and the brain

212

Figure 6. Orientation tuning with fast cortical oscillations is sharper than with spike rates of identical neurones. OI indicates the orientation index, resembling the “sharpness” of orientation. Microelectrode recordings of multiple unit activity (MUA, 1–10 kHz) from the upper layers of the primary visual cortex of an awake monkey.

3.4. Synchronized FCOs and Visual Perception During Binocular Rivalry The most convincing argument in favour of the proposed functional role of synchronized FCOs in visual perception would be a direct demonstration of their specific occurrence correlated with perception. We began experiments with trained monkeys that can directly show such correlation (Kottmann and Eckhorn, 1996). This is possible with stimuli that can be perceived alternatively in two different manners with the monkey giving an indication of the actual percept (by pressing a key). We used a paradigm of binocular rivalry in which either a horizontally or vertically oriented Gabor grating (visual stimulus of sinusoidal luminance distribution, windowed by a concentric Gaussian) was perceived while both were always presented (e.g. the visual stimulation remained the same in each trial). Our hypothesis of “feature-linking-by-synchronization” would gain support if synchronized FCOs would occur in those neurones representing the stimulus presently perceived, and not in the neurones representing the other possible percept. Indeed, our preliminary data support our hypothesis: neurones representing the orientation of the

Cortical processing by fast synchronization

213

presently perceived grating synchronized their FCOs at a high probability, while synchronization of neurones representing the orientation at right angles was seldom observed.

Figure 7a,b. Synchronized signals among cortical orientation detectors. Normalized cross-spectra. Left: Orientation-specific synchronization at high oscillation frequencies (35–90 Hz). Right: In the low frequency range (1–35 Hz) of the same recordings no orientation specificity is present. Recordings from sites with similar orientation preferences from primary visual cortex (V1) of an awake monkey. The correlation (grey level) at different cortical distances with respect to a reference position (distance from centre of polar plot) is plotted for various orientations (angle of polar plot) of a large oriented sinusoidal grating stimulus. The recordings were obtained with a onedimensional linear array of micoelectrodes, its orientation of penetration was randomly selected; 0.75 mm inter-electrode separation. (Figure modified from Frien et al., 1996).

3.5. Feature Representation by Classical Receptive Fields (cRFs) and Feature Relations by Association Fields (AFs) We are extending the concept of classical receptive fields (cRFs) to association fields (AFs), in order to bring perceptual feature-linking into correspondence with the local representation of basic sensory features (Eckhorn et al., 1990). In the retina, the visual thalamus, and the first visual cortical areas basic visual features are characterized by the cRFs of single neurones, which are represented in retinotopic maps (for example the local orientation of contrast edges in V1). The definition of AFs is based on our experimental observations. Spatial coherence of FCOs generally covered larger areas in visual cortical representations than the classical receptive field (cRF) of single neurones. However, coherence was laterally confined to a few millimeters of cortical surface (Figure 3;

Time and the brain

214

Eckhorn et al., 1988; Jordan, 1989; Gray et al., 1989; Eckhorn et al., 1992; Juergens et al., 1996) which means that synchronized cortical regions span only the representational range of small visual objects or parts of larger ones. The concept of AF extends that of cRF by including feature context. AFs of visual (cortical) neurones were defined by that range in visual space that is covered by the RFs of those neurones engaging in common synchronized FCOs. The AF of visual space is directly related to the cortical synchronization field by the cortical magnification factor (relating retinotopic to cortical distances of corresponding parts). Consequently, AFs span a larger range in the visual field than cRFs because they represent the spatial correlation structure among cRFs for an actual stimulus situation. We argue that spatial continuity of an object might be coded by a continuum of overlapping AFs, i.e. by overlapping synchronized regions, spanning the entire object representation. Alternatively, it might be coded by a hierarchy of AFs in which the assemblies engage in phase-locked oscillations at different frequencies: large objects (or larger parts) are represented by larger linking fields, defined by synchronization at lower frequencies, and smaller subparts are represented by smaller linking fields at higher frequencies (Schanze and Eckhorn, 1997). Our experiments have shown the presence of both types of synchronized states, including phase-locking among different frequencies. However, we do not yet know whether such codes are used for visual processing and perception. 3.6. Coding of Spatial and Temporal Feature Separation Feature binding is one of several operations required for visual scene segmentation. Generally, many objects are present in a current scene and they often overlap and partially occlude one another. Hence, spatial separation of features from different objects is required. While spatial binding may be coded by a short population burst during which neurones representing the same object fire within a few milliseconds together (a “population spike”), spatial separation may be characterized by population spikes occurring asynchronously in neurones representing different objects. This may occur either with a constant phase due to a delay circuit, or by decoupling the synchronization networks among the different representations. An additional requirement of visual processing is that of sharp contour coding, which seems necessary because the retinal images move continuously due to movements of the eyes and of visual objects. Hence, retinal image movements should cause “smearing” in perception, but this is generally not observed. Below we propose that the temporal “chopping” of neural activities into synchronized population spikes can provide the visual system with precise (“sharp”) internal representations of a scene’s spatiotemporal feature relations (see section 3.7.).

Cortical processing by fast synchronization

215

3.7. Suppression of FCOs by Stimulus-Locked Responses May Support Temporal Scene Segmentation Temporal segmentation of the continuous stream of visual information seems necessary for clear percepts, because retinal images are continuously moving. Even under conditions of rest and ocular fixation microsaccades and drifts of the eyes are always present. Cortical neurones, on the other hand, can persist in their responses over several hundreds of milliseconds, even to very short visual stimuli of 10–20 ms duration (e.g. Rolls and Tovee, 1994). However, if a figure with sharp contours has evoked cortical activations, which persist while the retinal image of that figure shifts across the retina, the contour’s perception might be smeared, because it activates successively many other neurones with neighbouring receptive fields. Smearing might be prevented if the continuously varying stream of afferent visual information were to be “chopped” by the cortex into short “image frames”. There are three modes with temporal interruptions: 1) Chopping in the representation range of an object, evoked by the sudden displacement of its retinal image; 2) chopping over a broader range of the visual representation by large and small ocular saccades; and 3) repetitive interruptions by synchronized FCOs. In the latter case optimal “frame rates” should be adapted to the rate of retinal image displacement (velocity) and the retinal size of the object’s image. Faster framing would be desirable with higher movement velocities and smaller size (according to the sampling theorem of signal theory see Fig. 2). Stimulus-dominated synchronizations were found to suppress ongoing oscillations partly or even fully. This makes sense in the framework of the “association-bysynchronization” hypothesis, because fast oscillations are generated during slow retinal image drifts, including periods of ocular fixation and smooth pursuit and, hence, may allow repetitive interactions with visual memories. Stimulus-dominated synchronization, on the other hand, is short-lasting and typically occurs after sudden object movements, and then suppresses oscillatory states (simulations to this topic in Gabriel et al., 1997). This may prevent super-position of contradictory scene segmentations by cutting down the persistence of oscillatory states. 3.8. Potential Influence of Oscillation Frequency on Visual Scene Segmentation In anaesthetized cats and especially in awake monkeys FCOs had on average higher oscillation frequencies when the stimulus moved faster or was smaller, compared with slower stimulus movements or larger size (Figure 2; Woelbern et al., 1994). This dependency of frequency suggests that it might be advantageous for precise spatiotemporal representations. A simple calculation shows that the spatial precision of perception based on series of such internal image representations is well in the range of optimal perceptual resolutions. Take the following simple calculation: in a typical recording experiment a stimulus object of 2° (visual angle) moving at l°/s induced a dominant oscillation frequency of about 60 Hz in the visual cortex of an awake monkey. This corresponded to a retinal image displacement of one minute of arc of visual angle

Time and the brain

216

during a single activation-inhibition cycle (17 ms). An even higher precision in spatiotemporal representation would exist during periods of ocular fixation. Assume residual drift movements of the eyes of about 0.1 °/s during ocular fixation tasks (in intervals between µ-saccades), and typical cortical oscillation frequencies around 40 Hz during fixation: this results in image displacements of 9 seconds of arc for a single oscillation cycle (25 ms “frame rate”). This is well in the range of the highest perceptual resolution for line displacements. In conclusion, changing visual scenes may be represented dynamically by synchronized FCOs, which offers the opportunity for an updated precise spatiotemporal representation in each FCO cycle. 3.9. Summary of Section 3 We can say that many experimental results favour the hypothesis of “feature-binding-bysynchronized FCOs”. However, additional recording experiments from awake monkeys, including their perceptual responses, will be necessary for a proof.

4. POTENTIAL MECHANISMS OF GENERATION AND SYNCHRONIZATION OF FCOs The question has often been asked whether FCOs are due to single neurones acting as local oscillators or (alternatively) occur as a consequence of network properties. We will show that this cannot yet be finally answered. However, recent experimental and modelling work has given many indirect hints how FCOs might be generated. In the next paragraphs the discussion of FCO generation is developed from oscillator properties of single neurones, to distributed synchronized assemblies generating FCOs as a network property. The latter may vary from local to wide distribution over the cortex. 4.1. Single Neurone Oscillators? 4.1.1. Spike encoder The analysis of spike trains shows the ability of neurones to generate rhythmic signals. Rhythmically discharging neurones might therefore be called “local oscillators”. Intracellular recordings with current injection has confirmed rhythmicity as a property of the spike encoder, which generally shows a monotonic increase of discharge rate with current or depolarizing potential (e.g. Llinás et al., 1991). Keeping the driving potential at a constant level results (after adaptation) in highly regular spike trains in many neurones. With strong depolarization, spike patterns can change into rhythmic burst firing (e.g. Gray and McCormick, 1996), especially in adapting spike encoders as found in cortical

Cortical processing by fast synchronization

217

pyramidal neurones. Regular spiking behaviour is well understood and was included in models of spike generating mechanisms (e.g. Fohlmeister and Miller, 1997). In conclusion, spike encoders are potential generators of FCOs. 4.1.2. Chattering cells A recently stated proposal for FCO generation by single neurones relies on a special subgroup of supragranular cells—the so called “chattering cells” (Gray and McCormick, 1996). When FCOs occurred in the visual cortex, chattering cells contributed to these synchronous discharges by firing high frequency spike bursts, recurring at intervals of 15–50 ms, i.e. at gamma frequencies. These properties were also apparent when injected with suprathreshold depolarizing current, showing that fast burst rates were a property of the single cell’s spike encoder (Gray and McCormick, 1996). These findings indicated, at least for visually induced FCOs, that a major source of their generation is of cortical origin, and not mainly due to thalamic or other extracortical sources, as has been argued by others (Ghose and Freeman, 1992). Cells receiving presynaptic input from chattering cells are expected to show membrane potential oscillations consistent with a synaptic rather than an intrinsic mechanism (Jagadeesh et al., 1992). In addition, FCOs synchronized across larger cortical populations may well be dominated by the influence of chattering cells, which may couple their bursts via lateral network connections. 4.1.3. Intrinsic membrane oscillations In addition to the spike encoder of a neurone, its input structures, including membranes of dendrites and soma, can favour certain frequency bands, as has recently been characterized by measuring their signal transfer properties with current injection (Connors and Amitai, 1997). The transfer function of cortical pyramidal cells showed multiple peaks, comprising high frequency ranges in which signals are less damped compared to other ranges. However, generation of subthreshold fast oscillations by dendro-somatic membranes themselves (distinguished from synaptically “injected” membrane oscillations) has not been convincingly shown in cortical neurones. In contrast, intracellular recordings indicated the synaptic origin of fast subthreshold oscillations in visual cortical neurones (Jagadeesh et al., 1992). In conclusion, even though single neurones have the properties of “local oscillators” the generation of FCOs is probably dominated by cortical network properties, because oscillatory signals in cortical neurones were synaptically supplied. This leaves the possibility that FCOs are selectively amplified by membrane resonances, so that the neurone’s spike encoder generates impulse sequences in which this frequency band is dominant.

Time and the brain

218

4.2. Local Network Oscillators As FCOs are probably supplied via synaptic inputs, they can be generated by either one or several remote oscillators (with intra- or extra-cortical locations) or they might emerge as a result of local cortical feedback circuitry introducing phase shifts due to delays and inhibition. There are several indirect arguments favouring, in my opinion, the generation of FCOs by cortical network properties instead of single neurone oscillators. Excitatory and inhibitory cortical neurones are present everywhere in the cortex. They establish local feedback circuits (McGuire et al., 1991) introducing phase shifts by sign inversion and transmission delays, and hence, are suitable for the generation of local FCOs (Freeman, 1959). On average, there are four times as many excitatory compared to inhibitory cortical neurones (Braitenberg and Schüz, 1991). It is therefore assumed that small groups of excitatory cells establish negative feedback via common inhibitory neurones (Douglas and Martin, 1991). Hence, such local groups of excitatory cells can be partially synchronized when they become activated (e.g. van Vreeswijk et al., 1994; Bush and Sejnowski, 1996). As their discharges are simultaneously chopped by their common interneurones they discharge well co-ordinated population bursts (Figure 8). Such bursts are a common observation in multiple unit (MUA) recordings in the visual cortex (e.g. Eckhorn et al., 1988; Gray et al., 1989). However, synchronization in such local groups may be sharpened by lateral coupling connections, which resemble local cRF relations as expected from Gestalt laws (see section 3; and Wertheimer, 1923; for models see e.g. Eckhorn et al., 1990; Juergens and Eckhorn, 1997; Wennekers and Palm, this volume). We argued in recent work on model networks for visual feature linking that the lateral linking connections should be symmetric and of a fast modulatory type (which means facilitatory, implemented by multiplicative interaction with feeding inputs: Eckhorn et al., 1990; Stoecker et al., 1996). Such linking connections can synchronize spikes of coupled neurones very quickly, and do not change the spike rates, while they avoid deterioration of cRF-properties by superposition (as would be the case with additively superimposed synaptic inputs used by most other models of spiking neurones). 4.3. Circuits for Synchronizing FCOs at Zero Correlation Delay We found FCOs’ phase differences narrowly distributed around zero, which means single central peaks in cross-correlograms. They can be explained by inputs from common sources, with approximately the same activation delays between these sources and cortical recording positions. This explanation is plausible because such common inputs guarantee zero delay

Cortical processing by fast synchronization

219

Figure 8. Common inhibition provides synchronized chopping of neurones discharging at different rates. E: excitatory, I: inhibitory neurones

independent of the type of signal (stochastic or rhythmic) and its frequency range (low or high). Relevant evidence was obtained in many experiments, for locations in several cortical areas of cat and monkey (Ts’o et al., 1986; Eckhorn et al., 1990; Engel, 1991a,b; Munk et al., 1992; Nelson et al., 1992; Frien et al., 1994). As most cortico-cortically projecting neurones only have a single remote target (Braitenberg and Schüz, 1991) it is probable that other neurones from the same synchronized population, in another layer of the same cortical column, supply the common input. Alternative or additional possibilities for establishing zero-phase FCOs are symmetric excitatory or modulatory feedback interactions among the cortical areas (Eckhorn and Frien, 1995; Saam and Eckhorn, 1998). Theoretical considerations and simulation models revealed that zero-delay phase locking requires special dynamic constraints, including activation delays around such loops to be shorter than about a third of the oscillation period (Gerstner et al., 1993; Ritz et al., 1994; Hansel et al., 1995; Crook et al., 1997), because this is the range in which constructive superposition of excitatory signals is possible. In particular, if the symmetric connectivity consists of inter- and intra-areal projections from the same local populations that activate their postsynaptic targets at about the same delay, the zero delay peaks in cross-correlograms can be explained for oscillatory and stochastic modes of activity (Eckhorn and Frien, 1995). Symmetric inhibitory connections among the inhibitory pools of spatially separate local population oscillators have also been proposed for establishing zero delay fast rhythms (Schillen and König, 1994). Even though this is a probable mechanism for synchronizing FCOs, it cannot explain the observed, central correlation peaks in states with stochastic activity. However, it cannot be ruled out on the basis of present experimental knowledge that mutual inhibition of inhibitory pools is an important mechanism for synchronizing FCOs.

Time and the brain

220

Common input from the visual thalamus (and the retina) is probably not the main source of FCOs, even though in anaesthetized cats oscillations were found there (Ghose and Freeman, 1992; Ito et al., 1994; Sillito et al., 1995; Neuenschwander and Singer, 1996). The first reason why such oscillations probably do not play a central role is that their oscillation frequencies were generally below those recorded in visual cortex (Neuenschwander and Singer, 1996). In addition, the distribution and phase relations of field potentials across the cortical layers is fundamentally different for signals coming from the visual thalamus and those of FCOs. The thalamic inputs always cause field distributions with a dipole character, showing phase reversal between upper and lower layers (zero at layer 4) independent of the frequency range (0–150 Hz), and with both electrical and visual stimulation (Creutzfeldt et al., 1966; Cracco and Cracco, 1978; Kraut et al., 1985; Mitzdorf, 1987). FCOs, in contrast, did not show phase shifts across cortical layers, their phases being narrowly distributed around zero (Jordan, 1989; Gray et al., 1989; Eckhorn, 1992). Coupling of cortical with other thalamic nuclei at FCO frequencies has also been found (Steriade, 1993; Steriade et al., 1996). However there are no indications yet whether these nuclei are involved in generating the FCOs. 4.4. Establishing Cortical Synchronization Fields by Hebbian Correlation Learning? We developed a biologically plausible network of spiking neurones, explaining the dependence of the width of cortical synchronization fields on lateral spike velocity, temporal dispersion of correlated input, duration of Hebbian learning window, and the period of the synchronized oscillatory driving input (Saam and Eckhorn, 1998). Spiking neurones were retinotopically organized in a map of feature detectors, and each neurone was laterally connected to all others by constant velocity axons and facilitatory synapses (Eckhorn et al., 1990). Systematic dependencies of synchronization field width on simulation parameters were derived (Figure 9): lower spike velocities led to narrower fields, because at larger distances the evoked PSPs of laterally travelling spikes came too late to coincide with spikes of the same oscillation period. Increasing the relative temporal jitter of the spike patterns in the neurone’s feeding inputs, from fully correlated and rhythmic to higher temporal dispersion (according to broader peaks in the pairwise cross-correlograms), increased the width of synchronization fields. This is because many input spikes occurred before the highest average spike correlation in the inputs so that longer lateral spike transmission delays could fulfil the Hebbian condition of simultaneity. If the input spike jitter exceeded a critical fraction of the rhythm’s period (about 1/3) the weight distributions lost their spatial structure. Conversely, reduction of the oscillation period narrowed the width of the synchronization field due to the restricted time available for lateral spike transmission and near-coincidence. In conclusion, our model implies that the width of cortical synchronization fields can become larger if lateral spike velocities are higher, correlation width of input signals broader, and periods of common oscillations longer. This model generated synchronization fields of the same width (3 to 5 mm) as found in visual cortex V1 and V2

Cortical processing by fast synchronization

221

of cat and monkey if biologically plausible values for lateral spike velocities (estimated average velocity 1 m/s) and oscillation frequencies (40 Hz) were used for the simulations. In

Figure 9. Width of association (synchronization) field (AF) in topographic neural map depends on lateral spike velocity and temporal dispersion of correlated input: a model with spiking neurones and Hebbian learning. a: Spatial distribution of correlation strength after learning (fixed axonal velocity), b: Dependence of association field width on axonal velocity, c: Dependence of AF width on temporal dispersion of correlated spikes. (Figure modified from Saam and Eckhorn, 1998).

addition, as in real cortices, the model produced (on average) zero phase delay among any neurones within the synchronization range. 4.5. Network Properties Explaining Influence of Visual Stimulation on Average FCO Frequency and Amplitude FCOs occur predominantly when cortical neurones are activated by visual stimulation in a sustained manner. Even with constant stimulation, oscillation frequency, amplitudes, and the cortical extent of synchronization fields vary over a broad range. However, their average values change in a systematic way with changes in stimulation (see section 3). These effects are compatible with a process of dynamic self-organization among coupled neurones receiving sustained activation by a common stimulus. With an increasing level of activation, local populations, having similar cRF-properties, start FCOs at high frequency due to their short, and hence fast, internal feedback loops. Synchronization among distant local populations does not occur because of the irregular nature of these FCOs, their low amplitude and the relatively long activation delays between populations more than a hypercolumn (1 mm) apart. At higher levels of activation, however, local populations activate their common inhibitory feedback circuits discharging longer bursts with stronger activation, and hence, inhibit the population over a longer period causing a lower frequency of the rhythm (Wennekers et al., 1994). At the lower FCO frequency mutual synchronization among neighbouring populations becomes possible via bidirectional connections, because the longer cycle period provides enough time for zerophase locking. In the context of this argument, a given local oscillation frequency limits the cortical coherence range due to lateral activation velocities (including axonal velocity and synapto-somatic activation delays) (Saam and Eckhorn, 1998). In monkey primary

Time and the brain

222

visual cortex the coherence range was 4 to 6 mm at about 40 Hz which is compatible with lateral activation velocities of 1 m/s (a biologically plausible value). 4.6. Mechanisms of Generation, Enhancement and Suppression of Signal Correlations by Local Populations of Coupled Neurones In order to understand the action of partially synchronized signals on a local group of laterally coupled neurones in different states of global activation, we developed a neural

Figure 10 A-C. Three Different Modes of Simulated Network Dynamics. They emerged by changing the lateral coupling strength, when the average

Cortical processing by fast synchronization

223

amplitudes of the random Gaussian white noise inputs were low, and a high constant value of the membrane potential caused relatively high sustained discharge patterns. A: Alternating stochastic and oscillatory activity (weak lateral coupling). B: Synchronized short oscillations, with random frequencies in successive events (medium coupling). C: Rhythmical bursts (stronger coupling). a: Feeding-, b: linking-, c: membrane potential, d: threshold potential, e: spike output patterns, f: multiple unit activity. ACF: auto-coincidence correlogram of output spikes, CCF: cross-coincidence correlogram of output spikes from two different model neurones. (Figure modified from Juergens and Eckhorn, 1997).

network model of spiking neurones (Juergens and Eckhorn, 1997). As in the visual cortex, in this network “FCOs” occurred only during enhanced sustained activations of the neurones (either by increasing the average input drive or by changing the threshold properties of all neurones equally). Three different patterns of synchronized FCOs emerged: enhancing the sustained drive sufficiently above threshold, so that medium frequencies of rather “regular” patterns were discharged by the neurones, led to relatively high frequency FCOs (Figure 10a). However, further increase in the sustained input level forced the neurones to discharge short high-frequency bursts, that occurred at lower repetition rates, and hence, lower FCO frequencies (Figure 10c) compared to the former oscillations (Figure 10a). At intermediate driving levels, spike trains were well synchronized, but these events occurred non-rhythmically (Figure 10b). However, low level inputs diminished the degree of synchronization, and FCOs were generally absent. In this state the neurones discharged only few action potentials, and partial synchronization of the input signals could be enhanced or suppressed gradually, depending on the lateral coupling strength among the neurones. These results fit our experimental observations quite well, even though the model was rather simple. 4.7. Networks Leading to Phase-Shift and Decoupling With Stimulation by Overlapping Visual Objects The situation in which several visual objects overlap, or are adjacent, has been analysed with neural network simulations by several groups (e.g. Schillen and König, 1994; Sompolinsky et al., 1990). For this situation, the synchronization hypothesis predicts separable oscillatory processes. However, this can be accomplished either by phase shift of the FCOs at the same common frequency, or by statistically independent FCOs. Nearly all models generated phase-shifted oscillations at the same frequency in neural representations of different scene segments. This was either due to a fast inhibitory feedback loop, that was common to all excitatory neurones (Schott, [1995], in Eckhorn, 1998 and see Fig. 8), and/or it was caused by mutual inhibition among spatially separate groups of excitatory neurones, as proposed by other model work. However, in the visual cortex this behaviour of phase-shifted coding at a common frequency has never been demonstrated, as far as the author knows the literature. Instead, several works have shown statistical decorrelation in their figures even though the

Time and the brain

224

authors do not explicitly state it. We recently designed an experiment in which this question was asked directly. We obtained statistically independent FCOs with two objects at two recording positions, while the FCOs synchronized if the neurones at each of these positions were stimulated by the same object. This demonstrates the potential for a functional disconnection of the lateral coupling. As synchronization was probably mediated via monosynaptic connections during these experiments, we propose that the neurones detecting the contrast separating two scene segments exert modulatory inhibition on the lateral coupling synapses and, by this means, functionally disconnect the coupling.

5. DO THE RELATIONS AMONG RECEPTIVE- AND ASSOCIATIONFIELDS GIVE HINTS TO A BASIC CORTICAL PROCESSING UNIT? We still do not know how most of the basic operations of vision are performed by the visual cortex. These operations include: (i) extraction and representation of relevant—and suppression of irrelevant—visual features; (ii) feature grouping into scene segments and objects; (iii) transformations into invariant representations, so that comparison with stored objects, and learning of new objects is possible; (iv) associative processing and memory; and (v) generation of stable and unambiguous percepts. It is not clear whether these operations each require special “smart” circuits, nor whether they are carried out more in a serial manner or more in a distributed way— stepwise by many similar modules. The latter possibility is supported by anatomical observations of similar local circuitry in the different areas of visual cortex. If this is actually the case we should already be able to identify basic modules in the primary visual cortex. The present approach tries to start with relations among simple-and complex-cell cRFs and AFs. 5.1. Interactions Among Simple and Complex Cells as an Example A simple cell in V1 responds well to an oriented contour if it is placed at a certain position, precisely defined by the locations and polarities of its cRF-subfields. The cRFs of complex neurones, on the other hand, show about the same width of orientation tuning, but the exact position of the contours and their polarity play a minor role for the cell’s activation. This inspired Hubel and Wiesel (1962) to propose that complex cells receive excitatory feeding inputs from a number of simple cells with similar orientation preferences, but with cRF positions spatially dispersed across the cRFs of the complex cell. In addition, simple cells should be driven by parvocellular neurones with aligned cRFs of appropriate position and polarity (ON- and OFF-centre). This connectivity seems convincing and is included in our scheme in Figure 11 a. Later experiments found direct and fast inputs from the magno-cellular pathway to complex cells (Hoffmann and von Seelen, 1978; Henry et al., 1979), while they and others confirmed the parvocellular input to simple cells (review in Henry et al., 1994). We therefore added excitatory feeding input from the magnocellular afferents to complex cells.

Cortical processing by fast synchronization

225

As FCOs are generated only in states of sustained input, it seems plausible that tonic parvocellular input to simple cells dominates in this situation, while the phasic magnocellular input is silent. With this and the above schema in mind, we predicted that FCOs would occur in simple cells, with a phase lead of a few milliseconds with respect to those in complex cells according to this activation delay. However, recordings from V1 of awake monkeys always revealed that FCO phase differences among simple and complex cells were narrowly distributed around zero (S.D. about 1 ms: Eckhorn et al, 1995). If we accept that complex cells receive activation by simple cells in this situation, we have to explain the zero phase delay by an additional mechanism. For this we propose a feedback loop, via an excitatory drive of simple and complex cells to a common inhibitory neurone (or pool of coupled neurones) that provides feedback inhibition to its input cells (circuit included in Figure 11 a). This circuit suppresses (simultaneously and repetitively) the activities of simple and complex cells by a fast inhibitory component (“chopped” synchronization as suggested in section 4.2). A longer-lasting inhibitory component supplied via the same connections may reduce activations of simple cells that are weakly driven by the stimulus, or are spontaneously active, and thus enhances contrast and signal-to-noise-ratio of an actual contour’s representation.

Figure 11, ab Proposed interactions among simple (S) and complex (C) cell; I: inhibitory cell; P: input from retino-thalamic parvocellular neurones, providing sustained activation during ocular fixation; M: input from retino-thalamic magnocellular neurones, signalling transient changes in local retinal luminance; S→C feeding: feeding connections from simple to complex cells, generating the cRF properties of complex cells by superimposing simple cRFs; S↔C linking: coupling connections providing fast spike synchronization, b: Range of

Time and the brain

226

synchronized simple cells association field (AF) influences classical receptive field (cRF).

This circuit also explains the suppression of static retinal information (supplied by the parvocellular pathway) by fast stimulus transients (transmitted by the magnocellular pathway; Lennie, 1980). The magnocellular input transiently activates complex cells, and these excite the inhibitory loop suppressing simple cell activities. In addition, this mechanism explains the suppression of FCOs in simple and complex cells by transient inputs (Kruse and Eckhorn, 1996) because any input transient will disturb and inhibit ongoing FCOs. Chopping by a common feedback inhibition also accounts for the zero phase delay among simple cells of similar orientation preference and overlapping cRFs. However, zero phase coupling among FCOs of simple cells with more separate cRFs cannot be explained by “local chopping”. For this we assume fast facilitatory connections, that can synchronize spike activities in remote populations without compromising their cRF properties (experimental and model results in Eckhorn et al., 1990). These “linking connections” are assumed to reflect Gestalt properties (as argued in section 3). Here in our example this is “similarity” (of orientation preference) and “proximity” (overlapping and neighbouring cRFs). Fast modulatory feedback connections from complex to simple cells are proposed, based on theoretical arguments and neural network simulations (Eckhorn et al., 1990). In the “transient mode” of stimulation they would sensitize, via the activated complex cell, the simple cells’ inputs (for example when a visual object suddenly moves). This can facilitate parvocellular inputs because they arrive delayed, and could explain involuntary attention and pop-out of transient stimuli. In the “sustained mode” of stimulation, when FCOs are generated by this circuit, complex cells would average the FCO phases of their inputs from simple cells and feed back this average signal for modulating and gating the simple cells’ feeding inputs from the previous stage. This operation would support spikes occurring synchronized at the FCOs rhythm, while out-of-phase activity, including spontaneous spikes, will be suppressed and therefore excluded as an input to the complex cell. It is probable from theoretical considerations and modelling studies that the width of the simple cells’ association field (defined by the superimposed cRFs of simple cells that can engage in a synchronized FCO) mainly determines the cRF width of complex fields (Figure 11b). This seems probable, because a synchronized input would be particularly successful with Hebbian learning, and it fits the observation that cRFs become larger in successive processing steps (e.g. from V1 to V2). 5.2. Generalized Functions of the Proposed Circuit If this circuit of simple/complex cell interactions is present in a similar way throughout other cortical areas, the following more general processing might be carried out by it. Extraction and representation of relevant features is performed by the convergence of feeding connections (with concentric cRFs) to the lower layer (simple cells) which are

Cortical processing by fast synchronization

227

superimposed to form new cRFs (preferring oriented contour segments). Feature grouping is supported in several ways: 1) by lateral linking connections synchronising (simple) cell spikes that were activated by the same stimulus object; 2) by feedback linking connections from the upper (complex) to the lower layer (simple) cells so that some averaged input is fed back supporting the nearly synchronized lower level cells; and 3) by the negative feedback loop that generates a rhythm (FCO) common to all lower level cells and suppresses dynamic feature representations occurring out-of-phase to it. Associative memory is incorporated in this coupling network by the distribution of synaptic weights. These weights are assumed to be shaped by Hebbian learning so that knowledge of relevant feature combinations is stored here (in our example the Gestalt rules of similarity and proximity of contour elements). “Learning” is supported by the repetitive high amplitudes of synchronized FCOs. “Recall” is executed when an input fits the requirements of activating a subset of cells in a synchronized manner (a partitioned oriented contour in a certain retinal position). “Associative completion” is made with partial or disturbed inputs, activating the same subset of units that is activated by the undisturbed input. Invariance and generalization can be assigned to the feedforward connections from lower to upper layer, by accepting only inputs with high similarity in some cRFproperties while other cRF properties are less strictly controlled but collected over a certain coding range and hence, are generalized (in our example, simple cells of equal orientation but distributed cRF positions converge on complex cells). Classification is made in the upper layer by a set of complex cells (with mutual synchronization and common inhibition?) with specific feeding inputs from simple cells, e.g. either sets of simple inputs with coaxially aligned cRFs (for contour-form representation) or sets of inputs with parallel cRFs (for grating-texture representation). In conclusion it has been argued how visual processing for object recognition might be carried out by a rather simple modular circuit, derived from interactions among simple and com-plex cells in the primary visual cortex. This module can, in principle, carry out stepwise basic visual operations, including feature extraction, associative grouping, processing of invariance and generalization, classification and associative memory and recall. These operations are also carried out by other sensory systems. However, more elaborate experimental and model investigations will be necessary before a generally acceptable module can be proposed, or before concepts of cortical modularity can be rejected.

ACKNOWLEDGEMENTS This article could not have been written without the numerous discussions with my colleagues and without their extensive help in experiments, data acquisition and processing, as credited in the text and figure captions. I want to thank Guenther Palm, Thomas Wennekers and Andrea Bibbig for years of fruitful discussions about the concepts proposed in this chapter. In addition, Robert Miller has considerably improved my work by pointing to unclear pathways of argumentation, by correction of my English

Time and the brain

228

and by his thorough editorial work. Expert help in care and preparation of our cats and monkeys, and help in experimental techniques came from U.Thomas, J.H.Wagner and W.Lenz. The financial support by the Deutsche Forschungsgemeinschaft is also greatly acknowledged (Ec 53/6, EC 53/7 and Ro 529/12, all to R.E.).

REFERENCES Abeles, M. (1991) Corticonics: Neuronal circuits of the cerebral cortex. Cambridge University Press. Adrian, E.D. (1950) The electrical activity of the mammalian olfactory bulb. Electroencephalography and Clinical Neurophysiology, 2, 377–388. Aertsen, A.M., Gerstein, G.L., Habib, M.K., and Palm, G. (1989) Dynamics of neuronal firing correlation: modulation of “effective connectivity”. Journal of Neurophysiology, 61, 900–917. Agmon-Snir, H., and Segev, I. (1993) Signal delay and input synchronization in passive dendritic structures. Journal of Neurophysiology, 70, 2066–2085. Baker, S.N., Olivier, E., and Lemon, R.N. (1997) Coherent oscillations in monkey motor cortex and hand muscle EMG show task-dependent modulation. Journal of Physiology (London), 501, 225–241. Earth, D.S., and MacDonald, K.D. (1996) Thalamic modulation of high-frequency oscillating potentials in auditory cortex. Nature, London, 383, 78–81. Bernander, O., Koch, C., and Usher, M. (1994) The effect of synchronized inputs at the single neuron level. Neural Computing, 6, 622–641. Bragin, A., Jando, G., Nadasdy, Z., Hetky, J., Wise, K., and Buzsaki, G. (1995) Gamma (40–100 Hz) oscillations in the hippocampus of the behaving rat. Journal of Neuroscience, 15:47–60. Braitenberg, V., and Schüz, A. (1991) Anatomy of the Cortex. Statistics and Geometry. Springer, Berlin. Brosch, M., Bauer, R., and Eckhorn, R. (1995) Synchronous high-frequency oscillations in cat area 18. European Journal of Neuroscience,7, 86–95. Brosch, M., Bauer, R., and Eckhorn, R. (1997) Stimulus-dependent modulations of correlated high-frequency oscillations in cat visual cortex. Cerebral Cortex, 7, 70–76. Bush, P.C., and Sejnowski, T.J. (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. Journal of Computational Neuroscience, 3, 91–110. Connors, B.W., and Amitai, Y. (1997) Making waves in the neocortex. Neuron, 18, 347– 349. Cracco, R.Q., and Cracco, J.B. (1978) Visual evoked potentials in man: early oscillatory potentials. Electroencephalography and Clinical Neurophysiology, 45, 731–739. Creutzfeldt, O.D., Watanabe, S., and Lux, H.D. (1966) Relation between EEGphenomena and potentials of single cells. Part I and II. Electroencephalography and Clinical Neurophysiology, 20, 1–37. Crook, S.M., Ermentrout, G.B., Vanier, M.C., and Bower, J.M. (1997) The role of axonal delay in the synchronization of networks of coupled cortical oscillators. J Computational Neuroscience, 4, 161–172. Douglas, R.J., and Martin, K.A.C. (1991) A functional microcircuit for cat visual cortex. Journal of Physiology (London), 440, 735–769.

Cortical processing by fast synchronization

229

Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H.J. (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biological Cybernetics, 60, 121– 130. Eckhorn, R., Reitboeck, H.J., Arndt, M., and Dicke, P. (1990) Feature linking among distributed assemblies: Simulations and results from cat visual cortex. Neural Computation, 2, 293–306. Eckhorn, R., and Schanze, T. (1991) Possible neural mechanisms of feature linking in the visual system: Stimulus-locked and stimulus-induced synchronizations. In: A.Babloyantz (ed.), Self-Organization, Emerging Properties and Learning. Plenum Press, New York, pp. 63–80. Eckhorn, R. (1992) Principles of global visual processing of local features can be investigated with parallel single-cell- and group-recordings from the visual cortex. In: A.Aertsen and V.Braitenberg (eds) Information Processing in the Cortex, Experiments and Theory, Springer-Verlag, Berlin Heidelberg, New York, pp. 385–20. Eckhorn, R., Schanze, T., Brosch, M., Salem, W., and Bauer, R. (1992) Stimulus-specific synchronizations in cat visual cortex: Multiple microelectrode and correlation studies from several cortical areas. In: E.Basar, and T.H.Bullock (eds), Induced Rhythms in the Brain. Brain Dynamics Series. Birkhäuser, Boston, Basel Berlin, pp. 47–82. Eckhorn, R., Frien, A., Bauer, R., Woelbern, T., and Kehr, H. (1993) High frequency (60–90 Hz) oscillations in primary visual cortex of awake monkey. NeuroReport, 4, 243–246. Eckhorn, R., and Obermueller, A. (1993) Single neurons are differently involved in stimulus-specific oscillations in cat visual cortex. Experimental Brain Research, 95, 177–182. Eckhorn, R., Rutschmann, R., Woelbern, T., Frien, A., and Bauer, R. (1995) Neurons responding with different latencies and time courses to a visual stimulus engage in near-zero delay mutual correlation in the visual cortex of awake monkey. Society for Neuroscience Abstracts, 21, 648.17. Eckhorn, R., and Frien, A. (1995) Neural signals as indicators of spatial and temporal segmentation coding in the visual system. Proceedings of the International Conference on Brain Processes, Theories and Models. J.Mira-Mira (ed). MIT-Press, Cambridge MA, pp. 1–10. Eckhorn, R. (1998) Neural mechanisms of scene segmentation: Recordings from the visual cortex suggest basic circuits for linking field models. IEEE Neural Networks. Engel, A.K.,König, P., Gray, C.M., and Singer, W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: Inter-columnar interaction as determined by crosscorrelation analysis. European Journal of Neuroscience, 2, 588–606. Engel, A.K., König, P., Kreiter, A.K., and Singer, W. (1991a) Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, New York, 252, 1177–1179. Engel, A.K., Kreiter, A.K., König, P., and Singer, W. (1991) Synchronization of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat. Proceedings of the National Academy of Sciences, U.S.A., 88, 6048–6052. Fohlmeister, J.F., and Miller, R.F. (1997) Mechanisms by which cell geometry controls repetitive impulse firing in retinal ganglion cells. Journal of Neurophysiology, 78, 1984–1964. Fox, K., and Daw, N. (1992) A model of the action of NMDA conductances in the visual cortex. Neural Computation, 4, 59–83.

Time and the brain

230

Freeman, W.J. (1959) Distribution in space and time of prepyriform electrical activity. Journal of Neurophysiology, 22, 644–666. Frien, A., Eckhorn, R., Bauer, R., Woelbern, T., and Kehr, H. (1994) Stimulus-specific fast oscillations at zero phase between visual areas V1 and V2 of awake monkey. NeuroReport, 5, 2273–2277. Frien, A., Eckhorn, R., and Reitboeck, H.J. (1996) Fast oscillations in V1 of awake monkey: synchronization depends on cortical distance, on angle between preferred orientations and on stimulus orientation. Society for Neuroscience Abstracts, 22, 255.5. Fries, P., Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997) Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proceedings of the National Academy of Sciences, U.S.A., 94, 12699–12704. Gabriel, A., Kopecz, K., and Eckhorn, R. (1997) Enhancement of stimulus-locked spatiotemporal contrast for scene segmentation in a computational model of spiking neurons. In: N.Elsner and H.Wässle (eds) Göttingen Neurobiology Report 1997. Thieme, Stuttgart New York, p. 1024. Gerstner, W., Ritz, R., and van Hemmen, J.J. (1993) A biologically motivated and analytically soluble model of collective oscillations in the cortex. I. Theory of weak locking. Biological Cybernetics, 68, 363–374. Ghose, G.M., and Freeman, R.D. (1992) Oscillatory discharge in the visual system: Does it have a functional role? Journal of Neurophysiology, 68, 1558–1574. Gilbert, C.D. (1993) Circuitry, architecture and functional dynamics of visual cortex. Cerebral Cortex, 3, 373–386. Gray, C.M., König, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, London, 338, 334–337. Gray, C.M., Engel, A.K., König, P., and Singer, W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: receptive field properties and feature dependence. European Journal of Neuroscience, 2, 607–619. Gray, C.M. (1994) Synchronous oscillations in neuronal systems: Mechanisms and functions. Journal of Computational Neuroscience, 1, 11–38. Gray, C.M., and Viana Di Prisco, G. (1997) Stimulus-dependent neuronal oscillations and local synchronization in striate cortex of the alert cat. Journal of Neuroscience, 17, 3239–3253. Gray, C.M., and McCormick, D.A. (1996) Chattering cells: superficial pyramidal neurons contributing to the generation of synchronous oscillations in visual cortex. Science, New York, 274, 109–113. Gray, C.M., Maldonado, P.E., Wilson, M., and McNaughton, B. (1995) Tetrodes markedly improve the reliability and yield of multiple single-unit isolation from multiunit recordings in cat striate cortex. Journal of Neuroscience Methods, 63, 43–54. Guettler, A., Eckhorn, R., Frien, A., and Woelbern, T. (1996) Influence of microsaccades on fast oscillations in monkey striate cortex. In: N.Elsner and H.-U.Schnitzler (eds) Brain and Evolution, (eds). Thieme, Berlin, New York, p. 421. Guettler, A., Eckhorn, R., Juergens, E., and Frien, A. (1997) Neural correlation contrast in visual cortex of monkey changes with stimulus contrast across an object-background border. In: N.Elsner and H.Wässle (eds) Göttingen Neurobiology Report 1997. Thieme, Stuttgart, New York, p. 551. Hansel, D., Mato, G., and Meunier, C. (1995) Synchrony in excitatory neural networks. Neural Computation, 7, 192–210.

Cortical processing by fast synchronization

231

Hāusser, M., and Roth, A. (1997) Estimating the time course of the excitatory synaptic conductance in neocortical pyramidal cells using a novel voltage jump method. Journal of Neuroscience, 17, 7606–7625. Henry, G.H., Harvey, A.R., and Lund, J.S. (1979) The afferent connections and laminar distributions of cells in the cat striate cortex. Journal of Comparative Neurology, 178, 725–744. Henry, G.H., Michalski, A., Wimborne, B.M., and McCart, R.J. (1994) The nature and origin of orientation specificity in neurons of the visual pathways. Progress in Neurobiology, 43, 381–437. Hoffmann, K.P., and von Seelen, W. (1978) Analysis of neuronal networks in the visual system of the cat using statistical signals—simple and complex cells. Biological Cybernetics, 31, 175–185. Hubel, D.H., and Wiesel, T.N. (1962) Receptive fields, binocular interaction, and functional architecture in the cat’s visual cortex. Journal of Physiology (London), 160, 106–154. Ito, H., Gray, C.M., and Viana Di Prisco, G. (1994) Can oscillatory activity in the LGN account for the occurrence of synchronous oscillations in the visual cortex? Society for Neuroscience Abstracts, 20.61.7. Jagadeesh, B., Gray, C.M., and Ferster, D. (1992) Visually-evoked oscillations of membrane potential in neurons of cat striate cortex studied with in vivo whole cell patch recording. Science, New York, 257, 552–554. Jordan, W. (1989) A model of global coding of feature relations in the visual cortex. Dissertation, Neurophysics Grp, Philipps-University Marburg. Juergens, E., Eckhorn, R., Frien, A., and Woelbern, T. (1996) Restricted coupling range of fast oscillations in striate cortex of awake monkey. In: N.Elsner and H.-U.Schnitzler (eds) Brain and Evolution. Thieme, Berlin, New York, p. 418. Juergens, E., Guettler, A., and Eckhorn, R., (1999) Visual stimulation elicits locked and induced gamma oscillations in monkey intracortical- and EEC-potentials, but not in human EEG. Exp. Brain Res., 129, 247–259. Juergens, E., and Eckhorn, R. (1997) Parallel processing by a homogenous group of coupled model neurons can enhance, reduce and generate signal correlations. Biological Cybernetics, 76, 217–227. König, P., Engel, A.K., Roelfsema, P.R., and Singer, W. (1995) How precise is neuronal synchronization? Neural Computation, 7, 469–485. König, P., Engel, A.K., and Singer, W. (1995) Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A., 92, 290–294. Kottmann, M., and Eckhorn, R. (1996) Perception related synchronized oscillations in monkey striate cortex in a binocular rivalry task. Society for Neuroscience Abstracts, 22, 255.10. Kraut, M.A., Arezzo, J.C., and Vaughan, H.G. (1985) Intracortical generators of the flash VEP in monkeys. Electroencephalography and Clinical Neurophysiology, 62, 300– 312. Kreiter, A.K., and Singer, W. (1996) Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. Journal of Neurophysiology, 16, 2381–2396. Kruse, W., and Eckhorn, R. (1996) Inhibition of sustained gamma oscillations (35–80 Hz) by fast transient responses in cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A., 93, 6112–6117.

Time and the brain

232

Kuramoto, Y. (1991) Collective synchronization of pulse-coupled oscillators snd excitable units. Physica D, 50, 15–30. Legatt, A.D., Arezzo. J., and Vaughan, Jr H.G. (1980) Averaged multiple unit activity as an estimate of phasic changes in local neuronal activity. Journal of Neuroscience Methods, 2, 203–217. Lennie, P. (1980) Perceptual signs of parallel pathways. Philosophical Transactions of the Royal Society, London, series B , 290, 23–37. Llinás, R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10–50 Hz frequency range. Proceedings of the National Academy of Sciences, U.S.A., 88, 879–901. MacKay, W.A., and Mendonca, A.J. (1995) Field potential oscillatory bursts in parietal cortex before and during reach. Brain Research, 704, 167–174. McGuire, B.A., Gilbert, C.D., Rivlin, P.K., and Wiesel, T.N. (1991) Targets of horizontal connections in macaque primary visual cortex. Journal of Comparative Neurology, 305, 370–392. Miller, R. (1994) What is the contribution of axonal conduction delay to temporal structure in brain dynamics? In: C.Pantev, T.Elbert, B.Lütkenhöner (eds) Oscillatory Event-Related Brain Dynamics. NATO ASI-Series A, Vol 271, Plenum Press London, pp. 53–58. Milner, P.M. (1974) A model for visual shape recognition. Psychological Review, 81, 521–535. Mitzdorf, U. (1987) Properties of the evoked potential generators: current source-density analysis of visually evoked potentials in the cat cortex. International Journal of Neuroscience, 33, 33–59. Munk, M.H.J., Nowak, L.G., Chouvet, G., Nelson, J.I., and Bullier, J. (1992) The structural basis of cortical synchronization. European Journal of Neuroscience, Suppl 5, 21. Munk, M.H.J., Roelfsema, P.R., König, P., Engel, A.K., and Singer, W. (1996) Role of reticular activation in the modulation of intracortical synchronization. Science, New York, 272, 271–274. Murthy, V.N., and Fetz, E.E. (1994) Effects of input synchrony on the firing rate of a 3conductance cortical neuron model. Neural Computation, 6, 1111–1126. Murthy, V.N., Aoki, F., and Fetz, E. (1994) Synchronous oscillations in sensorimotor cortex of awake monkeys and humans. In: C.Pantev (ed) Oscillatory Event Related Brain Dynamics. Plenum, New York, pp. 343–356. Murthy, V.N., and Fetz, E.E. (1997a) Oscillatory activity in sensorimotor cortex of awake monkeys: synchronization of local field potentials and relation to behavior. Journal of Neurophysiology, 76, 3949–3967. Murthy, V.N., and Fetz, E.E. (1997b) Synchronization of neurons during local field potential oscillations in sensorimotor cortex of awake monkeys. Journal of Neurophysiology, 76, 3968–3982. Nelson, M. (1994) A mechanism for neuronal gain control by descending pathways. Neural Computation, 6, 242–254. Neuenschwander, S., and Singer, W. (1996) Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature (London), 379, 728–732. Nowak, L.G., Munk, M.H.J., Nelson, J.I., James, A.C., and Bullier, J. (1995a) The structural basis of cortical synchronization. I. Three types of interhemispheric coupling. Journal of Neurophysiology, 74, 2379–2400.

Cortical processing by fast synchronization

233

Nowak, L.G., Munk, M.H.J., Girard, P., and Bullier, J. (1995b) Visual latencies in areas V1 and V2 of the macaque monkey. Visual Neuroscience, 12, 371–384. Nowak, L.G., and Bullier, J. (2000) Cross-correlograms for neuronal spike trains: Different types of temporal correlation in neocortex, their origin and significance. In: R.Miller (ed) Time and the brain. Harwood Academic Publishers, Reading, in press. Reitboeck, H.J. (1983) A multi-electrode matrix for studies of temporal signal correlations within neural assemblies. In E.Başar, (ed), Synergetics of the Brain, pp. 174–182. Ritz, R., Gerstner, W., Fuentes, U., and van Hemmen, J.L. (1994) A biologically motivated and analytically soluble model of collective oscillations in the cortex. II: Application to binding and pattern segmentation. Biological Cybernetics, 71, 349–358. Rolls, E.T., and Tovee, M.J. (1994) Processing speed in the cerebral cortex and the neurophysiology of visual masking. Proceedings of the Royal Society of London, Series B, 257, 9–15. Rougeul, A., Bouyer, J.J., Dedet, L., and Debray, O. (1979) Fast somato-parietal rhythms during combined focal attention and immobility in baboon and squirrel monkey. Electroencephalography and Clinical Neurophysiology, 46, 310–319. Saam, M., and Eckhorn, R. (1998) Width of association field in topographic neural map depends on lateral spike velocity and temporal dispersion of correlated input: a learning model with spiking neurons. In: N.Elsner and R.Wehner (eds) New Neuroethology on the Move. Thieme, Stuttgart, p. 767. Sanes, J.N., and Donoghue, J.P. (1993) Oscillations in local field-potentials of the primate motor cortex during voluntary movement. Proceedings of the National Academy of Sciences, U.S.A., 90, 4470–4474. Schanze, T., and Eckhorn, R. (1997) Phase correlation among rhythms present at different frequencies: Spectral methods, application to microelectrode recordings from visual cortex and functional implications. International Journal of Psychophysiology, 26, 171–189. Schillen, T.B., and König, P. (1994) Binding by temporal structure in multiple feature domains of an oscillatory neuronal network. Biological Cybernetics, 70, 397–405. Schneider, J., Eckhorn, R., and Reitböck, H.J. (1983) Evaluation of neuronal coupling dynamics. Biological Cyberneticsetics, 46, 129–134. Schott, U. (1995) Simultaneous unsupervised learning of phase-separated patterns in feed-back associative memories . Masters Thesis, Philipps University, Depart ment of Neurophysics, Marburg, Germany. Schuster, H.G., and Wagner, P. (1990) A model for neuronal oscillations in the visual cortex. Biological Cybernetics, 64, 72–82. Sillito, A.M., Jones, H.E., Gerstein, G., and West, D.C. (1995) Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London, 369, 479–482. Sompolinsky, H., Golomb, D., and Kleinfeld, D. (1990) Global processing of visual stimuli in a neural network of coupled oscillators. Proceedings of the National Academy of Sciences, U.S.A., 87, 7200–7204. Steriade, M. (1993) Central core modulation of spontaneous oscillations and sensory transmission in thalamo-cortical systems. Current Opinion in Neurobiology, 3, 619– 625. Steriade, M., Amzica, F., and Contreras, D. (1996) Synchronization of fast (30–40 Hz) spontaneous cortical rhythms during brain activation. Journal of Neuroscience, 16, 392–417.

Time and the brain

234

Stoecker, M., Reitboeck, H.J., and Eckhorn, R. (1996) A neural network for scene segmentation by temporal coding. Neurocomputing, 11, 123–134. Swadlow, H.A. (1983) Efferent systems of primary visual cortex: A review of structure and function. Brain Research Reviews, 6, 1–24. Tallon-Baudry, C, Bertrand, O., Wienbruch, C., Ross, B., and Pantev, C. (1997) Combined EEG and MEG recordings of visual 40 Hz responses to illusory triangles in human. NeuroReport, 8, 1103–1107. Traub R.D., Whittington M.A., Colling S.B., Buzsaki G., and Jefferys J.G.R. (1996) Analysis of gamma rhythms in the rat hippocampus in vitro and in vivo. Journal of Physiology (London), 493, 471–484. Traub R.D., Whittington M.A., Stanford I.M., and Jefferys J.G.R. (1996) A mechanism for generation of long-range synchronous fast oscillations in the cortex. Nature, London, 383, 621–624. Ts’o, D.Y., Gilbert, C.D., and Wiesel, T.N. (1986) Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by crosscorrelation analysis. Journal of Neuroscience, 6, 1160–1170. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London, 373, 515–518. van Vreeswijk, C., Abbott, L.F., and Ermentrout, G.B. (1994) When inhibition not excitation synchronizes neural firing. Journal of Computational Neuroscience, 1, 313– 321. Wennekers, T., Erb, M., Palm, G., and Eckhorn, R, (1994) Dependence of oscillation frequency on stimulus parameters in primary visual cortex of monkeys: simulations and experimental results. European Journal of Neuroscience, Suppl 7, 66.03. Wennekers, T., and Palm, G. (2000) Cell assemblies, associative memory and temporal structure in brain signals. In: R. Miller (ed) Time and the brain. Harwood Academic Publishers, Reading, In Press. Wertheimer, M. (1923) Untersuchungen zur Lehre von der Gestalt: II. Psychologische Forschung, 4, 301–350. Woelbern, T., Frien, A., Eckhorn, R., Bauer, R., and Kehr, H. (1994) Distributions of oscillation frequencies depend on stimulus size and velocity in monkey visual cortex. In: N. Elsner and H. Breer (eds), Proceedings of the 22nd Göttingen Neurobiology Meeting. Thieme Verlag, Stuttgart, New York, p. 518.

7 EEG Alpha and Cognitive Processes Wolfgang Klimesch Department of Physiological Psychology, Institute of Psychology, University of Salzburg, Hellbrunnerstr. 34, A-5020 SALZBURG Austria Tel: 0043 662 8044 5120 or 5100; Fax 0043 662 8044 5126 e-mail: [email protected]

The suppression of the alpha rhythm, which can be observed in response to a variety of different tasks, is the best known EEG phenomenon. Because the alpha rhythm is most evident in relaxed wakefulness, the implication is that alpha reflects mental inactivity rather than active cognitive processing. However, more recent evidence suggests that activity in the alpha frequency range reflects active information processing, and that different frequencies within the alpha frequency band have quite different functions. Upper alpha, which is a 2 Hz band above the individually determined alpha frequency (IAF), reflects the processing of sensory-semantic information, whereas lower-1 alpha, in the range of IAF-4 Hz to IAF-2 Hz, is associated with attentional processes. The lower-2 alpha band, which falls just below IAF (IAF-2Hz to IAF), seems to reflect expectancy. The general conclusion is that different EEG frequencies reflect different types of cognitive processes. KEYWORDS: Alpha; Brain oscillations; Pace maker; Attention; Memory

1. INTRODUCTION Alpha is an oscillatory EEG activity, within a range of about 8–13 Hz, which has been known since the early days of EEG research. It can best be observed over posterior regions of the head during relaxed wakefulness with closed eyes (cf. Niedermeyer, 1993a). As compared to other EEG phenomena it is characterized by the following outstanding features: (i) spectral analysis reveals that alpha is the dominant frequency in the human EEG, and with the exception of extremely low frequencies (below about 2.5 Hz) shows maximal power; (ii) it is well documented that alpha desynchronizes (becomes suppressed) in response to a variety of different tasks; and (iii) Alpha is the only frequency in the human scalp EEG that clearly and undoubtedly reflects an oscillatory process. Because of its outstanding features, many researchers have assumed that alpha may play a key role in understanding other EEG phenomena (Başar, 1997; Başar et al., 1997a,b). Before considering the functional meaning of alpha in more detail, we first

EEG Alpha and cognitive processes

237

review some of its most important properties.

2. ALPHA POWER AND FREQUENCY: BASIC FINDINGS Unlike frequency, power measurements are strongly effected by a variety of morphological factors of the head (such as the thickness of the skull or the volume of cerebrospinal fluid) and also by methodological and technical factors (such as interelectrode distance, or type of montage). If all of these factors could be perfectly controlled, or kept constant between individuals and different studies, alpha power would be an ideal measure to reflect the number of neurones that oscillate synchronously within the alpha frequency range. One way to compensate for the influence of these factors is to use relative power measurements (e.g. percentages, z-scores, differences or ratios between different conditions). Another way is to use large samples of subjects. In addition to these distorting influences, alpha power is well known to change with sensory, motor and cognitive task demands (see below) but also with age. Age-related changes in absolute and relative alpha power are not linear, but occur in several stages (Epstein, 1980; Hudspeth and Pribram, 1990). On the average, however, young children show an age related increase in alpha power up to about 9 years (Gasser et al., 1988; John et al., 1980; Katada et al., 1981; Marthis et al., 1980; Niedermeyer, 1993b) or even 15 years (John et al., 1980). For older subjects (about 50 years and older) a variety of studies have found age-related changes in EEG power (e.g. Obrist, 1954; Christian, 1984; review in: Van Sweden et al., 1993). These studies show a general slowing of the EEG, with a pronounced power increase in the slow frequency ranges of about 7 Hz and below, and a decrease in higher frequencies of about 7 Hz and higher (e.g. Obrist, 1954; Christian, 1984; Markand, 1990). A very similar pattern of results is obtained if cognitively impaired or demented subjects are compared with age-matched controls (Coben et al., 1985; Brenner et al., 1986; John and Prichep, 1993, [p. 995]). It should be noted that a few studies have reported that demented (Brenner et al., 1986) or old subjects (Könönen and Partanen, 1993) even show an increase in alpha power. In the study of Brenner et al. (1986) this effect was restricted to the lower alpha band. More recent studies have shown repeatedly that a selective increase in lower alpha power reflects a state of decreased attention (Klimesch et al., 1992, 1993; Crawford et al., 1995). Könönen and Partanen (1993) used a bipolar montage. Thus their findings reflect effects that are topographically restricted, and may therefore not be related to changes in absolute or relative power as reported in other studies. In summary, the reported findings suggest that alpha power (in the broad range of 8–13 Hz) changes with age in a nonlinear way. There is a strong increase from early childhood to adulthood. During adulthood alpha power remains stable, but beyond the age of 50 or 60 years (Breslau et al., 1989) a period of a decrease in alpha power can be observed, which may well be due to age-related neurological disorders, and not to age per se (Hubbard et al., 1976). In addition to alpha power, alpha reactivity or suppression is also known to change with age (e.g. Duffy et al., 1984). Many studies (including those reviewed below in the

Time and the brain

238

section about the functional meaning of alpha power) have shown that the extent of event-related alpha power changes (or event-related variability in alpha power) is indeed related to cognitive performance (Klimesch et al., 1997a,b). As an example, Sheridan et al. (1988) have reported that Alzheimer demented subjects, with normal alpha that is suppressed during eye opening, have significantly higher WAIS Performance IQ scores than patients with irregular alpha that does not change (or changes only weakly) during eye opening. Fuller (1978) has found that learning disabled children show less taskrelated alpha attenuation than an age-matched control group. Age-related changes in alpha frequency are well documented. During childhood, alpha frequency increases, from about 7 Hz at an age of about 2 years, to about 10 Hz during adulthood (e.g. Somsen et al., 1997; see also summary in Niedermeyer, 1993c). Then, with increasing age, alpha frequency starts to decline, down to as low as 7 Hz at an age of about 90 years. Köpruner et al. (1984) have found a linear relationship (alpha peak frequency=11.95–0.053 x age) within the age range of adult subjects. According to this relationship, a young adult (e.g. 20 years) has an expected peak frequency of 10.89 Hz, whereas a 70 year old subject shows a drop of 2.65 Hz, down to a frequency of 8.24 Hz. Even in a sample of age-matched subjects, interindividual differences are about as large as age-related differences. When interindividal variability is described in terms of a normal distribution (Klimesch, 1996), about a third of subjects are expected to show a difference in alpha frequency of more than 2 Hz.

3. THE FUNCTIONAL MEANING OF ALPHA FREQUENCY Several experiments reported by Surwillo (Surwillo, 1961, 1963a,b, 1964a,b, 1971) indicate that alpha frequency is significantly correlated with the speed of information processing, as measured by reaction times (RT). Subjects with high alpha frequency show fast RTs, whereas slow subjects have low alpha frequency. These findings are in good agreement with the results from a variety of experiments from our laboratory, which have revealed that alpha frequency of good memory performers is about 1 Hz higher than that of age-matched samples of bad performers (Klimesch et al., 1990a; Klimesch et al., 1990b; Klimesch et al., 1993a,b; Klimesch, 1995, 1996, 1997). Because good performers are faster in retrieving information from memory than bad performers (Klimesch, 1994), these data also indicate that alpha frequency is related to the speed of information processing or reaction time. It is important to note that all of these findings are based on the interindividual variability of alpha frequency, which was found to be significantly related to interindividual differences in the speed of information processing. It is well known that alpha frequency varies not only between individuals, but also within individuals. These intraindividual (or task-related frequency changes) may be as large as interindividual differences in alpha frequency (Klimesch et al., 1990b, Klimesch et al., 1993a; Köpruner et al., 1984).

EEG Alpha and cognitive processes

239

3.1. Task-Related Shifts in Alpha Frequency The interesting question is whether task-related or intraindividual differences in alpha frequency are also related to the speed of information processing. We have addressed this question in an experiment (Klimesch et al., 1996a) in which subjects had to perform a standard recognition task. The EEG was recorded, (i) during the study phase, in which the target words (to be remembered) were presented, (ii) during the recognition performance of the randomly presented targets and distractors and (iii) during a resting condition. RTs were measured in the recognition task. The time course of a single trial, and the intervals used for measuring alpha power are depicted in Figure 1. Because short intervals of only 1 sec were selected for detecting even fast intraindividual shifts in alpha frequency, “gravity” instead of peak frequency was measured. Gravity frequency has the disadvantage that it varies as a function of the selected frequency window. However, considering the large interindividual variability in alpha frequency, parts of the alpha power distribution will fall outside the relevant frequency range if a fixed frequency window (e.g. 8–13 Hz) were to be used. To avoid this problem, we have determined the frequency windows individually for each subject (i), by using peak frequency f(i) as an anchor point. As in studies performed earlier (Klimesch et al., 1990b), the measurement of alpha frequency was done in two steps. First, peak frequency f(i) was measured, during a period of 3 sec preceding the presentation of a word in the study phase (in which alpha power was comparatively large). Then, in a second step, mean (gravity) frequency was calculated within a frequency window which goes 4 Hz below and 3 Hz above f(i) for each subject. This method resulted in a frequency window from 6.4–13.4 Hz on the average. In order to avoid confusions with conventional measures we use the term “individual alpha frequency” (IAF). The variability in intraindividual differences was studied by comparing correctly- and incorrectly-identified targets and distractors. Interindividual differences were studied by correlating RT and alpha frequency over the sample of subjects. It is well known that correctly recognized words (targets or distractors) are identified much faster than incorrectly recognized words (e.g. Klimesch, 1994). Thus, the interesting question is whether alpha frequency is higher for correctly as compared to incorrectly identified words. The results have shown that this is not the case and, thus, lead to the rejection of the hypothesis that the speed of information processing is related to intraindividual, taskrelated shifts in alpha frequency. On the other hand, interindividual differences in alpha frequency were indeed related to RT, as the correlation coefficients in Table 1 indicate. In the resting phase, none of the correlations reached significance. Significant correlations were found for correctly identified words, and only at Pz and occipital recording sites over the left side of the scalp. It is interesting to see that the smallest correlations were obtained during the poststimulus interval of the recognition phase, when the degree of alpha power suppression reached a maximum. By comparing the spectra for the resting period with the reference,

Time and the brain

240

Figure 1. Structure of a typical trial for measuring event-related changes in band power. The use of a warning signal (WS) helps a subject to stay relaxed during a reference interval, and to increase his/her alertness before an imperative stimulus is presented. The time course (in sec) is indicated by numbers.

Table 1 Correlation coefficients between RT and IAF for remembered targets

Occipital, left hemisphere Study

Pz

Recognition

Study

Recognition

ref

pre

post

ref

pre

post

ref

pre

post

ref

pre

post

−.42*

−.43*

−.22

−.45*

−.31

−.11

−.39

−.42*

−.24

−.38

−.29

−.14

Note: *=significant beyond the 5%-level, one sided; ref, pre, post=reference, pre- and poststimulus interval. No significant correlations were found at frontal, central and parietal sites and over the right side of the scalp

pre- and poststimulus intervals, we have found that alpha power becomes increasingly suppressed or desynchronized if the task demands increase, a result that is known since Berger (e.g. Berger, 1929). Intraindividual or task-related shifts in alpha frequency may be due to an asymmetric desynchronization (suppression) of the alpha rhythm, as demonstrated in the two power spectra in Figure 2. Two different cases can be distingushed: desynchronization may favour either the lower (left spectra in Figure 2) or the upper (right spectra in Figure 2) alpha band. The first case leads to an increase, the second to a decrease in alpha frequency. Which of the two cases dominates depends on the type of task. As an example, if semantic task demands predominate, a selective desynchronization in the upper alpha band can be

EEG Alpha and cognitive processes

241

Figure 2. Different types of desynchronization (dotted line) during a test, in comparison to a reference interval (bold line). Asymmetric desynchronization, a very common finding, indicates that the lower and upper alpha band desynchronize independently from each other. The left spectrum shows a case where selective desynchronization of the lower alpha band leads to an increase in frequency. On the other hand, a selective desynchronization of the upper alpha band leads to a decrease in frequency as is shown in the right spectrum.

observed (see the section about the “functional meaning of the upper alpha band”, below). On the other hand, if attentional demands predominate, a selective desynchronization in the lower alpha band can be observed (see the section about the “functional meaning of the lower alpha band”, below). This selective desynchronization in the lower alpha band leads to an increase in frequency, which is frequently observed when task demands increase (e.g. Osaka, 1984; Klimesch et al., 1993a). When focusing on the relationship between RT and alpha frequency, the question arises, under which condition should alpha frequency be measured. Under specific task demands, alpha frequency may be distorted by a selective desynchronization of the lower or upper alpha band. During resting conditions, in which subjects are relaxed, the lower alpha band may even synchronize, showing an increase in band power, and consequently a decrease in alpha frequency. Thus, depending on the condition under which the EEG is measured, the frequency of the alpha rhythm may be distorted by a selective synchronization or desynchronization in the lower and or upper alpha band. We assume that a medium level of attentional (or task) demands shows a more symmetric desynchronization and, thus, gives a less distorted measure of alpha frequency as compared to a very difficult task (with a strong selective desynchronization of the lower

Time and the brain

242

alpha), or the resting condition in which subjects may already be drowsy (with a strong synchronization in the lower alpha band). In summarizing the results about alpha frequency and reaction time, we see that during a state of maximal desynchronization (which was found during the poststimulus interval of the recognition phase) the relationship between RT and alpha frequency is blurred. In a similar way, during a state of maximal alpha synchronization (which was found during the resting phase) no significant correlations between RT and alpha frequency were found. Only if alpha shows a medium level of desynchronization (which was found during the reference and prestimulus interval), did a significant relationship between alpha frequency and RT emerge (cf. Table 1). Our results are well in line with Surwillo’s findings (Surwillo, 1961, 1963a,b, 1964a,b, 1971) because he used only those epochs for measuring alpha frequency which preceded a subject’s response. Thus, his findings too indicate that alpha frequency is an indicator of processing speed only if it is measured during a prestimulus interval in which alpha oscillations are in a state of medium desynchronization. The important conclusion is that task related frequency shifts are not “real” and, thus, not related to the speed of information processing. Our findings contradict the view that there is a single alpha rhythm. They indicate (at least) that a lower and an upper alpha frequency component must be distinguished. It may even be hypothesized that there is an entire population of different alphas (Başar, 1997; Başar and Schürmann, 1997) which oscillate synchronously within a rather broad frequency range during a resting state. In a state of medium desynchronization reflecting increased alertness, the frequency window in which synchronous oscillations occur becomes smaller because lower and higher alpha frequency components already start to desynchronize. During this state of medium desynchronization, the dominant frequency of the alpha rhythm may reflect the activity of alpha pacemaker(s), possibly in the thalamus (e.g. Steriade et al., 1990). During strong desynchronization, different neural populations start to oscillate with different frequencies within the alpha band (cf. Klimesch, 1995, 1996). As a result, the dominant EEG alpha rhythm starts to disappear and the power spectrum becomes quite flat. 3.2. Brain Volume and Cognitive Maturity The findings just reported suggest that alpha frequency is an indicator of cognitive performance. This assumption is supported not only by the well documented relationship, with RT, but also by the fact that alpha frequency increases (although in a nonlinear way) from early childhood to adulthood, and then decreases with age over the remaining life span in a way which parellels that of brain volume and general cognitive performance (e.g. Bigler et al., 1995; Willerman et al., 1991). However, cognitive performance is not the only factor that affects alpha frequency, as the findings of Nunez et al. (1978) indicate. These authors have found that alpha frequency is negatively correlated with brain size. Nunez (1995) assumes that many different alpha waves travel from different places over the cortex. They may be generated in the cortex, or may be induced at least in part by thalamo-cortical feedback loops. The basic idea is that after substantial interference, standing waves are produced. An

EEG Alpha and cognitive processes

243

important prediction from his theory is that larger brains will, on the average, produce lower alpha frequencies. According to his theory (Nunez, 1995: p. 560ff.), the fundamental mode frequency, which may be interpreted as alpha frequency, can by estimated by:

where Ve is the velocity of action potentials (in cortico-cortical axons) and R is the brain radius of an idealized sphere (cf. Nunez, 1995: p. 82ff). Figure 3 illustrates this negative correlation between alpha frequency and brain size for different values of Ve. Nunez et al. (1978) report a correlation coefficient with head size (defined as the cube root of the product of three measures, ear to ear, nasion to inion, and circumference) of r =−.206 (p=.02, n=123).

4. THE FUNCTIONAL MEANING OF ALPHA POWER One of the most typical (and at the same time surprising) features of the alpha rhythm is its tendency to be reduced in amplitude (power) or even blocked when task demands increase. In some sense this behaviour of the alpha rhythm is the opposite of what one would expect. It would be plausible to assume a general increase in electrical brain activity with active mental processes. However, as far as the alpha frequency range is concerned an inverse relationship between band power and mental activity can be observed. This certainly holds true for more sophisticated measures of event-related alpha power changes (as we will see in the next section below). Another interesting finding is that alpha frequency is inversly related to alpha power. High alpha frequency is related to low alpha power, whereas low alpha frequency is related to high alpha power (Wieneke et al., 1980; Pigeau and Frame, 1992; see also review in Markand, 1990). High alpha frequency is related to good performance and is correlated with low alpha power if attentional demands are high (cf. Fig. 2). Thus, low alpha power should also be related to good performance. Studies measuring hemispheric differences in alpha power have demonstrated that alpha suppression is more pronounced over the task relevant hemisphere (Furst, 1976; Galin et al., 1982; Klimesch et al., 1990c, 1997b; but see Papanicolaou et al., 1986) and that over the right hemisphere alpha power is generally

Time and the brain

244

Figure 3. Hypothetical regression lines for head size and alpha frequency as predicted by Nunez (1995). Global (in contrast to local) alpha frequency is interpreted in terms of standing waves, and is estimated by f(alpha)=Ve/27πR, where Ve is the velocity of action potentials (in cortico-cortical axons) and R is the brain radius (in cm) of an idealized sphere (cf. Nunez, 1995, p. 82ff). A radius of R=14 cm represents a brain with a cortical surface of 2463 cm2- which is about the mean value for young adults. Because alpha frequency is about 10 Hz in young adults, the corresponding velocity Ve of action potentials is about 8.5 m/sec. Differences in the thickness of the myelin sheath play an important role in determining Ve, and thus have a strong effect on alpha frequency. A difference in Ve of 1 m/sec is related to a difference in alpha frequency of more than 1 Hz on the average. The lower alpha frequency in young children may be due to differences in the myelin sheath.

larger than over the left hemisphere (Markand, 1990; Niedermeyer, 1993a). Because even for left handed subjects, the left hemisphere has a probability of about p=.70 of being dominant (Kolb and Whishaw, 1985), these findings indicate that the dominant hemisphere is more active, and thus shows less alpha power. One important exception to this inverse relationship between mental activity (or cognitive performance) and alpha power refers to the finding that EEG power in the frequency range above 6 Hz decreases beyond an age of about 50 years, and that cognitively impaired or demented subjects also show a decrease in alpha band power. One likely interpretation is that not age per se but age-related neurological disorders (which may lead to a pronounced decrease in brain volume and loss of neurones) are the cause of this decline in alpha power see Klimesch, 1999 for a review. The validity of this

EEG Alpha and cognitive processes

245

assumption is supported by studies which report that healthy elderly subjects do not show a general slowing of the EEG (Hartikainen et al., 1992) or of the alpha rhythm (e.g. Hubbard et al., 1976; Torres et al., 1983) or changes in EEG power. 4.1. The Measurement of Event-Related Changes in Alpha Band Power Besides absolute alpha power, the measurement of alpha reactivity or alpha suppression is a promising method for studying the dynamics of event-related band power changes in selected frequency bands. A better understanding of these dynamics was provided by a new method, introduced by Pfurtscheller and Aranibar (1977), who coined the term ERD (event-related desynchronization). This method allows one to calculate the percentage of a event-related power change in different frequency bands. In more detail, the procedure is as follows. The EEG data for each epoch and each recording site are digitally bandpass filtered, squared (in order to obtain simple power estimates) and averaged separately for each lead, experimental condition and subject. The percentage of an event-related change in band power during a test interval is calculated by applying the following formula: ERD%={[(alpha band power, reference interval)—(alpha band power, test interval)]/(alpha band power, reference interval)}* 100. As reference, usually an interval of 1000 ms at the beginning of each trial is used. Test intervals usually are selected intervals (of 125 ms or more) preceding and following the presentation of a stimulus (cf. Figure 1). It should be noted that positive ERD-values indicate a state of desynchronization or power suppression, whereas negative ERD-values reflect a state of synchronization or increase in power. One critical question when using ERD is the selection of frequency bands. Due to large interindividual differences of alpha frequency, large portions of alpha power will fall outside a fixed frequency window and invite misleading interpretations. As an example, for a subject with a low mean alpha frequency of 8 Hz, the lower alpha band—defined as the frequency range below mean alpha frequency—already falls outside the frequency window of the fixed band (8–13 Hz; lower alpha: 8–10.5 Hz; upper alpha: 10.5–13 Hz). Thus, in this example the frequency band of 8–13 Hz would only cover the upper alpha and some portions of lower beta. As a consequence, changes in the (true) upper alpha band would be confused with those of the (fixed) lower alpha band. To avoid the problems of fixed-frequency windows, we (e.g. Klimesch et al., 1990c; Klimesch, 1997a,b,c) suggested the use of individually-adjusted frequency bands. Mean peak frequency f(i) of the dominant EEG frequency in the alpha band is used as an anchor point for the selection of frequency bands. Then, frequency bands with a width of e.g. 2 Hz can be defined: (f(i)–6) to (f(i)–4); to (f(i)−4) to (f(i)−2); (f(i)−2) to f(i); f(i) to (f(i)+2). For each subject, the ERD is calculated within these individually-determined frequency bands, which are termed theta, lower (1) alpha, lower (2) alpha, and upper alpha. Averaged over subjects, the cut-off points between the frequency bands for young adults yield the following typical values (e.g. Klimesch et al., 1996b): 4.7–6.7; 6.7–8.7; 8.7–10.7; 10.7–12.7. Adjusting frequency bands individually is important not only because alpha frequency varies as a function of several factors such as age, memory performance and attentional

Time and the brain

246

demands. What is even more important is that the frequency of the transition region between a task-related decrease in band power (as an indicator for the alpha band) and a task-related increase in band power (as an indicator for the theta band) also shows a tendency to vary as a function of alpha frequency (Klimesch et al., 1996b).

5. THE FUNCTIONAL MEANING OF THE LOWER ALPHA BAND It is important to note that alpha desynchronization is not a unitary phenomenon. If different frequency bands are distinguished within the range of the extended alpha band, two distinct patterns of desynchronization can be observed. Desynchronization of lower alpha (in the range of about 6–10 Hz) is obtained in response to a variety of (non-task specific) factors which may be best subsumed under the term “attention”. It is topographically widespread over the entire scalp, and probably reflects general task demands and attentional processes. Desynchronization of upper alpha (in the range of about 10–12 Hz) is topographically restricted and develops during the processing of sensory-semantic information, as recent evidence indicates (e.g. Klimesch et al., 1994; Klimesch et al., 1996c, 1997a,b). Since the early days of EEG research it has been well known (and has been shown repeatedly) that the alpha rhythm is sensitive to attentional demands (cf. Ray and Cole, 1985; Mulholland, 1969). Studies from our laboratory, however, indicate that it is particularly the lower alpha band which is responsible for this observation. Cognitive theories of attention distinguish between two basic aspects, divided attention and selective attention (for a review see e.g. Cowan, 1988; Kahneman and Treisman, 1984). These aspects refer to a subject’s processing capacity and the extent to which different tasks draw on the limited attentional resources. Besides these more cognitive aspects, Posner (e.g. Posner, 1975, 1995) considers alertness (a concept similar to arousal) another important component of attention. Alertness is further subdivided into phasic and tonic alertness. Phasic changes in alertness occur at a rapid rate, and are under volitional control, whereas tonic changes occur at a much slower rate and are not under direct volitional control. A good example of a phasic change is the increase in alertness after the presentation of a warning signal. The effects of phasic alertness and expectancy (as a special form of selective attention) were studied in two experiments reported by Klimesch et al. (1992). The basic design consisted of a warned stimulus presentation. In Experiment 1, subjects had to perform a reading task. Alertness was manipulated by randomly varying the interval between the appearance of the warning signal and the imperative stimulus. In Experiment 2, subjects had to perform two different category judgement tasks. In the semantic judgement task, they had to indicate to which category (animals versus tools) each of a series of words belonged. In the numerical judgement task, subjects had to judge whether the presented numbers were odd or even. Expectancy and alertness were varied in an orthogonal design. Whereas alertness was varied in the same manner as in Experiment 1, expectancy was manipulated by presenting the words and numbers either in blocks or in random order. Only under the blocked presentation did subjects know which type of task they had

EEG Alpha and cognitive processes

247

to perform on the next trial. The most important results of both experiments is that the effects of the warning signal showed up in the lower alpha band. Only in the lower alpha band was a pronounced desynchronization found in response to the appearance of the warning signal. This finding indicates that the lower alpha band reflects changes in phasic alertness. In contrast to the lower alpha band, the upper alpha band responds to changes in expectancy, and shows the most pronounced task-related power suppression from the preto the poststimulus interval. This indicates that the upper alpha band responds to the presentation of the imperative stimulus and to task-related cognitive processes. This interpretation agrees with the fact that changes in the upper alpha band are strictly localized, occurring at occipital, temporal and parietal recording sites. Because of the visual-verbal nature of the tasks, these brain areas would have been expected to exhibit the most significant increase of ERD. In more general terms, our results suggest that the lower alpha band reflects primarily non-task-specific processes (such alertness) whereas the upper alpha band reflects primarily task specific cognitive processes (such as perceptual processes and task related selective attention or expectancy). It is very important to emphasize that two different types of attention components (alertness and expectancy) are reflected in two different frequency bands (the lower and upper alpha band, respectively). However, in contrast to alertness, which is independent of the task type, expectancy represents a more cognitive aspect of attention, which is highly dependent on the type of cognitive operations a subject selects to prepare for a task. In everyday life the meaning of attention is much closer to the term “alertness” as compared to “expectancy”. For this reason we will prefer and use the term attention instead of alertness. The hypothesis that the lower alpha band reflects attention was supported by a variety of other studies. As an example, Klimesch et al. (1993a) have found that good memory performers (as compared to bad performers) show a significantly stronger desynchronization during encoding and retrieval in the lower alpha (for similar results see also Sterman et al., 1996). We have interpreted these findings by assuming that good memory performance may be due to a phasic increase in alertness, which is reflected by a strong desynchronization in the lower alpha band. Thus, a lack of desynchronization in the lower alpha band may be related to a lack of alertness or attention in general. In good agreement with this interpretation, Crawford et al. (1995) report that subjects with poor sustained attention, who have difficulty inhibiting distracting environmental stimuli, have significantly more lower alpha power than subjects not reporting attentional deficits. An even better analysis of attentional processes is achieved if an extended lower alpha band (divided in a lower-1, and lower-2 alpha with 2 Hz bandwidths) is used, as the findings of a more recent study (Klimesch et al., 1997d) indicate. (For details of method see the section about ERD measurement above.) The results of a visual oddball task, with a warned stimulus presentation (Klimesch et al., 1997d) demonstrate that the to-beattended stimulus (also termed the “target”) shows a particularly strong desynchronization in the lower-1 alpha band. What is even more important is the finding that a significant response (decrease in band power) to the warning signal could be observed only in the lower-1 alpha band, and when the warning signal preceded a target.

Time and the brain

248

Because targets are rare, and because no more than three targets or nontargets were allowed to occur in a row, it is likely that after a few trials subjects were well able to predict (expect) the occurrence of a target. Thus, as the upper two diagrams in Figure 4 demonstrate, the warning signal shows an alerting effect only when a target is expected. An interesting finding refers to the way the three alpha bands respond to the presentation of the warning signal. Only in the lower-1 alpha band, can a phasic response to the warning signal be observed (i.e. a decrease in band power that is interrupted and, thus, separable from the decrease in band power in response to the imperative stimulus). A tonic response (i.e. a steady decrease in band power, starting even before the onset of the imperative stimulus and lasting for the entire poststimulus period) was obtained in the lower-2 and upper alpha band. This tonic response may well be interpreted in terms of stimulus-related expectancy. According to this interpretation we can assume that the alerting effect of the warning signal should be particularly strong in those trials where a target is presented. In contrast to the lower-1 alpha band, the tonic decrease in power in the lower-2 and upper alpha band, starting as early as 1250 ms before the presentation of a

Figure 4. Event-related changes in induced band power (IBP; cf. Klimesch et al., 1998) for targets and nontargets at selected recording sites (Cz, Pz) and different frequency bands. IBP represents band power changes that are devoid of phase-locked EEG activity and are, thus, not influenced by event-related potentials (ERPs). In order to eliminate intersubject variations, band power was z-transformed. Negative z-values exceeding the dotted line (99% confidence interval with respect to the reference interval) represent a significant decrease in band power (desynchronization). It is important to note that in the upper alpha band a significant desynchronization can be observed

EEG Alpha and cognitive processes

249

primarily for targets, whereas the lower-1 and lower-2 alpha band also exhibit distinct responses to the warning signal.

target or nontarget, may reflect the way a subject prepares to respond to a particular type of stimulus. When expecting a nontarget, subjects prepare to ignore it, but when expecting a target subjects prepare to respond. Because expectancy plays a role during the prestimulus period for both types of stimulus, there is no reason to assume that the prestimulus period will differ between target and nontarget trials. This effect of expectancy seems to be reflected in the lower-2 alpha band (for the entire time course of the trial) and in the upper alpha band (but only up to the point when the imperative stimulus actually is processed). It is important to emphasize that the processing of targets and nontargets will be strikingly different. Because we have assumed that the upper alpha band reflects task specific effects, we expect that, particularly during the second half of the poststimulus interval (i.e. during the time the stimulus type is recognized), targets should show a significantly stronger decrease in upper alpha band power. This indeed is the case as inspection of the lower two diagrams in Figure 4 indicate. The interesting finding of this latter study is that the distinction of three bands within the extended alpha band proved useful for making a better distinction of different cognitive processes. The alerting effects of the warning signal is restricted to the lower-1 alpha band, whereas the processing of the task relevant stimulus shows up in the upper alpha band only. The effects of expectancy dominate in the lower-2 alpha band band, and can also be observed in the upper alpha band. Finally it should be mentioned that the reported findings are not due to (or distorted by) the influence of evoked potentials. By applying a method which is based on the “intertrial variance technique” (Kaufman et al., 1989; Kalcher and Pfurtscheller, 1995) we (Klimesch et al., 1997d) were ably to demonstrate that the above reported results are independent of phase-locked EEG activity and represent changes in “induced band power” or IBP. We assume that IBP reflects induced oscillations that are modulated by stimuli or events, and which (in contrast to evoked rhythms) do not respond in a phaselocked manner (for a similar but much broader definition see: Bullock, 1992).

6. THE FUNCTIONAL MEANING OF THE UPPER ALPHA BAND In the previous section we have suggested that the lower alpha band reflects attentional demands, whereas the upper alpha band is primarily associated with specific task demands such as the visual processing of the imperative stimulus. More recent experiments (e.g. Klimesch et al., 1994, 1997a,b,c) have shown that the upper alpha band may reflect the processing of sensory-semantic information. In Klimesch et al. (1994) subjects performed a semantic congruency task, in which they had to judge whether or not the sequentially presented words of concept-feature pairs (such as “eagle-claws” or “pea-huge”) are semantically congruent. Then, without prior warning, they were asked to perform an episodic recognition task. This was done to prevent subjects from using semantic encoding strategies, and thus to increase episodic memory demands. In the

Time and the brain

250

episodic task, the same concept-feature pairs were presented together with new distractors. Subjects had now to judge whether or not a particular concept-feature pair was one already presented during the semantic task. From reaction time experiments it is well known that episodic tasks of the kind used here are much more difficult than semantic tasks (see the review in Klimesch, 1994). This fact is important because we assume that upper alpha desynchronization selectively reflects semantic task demands regardless of general task demands, such as task difficulty. The results have shown that in spite of the fact that the semantic task is easier, the upper alpha band shows a significantly stronger desynchronization during the processing of the semantic task, as compared to the episodic task. In a similar study by Klimesch et al. (1997b) three experiments were carried out in which semantic processing demands were varied. The same sample of subjects was used in all of the three experiments. In Experiment 1 subjects had to judge whether pairs of sequentially presented words were semantically congruent (in a manner similar to that in the semantic task of Klimesch et al., 1994). In the following experiments subjects were presented the initial words of word-pairs from Experiment 1 and were asked to perform a free association task (in Experiment 2), and a cued recall task (in Experiment 3). The results of this study reveal that a significant increase in upper alpha desynchronization was found only during that time interval in which the semantic task actually was carried out (cf. Figure 5). Whereas the theta band does not respond to semantic task demands at all, the two lower alpha bands exhibit a stepwise increase in desynchronization that exceeds the level of significance even before a semantic relationship between the initial and second words of the word-pairs can be detected. Because the semantic task cannot be carried out until the second word of a pair is presented (during t4, in Figure 5), it becomes evident that only the upper alpha band responds selectively to semantic task demands, as the data in Figure 5 demonstrate. Furthermore, upper alpha desynchronization was significantly larger during t4 and t5 of Experiment 1, as compared to t2 and t3 of Experiment 2 and 3. Because semantic memory demands were highest in the congruency task, this finding provides further evidence for the hypothesis that the upper alpha band is associated with semantic memory processes. Upper alpha desynchronization during the semantic task (during t5) is strictly localized over the left hemisphere as Figure 6 shows. This finding is well in line with a variety of PET-studies and with Tulving’s HERA model (Tulving et al., 1994). The data also demonstrate that the upper alpha band responds not only to semantic but also to visual task demands. Topographical differences reveal that desynchronization is largest at occipital sites during t2 and t4 when the first and second words are presented. It seems plausible to assume that this highly selective increase in desynchronization reflects visual encoding processes. Comparing different recording sites shows an interesting dissociation between different brain areas. Whereas frontal, central, parietal and temporal regions do not respond to the presentation of a word, but only to semantic processing demands, occipital regions respond to both perceptual and semantic processing demands. Perceptual processes serve to extract the meaning from sensory information. It, thus, appears likely that upper alpha desynchronization responds not only to verbal-semantic but to any kind of sensory-semantic encoding processing. The two lower alpha bands show a flat topographical distribution of desynchronization,

EEG Alpha and cognitive processes

251

a generally higher level of desynchronization as compared to the upper alpha band, and a clear relationship to task difficulty. Thus, these findings support the view that the two lower alpha bands reflect rather unspecific cognitive processes such as general task demands and attention. We assume that the gradual increase in lower alpha desynchronization may reflect the gradual increase in attentional or general task demands from the beginning to the end of a trial.

Time and the brain

252

Figure 5. Event-related band power changes as measured by ERD in three different experiments (data from Klimesch et al., 1997b). Intervals t1-t5 represent consecutive intervals of 500 ms each. In Experiment 1, a feature word was presented during t2 whereas a concept word was presented during t4. In Experiment 2 and 3 only feature words were presented (during t2). In all of the three experiments, t1 represents the prestimulus period. In Experiment 1 subjects had to judge whether pairs of feature-concept words (e.g. “claws—eagle”, “blue—canary”) were semantically congruent. In Experiment 2, subjects were asked to respond with a free association to a feature word (presented during t2). In Experiment 3, the presentation of a feature word (during t2) served as cue to recall the respective concept word, presented in Experiment 1. Note that semantic memory demands are highest in Experiment 1, but only during t4 and t5 when the congruency judgment is carried out. Only the upper alpha band responds selectively to this type of task demand. Significant differences with respect to t1 are marked by asterisks.

EEG Alpha and cognitive processes

253

Figure 6. Hemispheric differences in upper alpha desynchronization (ERD) in the semantic task (Experiment 1 in Figure 5). A strong left hemispheric advantage (larger ERD-values over the left as compared to the right hemisphere) was found during semantic processing, particularly during t5.

Time and the brain

254

7. CONCLUDING REMARKS With respect to the functional meaning of the alpha rhythm, the easiest question to answer refers to the relationship between alpha frequency and the speed of information processing. Age-related differences, the slowing of the alpha rhythm as a function of neurological disorders, and studies documenting a significant relationship between alpha frequency and reaction time suggest that the frequency of the alpha rhythm is—in some way—related to the speed of information processing. According to Nunez’s “standing wave” theory (Nunez, 1995) we may assume that alpha waves, generated at many different locations, travel over the cortex, and, after interference, form a standing wave. The important prediction from this theory (supported by experimental results) is that brain size and alpha frequency are negatively correlated, which means that large brains tend to have a slower alpha frequency than small brains. This is paradoxical because several studies have shown that brain size is positively correlated with intelligence. More intelligent people process information faster than less intelligent people, and consequently also should have a higher frequency of the alpha rhythm. A possible solution of this paradox may come from reconsidering the distinction between a global alpha rhythm and local, or functional alphas (Başar, 1997). There is strong evidence for the view that there is no single alpha rhythm, but instead a population of different alpha rhythms (e.g. Lopes Da Silva et al., 1973, 1980; see Başar and Bullock, 1992, for a more recent review). A strong alpha that can be recorded with scalp electrodes documents the simple but important fact that a very large population of neurones oscillates synchronously with the same phase and with the same frequency, or at least within a narrow frequency window. During this type of “large scale” synchronous oscillation, which may also be termed “global” alpha (or “type 1” alpha oscillation reflecting a state of maximal synchronization: Klimesch, 1996), the brain is not actively processing information. During active information processing, alpha desynchronizes and the assumption is that different alpha rhythms (local alphas or functional alphas) start to oscillate with different frequencies. As a result, the dominant EEG alpha rhythm desynchronizes. Now, it is important to see that although the global alpha rhythm desynchronizes, each alpha subpopulation may still oscillate synchronously. This latter type of microscale synchronous oscillation is termed “type 2 oscillation”. Thus, the frequency of global alpha, as measured during a resting state, will not reflect properties of the working brain and should not be correlated with the speed of information processing. Global alpha is most pronounced during relaxed wakefulness, and studies trying to measure peak alpha frequency usually use a resting period in which the EEG is recorded. The measurement of alpha frequency by Nunez et al. (1978) was also obtained in this way. Thus, the relationship between head size and alpha frequency refers to global alpha. With respect to a resting situation it appears plausible to assume that the alpha rhythm needs more time to spread through a large (as compared to a small) brain. During actual information processing the global alpha (or type 1 large-scale alpha oscillation) disappears at least in part, and type 2 micro-scale alpha oscillations appear. In contrast to global alpha (reflecting maximal synchronization during a resting state), the frequency of

EEG Alpha and cognitive processes

255

local or functional alphas (type 2 micro-scale alpha oscillations) should be related to the speed of information processing. Thus, in a situation of medium desynchronization (possibly reflecting the interaction between global and local alphas) alpha frequency (as measured by scalp electrodes) should already be correlated with the speed of information processing. This is indeed the case as the results of Klimesch et al. (1996) indicate (cf. Table 1). As was expected, during a resting state with maximal alpha synchronization (reflecting global alpha activity) no significant correlation was obtained. However, during maximal desynchronization, in which type 2 micro-scale alpha oscillations predominate, no significant correlations were obtained either. This failure, however, most probably has a different explanation: because of technical and methodological limitations the frequency of type 2 micro-scale alpha oscillations can hardly be measured with scalp electrodes. Thus, only during a state of medium desynchronization, in which global alpha no longer dominates, and type 2 oscillations can still be detected by scalp electrodes, a significant correlation between alpha frequency and the speed of information processing emerges.

REFERENCES Başar, E. (1997) Towards a renaissance of “alphas”. International Journal of Psychophysiology, 26, 1–3. Başar, E., and Bullock, T.H. eds. (1992): Induced Rhythms in the Brain. Birkhäuser, Boston Basel Berlin. Başar, E., Hari, R., Lopes da Silva, F.H., and Schürmann, M. (1997a) Brain alpha acitvity—new aspects and functional correlates. International Journal of Psychophysiology, 26, 1–482. Başar, E., Schürmann, M., Başar-Eroglu, C., and Karakas, S. (1997b) Alpha oscillations in brain functioning: An integrative theory. International Journal of Psychophysiology, 26, 5–29. Başar, E., and Schürmann, M. (1997) Functional correlates of alphas: Panel discussion of the conference “Alpha Processes in the Brain”. International Journal of Psychophysiology, 26, 455–474. Berger, H. (1929) Über das Elektroenkephalogramm des Menschen. Archiv für Psychiatric und Nervenkrankheiten, 87, 527–570. Bigler, E.D., Johnson, S.C., Jackson, C., and Blatter, D.D. (1995) Aging, brain size, and IQ. Intelligence, 21, 109–119. Brenner, R.P., Ulrich, R.F., Spiker, D.G., Scalbassi, R.J., Reynolds, C.F., III, Marin, R.S., and Boller, F. (1986) Computerized EEG spectral analysis in elderly normal, demented and depressed subjects. Electroencephalography and Clinical Neurophysiology, 64, 483–492. Breslau, J., Starr, A., Sicotte, N., Higa, J., and Buchsbaum, M.S. (1989) Topographic EEG changes with normal aging and SDAT. Electroencephalography and Clinical Neurophysiology, 72, 281–289. Bullock, T.H. (1992) Introduction to induced rhythms: A widespread, heterogeneous class of oscillations. In: E. Başar and T.H. Bullock (eds) Induced rhythms in the brain. Boston: Birkhäuser, pp. 1–26. Christian, W. (1984) Das Elektroencephalogramm (EEG) im höheren Lebensalter. Nervenarzt, 55, 517–524.

Time and the brain

256

Coben, L.A., Danziger, W., and Storandt, M. (1985) A longitudinal EEG study of mild senile dementia of Alzheimer type: Changes at 1 year and at 2.5 years. Electroencephalography and Clinical Neurophysiology, 61, 101–112. Cowan, N. (1988) Evolving conceptions of memory storage, selective attention, and their mutual constraints within the human information-processing system. Psychological Bulletin, 104, 163–191. Crawford, H., Knebel, T., Vendemia, J., Kaplan, L., and Ratcliff, B. (1995) EEG activation patterns during tracking and decision-making tasks: Differences between low and high sustained attention. Paper presented at: 8th International Symposium on Aviation Psychology, Columbus, Ohio, 1995, April 24–27. Duffy, F.H., Albert, M.S., McAnulty, G., and Garvey, A.J. (1984) Age-related differences in brain electrical activity of healthy subjects. Annals of Neurology, 16, 430–438. Epstein, H.T. (1980) EEG developmental stages. Developmental Psychology, 13, 629– 631. Fuller, P.W. (1978) Attention and the EEG alpha rhythm in learning disabled children. Journal of Learning Disabilities, 11, 303–312. Furst, G.J. (1976) EEG alpha asymmetry and visuospatial performance. Nature, London, 260, 254–255. Galin, D., Ornstein, R., Herron, J., and Johnstone, J. (1982) Sex and handedness differences in EEG measures of hemispheric specialization. Brain and Language, 16, 19–55. Gasser, T., Verleger, R., Bächer, P., and Sroka, L. (1988) Development of EEG of school-age children and adolescents. I Analysis of band power. Electroencephalography and Clinical Neurophysiology, 69, 91–99. Hartikainen, P., Soininen, H., Partanen, J., and Helkala, E.L. (1992) Aging and spectral analysis of EEG in normal subjects: A link to memory and CSF AChE. Acta Neurologica Scandinavica, 86, 148–155. Hubbard, O., Sunde, D., and Goldensohn, E.S. (1976) The EEG in centenarians. Electroencephalography and Clinical Neurophysiology, 40, 407–417. Hudspeth, W.J., and Pribram, K.H. (1990) Stages of brain and cognitive maturation. Journal of Educational Psychology, 82, 881–884. John, E.R., Ann, H., Prichep, L., Trepetin, M., Brown, D., and Kaye, H. (1980) Developmental equations for the EEG. Science, New York, 210,1255–1258. John, E.R., and Prichep, L. (1993) Principles of neurometric analysis of EEG and evoked potentials. In: E. Niedermeyer and F.H. Lopes da Silva (eds) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn, Baltimore, Hong Kong, London, Munich, Sydney, Tokyo: Williams & Wilkins, pp. 989–1004. Kahneman, D., and Treisman, A. (1984) Changing views of attention and automaticity. In: R.Parasuraman and D.R.Davies (eds) Varieties of attention. London, Academic Press, pp. 29–61. Kalcher, J., and Pfurtscheller, G. (1995) Discrimination between phase-locked and nonphase locked event-related EEG activity. Electroencephalography and Clinical Neurophysiology, 94, 381–384. Katada, A., Ozaki, H., Suzuki, H., and Suhara, K. (1981) Developmental characteristics of normal and mentally retarded children’s EEG. Electroencephalography and Clinical Neurophysiology, 52, 192–201. Kaufman, L., Schwartz, B., Salustri, C., and Williamson, S.J. (1989) Modulation of spontaneous brain activity during mental imagery. Journal of Cognitive Neuroscience,

EEG Alpha and cognitive processes

257

2, 124–132. Klimesch, W. (1994) The structure of long-term memory: A connectivity model for semantic processing. Hillsdale, NJ: Lawrence Erlbaum. Klimesch, W. (1995) Memory processes described as brain oscillations in the EEG-alpha and theta band. Psycoloquy (electronic journal). Klimesch, W. (1996) Memory processes, brain oscillations and EEG sychronization. International Journal of Psychophysiology, 24, 61–100. Klimesch, W. (1997) EEG-alpha rhythms and memory processes. International Journal of Psychophysiology, 26, 319–340. Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Research Reviews. 29, 169–195. Klimesch, W., Schimke, H., and Pfurtscheller, G. (1990a) The topography of alpha frequency and memory performance. Journal of Psychophysiology, 4, 191–192. Klimesch, W., Schimke, H., Ladurner, G., and Pfurtscheller, G. (1990b) Alpha frequency and memory performance. Journal of Psychophysiology, 4, 381–390. Klimesch, W., Pfurtscheller, G., Mohl, W., and Schimke, H. (1990c) Event-related desynchronization, ERD-mapping and hemispheric differences for words and numbers. Journal of Psychophysiology, 8, 297–308. Klimesch, W., Pfurtscheller, G., and Schimke, H. (1992) Pre- and poststimulus processes in category judgement tasks as measured by event-related desynchronization (ERD). Journal of Psychophysiology, 6, 186–203. Klimesch, W., Schimke, H., and Pfurtscheller, G. (1993a) Alpha frequency, cognitive load, and memory performance. Brain Topography, 5, 241–251. Klimesch, W., Pfurtscheller, G., and Schimke, H. (1993b) ERD—attentional and cognitive processes in the upper and lower alpha band. Electroencephalography and Clinical Neurophysiology, 87, 133. Klimesch, W., Schimke, H., and Schwaiger, J. (1994) Episodic and semantic memory: An analysis in the EEG-theta and alpha band. Electroencephalography and Clinical Neurophysiology, 91, 428–441. Klimesch, W., Doppelmayr, M., Schimke, H., and Pachinger, T. (1996a) Alpha frequency, reaction time and the speed of processing information. Journal of Clinical Neurophysiology, 13, 511–518. Klimesch, W., Doppelmayr, M., Russegger, H., and Pachinger, T. (1996b) Theta band power in the human scalp EEG and the encoding of new information. NeuroReport, 7, 1235–1240. Klimesch, W., Schimke, H., Doppelmayr, M., Ripper, B., Schwaiger, J., and Pfurtscheller, G. (1996c) Event-related desynchronization (ERD) and the Dm-effect: Does alpha desychronization during encoding predict later recall performance. International Journal of Psychophysiology, 24, 47–60. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Ripper, B. (1997a) Brain oscillations and human memory: EEG correlates in the upper alpha and theta bands. Neuroscience Letters. 238, 9–12. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Russegger, H. (1997b) Event-related desynchronization in the alpha band and the processing of semantic information. Cognitive Brain Research, 6, 83–94. Klimesch, W., Doppelmayr, M., Schimke, H., and Ripper, B. (1997c) Theta synchronization in a memory task. Psychophysiology, 34, 169–176. Klimesch, W., Russegger, H., Doppelmayr, M., Pachinger, Th. (1998) A method for the calculation of induced band power: Implications for the significance of brain

Time and the brain

258

oscillations. Electroencephalography and clinical Neurophysiology, 108, 123–130. Kolb, B., and Whishaw, I.Q. (1985) Fundamentals of human neuropsychology. New York: Freeman. Könönen, M., and Partanen, J.V. (1993) Blocking of EEG alpha activity during visual performance in healthy adults. A quantitative study. Electroencephalography and Clinical Neurophysiology, 87, 164–166. Köpruner V., Pfurtscheller G., and Auer L. (1984): Quantitative EEG in normals and in patients with cerebral ischemia. In: G.Pfurtscheller, E.Jonkman, F.Lopes da Silva (eds) Brain Ischemia: Quantitative EEG and Imaging Techniques. Amsterdam, Elsevier, pp. 29–50. Lopes da Silva, F.H., Lierop, T.H., Schrijer, C.F., and Storm van Leeuwen, W. (1973) Organization of thalamic and cortical alpha rhythms: Spectra and coherences. Electroencephalography and Clinical Neurophysiology, 35, 627–639. Lopes Da Silva, F.H., Vos, I.E., and Van Rotterdam, A. (1980) Relative contributions of intracortical and thalamo-cortical processes in the generation of alpha rhythms, revealed by partial coherence analysis. Electroencephalography and Clinical Neurophysiology, 50, 449–456. Markand, O.N. (1990) Alpha rhythms. Journal of Clinical Neurophysiology, 7, 163–189. Marthis, P., Scheffner, D.M., Benninger, C., Lipinski, C., and Stolzis, L. (1980) Changes in the background activity of the electroencephalogram according to age. Electroencephalography and Clinical Neurophysiology, 49, 626–635. Mulholland, T.B. (1969) The concept of attention and the electroencephalographic alpha rhythm. In: C.R.Evans, & T.B., Mulholland (eds) Attention in Neurophysiology. London, Butterworths, pp. 100–127. Niedermeyer, E. (1993a) The normal EEG of the waking adult. In: E.Niedermeyer and F.H.Lopes da Silva (eds) Electroencephalography: basic principles, clinical applications, and related fields, 3rd edn. Baltimore: Williams & Wilkins, pp. 131–152. Niedermeyer, E. (1993b) Normal aging and transient cognitive disorders in the elderly. In: E.Niedermeyer and F.H.Lopes da Silva (eds) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn. Baltimore: Williams & Wilkins, pp. 329–338. Niedermeyer, E. (1993c) Maturation of the EEG: Development of waking and sleep patterns. In: E.Niedermeyer and F.H.Lopes da Silva (eds.) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn. Baltimore: Williams & Wilkins, pp. 167–191. Nunez, P.L., Reid, L., and Bickford, R.G. (1978) The relationship of head size to alpha frequency with implications to a brain wave model. Electroencephalography and Clinical Neurophysiology, 44, 344–352. Nunez, P.L. (1995) Toward a physics of neocortex. In: P.L. Nunez (ed.) Neocortical dynamics and human EEG rhythms. New York, Oxford: Oxford University Press, pp. 68–132. Obrist, W.D. (1954) The electroencephalogram of normal aged adults. Electroencephalography and Clinical Neurophysiology, 6, 235–244. Osaka, M. (1984) Peak alpha frequency of EEG during a mental task: Task difficulty and hemispheric differences. Psychophysiology, 21, 101–105. Papanicolaou, A.C., Loring, D.W., Deutsch, G., and Eisenberg, H.M. (1986) Task-related EEG asymmetries: A comparison of alpha blocking and beta enhancement. International Journal of Neuroscience, 30, 81–85. Pfurtscheller, G., and Aranibar, A. (1977) Event-related cortical synchronization detected

EEG Alpha and cognitive processes

259

by power measurements of scalp EEG. Electroencephalography and Clinical Neurophysiology, 42, 817–826. Pigeau, R.A., and Frame, A.M. (1992) Steady-state visual evoked responses in high and low alpha subjects. Electroencephalography and Clinical Neurophysiology, 84, 101– 109. Posner, M.I. (1975) Psychobiology of attention. In: M.S.Gazzaniga and C.Blakemore (eds) Handbook of Psychobiology. New York: Academic Press, pp. 441–480. Posner, M.I. (1995) Attention in cognitive neuroscience: An overview. In: M.S.Gazzaniga (ed.) The cognitive neurosciences. Cambridge, MA: Bradford, MIT Press, pp. 615–624. Ray, W.J., and Cole, H.W. (1985) EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes. Science, New York, 228, 750– 752. Sheridan, P.H., Sato, S., Foster, N., and Bruno, G. (1988) Relation of EEG alpha background to parietal lobe function in Alzheimer’s disease as measured by positron emission tomography and psychometry. Neurology, 38, 747–750. Somsen, R.J.M., Van-Klooster, B.J., Van-der-Molen, M.W., and Van-Leeuwen, H.M. (1997) Growth spurts in brain maturation during middle childhood as indexed by EEG power spectra. Biological Psychology, 44, 187–209. Steriade, M., Jones, E., and Llinás, R.R. (1990) Thalamic oscillations and signaling, New York, Chichester, Brisbane, Toronto, Singapore: John Wiley & Sons. Sterman, M.B., Kaiser, D.A., and Veigel, B. (1996) Spectral analysis of event-related EEG responses during short-term memory performance. Brain Topography, 9, 21–30. Surwillo, W. (1961) Frequency of the alpha rhythm, reaction time and age. Nature, London, 191, 823–824. Surwillo, W. (1963a) The relation of simple response time to brain-wave frequency and the effects of age. Electroencephalography and Clinical Neurophysiology, 15, 105– 114. Surwillo, W. (1963b) The relation of response-time variability to age and the influence of brain wave frequency. Electroencephalography and Clinical Neurophysiology, 15, 1029–1032. Surwillo, W. (1964a) Some observations of the relation of response speed to frequency of photic stimulation under conditions of EEG synchronization. Electroencephalography and Clinical Neurophysiology, 17, 194–198. Surwillo, W. (1964b) The relation of decision time to brain wave frequency and to age. Electroencephalography and Clinical Neurophysiology, 16, 510–514. Surwillo, W. (1971) Human reaction time and period of the EEG in relation to development. Psychophysiology, 8, 468–482. Torres, F., Faoro, A., Löwenson, R., and Johnson, E. (1983) The electroencephalogram of elderly subjects revisited. Electroencephalography and Clinical Neurophysiology, 56, 391–398. Tulving, E., Kapur, S., Craik, F.I.M., Moscovitch, M., and Houle, S. (1994) Hemispheric encoding/retrieval asymmetry in episodic memory: Positron emission tomography findings. Proceedings of the National Academy of Sciences U.S.A., 91, 2016–2020. Van Sweden, B., Wauquier, A., and Niedermeyer, E. (1993) Normal aging and transient cognitive disorders in the elderly. In: E. Niedermeyer and F.H. Lopes da Silva (eds) Electroencephalography: basic principles, clinical applications, and related fields, 3rd edn. Baltimore: Williams & Wilkins, pp. 329–338. Wieneke, G.H., Deinema, C.H.A., and Spoelstra, P. (1980) Normative spectral data on

Time and the brain

260

the alpha rhythm in male adults. Electroencephalography and Clinical Neurophysiology, 49, 636–645. Willerman, L., Schultz, R., Rutledge, J.N., and Bigler, E.D. (1991) In vivo brain size and intelligence. Intelligence, 15, 223–228.

8 Theta Frequency, Synchronization and Episodic Memory Performance Wolfgang Klimesch Department of Physiological Psychology, Institute of Psychology, University of Salzburg, Hellbrunnerstr. 34, A-5020 SALZBURG Austria Tel: 0043 662 8044 5120 or 5100; Fax 0043 662 8044 5126 e-mail: [email protected]

Based on findings from animal research, the hypothesis is examined that the encoding and retrieval of new information is reflected by a task related increase (synchronization) in theta power. The findings from several studies support this hypothesis, and show that only those items were correctly remembered that exhibited a significant increase in theta power during the encoding process. During the actual retrieval processes, a significant increase in band power was found only for correctly remembered words. In contrast to the theta band, remembered and not-remembered items revealed a different and more complex pattern of results in the lower and upper alpha band. These and similar findings support the view that synchronous neural activity within the theta frequency range reflects the encoding and retrieval of new (episodic) information, and is induced in the cortex via hippocampo-cortical feedback loops. KEYWORDS: Theta; Brain oscillations; Pace maker; Encoding; LTP; Episodic memory

1. INTRODUCTION In contrast to the alpha rhythm, most of what is known about the theta rhythm stems from animal research. Therefore, the most important properties of the hippocampal EEG will be reviewed first. Most interestingly, when studying results of experiments linking animal behaviour with the hippocampal EEG, it becomes evident that hippocampal theta power increases with increasing task demands just as does theta band power in the human scalp EEG. With respect to the functional meaning of the theta rhythm, evidence from divergent fields such as electrophysiology, anatomy, and neuropsychology point towards a specific relationship between hippocampal theta activity and the encoding of new information. Animal research shows that theta, as the dominant rhythm in the hippocampal formation, is closely related to the strength of long-term potentiation (LTP), which is generally considered a synaptic memory mechanism for the encoding of new

Theta frequency, synchronization and episodic memory performance

263

information. Scalp recordings from human subjects show that the encoding of new information, and the retrieval from short-term memory (STM), is accompanied by a significant increase in theta power. The reviewed evidence suggests that theta oscillations may provide the basis for the encoding of new information. It may even be suggested that—in a functional sense—memory traces are actually stored in the form of nested oscillations within the theta frequency range (Lisman and Idiart, 1995; Klimesch, 1996).

2. FINDINGS FROM ANIMAL RESEARCH In animal research, the frequency range of the theta rhythm recorded from the hippocampus of lower mammals is about 3 to 12 Hz (e.g. Lopes da Silva, 1992) and, thus, shows a much wider frequency range than in humans, where the theta frequency range lies between about 4 and 7 or 8 Hz. Its wide frequency range and large power make observation of its frequency and power changes easy in animals. This is in sharp contrast to the human scalp EEG, where—without the help of sophisticated methods—changes in theta frequency are very difficult or almost impossible to detect. The regular oscillatory pattern of the hippocampal EEG which also is termed Rhythmic Slow Activity (RSA) can be observed if animals make voluntary movements (e.g. Vanderwolf and Robinson, 1981; Vanderwolf, 1992) during exploratory behaviour (Buzsaki et al., 1994). In the absence of exploratory behaviour (e.g. in slow wave sleep, SWS, or during alert immobility) the hippocampal EEG shows Irregular Slow Activity (ISA) which also is called Large Irregular Activity (LIA) or Sharp Waves (SPW; Buzsaki et al., 1994). In simplifying terminology we will use the term SPW (instead of ISA and LIA) in what follows. RSA or theta is usually described in terms of local field potentials that are recorded from microelectrodes implanted in field CA1 or in the dentate layers of the hippocampus. It is induced from the septum which serves as a primary pacemaker (Petsche et al., 1962; cf. the reviews in Buzsaki et al., 1983, and Miller, 1991). The extracellular currents that underly theta activity reflect synchronous fluctuations in the membrane potentials of pyramidal and granule cells (e.g. Bland 1986; Buzsaki et al., 1983). Most interestingly, during theta activity, the majority of pyramidal and granule cells are silent, whereas interneurones exhibit a pronounced increase in their firing rate which is phase-locked to the theta rhythm. At the first glance this appears paradoxical, because theta activity is associated with exploratory behaviour and, thus, most likely with increased mental activity. However, when comparing the local field potentials with multi unit recordings, it can be observed that during high frequency (“type-1”: see below) theta the number of unit discharges is much higher than during low frequency (“type-2”: see below) theta, and that the number of discharges increases linearly with increasing theta frequency in both types of theta (Bland, 1986; Sinclair et al., 1982). Several authors have reported that unit discharges are phase-related, and occur during the negative peaks of theta waves (e.g. Buzsaki et al., 1983; Fox et al., 1986; Buzsaki et al., 1992). Particularly interesting is the finding from Sinclair et al. (1982) that discharges occur briefly after (i.e. about 30 degrees after) the theta wave has reached its negative

Time and the brain

264

peak (at 180 degrees). This result suggests that the negative peak of the theta wave may take an active part in triggering action potentials. 2.1. Theta Frequency, Phase and Behaviour Voluntary movements (e.g. walking, jumping, etc.) which can be observed during exploratory behaviour, are characterized by a highly regular theta oscillation which is termed type-1 theta. In rodents, type-1 theta ranges from about 6.5 to 12 Hz. A somewhat slower and more irregular theta frequency can be observed during a state of immobile alertness, if sensory stimuli are presented while animals are immobile but alert and, thus, in a state of arousal (Montoya et al., 1989). This type-2 theta frequency varies within a range of about 4–9 Hz. A third type of behaviour which is characterized by intermittent SPWs (of 40 to 120 ms duration) can be observed during sleep and “automatic” motor patterns. If the frequency of SPWs is measured, the following interesting relationship with behaviour emerges: with increasing frequency (ranging from about 0.02 to 3 Hz) SPWs occur during slow wave sleep (SWS), awake immobility, drinking, eating, face washing and grooming (Buzsaki et al., 1983). If we assume that attentional demands increase from SWS, awake immobility, automatic motor patterns, high alertness during immobility and finally to voluntary movements during exploratory behaviour, a perfect association between the frequency of the hippocampal EEG (within a range of 0.02–12 Hz) and attentional (or task demands) can be detected. With respect to the theta frequency range, it is important to note that with increasing task demands the hippocampal EEG becomes more regular, increases its frequency and power. Thus, as a result of increasing task demands, EEG-theta becomes synchronized within the small range of peak theta frequency (c.f. Leung et al., 1982; Buzsaki et al., 1983). Not only theta power and frequency, but also theta phase, is related to behaviour. Within the pyramidal cell layer of the rat hippocampus, cells have been found that, unlike theta cells, respond with a more irregular pattern of spikes. O’Keefe and Dostrovsky (1971) observed that this type of cell responds to the animal’s location in the environment. Phase-locked to the concurrent EEG-theta, these “place cells” fire several bursts of spikes as the rat runs through a particular location in its environment, which is called the place field of that cell. The most important result is that the firing of place cells began at a particular phase of theta frequency as the rat entered the place field. Within a particular place field the angle of theta phase was related to a particular spatial location (O’Keefe and Recce; 1993; see also review in O’Keefe, 1993). Thus, when considering the place field as a specific stimulus, theta frequency is phase-locked to the appearance of this stimulus. 2.2. The Functional Meaning of Hippocampal Theta The first evidence for the view that the hippocampal theta rhythm reflects memory processes, or the encoding of new information in particular, came from different research

Theta frequency, synchronization and episodic memory performance

265

fields. An important discovery was reported by Scoville and Milner (1957), who observed a severe anterograde amnesia for patient H.M., who had undergone a bilateral temporal lobectomy including the hippocampal formation. (For a review of related work see e.g. Squire, 1992). A few years earlier, Green and Arduini (1954) had found a dominant rhythmic electrical activity within the theta band in the hippocampus of rats (see the excellent review in Miller, 1991). Thus, it was plausible to assume that the hippocampal theta rhythm might be related to the encoding and/or retrieval of new information. Additional evidence came from more recent studies which have documented that long-term potentiation (LTP) occur preferentially in the hippocampal formation, and that theta activity induces, or at least enhances LTP (e.g. Larson et al., 1986; Greenstein et al., 1988; Maren et al., 1994). LTP is considered the most important electrophysiological memory mechanism for the encoding of new information. If it can be demonstrated that LTP is related in a specific way to some properties of the theta rhythm, an important argument would be at hand for considering theta to be an electrophysiological correlate of working memory. Such a relationship is supported by the finding that the induction of LTP is optimal with stimulation patterns that mimic the theta rhythm (Larson et al., 1986), whereas stimulations which are not repeated and short are usually ineffective. Another important result is that the induction of LTP depends on the phase of theta rhythm. Furthermore, the strength of the induced LTP was found to increase linearly with increasing theta power (Maren et al., 1994 [especially Figure 6 on p. 50]). These findings provide strong support for the view that hippocampal theta is related to the encoding of new information, just as is LTP. Comprehensive theories about hipocampal theta and memory can be found in Miller (1991), Buzsaki et al. (1994) and Klimesch (1996). Miller’s theory of resonant loops is of particular importance, because it suggests a rhythmic interplay between the hippocampus and cortex. The human scalp EEG primarily reflects cortical activity. Biophysically, theta frequency in the hippocampus, deep inside the brain, would be difficult to detect with scalp electrodes. If, however, hippocampal theta is transmitted to the cortex, it should be possible to detect memory related changes in theta activity even in the human scalp EEG.

3. FINDINGS FROM THE HUMAN SCALP EEG In the previous chapter (about the measurement of event-related changes in the alpha frequency range) it was emphasized that the individual determination of frequency bands is essential. The reason was that alpha frequency shows a large interindividual variability, and that, as a consequence, fixed frequency bands may include only parts of the “true” alpha frequency range. Will we face similar problems when attempting to measure bandpower changes in the theta frequency range?

Time and the brain

266

3.1. Theta Frequency The traditional view holds that theta frequency lies within a fixed range of about 4 to 8 Hz just below alpha frequency (of about 8–13 Hz). A fixed frequency band would be justified only if it can be demonstrated that theta does not vary between subjects. Thus, before considering findings from the human scalp EEG in more detail, it is of crucial importance to address the following two questions: (i) is there a physiological criterion that allows one to decide which frequency marks the transition between alpha and theta?; and (ii) does theta frequency vary in a manner similar to alpha frequency? The answer to the first question is surprisingly easy. It is a well established fact that, with increasing task demands, theta synchronizes whereas alpha desynchronizes. If EEG power in a resting condition is compared with a test condition, alpha power will become suppressed whereas theta power will increase. As Figure 1 illustrates, that frequency in the power spectra which marks the transition from theta synchronization to alpha

Theta frequency, synchronization and episodic memory performance

267

Figure 1. Power spectra for reference and test intervals intersect at the transition frequency (TF) between the theta and alpha band. During a reference interval (bold line) subjects are alert but not performing a task. In a test interval (dashed line), in which a subject performs some kind of task, alpha power decreases, but theta power increases. The intersection of the two power spectra, where theta synchronization gives way to alpha desynchronization, may be considered a physiological criterion for the transition region between the alpha and theta band.

Time and the brain

268

desynchronization may be considered the individual transition frequency (TF) between the alpha and theta band for each subject. When adopting this method to estimate TF, the second question can also be answered. Results document that TF shows a large interindividual variability (ranging from about 4 to 7 Hz), which is linked to the interindividual variability of alpha peak frequency. Preliminary evidence for a covariation between theta and alpha frequency was already found by Klimesch et al. (1994). By using the following method, further evidence for such a covariation was obtained in a more recent study (Klimesch et al., 1996). First, power spectra for the reference and test intervals were calculated for each subject and averaged over all trials and all leads. Then, the frequency of the transition region between the theta and alpha band was determined within a fixed frequency window of 3.5–7.5 Hz. For those few subjects who showed an asymmetric alpha desynchronization (with no desynchronization in the lower alpha), and thus failed to show an intersection (in the range of 3.5–7.5 Hz), the transition between the theta and alpha band was considered to be that frequency where the difference between the test and the reference interval reached a minimum. In Klimesch et al. (1996) Spearman’s rank correlation between alpha peak frequency and TF yielded a significant value of p=.64 (one sided: p < .02; two sided: p < .04). Similar results were reported by Doppelmayr et al. (1998a). These findings document the fact that theta frequency varies as a function of alpha frequency, and suggest the use of alpha frequency as a common reference point for adjusting different frequency bands, not only for the alpha, but in the theta range as well. For theta, the individual determination of frequency bands may be even more important, because otherwise the effects of theta synchronization are masked by alpha desynchronization, particularly in the range of TF. Results from our laboratory indicate that TF lies about 4 Hz below the individuallydetermined alpha frequency. As an example, in a sample of 10 subjects (Klimesch et al., 1996) with a mean alpha frequency of 10.7 Hz, mean TF is about 6.7 Hz. If we assume a bandwidth of 4 Hz, theta frequency would lie in a range of 2.7–6.7 Hz, which gives an estimate that appears much too low with respect to the lower frequency boundary of 2.7 Hz: frequencies below 4 Hz are considered to belong to the delta frequency range. To avoid an overlap with delta, we defined the theta frequency range as that band with a width of 2 Hz that falls below TF. All of the findings reported below are based on individually adjusted theta bands of 2 Hz width. These findings indicate that the theta frequency range in humans may be much smaller than originally assumed. 3.2. Event-Related Changes in Theta Power Measurements of event-related changes in theta power are based on the ERD-method which is described in more detail in the previous chapter on the alpha frequency range (Klimesch, this volume). Originally proposed by Pfurtscheller and Aranibar (1977), event-related desynchronization (ERD) is defined as the percentage decrease (positive ERD) or increase (negative ERD) in band power during a test, with respect to a reference interval. A negative ERD is also termed “event-related synchronization” or ERS

Theta frequency, synchronization and episodic memory performance

269

(Pfurtscheller, 1992). All of the experiments reported below were designed to test the hypothesis that a taskrelated increase in theta power reflects selectively the successful encoding and/or retrieval of new information. In reviews focusing on findings in functional neuroanatomy, electrophysiology, amnesia and memory research, Klimesch (1995, 1996) has proposed that the hippocampal theta rhythm reflects primarily the encoding and retrieval of episodic memory. It was suggested that the hippocampal theta rhythm may be induced in the cortex via hippocampo-cortical feedback loops, and thus may even be detected by scalp electrodes in the human EEG. The well known finding that theta power increases in a large variety of different tasks (Schacter, 1977; Arnolds et al., 1980) seems to contradict the suggested hypothesis of a specific relationship between theta and the processing of new information. However, if we consider the fact that the processing of new information is—in some way and to some extent—necessary for the performance of almost any type of task, it becomes obvious that specially designed experiments are needed to test the proposed hypothesis. Of particular importance is the experimental control of unspecific factors (such as attentional demands, task difficulty and cognitive load) which usually will go together—and, thus, will be confounded, with the actual processing of new information. 3.3. Theta Synchronization and Episodic Memory In most every-day situations, the processing of “new” information is synonymous with “episodic” information. According to Tulving (e.g. Tulving, 1984), episodic information is that type of contextual information which keeps an individual autobiographically oriented within space and time. In contrast to a sensory code which primarily reflects physical information, an episodic code which is created through the action of working memory processes (Baddeley, 1992), and reflects primarily subjective information, such as context, expectancy, emotion and autobiographic aspects. Because time changes the context permanently, there is a permanent and vital need to update and store new (or episodic) information. In a first study (Klimesch et al., 1994) the relationship between theta synchronization, alpha desynchronization and episodic memory demands was investigated. The experimental design consisted of two parts. Subjects first performed a semantic congruency task in which they had to judge whether the sequentially-presented words of concept-feature pairs (such as “eagle-claws” or “pea-huge”) are congruent. Then, without prior warning, they were asked to perform an episodic recognition task. This was done in an attempt to prevent subjects from using semantic encoding strategies, and to increase episodic memory demands. In the episodic task, the same word pairs were presented together with new distractors (generated by re-pairing known concept-feature words). In this task, subjects had to know whether or not a particular concept-feature pair had already been presented during the semantic task. Because distractors were semantically similar and generated by re-pairing already-presented words, subjects were able to give a correct response only if new episodic information (represented by a specifc combination of a concept and feature word) was actually stored in memory. The results of a reaction

Time and the brain

270

time study with the same experimental paradigma (Kroll and Klimesch, 1992, Experiment 4) indicated that semantic features speeded up semantic but slowed down episodic decision times (for an extensive review on this topic see also Klimesch, 1994). This finding demonstrates that semantic and episodic memory processes can be differentiated well by using the proposed design. Because pairs of items are presented, the episodic and semantic task can be performed only after the second item of a pair (i.e. the feature) is presented. Thus, the critical issue is to compare the amount of theta synchronization during the presentation of the concept and feature word in the episodic and semantic task. Only correctly identified conceptfeature pairs were analyzed. The results show that only in the episodic task, and only when the feature word is processed can the expected increase in theta power be observed. During the processing of the concept words theta power shows almost identical values in the semantic as in the episodic task. Because the same words were presented in both tasks, and all other variables (such as exposure time, length of interstimulus interval, etc.) were kept constant, the findings support the hypothesis of a specific relationship between memory (for new episodic information) and theta synchronization. The fact that the theta band responds selectively to episodic task demands is also demonstrated by the finding that the lower and the upper alpha band show quite different results. Whereas event-related changes in the lower alpha band were comparatively small, the upper alpha band shows a much larger degree of desynchronization during the semantic as compared with the episodic task. Thus, there is a dissociation between theta synchronization, which is maximal during the processing of new information, and upper alpha desynchronization, which is maximal during the processing of semantic information (see also the previous chapter about the alpha frequency range). In confirming and extending these findings, in a more recent study (Klimesch et al., 1997a) we were able to demonstrate that the extent of upper alpha desynchronization is significantly correlated with semantic memory performance, whereas the extent of theta desynchronization is significantly but negatively correlated with episodic memory performance. Thus, in the theta band, episodic memory performance is correlated with an event-related increase in power whereas the opposite holds true for semantic memory and the upper alpha band. The topography and time course of event-related power changes in the theta band during encoding are depicted in Figure 2. Theta synchronization occurs in two phases. In the first third of the poststimulus interval (of 1000 ms) occipital leads show maximal synchronization, whereas frontal leads show maximal synchronization during the last third. This typical topography and time course of theta synchronization did not vary as a function of semantic or episodic task demands. However, the overall extent of synchronization was stronger during the episodic task. It should be noted that in the lower and upper alpha band the topography and time course of desynchronization is strikingly different. In contrast to theta, alpha desynchronization gradually increases after the presentation of a stimulus and (depending on the type and difficulty of a task) reaches a maximum about 300 ms after the stimulus. In a study with a series of three experiments (Klimesch et al., 1997b) we obtained similar results. In all of the three experiments, a shortlasting synchronization was

Theta frequency, synchronization and episodic memory performance

271

observed, but only at occipital sites, and only during the first 500 ms after the stimulus when a word was presented. This increase in theta power which is particularly strong at occipital areas is due, most probably, to lambda waves, which are known to exert maximal power in the theta band (Billings, 1989a,b). According to Billings (1989b), lambda waves result from retinal afferents and are observed particularly at occipital regions during the processing of visual information. It appears plausible to assume that lambda waves are phase-locked theta activity that represents the encoding of new visual information. 3.4. Theta Synchronization during Successful Encoding and Retrieval The purpose of more recent studies was to investigate the functional meaning of theta synchronization with respect to the formation of a memory code. An interesting prediction of our hypothesis is that, during encoding, those words that will later be remembered (e.g. in a recognition task) should exhibit a significantly stronger theta synchronization than words

Figure 2. Time course and topography of theta synchronization during the presentation of a word in an episodic memory paradigm (data from Klimesch et al., 1994). Occipital recording sites show an early phase of pronounced synchronization (from 125 to 375 ms poststimulus), whereas frontal sites exhibit a late phase of maximal synchronization (from 750 to 1000 ms poststimulus). Intervals t1-t8 represent consecutive time periods of 125 ms each. Note that negative ERD reflects an increase in band power.

that cannot be remembered later. Another prediction is that during the actual recognition process correctly recognized targets will show a significantly stronger increase in theta power than targets which are not recognized. Furthermore, because an episodic memory

Time and the brain

272

trace is available for remembered targets only, distractors are not expected to show a significant increase in theta power. In a study of Klimesch et al. (1997c), a set of 96 words was used as targets. Subsamples of 16 words each belonged to one of 6 categories (birds, fruits, vegetables, vehicles, clothes and weapons). The 96 distractors were selected in a way that, for each target (e.g. robin), a semantically similar distractor (e.g. sparrow) was obtained. Thus, as for the targets, the 96 distractors are subdivided into the same 6 semantic categories, each comprising 16 words. This similarity between distractors and targets guarantees that subjects have to rely on episodic information (and not for example, on semantic familiarity) in order to make a correct decision in the recognition task. If, for instance, a subject has to distinguish the target word “robin” from the distractor “sparrow”, semantic information representing the meaning of the words will not be helpful. For a correct response, the subject has to remember which of the two words was presented in the context of the study list. The results of the study and recognition task show clearly that the theta band responds with a task-related increase in band power (synchronization), whereas the alpha bands respond with a task-related decrease in band power (desynchronization). However, theta synchronization was significant only under those conditions in which a word was either successfully encoded or retrieved. Thus, in the study phase, only those words show a significant synchronization in the theta band which can be remembered in the later performed recognition task (cf. Figure 3). In comparison to Figure 2, inspection of Figure 3 reveals a similar time course and topography of theta synchronization during encoding.

Theta frequency, synchronization and episodic memory performance

273

Figure 3. Successful encoding in an explicit memory paradigm: theta synchronization is plotted during encoding in an experiment where subjects knew that their memory would be tested later (data from Klimesch et al., 1997). Words that could be remembered in a later recognition test (bold lines) show a significant stronger synchronization during encoding than words which could not be remembered later (dashed line). As in Figure 2, frontal sites exhibit a late phase of maximal synchronization (from 750 to 1000 ms poststimulus). Intervals t1-t4 represent consecutive time periods of 250 ms each.

Whereas at frontal leads theta synchronization is largest at the end of the poststimulus interval, the opposite holds true for occipital sites (cf. Figure 2). During the actual recognition process, only the remembered words exhibit a significant increase in theta power, as Figure 4 indicates. Not-remembered words, as well as distractors show no increase in theta power. The findings obtained support the proposed hypothesis that theta synchronization is related selectively to the encoding and retrieval of episodic information. Comparing the topography and time course of theta synchronization during successful encoding (Figure 3) and retrieval (Figure 4) again shows that, during the end of the poststimulus interval, synchronization is maximal at frontal leads, but minimal at occipital sites. Thus, frontal regions play an important role in the explicit formation and retrieval of a memory trace. Because subjects knew that their memory would be tested later, the finding that theta synchronization is significantly stronger during the encoding of those words which can later be rememberd could be due to specific encoding strategies. If this were true, theta synchronization during encoding would reflect a rather nonspecific factor, and not the actual encoding of new information. One way to avoid the possible influence of encoding strategies is to use an incidental memory paradigm (instead of an intentional one). During the encoding phase of an incidental memory paradigm, subjects do not know that

Time and the brain

274

memory performance will be tested later, because they are performing some type of distractor task. Therefore, specific mnemonic techniques and specific attentional factors will not play a significant role during encoding. In the study of Klimesch et al. (1996), we

Figure 4. Successful retrieval in an explicit memory paradigm: theta synchronization is plotted during the actual recognition process, in an experiment where subjects knew that their memory would be tested (data from Klimesch et al., 1997). Words that were correctly remembered (bold lines) show a highly significantly stronger synchronization during actual recognition than words which could not be remembered later (dashed line). Note the large difference in the strength of synchronization between occipital and frontal recording sites and the late maximum of synchronization (during 750 to 1000 ms poststimulus). Intervals t1–t4 represent consecutive time periods of 250 ms each.

used an incidental memory paradigm that consisted of two parts. In the first part subjects were asked to categorize a series of words, and to respond with “yes” if a word belonged to the category “living”, and with “no” if it belonged to the category “nonliving”. At this point of the experiment, subjects did not know that in the second part of the experiment they would have to recall the words which were presented in the judgement task. Band power values during the encoding stage were calculated, and words which could be remembered later were compared with those words which could not be remembered later. The results, which are depicted in Figure 5, indicate again that theta synchronization reflects the actual encoding of new information. However, as compared to the results depicted in Figures 2, 3 and 4, differences in the time course and topography of theta synchronization can be observed during the implicit encoding processes in an incidental memory paradigm: Early time intervals show a particularly strong synchronization for words which can later be remembered. Possibly, this finding reflects the readiness of the “theta system” to encode new information (during the beginning of the early poststimulus

Theta frequency, synchronization and episodic memory performance

275

period) which obviously fluctuates between trials.

4. CONCLUDING REMARKS The reported findings from animal research and the human scalp EEG clearly support the hypothesis that theta synchronization is related to the encoding and retrieval of new episodic information. In an attempt to explain the possible functional signifi-

Figure 5. Successful encoding in an implicit memory paradigm: theta synchronization is plotted during encoding, in an experiment where subjects did not know that their memory would be tested later (data from Klimesch et al., 1996). Words that were correctly remembered in a later recall test (bold lines) show a highly significant stronger synchronization during implicit encoding than words which could not be remembered later (dashed line). Note the early maximum of synchronization (during 0 to 500 ms poststimulus) and the differences in topography as compared to Figures 2, 3 and 4. Intervals t1–14 represent consecutive time periods of 250 ms each.

cance of the theta rhythm in the human EEG, we assume that synchronized bursts of a small set of hippocampal pyramidal cells induce theta activity in selected (but distributed) cortical regions, which are relevant for performing a particular task. Experimental findings support this view, and indicate that theta band power increases with increasing (episodic) task demands (Klimesch et al., 1994, 1996, 1997 a,c). Evidence for the view that only a small percentage of hippocampo-cortical feedback loops is synchronized comes from a re-examination of the pacemaker role of the septum in the production of the hippocampal theta rhythm (Petsche et al., 1962; Stewart and Fox, 1990). A large fraction of the septo-hippocampal projections terminate on inhibitory (GABA-ergic) hippocampal interneurones (Freund and Antal, 1988; see also the reviews

Time and the brain

276

in Lopes da Silva, 1992). Stewart and Fox (1990) assume that the septal input might organize the hippocampal theta activity, via rhythmic inhibition of hippocampal interneurones. Thus, interneurones are particularly likely to behave as theta cells (Fox and Ranck, 1981; Traub et al., 1989). The finding that only a small percentage of the pyramidal neurones displays synchrony, agrees with the idea that hippocampo-cortical feedback loops induce theta activity into selected cortical areas, where new information is encoded. Considering the fact that theta frequency induces, or at least enhances LTP (see also Lopes da Silva, 1992), it seems tempting to assume that theta activity, induced into selected cortical areas, reflects a process of encoding or retrieving new information, by keeping or putting selected cortical areas into a state of resonance (cf. Miller, 1999; Klimesch, 1996). Finally, an apparent contradiction with well known results should be discussed. Many studies have reported that absolute power in the theta frequency range increases with age (e.g. Christian, 1984; Niedermeyer, 1993b) and is increased in demented subjetcts (Brenner et al., 1986; Coben et al., 1985). Because memory performance decreases with age (and is decreased in demented subjetcs), these findings seem to contradict our hypothesis that an event-related increase in theta power reflects episodic memory performance. However, in a study by Doppelmayr et al. (1998b) it was demonstrated, that subjects with large absolute theta power show significantly smaller theta ERS than subjects with smaller theta power, who show large ERS. Thus, absolute theta power and the extent of event-related changes in theta power are negatively correlated. Consequently, the finding of an age related increase in theta power is in good agreement with our results, which are based on relative or event-related theta power.

REFERENCES Arnolds, D.E.A.T., Lopes Da Silva, F.H., Aitink, J.W., and Boeijinga, W. (1980) The spectral properties of hippocampal EEG related to behaviour in man. Electroencephalography and Clinical Neurophysiology, 50, 324–328. Baddeley, A. (1992) Working Memory. Science, New York, 255, 556–559. Billings, R.J. (1989a) The origin of the initial negative component of the averaged lambda potential recorded from midline electrodes. Electroencephalography and Clinical Neurophysiology, 72, 114–117. Billings, R.J. (1989b) The origin of the occipital lambda wave in man. Electroencephalography and Clinical Neurophysiology, 72, 95–113. Bland, B.H. (1986) The physiology and pharmacology of hippocampal formation theta rhythms. Progress in Neurobiology, 26, 1–54. Brenner, R.P., Ulrich, R.F., Spiker, D.G., Scalbassi, R.J., Reynolds, C.F., III, Marin, R.S., and Boller, F. (1986) Computerized EEG spectral analysis in elderly normal, demented and depressed subjects. Electroencephalography and Clinical Neurophysiology, 64, 483–492. Buzsaki, G., Horvath, Z., Urioste, R., Hetke, J., and Wise, K. (1992) High-frequency network oscillation in the hippocampus. Science, New York, 256, 1025–1027. Buzsaki, G., Leung, L., and Vanderwolf, C. (1983) Cellular bases of hippocampal EEG in the behaving rat. Brain Research Reviews, 6, 139–171. Buzsaki, G., Bragin, A., Chrobak, J.J., Nadasdy, Z., Sik, A., Hsu, M., and Ylinen, A.

Theta frequency, synchronization and episodic memory performance

277

(1994) Oscillatory and intermittent synchrony in the hippocampus: Relevance to memory trace formation. In: G. Buzsaki, R. Llinás, W. Singer, A. Berthoz and Y. Christen (eds) Temporal coding in the brain. Berlin, Heidelberg: Springer-Verlag, pp. 145–172. Coben, L.A., Danziger, W., and Storandt, M. (1985) A longitudinal EEG study of mild senile dementia of Alzheimer type: Changes at 1 year and at 2.5 years. Electroencephalography and Clinical Neurophysiology, 61, 101–112. Christian, W. (1984) Das Elektroencephalogramm (EEG) im höheren Lebensalter. Nervenarzt, 55, 517–524. Doppelmayr, M., Klimesch, W., Pachinger, Th., and Ripper, B. (1998a) Individual differencies in brain dynamics: Important implications for the calculation of eventrelated band power measures. Biological Cybernetics, (in press). Doppelmayr, M., Klimesch, W., Pachinger, Th., and Ripper, B. (1998b) The functional significance of absolute power with respect to event-related desynchronization. International Journal of Psychophysiology, (in press). Fox, S.E., and Ranck, Jr., J.B. (1981) Electrophysiological characteristics of hippocampal complex-spike cells and theta cells. Experimental Brain Research, 41, 399–410. Fox, S.E., Wolfson, S., and Ranck, J.B.Jr. (1986) Hippocampal theta rhythm and the firing of neurons in walking and urethane anesthetized rats. Experimental Brain Research, 62, 495–508. Freund, T., and Antal, M. (1988) Hippocampal interneurons. Nature, London, 336, 170– 173. Green, J.D., and Arduini, A.A. (1954) Hippocampal electrical activity in arousal. Journal of Neurophysiology, 17, 533–547. Greenstein, Y.J., Pavlides, C., and Winson, J. (1988) Long-term potentiation in the dentate gyrus is preferentially induced at theta rhythm periodicity. Brain Research, 438, 331–334. Klimesch, W. (1994) The structure of long-term memory: A connectivity model for semantic processing. Hillsdale, NJ: Lawrence Erlbaum. Klimesch, W. (1995) Memory processes described as brain oscillations in the EEG-alpha and theta band. Psycoloquy (electronic journal). Klimesch, W. (1996) Memory processes, brain oscillations and EEG synchronization. International Journal of Psychophysiology, 24, 61–100. Klimesch, W., Schimke, H., and Schwaiger, J. (1994) Episodic and semantic memory: An analysis in the EEG-theta and alpha band. Electroencephalography and Clinical Neurophysiology, 91, 428–441. Klimesch, W., Doppelmayr, M., Russegger, H., and Pachinger, Th. (1996) Theta band power in the human scalp EEG and the encoding of new information. NeuroReport, 7, 1235–1240. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Ripper, B. (1997a) Brain oscillations and human memory performance: EEG correlates in the upper alpha and theta bands. Neuroscience Letters, 238, 9–12. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Russegger, H. (1997b) Event-related desynchronization in the alpha band and the processing of semantic information. Cognitive Brain Research, 6, 83–94. Klimesch, W., Doppelmayr, M., Schimke, H., and Ripper, B. (1997c) Theta synchronization in a memory task. Psychophysiology, 34, 169–176. Kroll, N.E.A., and Klimesch, W. (1992) Semantic memory. Complexity or connectivity? Memory and Cognition, 20, 192–210. Larson, J., Wong, D., and Lynch, G. (1986) Patterned stimulation at the theta frequency

Time and the brain

278

is optimal for the induction of hippocampal long-term potentiation. Brain Research, 368, 347–350. Leung, L., Lopes da Silva, F., and Wadman, W. (1982) Spectral characteristics of the hippocampal EEG in the freely mooving rat. Electroencephalography and Clinical Neurophysiology, 54, 203–219. Lisman, J.E., and Idiart, M.A.P. (1995) Storage of 7 +−2 short-term memories in oscillatory subcycles. Science, NewYork, 267, 1512–1515. Lopes da Silva, F.H. (1992) The rhythmic slow activity (theta) of the limbic cortex: An oscillation in search of a function. In: E. Başar and T.H. Bullock (eds) Induced rhythms in the brain. Boston: Birkhäuser, pp. 83–102. Maren, St., DeCola, J., Swain, R., Fanselow, M., and Thompson, R. (1994) Parallel Augmentation of hippocampal long-term potentiation, theta rhythm, and contextual fear conditioning in water-deprived rats. Behavioral Neuroscience, 108, 44–56. Miller, R. (1991) Cortico-hippocampal interplay and the representation of contexts in the brain. Berlin: Springer. Montoya, Ch., Heynen, A., Faris, P., and Sainsbury, R. (1989) Modality specific type 2 theta production in the immobile rat. Behavioral Neuroscience, 103, 106–111. Niedermeyer, E. (1993) Normal aging and transient cognitive disorders in the elderly. In: E. Niedermeyer and F.H. Lopes da Silva (eds) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn. Baltimore: Williams & Wilkins, pp. 329–338. O’Keefe, J. (1993) Hippocampus, theta and spatial memory. Current Opinion in Neurobiology, 3, 917–924. O’Keefe, J., and Dostrovsky, J. (1971) The hippocampus as a spatial map: preliminary evidence from unit activity in freely moving rats. Brain Research, 34, 171–175. O’Keefe, J., and Recce, M. (1993) Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus, 3, 317–330. Petsche, H., Stumpf, C., and Gogolak, G. (1962) The significance of the rabbit’s septum as a relay station between the midbrain and the hippocampus. Electroencephalography and Clinical Neurophysiology, 19, 25–33. Pfurtscheller, G., and Aranibar, A. (1977) Event-related cortical synchronization detected by power measurements of scalp EEG. Electroencephalography and Clinical Neurophysiology, 42, 817–826. Pfurtscheller, G. (1992) Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest. Electroencephalography and Clinical Neurophysiology, 83, 62–69. Schacter, D. (1977) EEG theta waves and psychological phenomena: A review and analysis. Biological Psychology, 5, 47–82. Scoville, W., and Milner, B. (1957) Loss of recent memory after bilateral hippocampal lesions. Journal of Neurology, Neurosurgery and Psychiatry, 20, 11–21. Sinclair, B.R., Seto, M.G., and Bland, B.H. (1982) Theta-cells in CA1 and dentate layers of hippocampal formation: Relations to slow-wave activity and motor behavior in the freely moving rabbit. Journal of Neurophysiology, 48, 1214–1225. Squire, L.R. (1992) Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans. Psychological Review, 99, 195–231. Stewart, M., and Fox, S.E. (1990) Do septal neurons pace the hippocampal theta rhythm? Trends in Neuroscience, 13, 163–168. Traub, R., Miles, R., and Wong, R. (1989) Model of the origin of rhythmic population oscillations in the hippocampal slice. Science, New York, 243, 1319–1325.

Theta frequency, synchronization and episodic memory performance

279

Tulving, E. (1984) Precis of elements of episodic memory. The Behavioral and Brain Sciences, 7, 223–268. Vanderwolf, C.H. (1992) The electrocorticogram in relation to physiology and behavior: a new analysis. Electroencephalography and Clinical Neurophysiology, 82, 165–175. Vanderwolf, C., and Robinson, T. (1981) Reticulo-cortical activity and behavior: A critique of the arousal theory and a new synthesis. Behavioral and Brain Sciences, 4, 459–514.

9 Distributed Assemblies, High Frequencies and the Significance of EEG/MEG Recordings Friedemann Pulvermüller Department of Psychology, Universität of Konstanz, PO Box D23, 78457 Konstanz, Germany and Cognition & Brain Sciences Unit Medical Research Council, 15 Chaucer Road, Cambridge CB2 2EF, England e-mail: [email protected]

According to recent proposals, coherent high-frequency activity in cortical neurones may be a specific indicator of cognitive processes in the brain, including those related to the perception of objects, the programming of actions, and the processing of language. One may therefore speculate that cell assemblies involved in higher cortical functions primarily generate high-frequency rhythms. In contrast, it has been argued that such neuronal assemblies may also generate equally strong spectral activity in various frequency bands, high and low, when active. It is proposed here that the controversy can be decided on the basis of knowledge about axonal conduction times inferred from fibre calibres measured in the human cortex. These data suggest that conduction times of most long axons are around 5–10 m/s, and even loops between neurones in distant areas can be travelled in a few hundredths of a second. Reverberation of excitation in assemblies would thus primarily lead to high-frequency cortical activity above 20 Hz. This leads to an explanation of recent findings from large-scale neurophysiological recordings during cognitive processing in humans. KEYWORDS: MEG; EEG; Cell assembly; Axonal conduction time; 30 Hz; 40 Hz; Gamma; Cognition; Language Cortical neurones usually fire a few spikes per second, that is, with frequencies of a few Hertz (see, for example, Hubel [1988] and Fuster [1995]). However, much faster activity, ranging from tens to hundreds of Hertz, has been found not only in cortical neurones, but in subcortical structures as well (Bressler and Freeman, 1980; Llinás and Ribary, 1993; Nuñez et al., 1992; Singer and Gray, 1995; Steriade, 1993). There are several types of high-frequency activity, including rhythms generated by individual neurones (Llinás et al., 1991; Gray and McCormick, 1996), by subcortico-cortical loops involving many neurones, which force them to fire synchronously (Llinás and Ribary, 1993; Steriade, 1993), and by interactions between excitatory and/or inhibitory cortical cells which also evoke coherent activity in these neurones (Singer and Gray, 1995; Traub et al., 1996). Apparently, several types of rhythms can be distinguished that play different

Time and the brain

282

roles in arousal and attention, perception, and higher cognitive processing (Pulvermüller et al., 1997). One possible role is the maintenance of active memory in the neocortex. Accordingly, a specific engram would correspond to a network of neurones in which activity reverberates, and thereby evokes a complex spatiotemporal activity pattern in its many neurons (Abeles, 1991; Fuster, 1995; Hebb, 1949). Figure 1 illustrates Donald Hebb’s idea about an assembly consisting of small groups of neurones connected in such a way that a wave of activity takes a complex path through the network, producing multiple reverberations. The loops in the assembly give rise to repeated well-timed activity in many neurones, and if multiple reverberations are allowed in each of these loops, several activation-deactivation cycles with similar frequencies will be generated when the assembly is active. This would be interesting for cognitive brain research, because if cell assemblies are sufficiently large and “influential” their high-frequency dynamics may be observable by large-scale non-invasive recording techniques, that is, by EEG and MEG, making it possible to scrutinize an important correlate of cognitive processes in humans. However, this experimental strategy depends crucially on the processes one envisages to occur during reverberatory activity of an assembly. If, for example, an assembly includes many neurones with slowly conducting axons connected in sequence, reverberations may occur only once a second or even less frequently. In contrast, if the assembly looks like the diagram in Figure 1 and therefore includes loops formed by only three smaller neurone populations (see the cycle 1–2-3), and if axonal conduction times in these neurones are rather fast, one may even expect 100 Hz or faster activity to be generated by the network, when it is active. Robert Miller (1994; 1999) has recently argued that cell assemblies should produce neuronal activity at various frequencies, and that no specific predictions can therefore be drawn about the frequency range primarily affected when an assembly becomes activated. However, one may ask whether there are indications that cell assembly activation affects certain frequency bands more than others, and whether there is a range where investigation appears to be particularly fruitful. Let us take the Hebbian idea illustrated in Figure 1 as a starting point: cell assemblies consist of neuronal subsets connected by orderly connections, which include loops. Thus, the subsets whose neurones fire synchronously become active in a fixed order, and some of the subsets (those included in loops) can become active repeatedly. For determining the frequency of these repetitions, three characteristics of the assemblies are of particular importance: (i) the distances to be bridged and, thus, the distributedness of the assemblies; (ii) the number of subsets making up an internal loop within an assembly; and (iii) the conduction times of the axons involved. Ad (i): It has been argued that the neurones of one assembly may well be spread out over distant areas in one hemisphere, and even over both hemispheres (Miller, 1996; Pulvermüller and Mohr, 1996). However, assembly neurones probably are not dispersed over the entire cortex, but include neurones in a small set of well-defined areas (Posner and Raichle, 1994). Figure 2 shows a sketch of an assembly, assumed to represent a word referring to an action, for example “to write”, whose neurones are distributed over three cortical loci, the anterior and the posterior language areas and motor cortices relevant for

Distributed assemblies, high frequencies and the significance of EEG/MEG

283

hand movements. In humans, the distances from the anterior to the posterior language region, from dorsal motor cortices to sites close to the Sylvian fissure, or between homologous areas of the hemispheres are in the order of 10 cm.

Figure 1. Hebb’s idea of a reverberatory cell assembly is illustrated. Arrows represent neurone sets and numbers indicate the sequence in which they become active.

Ad (ii): Although Hebb’s diagram suggests that only a few subsets form a loop within the assembly, there is no a priori argument against assuming longer loops. The smallest loops possible would be a neurone set projecting onto itself, and two sets with reciprocal connections. However, longer loops including 10 or even 20 neurone sets are possible. When estimating “round trip” times and frequencies, several options must be taken into account here. The time for travelling a loop also varies as a function of the number of steps, because synaptic transmission and the spreading of the postsynaptic voltage change towards the axon hillock, (where action potentials are generated) take time. About 1 ms after the action potential arrives at the presynapse, the excitatory postsynaptic potential reaches its maximum at the postsynaptic membrane, and it may take about another

Time and the brain

284

millisecond before voltage changes start to be detectable at the axon hillock. These synaptic and dendritic delays will be assumed to lie around 2 ms (1 ms synaptic and 1 ms dendritic delay). Accordingly, if all neurones of a subset A of an assembly fire, so that action potentials arrive synchronously at the neurones included in the next subset B, the B neurones will synchronously fire 2 ms later. If a loop includes n subsets, the synapticdendritic delay can be estimated to lie around 2n ms.

Figure 2. Hypothetical cell assembly representing a word referring to an action (e.g. to write). The cell assembly includes neurones in the anterior and posterior language regions and in motor and premotor areas. Circles indicate local neurone clusters, the short lines are called “within-area” connections, and the long lines are between-area connections.

Ad (in): Precise calculations of round trip times are possible based on actual data about axon calibres (Aboitiz et al., 1992; LaMantia and Rakic, 1990): The larger the diameter of an axon, the faster it conducts action potentials. The fibre calibres allow one to estimate conduction times (see below). Figure 3 presents data from macaca (left) and from homo sapiens (right). In both studies, fibres from the Corpus callosum were evaluated. It must be emphasized that the interpretation of these data rests on the assumption that the fibre composition of the callosum also gives an adequate picture of the axons connecting neurones within one hemisphere. This assumption may turn out to be wrong, in which case the estimations below would have to be revised. The histograms obtained from ape and monkey do not differ in their general shape and the most numerous fibre type differs between species by only a few tenths of a µm (ca. 0.5 µm in macaca vs. 0.8–0.9 µm in homo). This relatively good agreement may be taken as an argument in favour of the reliability of the results from humans. Both columns of histograms in Figure 3 plot the number (or percentage) of myelinated

Distributed assemblies, high frequencies and the significance of EEG/MEG

285

axons as a function of their calibres. The slow unmyelinated fibers were not included. They represented only around 10% of the total number of fibres, or even less. In addition, calibres, and therefore conduction times, of these few unmyelinated fibres vary greatly (see Table 1). Therefore, loops involving these fibres cannot generate predominant rhythms. In contrast to this view, Robert Miller (1994; 1996) has argued that the data from humans (Aboitiz et al., 1992) are questionable, because fixation may have been a problem and the number of unmyelinated axons may thus be underestimated. Miller therefore advocates interpretation of data from cat, rabbit and rat revealing higher percentages of unmyelinated fibers. However, consistent with the results in humans

Figure 3. Histograms from two studies of fibre calibres in the corpus callosum. Data from macaca are presented on the left (adapted from LaMantia

Time and the brain

286

and Rakic, 1990), and data from humans are on the right (adapted from Aboitiz et al., 1992). Normalized numbers of axons or their percentage is plotted for different calibers of myelinated axons.

(Aboitiz et al., 1992), the study by LaMantia and Rakic (1990) yielded small percentages of unmyelinated fibers in macaca indicating that in monkeys or even in higher mammals the slow fibres play a minor role. This would make good sense, in particular because the brains of most higher mammals are relatively large. While unmyelinated fibres certainly also participate in generating physiological responses in the human brain, the predominant physiological activity is probably related to the predominant anatomical substrate, that is, to the myelinated fibres slightly less than 1 µm in diameter. It is important to note that more than 60% of the axons have diameters in the range between 0.5 and 1 µm. Loops in the human cortex are therefore most likely to be formed by neurones with these frequent axon types. How rapidly would these axons conduct their action potentials? Table 1 presents estimated conduction times for fibres with different diameters. These estimates are based on measurements of peripheral nerves (e.g., Lee et al., 1986). (The fact that tissue shrinks during fixation has been taken into account.) Obviously, there are fibres with very fast conduction velocities which could bridge the gap between cortical areas 10 cm apart in less than 2 ms. On the other hand, there are very slow unmyelinated fibres taking almost half a second for this distance, and also thin myelinated axons along which an action potential would take around 20 ms. However, the histograms in Figure 3 indicate that both very fast and very slow fibers are relatively rare. The conduction velocities of the frequent types—with diameters between 0.5 and 1 µm—are between 5 and 10 m/s. Therefore, a 10 cm distance would be travelled in around 10 to 20 ms. When estimating predominant activity dynamics in cortical networks, it makes sense to consider two types of common axons, the fast common type (velocity 10 m/s) and the slow common type (velocity 20 m/s), to obtain an upper and a lower boundary for the estimates. Based on the numbers discussed above, it is easy to calculate round trip times in loops formed by various numbers of neurone sets connected by fibres with different calibres. However, it is adequate to restrict the considerations regarding the number of subsets connected in loops. In particular, for neurone populations located far apart, it appears likely that the loops they form do not include too many steps. Investigations in macaca indicate that the likelihood of two distant areas being connected is only around 30% (Young et al., 1994, 1995). However, connections are usually topographic and reciprocal. This means that if two neurone populations are located in two distant cortical areas, respectively, and if these subsets exhibit connections in one direction, the probability of the reciprocal connections existing is high (Young et al., 1995). On the basis of this reciprocity principle, two-step reverberations between distant subsets are likely. If one local population is connected to two other ones, the probability of the latter two exhibiting strong direct connections is low, and reverberations would again occur between the connected pairs. In contrast to the long-range connections, local con

Distributed assemblies, high frequencies and the significance of EEG/MEG

287

Table 1 True minimal diameter, corrected diameter, estimated conduction velocity and estimated travelling time for a distance of 100 to 130 mm listed for different fiber populations. Adapted from Aboitiz et al., (1992).

fibre group

corrected diameter (µm)

velocity axonal m/s

conduction delay 10–13 cm (ms)

unmyelinated

0.1–1

0.3–3.2

50–433

0.4 µm

0.6

5.2

19.2–24.9

1 µm

1.5

13.1

7.6–9.9

3 µm

4.6

40

2.5–3.2

5 µm

7.7

67

1.5–1.9

Table 2 Estimated “round trip” times and frequencies produced by three types of loops of neurones exhibiting conduction times around 10m/s (fast frequent type).

length of individual axon branches involved and conduction delays

overall conduction time (sum)

synaptic and dendritic delays (sum)

round trip time

round trip frequency

(mm|ms)

(ms)

(ms)

(ms)

(Hz)

local, 2 steps

0.1|0.01

0.02

4

4

250

local, 10 steps

0.1|0.01

0.1

20

20

50

local, 20 steps

0.1|0.01

0.2

40

40

25

withinarea, 2 steps

10|1

2

4

6

167

withinarea, 10 steps

10|1

10

20

30

33

betweenarea, 2 steps

100|10

20

4

24

42

loop type

nections between sets of adjacent spiny cells, and between neurones in adjacent cortical areas have a higher probability of existing, and therefore the possibility of longer loops,

Time and the brain

288

including, for example, 10 or even 20 steps, should be considered. Tables 2 and 3 present estimates of “round-trip” times for neuronal activity in local loops and between-area loops, together with the corresponding frequencies generated by reverberating activity. In addition to local loops between adjacent neurone populations and between-area loops, semi-local reverberations in neurone populations within one area are examined (distances around 1 cm). In Figure 2, this would correspond to activity dynamics assumed to occur in the local neurone clusters depicted as small circles (local loops), in the groups of 4 adjacent local clusters (within-area loops), and in the reciprocal long range connections that would allow for two-step long-range loops (between-area loops). Axonal conduction times are calculated as the sum of axonal conduction times and synaptic and dendritic delays. The two tables present data separately for the slow and the fast common axon types, evidenced by the neuroanatomical study. Synaptic and dendritic delays are estimated, as detailed above. Surprisingly, for all the parameter choices considered the reverberation frequencies are above 20 Hz. This means that a coherent wave of neuronal activity travelling along these

Table 3 Estimated “round trip” times and frequencies produced by various loops of neurones with axons exhibiting conduction times around 5 m/s (slow frequent type).

length of individual axon branches involved and conduction delays

overall conduction time (sum)

synaptic and dendritic delays (sum)

round trip time

round trip frequency

(mm|ms)

(ms)

(ms)

(ms)

(Hz)

local, 2 steps

0.1|0.02

0.04

4

4

250

local, 10 steps

0.1|0.02

0.2

20

20

50

local, 20 steps

0.1|0.02

0.4

40

40

25

withinarea, 2 steps

10|2

4

4

8

125

withinarea, 10 steps

10|2

20

20

40

25

betweenarea, 2 steps

100|20

40

4

44

23

loop type

loops takes less than 50 ms for one cycle. The predominant rhythms generated by

Distributed assemblies, high frequencies and the significance of EEG/MEG

289

between-area loops lie between 23 and 42 Hz, and for more local loops the frequency range extends towards even higher frequencies. One may argue that it is necessary to consider longer loops involving many more steps, and these would undoubtedly generate longer “round-trip” times and lower frequencies. However, as already mentioned, the not-so-frequent but reciprocal and topographic connections between-areas argue in favour of two-step loops, and against a larger number of steps. The local neurone clusters are linked through local axon collaterals synapsing on basal dendrites of adjacent cells. But both axon collaterals and dendritic trees have only a very limited range, so that after more than 10 or 20 steps the neurones activated probably cannot project back through their local connections. This also argues against longer local loops. Another point one may want to make is that, in many cases, local axon collaterals are not myelinated while the axon branches reaching the white matter are. This would lead to longer conduction times for the axons involved in local loops, so that the respective numbers in Tables 2 and 3 should be increased, probably by a factor of 10. However, because estimated conduction times are very low (below half a millisecond), such an increase would only lead to a slight increase of round-trip times, and to a minimal reduction of the frequencies. (Notice that the present estimates of round-trip times in local loops are entirely determined by synaptic and dendritic delays.) This adjustment would not change the overall conclusion. From these calculations, one may develop an optimistic perspective on the investigation of cognitive brain processes by means of non-invasive physiological recordings: if there are cell assemblies, including subsets of neurones connected in sequence and in loops, so that coherent reverberatory activity in many neurones is generated by the loops whenever the assembly is active, it may be possible to capture the gist of these reverberations at the surface of the head. In this case, the frequency range to which one may want to be particularly attentive is above 20 Hz and, if one is specifically interested in widely distributed neurone populations, in the 20–40 Hz range. Recent results from investigations of high-frequency dynamics during cognitive processing indicate that spectral dynamics between 20 and 40 Hz indeed co-occur with processes such as perception of a Gestalt-like figures or access to a word form or to its meaning (Pulvermüller et al., 1997). Interestingly, specific topographic patterns can sometimes be observed that may allow for conclusions on assembly topographies. (Pulvermüller, 1999) Let me finally stress again that the estimates above were made for the most frequent fibre types. Without any doubt, there are faster and slower fibres and, therefore, an assembly may well generate various rhythms, high and low, when active. However, the fast rhythms relying on the most common fibres probably predominate.

ACKNOWLEDGEMENTS I am grateful to Valentino Braitenberg and Robert Miller for ample discussion of ideas related to this article. Special thanks go to Detlef Heck and Almut Schüz who gave me advice on the neurophysiological and neuroanatomical parameters discussed. However, not they but the author is to be blamed for any remaining errors or inappropriatenesses.

Time and the brain

290

Supported by grants Pu 97/2, Pu 97/5, Pu 97/10 and Pu 97/11 of the Deutsche Forschungsgemeinschaft (DFG).

REFERENCES Abeles M. (1991) Corticonics—Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Aboitiz, F., Scheibel, A.B., Fisher, R.S., and Zaidel, E. (1992) Fiber composition of the human corpus callosum. Brain Research, 598, 143–153. Bressler, S.L., and Freeman, W.J. (1980) Frequency analysis of olfactory system EEG in cat, rabbit and rat. Electroencephalography and Clinical Neurophysiology, 50, 19–24. Fuster, J.M. (1995) Memory in the cerebral cortex. An empirical approach to neural networks in the human and nonhuman primate. Cambridge, MA: MIT Press. Gray, C.M., and McCormick, D.A. (1996) Chattering cells: superficial pyramidal neurns contributing to the generation of synchronous oscillations in the visual cortex. Science, New York, 274, 109–113. Hebb, D.O. (1949) The organization of behavior. A neuropsychological theory. New York: John Wiley. Hubel, D. (1988) Eye, brain, and vision. New York: Freeman. LaMantia, A.S., and Rakic, P. (1990) Cytological and quantitative characteristics of four cerebral commissures in the rhesus monkey. Journal of Comparative Neurology, 291, 520–537. Lee, K.H., Chung, K., Chung, J.M., and Coggeshall, R.E. (1986) Correlates of cell body size, axon size, and signal conduction velocity for individually labelled dorsal root ganglion cells in the cat. Journal of Comparative Neurology, 243, 335–346. Llinás, R.R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in mammalian cortex layer 4 exhibit intrinsic activity in the 10 to 50 Hz frequency range. Proceedings of the National Academy of Sciences, USA, 88, 897–901. Llinás, R.R., and Ribary, U. (1993) Coherent 40-Hz oscillation characterizes dream state in humans. Proceedings of the National Academy of Sciences, USA, 90, 2078–2081. Miller, R. (1994) Cognitive Processing, but not cell assembly ignition. Commentary on Pulvermüller et al. on brain-rhythms. Psycoloquy, 5 (50), 1–7. Miller, R. (1996) Axonal conduction times and human cerebral laterality. A psychobiological theory. Amsterdam: Harwood Academic Publishers. Miller, R. (1999) Unifying cell assembly theory with observations from brain dynamics. Behavioral and Brain Sciences, 22, 297–298. Nuñez, A., Amzica, F., and Steriade, M. (1992) Voltage-dependent fast (20–40 Hz) oscillations in long-axoned neocortical neurons. Neuroscience, 51, 7–10. Posner, M.I., and Raichle, M.E. (1994) Images of mind. New York: Scientific American Library. Pulvermüller, F., Birbaumer, N., Lutzenberger, W., and Mohr, B. (1997) High-frequency brain activity: Its possible role in attention, perception and language processing. Progress in Neurobiology, 52, 427–445. Pulvermüller, F., and Mohr, B. (1996) The concept of transcortical cell assemblies: a key to the understanding of cortical lateralization and interhemispheric interaction. Neuroscience and Biobehavioral Reviews, 20, 557–566. Pulvermüller, F., (1999) Words in the brain's language. Behavioral and Brain Sciences, 22, 253-366.

Distributed assemblies, high frequencies and the significance of EEG/MEG

291

Singer, W., and Gray, C.M. (1995) Visual feature integration and the temporal correlation hypothesis. Annual Review in Neuroscience, 18, 555–586. Steriade, M. (1993) Central core modulation of spontaneous oscillations and sensory transmission in thalamocortical systems. Current Opinion in Neurobiology, 3, 619– 625. Traub, R.D., Whittington, M.A., Stanford, I.M., and Jeffreys, J.G.R. (1996) A mechanism for generation of long-range synchronous fast oscillations in the cortex. Nature, London, 383, 621–624. Young, M.P., Scannell, J.W., Burns, G., and Blakemore, C. (1994) Analysis of connectivity: neural systems in the cerebral cortex. Reviews in Neuroscience, 5, 227– 249. Young, M.P., Scannell, J.W., and Burns, G. (1995) The analysis of cortical connectivity. Heidelberg: Springer.

Time and the brain

292

This page intentionally left blank.

10 Cell Assemblies, Associative Memory and Temporal Structure in Brain Signals Thomas Wennekers and Günther Palm Abteilung Neuroinformatik, Fakultät für Informatik, Universitat Ulm, Oberer Eselsberg, D-89069 Ulm, Germany Tel: (0049)–731-502–4151; FAX: (0049)–731–502–4156; e-mail: (thomas,palm)@neuro.informatik.uni-ulm.de

In this work we discuss Hebb’s old ideas about cell assemblies in the light of recent results concerning temporal structure and correlations in neural signals. We want to give a conceptual (and necessarily only rough) picture of how ideas about “binding by synchronization”, “synfire chains”, “local and global assemblies”, “short and long term memory” and “behaviour” might be integrated into a coherent model of brain functioning based on neuronal assemblies. KEYWORDS: Cell assemblies; Synchronization; Gamma-oscillations; Synfire chains; Memory; Behaviour

1. CELL ASSEMBLIES AND ASSOCIATIVE MEMORIES 1.1. Cell Assemblies Cell assemblies were introduced by Donald Hebb with the intention of providing a functional, and at the same time a structural model for cortical processes and neuronal representations of external events (Hebb, 1949). According to Hebb’s ideas, stimuli, objects, things, but also more abstract entities, like concepts, contextual relations, ideas (and so on) are considered to be represented in the brain by simultaneous activation of large groups of neurones, which are connected by relatively numerous and/or strong mutual excitatory synapses. Each single neurone may belong to many different cell assemblies. The determinant of an assembly is the connectivity structure between cells. This defines which cells lend support to each others’ firing, and hence have a higher probability of becoming coactivated in a reliable manner in response to different versions of the same stimulus. If an external stimulus excites a sufficiently large subset of cells of an assembly, then the whole assembly can “ignite” or “fire”, because recurrent activity,

Time and the brain

294

distributed via the specific mutual connections, can also raise above threshold those cells which are not (or are only weakly) stimulated externally. This can be viewed as an elementary associative process, where the firing of externally driven cells represents the key information, and triggers the firing of cells representing information addressed by, but not yet contained in the key. In a similar way, some kind of short term memory is also supported. If continuously changing sub-groups of cells produce maintained activation of other groups, the activation within the assembly may survive for some time after its ignition, even if the external stimulation has already vanished. The assembly concept furthermore proposes a mechanism for long term memory, that is, the formation of new assemblies in cortical tissue, under the influence of electrical activation. Learning is believed to be expressed in activity-dependent changes of synaptic efficacies, and the “Hebbian learning rule”, which is nowadays well known, states that synapses become strengthened when both connected cells are activated simultaneously within a certain time window. This hypothesis was motivated by the observation that events which repeatedly occur together should somehow belong together. Every time they appear in conjunction, they drive certain subgroups of cortical cells; the correlated firing of these sets of neurones (which are not yet assemblies!) should be learned and by that, the respective groups should become associatively connected. 1.2. Associative Memories The classical associative memory is an abstraction of the assembly concept (Willshaw et al., 1969; Palm, 1980, 1982), and provides a model based on single neurones as simple threshold elements. Figure 1 shows an example of an associative memory comprising 6 neurones with 6 inputs. Circles at the bottom, and vertical lines, represent threshold neurones and their dendrites, horizontal lines are axonal input fibres, and small filled circles are synapses, which are assumed to have an initial weight of zero. Neuronal excitation patterns are represented by binary {0,1} sequences reflecting whether or not cells fire in a given time window. A set of associations, that is, pairs of patterns (xi,yi), i=1,2,..P, is stored in the synaptic connectivity matrix. Special cases are the so-called auto-association, where xi equals yi for all pairs, or hetero-association, where xi and yi can be different (Palm, 1991). In Figure 1a two associations are stored: first the vector x1=(100101) activates the input lines, and y1=(110100) activates the neurones. All synapses, which simultaneously receive pre- and postsynaptic activity are set to a value of 1, indicated by the black filled synapses. Afterwards the second association, (x2,y2), is stored in the same way (grey synapses). Figure 1b displays an elementary retrieval process. The address pattern x=(111000) activates some of the previously strengthened synapses (squares), via the first three input lines. Each threshold-neurone sums over its activated synapses; this determines the “potentials” of the cells, shown as small numbers above the neurones. A neurone generates an output signal of 1, if its potential is larger than a certain threshold θ. Here θ=2, and thus, the output vector is y=(010011), which is obviously exactly the stored vector y2. The input pattern x during retrieval is more similar to x2 than to x1, because the number of common “ones” is 2 in the first, but only 1 in the second case. Hence this is a

Cell assemblies, associative memory and temporal structure in brain signals

295

correct association. In auto-associative networks the output vector can be fed back to the input for further recurrent iteration steps. For best performance in networks of realistic size the stored patterns should contain only a small fraction of active neurones, which is well in accordance with the situation in the cortex (“sparseness”, see Palm 1980). Further theoretical results on associative memories can be found in Willshaw (1967), Palm (1980, 1982, 1991), and Schwenker et al. (1996).

Figure 1. Simple associative memory model comprising 6 inputs and outputs, a) Learning of 2 associations (x1,y1) and (x2,y2). At first, the first pattern-pair is applied to input and output; then all synapses (crossings of vertical and horizontal lines) which are pre- and postsynaptically activated are set to a value of one (black circles). Afterwards the second pattern-pair is learned in the same way (grey synapses), b) Retrieval from an erroneous incomplete key, x= (111000). This input activates synapses that are drawn as black rectangles. The sum over activated synapses of a neurone determines its potential (small numbers above threshold neurones, (130122)); if this is larger than or equal to θ=2, the neurone generates an output of one, otherwise of zero. The output vector y is equal to the stored vector y2. Because the address pattern x has a larger overlap (common ones) with x2 than with x1 this is a proper association.

1.3. Spiking Neurones and Threshold Control The classical associative memory presumably covers basic aspects of pattern storage in real brains. Nonetheless, compared with biological neurones and synapses, the inclusion of further physiological details seems desirable, particularly, if we ask for temporal properties of neural systems. Conventional associative memory models employ simple threshold neurones and an iterative, time-quantized update scheme. Therefore they can give only a very rough insight into dynamical processes in cell assemblies. For this

Time and the brain

296

reason we add some features to the classical model, which mimic the spiking behaviour of real neurones as well as properties of spatiotemporal integration on dendrites. First, we choose a “continuous time” model in all simulations shown later.1 Single cells are modelled as “spiking neurones”: dendrites and soma of a cell are lumped into a single potential value. Every time this potential reaches a certain threshold θ a pulse-like “action-potential” is generated. Afterwards a suitable refractory mechanism prevents the cell from firing immediately afterwards. Action potentials, when they arrive at a target cell, evoke “post-synaptic potentials” of realistic form and time-constants, and the responses of different synapses are supposed to add up linearly. 1. We only give informal model descriptions in this paper, because the arguments and results in the sequel are to a large degree independent of implementational details. More precise descriptions and equations can be found in Wennekers et al. (1995) and Bibbig et al. (1995). For comparable spiking neurone models see Stein (1967) and Gerstner and v. Hemmen (1993)

Investigations of recurrent auto-associative memories by Schwenker et al. (1996) reveal the need for a proper continuous and global threshold control, in order to avoid the network being insensitive to any input (θ in Figure 1b being too large) or that its activity explodes (θ too small). In accordance with common ideas about cortical functioning (e.g. Braitenberg and Schüz, 1991), we assume that pyramidal cells and their excitatory connections carry out essential information processing tasks, like feature extraction, pattern storage, pattern retrieval, etc., and that inhibitory interneurones have mainly regulatory functions, one of which is probably some kind of activity control, similar to the threshold regulation required on theoretical grounds. Therefore, we start from an associative network of spiking neurones linked by excitatory connections, and embed interneurones within the network, which measure local firing rates and inhibit the excitatory cells accordingly. Networks of this architecture typically reveal collective oscillations when external excitation exceeds some critical level, and assuming excitatory and inhibitory synapses are strong and/or numerous enough. We have shown (Wennekers et al. 1995) that with reasonable assumptions concerning network parameters, firing can be sparse even in states of collective oscillations, that is, single neurones need not fire in every period (see also next section). This work also shows that the more realistic model neurones, with a threshold and refractory period, support synchronization more efficiently than models with a purely probabilistic spike encoder (e.g. rate modulated Poisson processes). This is because the threshold-crossing process determines precise firing times, whereas those are necessarily imprecise in modulated Poisson models.

2. LOCAL ASSEMBLIES Experiments on anaesthetised cats and alert behaving monkeys have shown that local populations of cells in primary visual areas often respond rhythmically with frequencies in the gamma-range (30–90 Hz) (cf. Eckhorn, this volume; experimental review in Singer

Cell assemblies, associative memory and temporal structure in brain signals

297

and Gray, 1995). Most interestingly, those “oscillations”, when observed at two distant cortical sites, can reveal a considerable amount of synchronization, which depends strongly on certain non-local stimulus properties, and seems to follow simple “Gestalt”rules like proximity, colinearity, etc., even though the recording sites may be located in different cortical areas or hemispheres. These findings have been taken as evidence for the so-called temporal correlation hypothesis of sensory integration in the mammalian cortex (von der Malsburg, 1981; Eckhorn et al., 1988; Singer and Gray, 1995), which states, that neurones which fire in response to stimulation by the same external entity should display correlated—in particular synchronized—firing. This way the cohesion of different parts of a single object can be signalled, even if those parts are processed over distributed regions of the brain (binding by synchronization). Many attempts have been made to model this “binding” process (c.f. chapters by Eckhorn, and Borisyuk, [this volume]; see also Wennekers and Palm [1997] for an overview and a discussion of some principal assumptions of similar models). In the sequel to this section we present our own simulations of oscillatory assemblies in localized patches of cortex, and discuss the possible role of fast gamma rhythms for information processing. 2.1. Local Dynamics in a Primary Area We consider a local patch of cortical tissue in a primary cortical area of (roughly) the size of a cortical column. This patch will contain cells tuned to different stimulus orientations, directions, velocities, etc. For simplicity we assume that cells can be ordered by their orientation preference and that similarly oriented cells have a higher probability of being connected. We neglect other tuning properties of cells which might be reflected by the local cortical circuitry, and also we do not model the laminar structure of real cortices. This means that, to a first approximation, we consider a one-dimensional topographically ordered model network of N=128 excitatory spiking neurones. Connectivities are restricted to some neighbourhood of a given cell (roughly N/4, with Gaussian spatial decay of probabilities of synapses). Here, topography is meant to represent orientation tuning, but note that a spatial interpretation is also possible. Embedded within the network of excitatory cells are inhibitory interneurones, which receive input from excitatory cells in a neighbourhood (of roughly N/8) and inhibit the excitatory cells accordingly. Inhibitory cells in the model have graded responses, and represent local pools of interneurones. All synaptic response functions have transmission delays of 1ms, a rise time of t,. and fall time of tf, where tr=1ms, tf =3ms for excitatory and tr=2ms, tf=5ms for inhibitory synapses. Axonal conduction delays are not included, since we only consider a localized patch of cortex. An external “bar” or “grating”-stimulus will excite cells in a local cortical network in different ways, depending strongly on their orientation (and maybe on other properties). This is represented as an external input current into the excitatory cells, which is centred at some “orientation” (say neurone N/2), and falls off in a Gaussian way. Furthermore each neurone also receives a certain amount of white noise background activity, corresponding to spontaneous spiking of background cells. Figure 2a displays the activation dynamics of our network for a “bar” stimulus, as

Time and the brain

298

explained above. Neurone N/2=64 receives the strongest input (e.g. is well tuned to the bar), and the input strength decays for smaller and larger neurone numbers (e.g. to less well-tuned cells). Shown as “LFP” in the figure is the ensemble average over excitatory postsynaptic potentials, averaged over all cells. This corresponds roughly to the physiologically derived local field potentials (LFP). In addition the spike trains of all excitatory cells are displayed as a raster plot over time (SUA, single unit activity). The LFP signal clearly shows oscillatory activity, although in a waxing and waning manner similar to physiological recordings. Diamonds above the LFP indicate spike-times of an arbitrary neurone (here neurone number 49, also indicated by the dashed horizontal line in the raster plot). Apparently the neurone is not rhythmic, but emits spikes only in some periods of the collective rhythm. A closer investigation of all single unit spike trains reveals that this is the generic case in our model. No neurone fires periodically. In particular we find the same categories of cells as reported by Eckhorn and Obermüller (1993) for experimental data: “locked” cells, whose spikes display auto-correlation histograms (ACH’s) with oscillatory side-peaks and which are also correlated with the collective rhythm, “lock-in” cells, which are coupled to the LFP but are not rhythmic by themselves (as revealed by the flatness of their ACH), and “non-participating” cells that are neither rhythmic nor coupled to the global oscillation, although they are driven by the stimulus and fire significantly! (see also the chapter by Eckhorn, this volume [especially Figure 1]).

Cell assemblies, associative memory and temporal structure in brain signals

Figure 2. Activation dynamics of a local cell assembly in response to a bar stimulus, a) Local field potentials (LFP; computed as a low-passed version of the total number of spikes per millisecond) reveal clear oscillations in the gamma-range. The raster plot of single unit activity of the 128 excitatory neurones (SUA) nonetheless shows that cells are only loosely synchronized, b) Firing rates of all single neurones (dashed curve; average taken over five seconds) reveal “tuning” properties, but most cells fire at rates well below the frequency of the LFP oscillation (51 Hz). Thick line in b): further spatial average over seven nearest neighbours, c) On average sub-optimally stimulated cells fire later in each LFP-cycle than optimally-stimulated ones. Here, lag is measured relative to maxima of LFP-oscillation by spiketriggered averaging.

299

Time and the brain

300

As a consequence of this irregular firing, single cells in our model can be tuned, which means that they change their firing frequencies when the stimulus orientation (and the corresponding centre of the Gaussian shaped input) changes. This can be seen in Figure 2b, where the firing rate of all neurones is displayed (dashed line; rates are averaged over a total sample length of five seconds) together with a further average over local neighbourhoods (thick line, roughly equivalent to multiple unit activity, MUA). The horizontal line at 51/s indicates the “oscillation” frequency derived from the power spectrum peak of LFP. Almost all cells have lower rates in accordance with for instance experimental results by Kreiter and Singer (1996). Furthermore the shape of the rate function in Figure 2b reflects the Gaussian input strength, and this in turn reflects the single cell tuning; deviations from the Gaussian shape are mainly due to random lateral connections. Although tuning is one of the most basic properties of cortical neurones, it cannot be observed in many network simulations, which aim to explain “binding by synchronization”. Those often operate in parameter-regimes, where single cells (or other kinds of simulated “units”) have very similar firing frequencies, equal to the oscillation frequency of the collective rhythm, and almost perfectly synchronize, when they are sufficiently strongly connected. As a consequence “tuning curves” are flat in the synchronized regime. In contrast to this “tight-binding” situation, in our simulations synchronization is only rather loose. Nonetheless, we should mention another property well in accordance with experimental results: König et al. (1995a) investigated precisely how sub-optimally driven cells lock into the collective rhythm. Measuring peak-shifts in cross-correlograms between two sites with orientation preference (ILM)1 and (ILM)2 and varying the orientation $ of a stimulus, they found, that sub-optimal cells reveal a systematic phaselag relative to optimally stimulated cells. This depends linearly on the stimulus orientation (ILM) and is, in addition, proportional to the difference (ILM)2–(ILM)1 between the preferred orientations of the recorded cells. This implies that the lag of cells with a particular orientation ψ relative to the best matching cells (or alternatively to some reference oscillation like the LFP in our simulation) must depend quadratically on ψ. This is clearly seen in Figure 2c, where phase-lags obtained from fitted peaks of spike triggered averages of the LFP signal are displayed together with a quadratic fit of the lags estimated from the same 5 second data set as before. Of course, the strong scatter in Figure 2a shows that, at least in our simulations, this lag is only an average property of cell firing, and by no means implies deterministic delays between firing times of different neurones. What actually happens in our network is that the best matching cells on average fire more often. They can ignite waves of activation spreading to less excited neurones, which have not yet fired. Therefore, these cells fire later. This may be used as a coding principle: the later a cell fires the less directly is it supported by the external stimulus, but instead may reveal properties of the cortical connectivity matrix. The inhibitory interneurones recognize the increase in activity and suppress further firing for some milliseconds. Due to different sources of randomness, these cycles of excitatory amplification and subsequent inhibition appear to be rather imprecise in amplitude and period duration. Nevertheless they reveal elementary properties, which cannot be explained by tight-binding theories. Two points are particularly important: First, there is a

Cell assemblies, associative memory and temporal structure in brain signals

301

pronounced global gamma-oscillation, but single cells show a broad spectrum of typically slower firing rates, and are coupled only loosely into the global rhythm. Second, in response to different stimulus conditions cells show systematic shifts in their firing times relative to other cells or local field potentials. These shifts, however, are observable only in cross-correlograms computed over long times; within single gamma-periods the firing of cells appears to be unreliable and the relative timing imprecise. After some reflection this suggests that it is less the “phase coupling” of oscillatory cortical activity in the gamma range that matters in our simulations, than the temporal synchronicity and fine spike timing shifts of sets of similarly tuned cells within single periods. Sets of cells are required for reasons of proper signalling since the single units fire unreliably. The oscillation appears as a by-product of repeated local processes, characterized by a fast spread of activity in excitatory subnetworks, followed by a subsequent inhibition phase, which is somewhat delayed, because inhibitory potentials are typically slower than excitatory ones, and the excitatory activity is itself needed to evoke the inhibition. Properties of the collective rhythm—amplitudes, frequencies—are rather imprecise. Therefore, it seems unlikely that they code for particular stimulus features. Similarly, due to the strong fluctuations, correlations decay quickly in time— typically exponentially on a time scale of several tens to less than 100 milliseconds—just as is found in experiments. This supports the idea that only short epochs of the signals are relevant for information processing, and long coherent wave trains (as used for example, in holography) are not necessary. In this context experiments by König et al. (1995b) are quite interesting: they show, that for electrode distances above roughly two millimetres (within or between areas) cross-correlograms have, almost exclusively oscillatory sidebands, whereas at shorter distances those can be completely missing. It seems that distant sites require (at least) two gamma-periods for effective mutual interactions. 2.2. Pattern Completion and Gamma-oscillations The previous section considered local assemblies in primary sensory areas. Higher association areas are usually small. Therefore, we may assume that those areas can be modelled reasonably well as fully connected associative memories (Palm, 1982). The above interpretation of gamma-oscillations viewed as fast excitatory amplification processes followed by inhibited phases also applies in this case. Here, the spread of activation proceeds from externally driven neurones (the address pattern) to associated neurones of the stored assembly or memory pattern. This may involve one or more steps of synaptic transmission, which might be identified with iterative feedback steps in the associative memory model. However, the growing inhibition suppresses the activation after a short time, interrupts the retrieval process, and restarts it after a short relaxation phase. This always keeps the network sensitive to changes in the input signal. Now, it is most notable that Schwenker et al. (1996) have shown that iterative retrieval in sparsely coded associative memories is extremely fast, provided that firing thresholds are adapted to the network activity at each step; then at most three feedback steps (and most often only one or two) suffice for perfect retrieval. In the current context this means the following: taking a few ms for a single associative feedback step in the cortex, which

Time and the brain

302

is determined by synaptic and axonal delays, perfect pattern completion can be performed in less than about 10ms, which well fits into the observable gamma-periods. Furthermore the work of Schwenker et al. (1996) shows that pattern completion is most efficient (in terms of storage capacity) when the number of ones in the address pattern is about half that of the stored patterns. In practice this means that the active input synapses to any relevant neurone that is about to be “addressed”, should not be less than half the synapses that could be activated by the complete pattern. This implies further, that spikes must be synchronous if efficiency is required. Since real neurones have an integration time of a few ms, all relevant information should be present during those short time intervals. These arguments show that the hypothesis that local information processing is essentially restricted to population bursts of cell pools in single gamma periods goes together very well with iterative retrieval in associative memories. Not only do synchronized spikes support a high memory capacity, but also frequencies in the gamma range are almost the fastest possible operation speed for rhythmic retrieval. A more thorough discussion of this and related topics can be found in Wennekers and Palm (1997).

Figure 3. Rhythmic associative retrieval in a fully connected memory, comprising 64 spiking neurones. Three input patterns (a-c)—coarse-

Cell assemblies, associative memory and temporal structure in brain signals

303

grained to a size of 8 times 8 bins—were used as input to the memory neurones. During a preceding learning phase, each pattern was individually presented to the network for 500 steps and learned, by means of a Hebbian coincidence rule (data not shown, cf. Figure 5). Test pattern (d) was applied during the displayed simulation run. Shown in e) are raster plots of only those neurones, which fire in response to the stimulus; these cells are ordered (as well as possible for three overlapping patterns) by their membership to the different patterns (roughly the upper third of units represents the triangle; middle part: square; lower part plus some cells above: circle). Observe the overall rhythmicity, but a non-periodic activation of single patterns. The mixed patterns in d) are segregated and completed, but not all three are phase-coded within single periods. Instead usually only one pattern is processed completely in every period.

Figure 3 shows a simulation of this general principle applied to associative retrieval in a higher area. Three different patterns, e.g. local assemblies, have been stored in a fully connected coupling matrix. Apart from this the network structure is very similar to that in Figure 2, but the time constants of synaptic potentials are somewhat different; therefore, the observed oscillation has a different frequency. Note that during the retrieval phase in Figure 3e single perfect patterns are recovered from the composite and incomplete input pattern (d) in a very short time. Usually only one pattern is retrieved per elementary associative process. Which pattern becomes amplified depends largely on the noise level, hence random segregation takes place, but not phase-segregation of different patterns within single periods. The “oscillation” itself has no direct functional significance except in keeping the network sensitive: as soon as a pattern has been retrieved it is suppressed by the somewhat delayed inhibitory response. Afterwards a new retrieval process can take place. This way long-lasting stationary firing states are avoided in favour of fast and more flexible recognition processes. Of course, a more orderly (apparently non-random) retrieval of the three patterns could be achieved, if we had added a further adaptation in every single cell with a much lower gain and a larger time constant (say 50 ms) as in the simulations of binocular rivalry by Fahle and Palm (1991). In that case, cells belonging to a pattern that previously fired most are suppressed most strongly, and the chance for the firing of a pattern increases with the time elapsed since it has fired the last time.

3. SYNFIRE CHAINS So far we dealt with temporal synchrony and gamma-oscillations, which often, although not necessarily, co-occur in primary visual areas. Abeles (1991) has described another type of spatiotemporal correlation in frontal areas, which (at first glance) is not characterized by synchronized firing of cells, but consist of precisely timed sets of spikes of one or several cells, with well-defined relative time-delays; those spike-patterns (repeating triplets, quadruplets, etc.)—also termed synfire activity in the sequel—occur

Time and the brain

304

significantly more often than it would be expected by chance, given the hypothesis that spike trains are independent Poisson processes (Abeles, 1991; Abeles et al., 1993b; see also Villa, this volume). The occurrence of synfire activity in frontal cortical areas is furthermore clearly correlated with behavioural events (Abeles et al., 1993b). This observation relates the phenomenon to cognitive processes, although the explicit relation is still a matter of discussion. In order to explain those precisely correlated spike events, which can extend over timescales of up to hundreds of milliseconds, Abeles (1991) introduced the concept of “synfire chains” (SFC’s). The main idea is that they arise from ordered sequences of synchronously firing pools of neurones, which iteratively excite other well-defined pools, whereby a chain of activation evolves and propagates through the network. This idea can be formalized within the framework of associative memories: to this end it suffices to envisage every single, synchronously firing pool of cells as a memory pattern and to store the whole set of linearly ordered (not necessarily non-overlapping) patterns, pair by pair, just as described in Figure 1. The contribution of Villa in this volume considers experimental and biophysical properties of synfire chains. We will discuss two theoretical aspects, which focus on the tentative role of synfire activity for cortical information processing. Both view synfire chains as storage elements: the first describes SFC’s as a long term store for learning, recognition and replay of spatiotemporal patterns (cf. also Wickelgren, 1969); the second takes them as a possible physical substrate for short term memory (STM). 3.1. Synfire Chains as Temporal Storage Elements As mentioned above, synfire chains can be interpreted as an extension of the standard associative memory, from static or structural patterns to spatiotemporal ones (cf. Palm, 1982, [Chapter 11]). The examples in this section are a consequent elaboration of this idea. In fact, the regeneration of ordered sequences of patterns has been demonstrated repeatedly in earlier SFC-models (Abeles et al., 1993a; Aertsen et al., 1996; Bienenstock, 1995). Nonetheless in this section we go a step farther, and show that SFC’s can be used not only to recover sequences, but also to learn time-patterns and recognize them in a fault-tolerant manner.

Cell assemblies, associative memory and temporal structure in brain signals

305

Figure 4. Synfire chains can store, recognize and replay spatiotemporal patterns, a) and b): Time patterns to learn are derived from simple line-drawings: in a) a spot moves along the curve r(t) with constant velocity; the two components of the velocity vector, proportional to dr(t) in b), provide inputs to each neurone in a synfire chain (SFC). c) displays learning of a trajectory: to the left a snapshot of the input space is shown and to the right the spike-raster of the synfire chain neurones. Learning in c) is Hebbian: movement (the spot) and SFC both start at t=0. Each time an SFC-neurone fires, it stores the actual input values dr(t) in its synapses. The finally-learned synaptic input matrix C is shown to the right, d) displays a two-fold replay of the trajectory by repeated activation of the SFC with two different gain values. Here, the previously learned synapses are used in reverse direction and control the movement (velocity vector) in input space, e) Recognition of a distorted input trajectory, f) Recognition fails for a different pattern. For explanations see text.

We demonstrate the main ideas in the form of an example. To this end, imagine an associative network of spiking neurones, linked by excitatory connections, as considered before. Inhibitory interneurones do no harm as long as the inhibition is not so strong that it prevents the stable propagation of synfire activity. Assume that a sequence of P patterns has been stored in linear order in the coupling matrix. For the sake of simplicity, we further replace each pattern by a single representative cell; hence, the synfire chain network can be thought of as consisting of P cells coupled in linear feedforward order. Some mathematical analysis reveals, that this structure can show stable propagation of activity, moving from the first to the last neurone in the chain, provided that some global

Time and the brain

306

threshold level is chosen appropriately (Wennekers and Palm, 1996). Now, we want to learn, recognize and replay a certain time-pattern a(t), where a(t) may have more than one component (for example m components). To do this, we assume that synaptic connections Cij, i=1…P, j=1..m exist from each component of a(t) to each neurone in the SFC. These synapses are used to store samples of the pattern a(t) at certain times provided by the ordered firing of the SFC neurones. Learning can proceed in a Hebbian way: Suppose that, by some mechanism, the learned pattern a(t) and the activation of the SFC (node 1) start simultaneously. Then, each time ti, a neurone of the SFC fires, it suffices to store the actual values a(ti) in the synapses of the respective neurone i. This works in a single trial (“one-shot” learning). Figure 4c displays an example of such a simulation (with P=30 and m=2). The spikeraster of the SFC-neurones is shown to the right. These spikes represent the postsynaptic part of the Hebbian learning rule. Generation of the (presynaptic) input time-signal a(t) requires further explanation: In principle arbitrary, sufficiently smooth signals can be used as input. Here, these are extracted from simple line drawings as indicated in Figure 4a/b. A spot (rectangle in Figure 4a) starts at time zero at some corner of the object, and travels along it with constant speed v. The velocity vector along the curve r(t) is taken as the two-dimensional input function a(t). Hence, every neurone in the SFC has two external inputs, which represent velocity in x and ydirections. When, during learning, neurone i fires at time ti, its synapses Ci1 and Ci2 are resulting from such a set respectively to dx(tj)/dt and dy(ti)/dt. The matrix learning process is shown to the right in Figure 4; obviously it represents the derivatives of the curve r(t) in 4a (white codes for movement in positive and black for movement in negative x or y direction).2 Pattern regeneration is shown in Figure 4d. Again the SFC has to operate in a stable manner, but now without external input. Instead, the formerly learned synapses are now interpreted as “output”-synapses controlling the movement in the output space. If a neurone in the SFC fires, its synapses determine the instantaneous velocity vector, with which a movement is performed. In Figure 4d the SFC is activated twice—only the second spike-raster is shown. By choosing different absolute starting positions and different (arbitrary) gain factors the previouslystored object is recovered in two sizes in the input space. Finally, Figures 4e and f display examples for pattern recognition. To this end the thresholds in the SFC-network must be high enough to avoid stable propagation of excitation 2. We should mention that the model is not intended as a concrete example for visuo-motor coordination or related tasks, although similarities might exist. It is intended to provide a purely abstract view, just as the standard associative memory at first is an abstract paradigm. Both models may be suited as building blocks for more concrete and complex networks, incorporating static as well as temporal properties of stored entities.

without a further external input. An additional temporal input into those neurones which are actually firing can then lead to complete recovery of the stored sequence, provided the input pattern matches the synaptic pattern of the actually firing neurones sufficiently closely. At any step, only the conjunction of the additional input and that from the

Cell assemblies, associative memory and temporal structure in brain signals

307

previously firing SFC-neurone should lead to the firing of the next cell. If at any position in the SFC, the stored and the externally applied patterns do not mach, the synfire chain in the recognition network dies out. An example for proper recognition of a distorted version of the stored pattern is shown in Figure 4e. Note that: 1) the different parameter settings lead to a faster SFC sequence than before (cf. Wennekers and Palm [1996] for speed control of SFC’s); 2) the distortions in the test pattern lead to slight fluctuations in the instantaneous speed of synfire propagation; and 3) the stored pattern effectively uses only 27 of the total of 30 cells in the SFC (cf. matrix C in Figure 4). Hence, under recognition conditions, the last few nodes do not fire in Figure 4e. Finally, Figure 4f shows failure of recognition for a completely different input pattern. The above interpretation of synfire activity explicitly takes account of temporal information stored in the network structure. Nonetheless, we should note, that it is not very likely that synfire networks of this kind provide a reasonable substrate for arbitrary time patterns which an individual might learn, for example, complex movement patterns. The main reason for this is that sequences longer than some hundred milliseconds need exceedingly large hardware resources (Bienenstock, 1995) and, furthermore, the resulting synfire structure is very inflexible. Motor control (for movements or speech, etc.) certainly needs more flexible, probably modular and hierarchically organized structures (cf. also Wickelgren, 1969). However, we believe that chains with roughly some 100 nodes and perhaps 100 neurones per node may be useful storage devices for elementary “spatiotemporal features” in such architectures: the complete information can still be retrieved in a short time, and a time-span of 200 to 500 ms would indeed make sense, since this is roughly the duration of syllables or morphemes, which organize speechproduction; similar timing intervals have also been proposed to organize other cognitive tasks (see Gibbon and Allan [1984] for a collection of related articles). 3.2. Synfire Chains and Short Term Memory A primary function of the classical associative memory and its variants is that of a content-addressable storage device. Since information is laid down in synapses, this function may be identified as long term storage (LTM). A second function—possible in recurrent architectures—relates to the short term storage of information: because memory patterns present attractors of the network dynamics (at least in the classical model, see below) it is possible to keep them activated, even if the initial activating stimulus or address pattern has already vanished. Different patterns or assemblies, stored permanently in the coupling matrix, may become selectively excited by appropriate stimuli. After this, persistent firing of the related assemblies represents the information that the particular stimulus previously occurred in the current behavioural context, either as an external event or internal “idea”. This aspect of short term storage in associative memories has recently been reviewed by Amit (1995). In the light of the interpretation of cortical gamma-oscillations (outlined earlier) as rhythmic fast associative processes followed by a period of inhibition, we face some problems. One of the main points of this interpretation is that the retrieval state is destroyed almost immediately after it has been reached: it lasts just a few milliseconds,

Time and the brain

308

long enough to get signalled to target structures. Thereby we avoid the usually rather stable attractor states (which are hence hardly perturbable) and thus permit very fast responses to environmental changes. This model obviously cannot serve directly as an STM device, because it is not clear how the activation can be transferred from one synchronous population burst to the next in the absence of sufficient external stimulation, without the assumption of exceedingly long excitatory post-synaptic potentials, or other facilitating mechanisms on the time-scale of some ten ms (cf. Bibbig and Wennekers, 1996). STM in associative memories has been demonstrated repeatedly in networks of timediscrete noisy or noise-free threshold neurones (Little, 1974; Palm, 1982; Amit, 1995). Also, in networks of more realistic spiking neurones, persisting attractor states can be stable (Abbott and van Vreeswijk, 1993; Gerstner and van Hemmen, 1993). In these models STM is possible because the retrieval states utilize asynchronous firing of cells, which can be reached by appropriate choices of network parameters. However, long-time correlations of the synfire type in states of persisting activation are not possible in such networks. It is known that, in networks of noisy threshold neurones, spikes of different cells in different time-steps become uncorrelated when the system size gets large. This is because the relative contribution of any single neurone to the potential of a target neurone becomes vanishingly small in comparison with the total input (cf. Wennekers and Pasemann, 1996, and references therein). On the contrary, in networks of spiking neurones with refractoriness (as far as demonstrated to date) cells in persistent attractors fire more or less periodically (depending on the noise level). This can lead to long-time correlations, but of a different kind in comparison to synfire activity. In fact, only relatively few cortical cells, especially in prefrontal areas, fire periodically. It may well be that different areas of the cortex operate in different parameter regimes, such that association areas subserve STM functions with low rate firing, weak synchronization and synfire activity, whereas sensory areas show gamma-activity with the possibility of quickly changing patterns. But it is also not unreasonable to assume that all three phenomena—cortical gamma, synfire activity and STM—co-exist in one and the same local network. This has not yet been shown explicitly in a single experimental setup, but gamma activity is known to be a rather prominent rhythm in many cortical structures (Steriade et al., 1996; Gray, 1994) and synfire activity has also been demonstrated in the visual cortex and auditory thalamus (Lestienne and Strehler, 1987; Villa and Abeles, 1990). To date, the least evidence has been found for physiological signs of STM in reverberating loops of activity (see perhaps Miyashita and Chang, 1988; Fuster and Jervey, 1980; Fuster, 1989), but if this concept of short term storage is biologically relevant at all, it somehow should be expected to exist at least in wide parts of the association cortices. None of the above cited models—including our own—can explain gamma-oscillations, persistent activity and synfire patterns at the same time without modifications; hence we may ask whether or not there exists a unified view. To this end synfire chain models seem to be a natural starting point. It is clear, that they can serve as a temporal short term store. Bienenstock (1995) argues, that the control of activity in such networks may even be easier than in more homogeneously connected networks. Whether or not synfire chains are also consistent with gamma-oscillations has been investigated by Abeles et al. (1993a). Their simulation studies show that oscillatory

Cell assemblies, associative memory and temporal structure in brain signals

309

behaviour can indeed be found in two mutually connected networks of reverberating synfire chain models. Our own preliminary investigations of a single reverberating SFCmodel with global inhibitory activity control (similar to the networks already described above) show oscillations only near stability boundaries of stable propagation of synfire waves. Here, the mechanism is essentially an instability of the inhibitory control loop while suppressing randomly firing cells that are uncorrelated with the synfire activity. A second integrative scenario, perhaps with more physiological significance, assumes, that Hebbian cell assemblies are distributed over wide parts of a single or several cortical areas. In such a framework it is possible that rhythmic associative processes may occur locally by the same mechanism and with the same interpretation as discussed above. However, to bridge the inhibitory phases, and to obtain short term memory, it is necessary that local synchronously firing pools of cells excite other cell groups at sufficiently large distances. The locally synchronous activity, together with the patchy structure of cortical long-range connections support this hypothesis, in as much as the high convergence of activity in target columns under the influence of synaptic plasticity should lead cell pools in those columns to be excited in a rather specific and reliable manner, similar to synfire nodes (Sommer et al., 1998). Finally a sufficiently distributed ensemble of such mutually connected patches should lead to persistent reverberating synfire-type activity in conjunction with local gamma-oscillations (which in this case are not necessarily globally synchronized; see our discussion in Wennekers and Palm, 1997). Furthermore, the results in Sommer et al. (1998) indicate that associative modules, connected and operating in a bi-directional manner, provide a means for associative storage that can be much more efficient and advantageous than single localized autoassociative memories.

4. GLOBAL ASSEMBLIES At the end of the last section we arrived at the concept of “global cell assemblies”. Those widely distributed assemblies may include sub-assemblies in different sensory modalities as well as in higher association areas. Therefore, they may serve as representational schemes for virtually any kind of entity including things, situations, contexts, concepts, etc. However, in some sense the concept of global assemblies presents only a generalization of localized associative memories, to networks of such networks, where the “super-network” may itself reveal associative properties. This still holds, if we include spatiotemporal features as exemplified in Figure 4, although the possible physical modes of temporal behaviour will certainly become very complex in that case. Nonetheless, the main purpose is intrinsically that of representation and association of patterns of excitation, and less that of cognitive reasoning, planning and complex behaviour in time. In this section we outline some ideas related to the latter topics.

Time and the brain

310

4.1. Cortico-hippocampal Interplay In a series of computational studies we investigated global cell assemblies in some detail (Palm, 1993; Bibbig et al., 1995, 1996; Bibbig and Wennekers, 1996; Wennekers and Palm, 1997; Sommer et al., 1998) In particular we were interested in the role of the hippocampal formation during memory consolidation and retrieval of information in networks of higher neocortical associative areas (see also Chapter 8 by Klimesch, this volume; Klimesch, 1994; Miller, 1991). Two sensory pathways, say visual and acoustic, were considered in our model, which both consisted of a primary “pre-processing” area and a higher uni-modal “associative” area. The higher, more central areas, A1 and A2, were bi-directionally connected with a further associative structure, supposedly the hippocampus H (more generally, this could also be a higher cortical association area, maybe a “convergence zone” in the sense of Damasio, 1989). The model structure of the individual sub-networks A1, A2 and H was virtually the same as that considered in earlier sections. External inputs to the sensory streams were static, simple geometrical line-figures for the visual and abstractions of tones or frequency combinations for the acoustic branch. Several inputs could be applied in different combinations. The “pre-processing” areas mainly transformed a specific unimodal input into a local representation in the respective uni-modal association cortex, A1 or A2. Spatiotemporal properties of these peripheral parts of the model are similar to those of the local assemblies described above (cf. also Wennekers and Palm, 1997; and Bibbig et al., 1995 for details). Therefore, we focus on the combined behaviour of the neocortical memories and the hippocampus in what follows. It is known, that activity in higher association areas is often rather sparse. Furthermore, those areas are usually not densely connected with each other. Estimates by Palm (1993) suggest, that the probability of synchronous pre- and postsynaptic activation sufficient for Hebbian strengthening of a synapse is only very small. Without further supporting input, learning of local assemblies and global assemblies including those areas seems unlikely. Because the hippocampus receives input from virtually all higher neocortical areas, and can in turn also influence them, we have hypothesized that this brain structure plays a key role in the learning of global assemblies by providing such support (Palm, 1993). Even if it sends out unspecific activity to the higher areas, probabilities of coincidences increase steeply, because the neuronal threshold process is highly non-linear. Furthermore, since the hippocampus is itself an associative structure, which at any moment receives a reduced but global picture of the ongoing neocortical activation, the hippocampal activity will by no means be random, but may organize into reduced representations, say specific local assemblies or chunks, which store the information about the conjunctive occurrence of subevents in different areas of associative cortex (Wickelgren, 1992). Hebbian strengthening of feedback connections from hippocampal to neocortical local assemblies then may lead to stable representations of global cortical states in cortico-hippocampal loops (cf. Miller, 1991). This may be the basis for more complex hierarchical representations of ideas and concepts that are less clearly related to sensory impressions. Figure 5 displays a computer simulation of this process. A1, A2 and “Central” reveal (respectively) raster plots of spikes in two neocortical areas and the hippocampus. The

Cell assemblies, associative memory and temporal structure in brain signals

311

above two black lines indicate that certain input stimuli are supplied to each of the sensory pathways during the whole simulation run. From step 0 to 500 these lead to apparently random firing of a fraction (here 50%) of the cells in both areas A1, A2. The curve “Learning” measures cumulative changes in synaptic efficacies due to synaptic plasticity (using the Hebbian coincidence rule) in A1 (solid) and A2 (dashed). Obviously learning is very slow during the first 450 steps of the simulation. Then, the hippocampal

Figure 5. Hippocampal support of learning of a global cell assembly. Two associative areas/memories A1 and A2 (say visual and acoustic) are bidirectionally connected with the hippocampus (“Central”); this is further controlled in the theta range by the septal pacemaker. Input patterns presented to A1 and A2 evoke apparently random firing of cells as long as the pacemaker does not drive hippocampal cells (steps 0 to 400). Therefore, spike coincidences and learning rates are small (solid learning curve: A1, dashed: A2; displayed are accumulative changes in overall synaptic efficacies). Activation of the septal pacemaker supports selective firing of some hippocampal cells, which represent the current stimulus combination. These spikes propagate back to A1 and A2 and organise the activity in these areas into synchronized population bursts accompanied by a strong increase in learning rates and oscillatory activity (similar to Figure 3). In this way, local assemblies are consolidated in A1 and A2 and the global contextual information is stored in the cortico-hippocampal loop.

Time and the brain

312

area receives some further pacemaker signal, which in real brains may be supplied by the septal region. This signal in the theta-frequency range serves as a rhythmic threshold control for the hippocampus and—together with input from A1, A2—leads to spikes of some hippocampal cells, which, after being transferred back to the neocortical areas, start, almost immediately, to organize the activity in those areas into synchronized population bursts accompanied by a significant increase in learning rates. After several theta periods synaptic efficacies saturate at some maximum level, and the neocortical areas reveal a pronounced gamma-rhythm similar to that in Figures 2 and 3.3 3. Due to technical constraints (limited computer time) the time-scales of gamma and theta rhythms and probably also the learning rates are not realistic.

After learning the hippocampal formation binds distributed sets of local assemblies into global ones, which can now be retrieved from only partial information in a single sensory stream: first, the respective local assembly is completed in its unimodal association area, and then—via the cortico-hippocampo-cortical loops—the hippocampal contextual chunk is reactivated; thereby it addresses and restores the complete, widelydistributed information in neocortex (Miller, 1991; Wickelgren, 1992; Bibbig et al., 1995). We have also demonstrated, that in the case of learning of very many contextual situations, generalization across stimulus properties can take place in the hippocampus, with a qualitative change of its influence from supporting highly specific chunks, to a more coarse-grained threshold control (Bibbig et al., 1996). Finally, we should emphasize that the neocortical areas themselves were not mutually connected in these simulations. Obviously such connections would also become consolidated during learning, with the consequence that, at the end, cross-modal retrieval may rely solely on neocortical interactions as in the work of Sommer et al. (1998). 4.2. Associative Brain Models Several brain theories based on assembly- or associative memory models as outlined above have been devised (Hebb, 1949; Braitenberg, 1978; Wickelgren, 1981; Palm, 1982; Damasio, 1989; McGregor, 1993; and others). Interestingly, within such general frameworks, the question whether persistent assemblies are supported in the brain by stochastic or correlated activity has also been discussed in early work (e.g. Hebb, 1949; Eccles, 1958). McGregor (1993) called the two alternatives “stochastic” and “sequential configuration hypothesis” and outlined a theory of the neocortex, according to which entities are represented locally by “sequential configurations” or “dynamical modes” very similar to reverberating synfire chains in local networks. Similar modes in different local modules can support each other by long-range connections (the “super-network”, see above), thereby leading to specific collective excitation patterns, distributed over almost the whole cerebral cortex. Unfortunately the theory has not yet been developed far enough to suggest a functional role for synfire activity or temporal structure in brain signals. Most effects in McGregor’s paper also occur, for example, in coupled associative modules with exclusively static attractor states or “modes”.

Cell assemblies, associative memory and temporal structure in brain signals

313

Bienenstock (1995) is more explicit with regard to the role of “sequential configurations”. He proposed a model of the neocortex, focusing on “cognitive compositionality”.. This takes synfire chains as basic, quasi-atomic, functional elements or excitation modes, which are not necessarily restricted to local modules as in McGregor’s work. According to Bienenstock, elementary or “narrow” chains themselves carry very little information, but can aggregate or synchronize with each other by interactions via mutual, relatively weak, and plastic synapses. Only conjunctive “broad chains”, which are composed of flexible sets of synchronized narrow chains carry meaningful information. Those broad chains are again thought of as being distributed over large parts of the cortex. In this way, an almost infinite variety of complex excitation patterns seems possible, an argument, which has also been put forward by Palm (1982) as an advantage of earlier assembly theories in favour of theories based on grandmother neurones. In some sense Bienenstock’s model is also essentially a hierarchical associative memory model; the difference from earlier theories is, that he replaces stationary attractors by SFC’s, and includes ideas concerning “binding by synchronization”, now applied to coupled SFC’s and not coupled oscillators. 4.3. Towards Cognitive Operations Experiments performed by Abeles et al. (1993b) on trained, awake monkeys show, that synfire activity in frontal cortical areas is correlated with behavioural events. Those areas are known to contribute to short term and temporal memory, problem solving and planning in complex behavioural tasks (Fuster, 1989). Therefore, it seems reasonable to ask how cognitive capabilities might be included in assembly theories, and what particular role synfire activity may play in this context. Simple stimulus-response schemes may be easily explained: it suffices that an assembly that represents a particular stimulus is learned during training, and triggers the corresponding response in the case that it becomes activated in subsequent tasks. In this way, conjunctions or other logical combinations of different simultaneous stimuli may also become associated with a certain response. Those mappings would implement basic “rules” of reactive behaviour. Time-order relations between cues may still be represented statically in the form of assemblies “A” and “B” for events, and other assemblies “AB” and “BA” for their occurrence in different temporal order. The latter assemblies may then trigger the desired response. Nonetheless, we face the problem how “AB” is excited from stimuli A and B. Possible solutions require short term storage of the first stimulus and subsequently the excitation of assembly “AB” by both the external input B as well as the internal representation of A. In general terms there is a requirement both for short term memory of the temporal stimulus context, and operations on (or influenced by) those internal representations. Clearly, both these aspects exceed the simple stimulus-response scheme and are central to any theoretical framework of “cognition”. Our assembly theory is well suited for representional purposes, but still lacks elaborated operational components. This should not be a serious problem, in principle, because arbitrary finite automata can be built from very simple neural networks (McCulloch and Pitts, 1943; von Neumann,

Time and the brain

314

1958). Therefore, a general and rough picture for cognitive operations based on assemblies is developed in what follows: First, we assume that external stimuli or events relevant to a certain task are represented in the form of assemblies, which comprise local sub-assemblies in frontal areas; those may have short-term storage properties and reveal synfire type activity as discussed earlier. This set of assemblies or “memory chains”, perhaps together with further ones for the storage of internal events (like “AB”, “BA”, see above), can be viewed as a set of “(logical) variables” representing entities relevant to the particular experimental situation. Storage assemblies can be activated or silent; they represent propositions about the experimental situation. For example, activation of assembly “AB” means that event A has occurred and afterwards also B; its inactivation implies that this is not true. Second, we assume the existence of control structures, which are able to perform operations on memory chains (e.g. the activation of “AB” if “A” is already active and B occurs). Those structures may implement logical and/or procedural knowledge; they present neuronal “programs”. Interestingly, such structures can also be implemented by synfire chain networks. It turns out that a simple modification leads from reverberating synfire chains to models of spiking neurones able to implement arbitrary finite state automata. All that is required are several possible successor states at any node of the chain, which are activated not only by the firing of predecessors, but also in dependence of specific input-patterns. In this way, complex graph-like synfire structures can be built, which implement the desired control components. Third, memory and control networks are assumed to interact with each other and also with further input/output structures like sensory-motor or higher associative areas: Firing of specific nodes in the “program”-network may trigger elementary motor patterns, which are perhaps themselves represented by synfire chain-like activity as discussed in section 3. They also can induce sampling of new input by threshold control in appropriate sensory areas. More locally, firing of program-nodes may excite or inhibit activity in some local memory chains, or gate the transfer of activity from one storage element to another. On the other hand, the flow of activity in the program network can be influenced by the actual state of one or more of the memory chains as well. Also external input may induce conditioned transitions in the program-network, such that different behavioural procedures can be performed in different situations in time. The above assumptions suffice to model tasks of arbitrary complexity. For example, we have implemented a network of spiking neurones that is capable of performing arithmetic based on generalized synfire chains as described above (Wennekers, 1998). Similarly other behavioural or cognitive tasks can also be realized. In many behavioural experiments, monkeys need a rather long time to achieve the skills required for good performance. It is plausible to assume that the structures for storage and processing (the “cognitive modules”) are acquired during this training phase and are essentially fixed afterwards. This suggests that these structures presumably are quite specific, that is, built and usable for the particular task only. With respect to local assemblies in frontal areas this would mean that those are not “universal variables”, but can only represent specific external events, expressed by their connectivity pattern to a corresponding global assembly. Now it seems possible that repeated training of the same

Cell assemblies, associative memory and temporal structure in brain signals

315

task, but with many different, interchangable stimuli may lead to some kind of generalisation across stimuli, expressed, for example, as a dissociation of the local assembly from particular features of the stimuli but with remaining connections to general stimulus properties. In that case the “cognitive module” loses the ability to refer back to a specific actual global assembly; it becomes more universal by generalisation across stimulus properties. A similar situation occurs also in delayed-match-to-sample experiments, because the sample changes from trial to trial and may even be completely new to the animal. Hence, there might not even exist a fixed global representation for it. Both cases are characterized by the existence of local frontal assemblies which do not correspond to fixed and unique global assemblies. Therefore we need some mechanism that transiently binds the local storage assembly, which is part of the well-trained cognitive module responsible for the general class of tasks, to the respective representations of the actual sample stimuli. Apparently, the mechanism of memory formation based on corticohippocampal interplay (see Section 4.1) is able to solve this problem. It is interesting in this context that tasks which require the continuous learning of new representations appear to be in some sense “more difficult” than those which solely rely on fixed sets of stimuli (cf. Klimesch, 1994; Mishkin et al., 1984). The latter experimental paradigms may only involve associations along neocortical pathways, which have been strengthened during previous training by repeated presentation of the fixed stimulus set; this should lead to fast and reliable recognition under test conditions. On the other hand, the former paradigm requires a more complex architecture and the acquisition of a new global representation from a single stimulus; hence the recognition may be less reliable and slower.

5. CONCLUSIONS In summary, we have developed a rough picture of cortical function based on cell assemblies incorporating the time-structure in neural signals as well as operational components. Local information processing has been characterized by fast recurrent associative amplification processes serving feature binding and “Gestalf”-principles in sensory areas, and pattern recognition or segregation in higher association areas. Local assemblies in a modular architecture can be integrated into global ones by means of numerous uni- and bi-directional synaptic pathways. The formation of such assemblies— that is, the integration of information from different modalities and other internal sources—as well as their consolidation and cross-modal retrieval can well be supported by cortico-hippocampal loops. Synfire chains, i.e. sequential synchronized activity along specific synaptic pathways, have further been shown to provide a basis for short term memory as well as the storage of spatiotemporal features of internal or external events. Finally, by extending the synfire chain concept to “synfire graphs”, the controlled interaction of assemblies distributed over many cortical areas can serve as a basis for operational short term memory, in principal organizing behavioural responses of arbitrary complexity.

Time and the brain

316

ACKNOWLEDGEMENT We are grateful to Andrea Bibbig, Reinhard Eckhorn and Axel Frien for valuable discussions, and to Robert Miller for many comments that led to improvements of the manuscript. This work has been supported by the German Research Foundation DFG (grants Pa 268/8–1 and 268/12–1).

REFERENCES Abbott, L.F., and van Vreeswijk, C. (1993) Asynchronous states in networks of pulsecoupled oscillators. Physical Review E, 48, 1483-1490. Abeles, M. (1991) Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Abeles, M., Vaadia, E., Bergman, H., Prut, Y., Headman, I., and Slovin, H. (1993a) Dynamics of Neuronal Interactions in the Frontal Cortex of Behaving Monkeys. Concepts in Neuroscience, 4, 131-158. Abeles, M., Bergman, H.Margalit, E., and Vaadia, E. (1993b) Spatio-temporal firing patterns in frontal cortex of behaving monkeys. Journal of Neurophysiology, 70, 16291643. Aertsen, A., Diesmann, M., and Gewaltig, M.O. (1996) Propagation of synchronous spiking activity in feedforward neural networks. Journal of Physiology (Paris), 90, 243-247. Amit, D.J. (1995) The Hebbian Paradigm Reintegrated: Local Reverberations as Internal Representations. Behavioural and Brain Sciences. 18, 617-657. Bibbig, A., Wennekers, Th., and Palm, G. (1995) A Neural Network Model of the Cortico-Hippocampal Interplay and the Representation of Contexts. Behavioural and Brain Research, 66, 169–175. Bibbig, A., Wennekers, Th., and Palm, G. (1996) A Neural Network Model of the Cortico-Hippocampal Interplay: Contexts and Generalization. Engineering Applications of Artificial Intelligence, 9, 145–151. Bibbig, A., and Wennekers, Th. (1996) Hippocampal two-stage Learning and Memory Consolidation. In: Proceedings of the 13th European Meeting on Cybernetics and Systems Research, Vienna, 1996. Bienenstock, E. (1995) A model of the neocortex. Network, 6, 179–224. Braitenberg, V. (1978) Cell Assemblies in the Cerebral Cortex. In: R.Heim and G.Palm (eds) Theoretical Approaches to Complex Systems, Berlin: Springer, pp. 171–188. Braitenberg, V., and Schüz, A. (1991) Anatomy of the cortex. Berlin: Springer. Damasio, A.R. (1989) Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition. Cognition, 33, 25-62. Eccles, J.C. (1958) The physiology of imagination. Scientific American, 199, 135-146. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H.J. (1988) Coherent oscillations: A mechanism of feature linking in the visual cortex? Biological Cybernetics, 60, 121–130. Eckhorn, R., and Obermüller, A. (1993) Single Neurons are Differently Involved in Stimulus-Specific Oscillations in Cat Visual Cortex. Experimental Brain Research, 95,

Cell assemblies, associative memory and temporal structure in brain signals

317

177–182. Fahle, M., and Palm, G. (1991) Perceptual rivalry between illusory and real contours. Biological Cybernetics, 66, 1–8. Fuster, J.M. (1989) The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe. second edition. New York: Raven. Fuster, J.M., and Jervey J.P. (1980) Inferotemporal neurons distinguish and retain behaviourally relevant features of visual stimuli. Science, 212, 952–955. Gerstner, W., and v.Hemmen, J.L. (1993) Associative memory in a network of "spiking" neurons. Network, 3, 139–164. Gibbon, J., and Allan, L. (eds.) (1984) Timing and Time Perception. Special Issue of Annals of the New York Academy of Sciences, 423. Gray, C.M. (1994) Synchronous Oscillations in Neuronal Systems: Mechanisms and Functions. Journal of Computational Neuroscience, 1, 11–38. Hebb, D.O. (1949) The organization of behaviour. A neuropsychological theory. New York: Wiley. Klimesch, W. (1994) The structure of long-term memory: A connectivity model of semantic processing. Hillsdale, NJ: Lawrence Erlbaum. König, P., Engel, A.K., Roelfsema, P.R., and Singer, W. (1995a) How Precise is Neuronal Synchronization. Neural Computation, 7, 469–485. König, P., Engel, A.K., and Singer, W. (1995b) Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Science, U.S.A., 92, 290–294. Kreiter, A.K., and Singer, W. (1996) Stimulus-Dependent Synchronization of Neuronal Responses in the Visual Cortex of the Awake Macaque Monkey. Journal of Neurophysiology, 16, 2381–2396. Lestienne, R., and Strehler, B.L. (1987) Time structure and stimulus dependence of precisely replicating patterns present in monkey cortical neuronal spike trains. Brain Research, 437, 214–238. Little, W.A. (1974) The existence of persistent states in the brain. Mathematical Bioscience, 19, 101–120. McCulloch, W.S., and Pitts, W. (1943) A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115–133. MacGregor, R.J. (1993) Composite cortical networks of multimodal oscillators. Biological Cybernetics, 69, 243–255. Malsburg, C. von der (1981) The correlation theory of brain function. Internal Report 81–2. Max-Planck Institut für Biophysikalische Chemie, Göttingen. Miller, R. (1991) Cortico-Hippocampal Interplay and the Representation of Contexts in the Brain. Berlin: Springer. Mishkin, M., Malamut, B., and Bachevalier, J. (1984) Memories and habits: Two neural systems. In: G.Lynch, J.L.Gaugh, and N.M.Weinberger (eds) Neurobiology of Learning and Memory. New York: Guilford Press, pp. 65–77. Miyashita, Y., and Chang, H.S. (1988) Neural correlate of pictorial short term memory. Nature, London 331, 68–70. Neumann, J.von (1958) The computer and the brain. Yale University Press, New Haven, London. Palm, G. (1980) On associative memories. Biological Cybernetics, 36, 19–31. Palm G. (1982) Neural Assemblies. An Alternative Approach to Artificial Intelligence. Berlin: Springer. Palm, G. (1991) Memory Capacities of Local Rules for Synaptic Modification. Concepts

Time and the brain

318

in Neuroscience, 2, 97–128. Palm, G. (1993) Cell Assemblies, Coherence and Cortico-Hippocampal Interplay. Hippocampus, 3, 219–225. Schwenker F., Sommer F.T., and Palm G. (1996) Iterative Retrieval of Sparsely Coded Associative Memory Patterns. Neural Networks, 9, 445–455. Singer, W., and Gray, C.M. (1995) Visual feature integration and the temporal correlation hypotheses. Annual Review of Neuroscience, 18, 555–586. Sommer, F. T., Wennekers, T., and Palm, G. (1998) Bi-directional completion of cell assemblies in the cortex. To be published in: J.M. Bower (ed.) Computational Neuroscience: Trends in Research, Plenum Press, New York. Stein, R.B. (1967) Some Models of Neuronal Variability. Biophysical Journal, 7, 37–68. Steriade, M., Amzica, F., and Contreras, D. (1996) Synchronization of fast (30–40Hz) spontaneous cortical rhythms during brain activation. Journal of Neuroscience, 16, 392-417. Villa, A.E.P., and Abeles, M. (1990) Evidence for spatiotemporal firing patterns within the auditory thalamus of the cat. Brain Research, 509, 325–327. Wennekers, T. (1998) Synchronisation und Assoziation in Neuronalen Netzen. PhD Thesis in Computer Science, University of Ulm, Germany. Wennekers, Th., Sommer, F.T., and Palm, G. (1995) Iterative Retrieval in Associative Memories by Threshold Control of Different Neural Models. In: H.J.Hermann (ed) Proceedings of the Workshop on Supercomputers in Brain Research, World Scientific Publishing Company, Singapore. Wennekers, T., and Palm G. (1996) Controlling the Speed of Synfire Chains. In: C.von der Malsburg, et al. (eds) Proceedings of the International Conference on Artificial Neural Networks ICANN96. Berlin: Springer, pp. 451–456. Wennekers, T., and Palm G. (1997) On the relation between neural modelling and experimental neuroscience. Theory in Biosciences, 116, 273–289. Wennekers, T., and Pasemann F. (1996) Synchronous Chaos in High-Dimensional Modular Neural Networks. International Journal of Bifurcation and Chaos, 6, 2055– 2067. Wickelgren, W.A. (1969) Context-sensitive coding, associative memory, and serial order in (speech) behaviour. Psychological Review, 76, 1–15. Wickelgren, W.A. (1981) Human Learning and Memory. Annual Review of Psychology, 32, 21–52. Wickelgren, W. (1992) Webs, cell assemblies, and chunking in neural nets. Concepts in Neuroscience, 3, 1–53. Willshaw, D.J., Buneman, O.P., and Longuet-Higgins, H.C. (1969) Non-holographic associative memory. Nature. London. 222, 960.

11 The Relation between EEG and Evoked Potentials Erol Başar2, Sirel Karakas1, Elke Rahn1, and Martin Schürmann1 Institute of Physiology, Medical University Lübeck, Ratzeburger Allee 160, 23538 Lübeck, Germany Tel: (+49) 451/500–4170; FAX: (+49) 451/500–4171: e-mail: [email protected] 2TÜBITAK Brain Dynamics Research Unit, Ankara, Turkey

In order to analyze the relation between EEG and evoked potentials (EPs), we studied frontal visual EPs (VEPs) by means of a recently introduced algorithm for selective averaging. This algorithm is based on the inverse relationship between amplitudes of alpha or theta components of the spontaneous EEG activity and EP amplitudes. Stimuli were only applied if the root mean square (RMS) value of the ongoing EEG at the lead F4 was below an individual threshold level (“selective stimulation”). For this comparison, the EEG was filtered in one of the frequency ranges “alpha”, “theta” and “alpha & theta”, respectively. “Alpha- ” and “alpha & theta-dependent” selective stimulation conditions resulted in significant amplitude increases (p<.05) at the input reference channel F4 and partly at ipsihemispherical temporal and parietal leads and at Cz. The largest increase, of 35% at F4 (p<.01), was obtained with visual stimulation during low prestimulus theta activity. We conclude that spontaneous theta activity of the frontal cortex may be a factor influencing the amplitudes of frontal VEPs. A sophisticated analysis of frontal EPs, mainly in the framework of cognitive studies, should consider the theta activity prior to stimulation. KEYWORDS: VEP; Theta response; Frontal cortex; Prestimulus EEG; Selective stimulation

1. INTRODUCTION Interactions between prestimulus EEG parameters and the configuration and amplitude of evoked potentials (EPs) seem to attract increasing attention in recent years (cf. Arieli et al., 1996). Although recording of EPs and averaging them in the time domain usually does not take account of differences or changes in the ongoing EEG, there is strong

The relation between EEG and evoked potentials

321

evidence that the spontaneous activity immediately before an external stimulus has a considerable influence on the response obtained to that stimulus. Several parameters of the prestimulus EEG have been related to the amplitudes and latencies of components of evoked potentials, as well as of event-related potentials (McDonald, 1964; Jasiukaitis and Hakarem, 1988; Başar et al., 1989). Special emphasis has been laid on the influence of prestimulus spectral EEG patterns reflecting CNS levels of activation (Fruhstorfer and Bergström, 1969; Pritchard et al., 1985; Romani et al., 1988; Brandt et al., 1991; Brandt and Jansen, 1991) or the phase angle of the spontaneous activity at the time of stimulation (Jansen and Brandt, 1991). Other workers have studied the effects of the presence or absence of occipital alpha activity (Maras et al., 1990; Brandt et al., 1991) or have applied stimulation during defined levels of minimal background activity (McDonald, 1964). It is difficult to compare these studies, due to differences in the methodologies utilized for relating the prestimulus EEG characteristics to the evoked responses, but all of them have hinted at a consistent relationship between these two phenomena representing neuronal activity. Our own working hypothesis interprets EPs as stimulus-induced synchronization and enhancement of the spontaneous EEG activity. Accordingly, the compound EP/ERP arises from a superposition of evoked rhythmicities in several frequency channels which might bear different functional significance (Demiralp and Başar 1992; Başar-Eroğlu et al., 1992). For these frequency channels, e.g. alpha or theta, an inverse relationship between the root-mean-square (rms) voltage level of the prestimulus EEG and the maximal poststimulus EP-amplitude has been established by single-trial analyses in various experiments in human subjects and cats (Başar, 1980; Başar et al., 1989). It has been shown that spontaneous oscillations with smaller magnitudes can be enhanced more efficiently by stimulation signals than spontaneous oscillations with larger magnitudes. The existence of a synchronized EEG pattern reduces the probability of marked, timelocked responses upon sensory stimulation. Accordingly, the amplitudes of the background EEG are considered as one of the main factors influencing the evoked potential amplitudes. Evoked potential components are thus predictable from the EEG.

2. A NEW APPROACH: THE ALGORITHM FOR SELECTIVE AVERAGING The major aim of the studies to be described in the following was to evaluate EPs as a function of the spontaneous EEG. In contrast to the conventional method of simply averaging single epochs over a defined period of time, the new algorithm studies the responses to stimuli given under periods of low spontaneous EEG activity. • As a first step, the dependence of single responses on the prestimulus epoch was tested by averaging two subsets of single trials classified by prestimulus EEG voltage: the resulting EPs showed different amplitudes which were higher in the case of low prestimulus EEG activity (Başar et al., 1989). • As a second step, this kind of selective averaging was extended from an a posteriori to an a priori approach. In on-line experiments, stimulation was applied selectively

Time and the brain

322

during stages of low prestimulus activity. To achieve stimulation contingent upon the root mean square (rms) values of the amplitudes of spontaneous oscillations, a computer control system was developed in our laboratories. Spontaneous EEG activity at the vertex electrode—which was chosen as the input channel—was fed into a microprocessor able to quantify the rms values of the alpha and theta frequency bands. The rms-dependent averaged responses were then compared to conventionally evoked responses obtained during the same session. For this, two paradigms were used as schematically shown in Figure 1. (a) In standard EP recordings—designed as controls—the stimulus sequence was fixed a priori (random inter-stimulus interval [ISI]). That means, the rms values immediately preceding the stimuli were randomly distributed, varying from very low to rather high levels. (b) In the selective-stimulation paradigm, stimulation was EEG-dependent: on-line, and the rms values of either the alpha band (8–14 Hz), the theta band (4–8 Hz) or of a combination of both (4–14 Hz) were computed. The activity was checked to see whether it exceeded a predetermined rms level in a 1000 ms time period. If a stimulus was triggered, the next one could occur after a specified time interval which was identical to the mean ISI of the controls. Frequently, five or six stimuli occurred in series. All three frequency bands were used to control stimulus application, each with its respective control experiment. To test the influence of the different ISIs inherent in these paradigms, a second experiment was done using the given stimuli with “Zlong IS”s, imitating the irregular, often long, interstimulus intervals of the selective-stimulation visual EPs. However, in this experiment the stimulation was not rms-dependent (the procedure of comparing the rms value to a predetermined EEG level was not applied).

The relation between EEG and evoked potentials

323

Figure 1. Concept of selective stimulation: stimulation is triggered only during absence of highly synchronized alpha or theta EEG activity. Selective stimulation in a defined frequency band is supposed to result in a reduction in the number of stimuli required to produce a clearly recognizable, measurable EP (from Rahn, E. and Başar, E. [1993] International Journal of Neuroscience, 69, 207-220)

3. DEPENDENCE OF EP AMPLITUDES AND WAVEFORMS ON THE PRESTIMULUS EEG 3.1 Vertex Recordings 3.1.1. Auditory evoked potentials The new algorithm was applied first in the auditory modality, using the vertex as the input channel for rms evalution. This study confirmed the predicted effects of prestimulus EEG amplitudes on the subsequent auditory EP (AEP); selective stimulation increased EP amplitudes (N1-P2) by 30%–40% in comparison to conventional stimulation (Rahn and Başar 1993a).

Time and the brain

324

3.1.1.1. Difference between ISI-matched controls and standard controls Because we aimed to introduce a new algorithm for averaging of EPs by taking the prestimulus EEG states into consideration, the resulting EPs were compared to those obtained by the standard averaging method. The concept of selective averaging implies that interstimulus intervals diverged from the usual ones in two aspects: (1) they were longer on the average and (2) the stimuli were triggered in an irregular and unpredictable way. To what degree were the results reported affected by different ISIs? This question had to be answered before interpreting the differences between EEG-dependent and conventional EPs. It was seen that “long ISI” control AEPs, imitating the irregular, often long interstimulus intervals of the selective-stimulation AEPs (but without taking into account prestimulus amplitudes) depicted N1-P2 amplitudes that were 12% higher in comparison to the conventionally obtained averaged evoked potentials. 3.1.1.2. Broad band filtered data Figure 2 allows a direct comparison between selectively averaged AEPs and control AEPs. Superimposed evoked responses of 10 subjects according to the different experimental conditions are shown. Conventionally averaged EPs showed quite uniform amplitude values, with no tendency of amplitudes to decrease or increase during the course of the experimental session. Therefore only one randomly chosen control is displayed here. A clear global tendency to increase in amplitude in EPs in the prestimulus alpha- and theta-contingent condition compared to the conventionally-averaged EPs is seen. Although distinct, a minor effect was observed if both frequency bands were evaluated at the same time. The grand averages also depict clear amplitude differences between selective and conventional stimulus conditions. The percent gain in amplitude of the EPs of the selective-stimulation experiments differed for the three frequency bands: EPs with prestimulus amplitude restrictions in a single frequency band—alpha or theta—showed comparable mean increases in amplitude, 47% (p<.01) and 41% (p<.05), respectively. The EPs with low prestimulus activity in the broad range from 4–14 Hz did not differ significantly (a mean of 28%) from the control condition (Wilcoxon-Wilcox test). If the ISI correction is taken into account, alpha- or theta-dependent stimulation produce mean amplitude increases of 30%–35%, the third condition of nearly 20%.

The relation between EEG and evoked potentials

325

Figure 2. Superimposed individual auditory EPs (top) and grand averages (bottom) of all subjects under study (N=10), obtained during different experimental conditions of conventional or selective stimulation (from Rahn, E. and Başar, E. [1993] International Journal of Neuroscience, 69, 207–220)

3.1.1.3. Alpha and theta components For the filtered data, theta and alpha components are of nearly the same magnitude across all control experiments. On the other hand, the strength of alpha and theta responses, respectively, in the different selective-stimulation conditions was quite different. When stimulus application was conditional on low prestimulus alpha band activity, there was an increase not only in the alpha component, but also in the theta response. Vice versa, an alpha enhancement was observed in the trials contingent on theta band activity. Both components revealed strong resonant behaviour independent of prestimulus theta or alpha band-contingent stimulation.

Time and the brain

326

3.1.1.4. Single trial analysis Averaged EPs tended to show almost regular oscillatory waveforms of, for example, 6 or 9 Hz. Single responses tended to form similar patterns of damped oscillations in the same frequency range. Comparative analyses in all subjects revealed that, in the selectivestimulation conditions, the single sweeps often exhibited replicable patterns of timelocked, large-amplitude wave packets. In the control condition, the variability of single responses was higher. To quantify these observations on single trials giving small standard deviation, correlation coefficients between single trials and averaged EPs (both filtered in the 0.55– 30 Hz range) were evaluated within 300 ms after stimulus onset: median correlation coefficients for AEPs contingent on alpha were 0.66, on theta were 0.67, and on alpha and theta combined were 0.58, compared with 0.52 for the control condition. Significant differences were found between the alpha- and theta-contingent experimental EPs and the control EPs (p<0.01 and p<0.05 respectively; Wilcoxon-Wilcox test). We conclude that standardization of prestimulus EEG conditions by quantification of prestimulus frequency band activities seems to decrease the variability of single trials. 3.1.2. Visual evoked potentials The paradigms for the auditory modality were also used for visual stimulation, to test (i) whether similar amplitude enhancements occur in visual evoked potentials and, if so, (ii) to what extent these amplitude enhancements appear on several different recording sites. The results confirmed the findings in the auditory modality: we found a marked increase (about 35% at the vertex electrode) in bioelectrical activity due to the prestimulus EEG amplitude measurements. Moreover, this effect was observed around Cz in frontal, temporal and parietal sites (Rahn and Basar, 1993b). 3.1.2.1. Broad band filtered data Figure 3 shows the transient responses in both conditions for all 12 subjects under study, with the selective-stimulation visual EPs below, and the matching controls above. There is an obvious increase in the maximal peak-to-peak amplitudes in the poststimulus range if the stimulus application was contingent upon prestimulus activities. Each of the alpha, theta and alpha/theta band-contingent visual EPs depicted significant increases at the vertex. The visual responses to the rms-contingent stimuli at the vertex location showed marked increases, in the range of 35% in the amplitudes of the N1-P2 complexes, compared to the standard visual EPs. The interstimulus interval differences between controls and selective-stimulation visual EPs do not contribute appreciably to these amplitude changes. Standard visual EPs were recorded as before and compared to “long ISI” controls: Without taking into account any

The relation between EEG and evoked potentials

327

parameters of the EEG, just the interstimulus intervals of various selective-stimulation visual EPs were just simulated. At the vertex position, the amplitudes of the long ISI controls were 11% higher than those of the standard controls. Over the entire scalp there were only small differences in the range of 5 to 12%. 3.1.2.2. Alpha and theta components For further analysis, the time courses of the evoked signals were filtered digitally. The bandpass limits of the filters were selected according to the alpha and theta EEG bands, as done for prestimulus rms evaluation. The increases of the broad band filtered time series of around 35% were accompanied by increases of the alpha, the theta or the alpha/theta band, respectively. Both alpha and theta components increased with respect to the prestimulus alpha and theta amplitudes. The alpha amplitudes remained unchanged but the theta band increased after low prestimulus theta. With low alpha in the prestimulus period, the alpha band as well as the 7 Hz theta component increased substantially.

Figure 3. Superimposed averaged visual EPs of 12 subjects recorded at the vertex, filtered for 1–45 Hz. Standard visual EPs at the top and corresponding selective stimulation visual EPs (with low prestimulus amplitude in the frequency range indicated) below. Control parameters: prestimulus alpha activity (A); prestimulus theta activity (B); prestimulus alpha and theta activities (C) (from Rahn, E. and

Time and the brain

328

Başar, E. [1993] International Journal of Neuroscience, 72, 123–136)

3.1.2.3. Amplitude-frequency characteristics Figure 4 illustrates grand average amplitude frequency characteristics, obtained from standard visual EPs (dashed lines), and from selective-stimulation visual EPs (solid lines) with three bandwidths, recorded at the vertex location. Almost no difference can be seen in the frequency distribution of the standard visual EPs recorded as controls to the three selective-stimulation conditions. The graph in section A represents the amplitudefrequency characteristics when prestimulus alpha activity served as the control parameter for stimulus application. An alignment of the frequencies around 10-Hz is seen, which is not present in sections B or C. However a similar alignment is shifted to the frequencies around 6-Hz in section B, where the control parameter is prestimulus theta. In section C, where prestimulus alpha/theta amplitudes were low, the dominant frequency range was between 5- and 15-Hz peaking around 8-Hz. In all three sections, the selective-stimulation visual EPs show stronger delta and beta components. The differences in the centre frequencies lead to the conclusion that EEGdependent stimulation affects the visual EPs not only quantitatively—as to potential amplitudes—but also qualitatively, as to their frequency content.

Figure 4. Grand average amplitude frequency characteristics from standard visual EPs (dashed lines) and selective stimulation visual EPs (with low prestimulus amplitude in the frequency range indicated by solid lines), N=12, vertex recordings. Control parameter: prestimulus alpha (A), prestimulus theta (B), prestimulus alpha plus theta (C) (from Rahn, E. and Başar, E., [1993] International Journal of Neuroscience, 72, 123–136)

The relation between EEG and evoked potentials

329

3.1.2.4. Topographic aspect Amplitude enhancements comparable to those observed at the vertex were present in a widespread area, forming a circle around Cz which included F3, F4, T3, T4, P3 and P4 electrodes. The mean prestimulus rms values at these electrode sites were also lower in selective-stimulation experiments than in controls. The occipital leads showed a slight, but nonsignificant amplitude gain. To illustrate the topography of amplitude changes, the percent changes of the amplitudes of the selective-stimulation visual EPs from those of controls were computed. Figure 5 represents, as histograms, the medians and 95% confidence intervals of percent amplitude increases of the selective-stimulation visual EPs compared to the controls, sorted according to the experimental conditions and recording sites. The representation as histograms allows a direct visual comparison of the amplitude increases, together with the scalp distribution, under the three different prestimulus conditions. Open bars correspond to alpha as control parameter, the diagonally striped ones to theta, and the laterally striped ones to both frequency bands as the control parameters. At the vertex, which was used as the reference point for the EEG filtering, rms value computation and evaluation, the rms-contingent stimulation led to an amplitude gain of roughly 35%. The amplitudes of the visual EPs obtained by prestimulus alpha-contingent stimulation showed an increase of 37% (p<.01), the theta-contingent visual EPs of 35% (p<.05) and the alpha/theta-contingent of 38% (p<.05). It can also be seen that the frontal, temporal and parietal recording sites show amplitude enlargements comparable to those at the vertex, whereas there is little effect in the occipital region.

Time and the brain

330

Figure 5. Percent presentation of the amplitude increase of selective stimulation visual EPs versus standard visual EPs (set to 100%), based on the median values of 12 subjects (from Rahn, E. and Başar, E., [1993] International Journal of Neuroscience, 72, 123–136)

3.2. Frontal Visual Evoked Potentials In the investigation reported in the previous section (Rahn and Başar, 1993a,b), the central electrode (vertex, Cz) was taken as the input electrode for EEG evaluation (input reference channel). If stimuli were triggered selectively in low-activity EEG states—with respect to alpha or theta components—the N100-P180 complex of auditory and visual evoked potentials showed an increase of 30–40%. The next question was whether such an inverse relationship exists for different recording sites? Besides checking the validity of the new algorithm, frontal evoked potentials were also studied to gain further insight into the response-susceptibility of frontal lobes in theta

The relation between EEG and evoked potentials

331

frequency ranges. The F4 lead was used as the input channel directing selective stimulation. In three series, alpha as well as theta components and alpha/theta components were used as control parameters for stimulus triggering. We found that alpha- and alpha/ theta-contingent selective-stimulation conditions resulted in significant amplitude increases (p<.05) at the input reference channel F4 and partly at ipsilateral temporal and parietal leads, and at Cz. The most significant increase of 35% at F4 (p<.01) resulted if stimulation was contingent on prestimulus theta components. It was concluded that the major operating rhythm (MOR) of frontal lobes is the theta frequency band, and that frontal evoked potential analysis should consider the theta states of the brain for interpretation of the variability of frontal EPs (Başar et al. 1998). 3.2.1. Broad band filtered data (including topography) The percent changes of amplitudes of the selective-stimulation visual EPs from those of controls were computed for all recording sites. Figure 6 shows a histogram representing the medians of percent amplitude increases of the selective-stimulation visual EPs compared to the controls, sorted according to experimental conditions and recording sites. The black bars correspond to theta as control parameter. At the F4 location (which was also the input channel for EEG filtering, rms value computation and evaluation) all three selective-stimulation conditions effected amplitude increases, in comparison to the respective controls. The most distinct difference can be observed between the thetadependent visual EPs and the corresponding controls. The theta-contingent stimulation led to an amplitude gain of about 35% (p<.01). This high significance level was found only for the theta condition at F4. Selective stimulation also effects global amplitude enhancements at the right temporal and parietal leads. In contrast, for none of the selective-stimulation conditions, could an amplification in the occipital visual EPs be observed. Indeed, in occipital recordings, standard visual EPs seemed, if anything, to have a slight tendency to higher amplitudes. Accordingly, there is a significant site effect, the amplification effects at F4 as well as at the right and more anterior leads being larger than those at other sites. In all frontal, temporal and central recordings, the prestimulus theta-contingent visual EPs (black bars) display the most prominent differences between controls and selective-stimulation visual EPs. By contrast, differences were smallest if alpha activity served as control parameter. The alpha- and theta-contingent experiments show mean amplitude enlargements. The situation is different for parietal and even more for occipital recordings. While at the right parietal lead the selective-stimulation visual EPs show sligthly higher amplitudes than controls, no difference can be found at the left parietal lead. In the occipital lobe, the amplitude ratio is the opposite: control experiments depict higher amplitudes than the frontal EEG-dependent visual EPs. As mentioned earlier in this chapter (section 3.1.2: visual EP with vertex reference), we performed a parallel study to test the influence of the different interstimulus intervals inherent in the paradigms on amplitude behaviour. The comparison was designed to test the common finding that EP amplitude is dependent on interstimulus variations. It in fact found amplitude enlargements, depending on the lead, in the range of 5–12% due to ISI

Time and the brain

332

differences. For the F4 position, the “long ISI” control differed from the standard control by 9%. Focusing on the alpha condition, amplitude changes do not exceed the 10% level, and there is no clear effect of the alpha band on the visual EP amplitude. In contrast, the theta dependent visual EPs show 35% higher amplitudes than controls. The difference between the alpha- and the theta-contingent selective-stimulation visual EPs, which were

Figure 6. Effect of selective stimulation on amplitudes of wide-band filtered evoked potentials. The changes in the amplitudes of the selectivestimulation visual EPs as the percentage of standard visual EP amplitudes (set to 100%), filtered for 1–30 Hz. The different bar styles refer to control parameters ALPHA, THETA, and ALPHA plus THETA, respectively. Median values of 9 subjects are presented (from Başar, E., Rahn, E., Demiralp, T., and Schürmann, M. [1998] Electroencephalography and Clinical Neurophysiology, 108, 101– 109)

The relation between EEG and evoked potentials

333

both recorded using the paradigm with irregular and long ISIs, demonstrates very clearly that the parameters of the prestimulus EEG are critical for amplitude changes. This difference further indicates that the amplitude enhancements are mainly due to the different prestimulus conditions, and not to ISI differences. 3.2.2. Amplitude-frequency characteristics (including topography) The grand averages also indicate increases in visual EP amplitudes in frontal and parietal locations, and an inverse behaviour for the occipital lobe (Figure 7). In both conditions,

Figure 7. Time-domain (top) and frequency-domain (bottom) representations of grand averages of standard visual EPs and responses to selectively applied stimulation (selective stimulation visual EPS) obtained in frontal, parietal, and occipital recording sites. Control parameter: prestimulus THETA activity, N=9 (from Başar, E., Rahn, E., Demiralp, T., Schürmann, M. [1998] Electroencephalography and Clinical Neurophysiology, 108, 101–109)

the wave shapes were similar, and any components which differed from the peaks and valleys of the control visual EPs were not detected. The amplitude-frequency characteristics (AFCs) show differences in the frequency contents of visual EPs obtained from various recording areas: In parietal and occipital recordings, the alpha band activity is most prominent, peaking at 9–10 Hz. In frontal visual EPs, the dominant frequencies are below 8 Hz with a greater maximum at 6–7 Hz and a smaller one at 3 Hz. As to the peak location, only minor differences were found between selective-stimulation and control conditions. It should be mentioned in this context that previous results of Başar et

Time and the brain

334

al. (1992) showed that the responses of nonprimary areas to visual (as well as to auditory) stimulation consisted mainly of theta rhythmicities, while those of primary sensory areas were predominantly in the alpha range.

4. DISCUSSION 4.1. The Inverse Relation Between EEG and Visual EP May Lead to a New Standardization in EP Measurements Systematic analyses of combined epochs of EEG and evoked potentials in the auditory and visual pathways of the cat brain have led to the concept that an inverse relationship exists between the amplitudes of the prestimulus and poststimulus activities (Başar, 1980). Recent studies (Rahn and Başar, 1993a,b) using visual and auditory stimulation strongly suggest that different states of EEG rhythms contribute directly to differences in amplitudes of the evoked cortical responses. The following pieces of evidence are pertinent to this statement: (1) Vertex EPs elicited by auditory stimulation during periods of low amplitude alpha or theta band activity showed about 40% higher amplitudes than conventional AEPs. (2) The findings in the auditory modality were confirmed and extended in the visual modality. At the vertex, the visual EP amplitudes increased about 35% compared to controls when prestimulus alpha or theta amplitudes did not exceed certain amplitude levels. Visual EP amplitudes increased also at neighbouring electrode sites. (3) An inverse relationship between the visual EP amplitudes and the spontaneous EEG immediately preceding stimulation also exists for frontal recordings. Selective stimulation in the frontal area differed in two major aspects from the vertex-contingent stimulation used in previous studies: (i) The amplitude enhancement due to selective stimulation was not consistently found all over the head, but was more localized, and (ii) the alpha activity seems to be of less importance for frontal EP generation than the theta activity. In fact, the most prominent amplitude enhancement was observed for the theta-dependent visual EPs, which indicates a preferred response susceptibility of frontal lobes in theta frequency ranges. While for the auditory EPs the measurements were confined to the vertex, for the visual modality, several electrode sites were included to analyze spatial relationships. Although the bioelectrical activity was evaluated at the central electrode, the investigation of multiple sites revealed that locations in a circumference centred on the vertex showed similar, but smaller amplitude enhancements. This was not the case for the most distant occipital recordings. The findings in two different modalities, which complement each other, suggest that the vertex may be appropriate as a reference for EEG-sensitive, selective stimulation. If these findings can be generally confirmed, this may be an alternative method of averaging that may be more appropriate than the conventional one, and also useful for

The relation between EEG and evoked potentials

335

clinical applications. Although mean interstimulus intervals increase, the recording time does not need also to increase, because a smaller number of single trials is needed to compute clearly recognizable averaged EPs. The method partly eliminates the variability (induced by spontaneous activity) from the evoked potential, and thus produces more consistent recordings. Moreover, a smaller number of single trials is needed to compute a clearly recognizable averaged EP. 4.2. Comments on Experimental Design It is a well known fact that, for example, the auditory vertex response is affected by the interstimulus interval (Davis et al., 1966; Hari et al., 1982). Nevertheless, since we aimed to introduce a new algorithm for averaging of EPs by taking the prestimulus EEG states into consideration, the conventional averaging method was chosen as a base for comparison. In order to determine the degree to which the ISI incongruence of the two conditions contributed to the amplitude differences, additional “long ISI” controls were performed with the same mean interstimulus interval as in the selective-stimulation conditions, both in the auditory and visual modalities. Depending on the electrode site, a 5–15% increase could be attributed to ISI alone. In other words, the new algorithm effects an amplitude gain of about 30% when an ISI correction factor is taken into account. The minor importance of ISI incongruency was also nicely demonstrated in the experiments with a frontal electrode as input channel: prestimulus alpha, prestimulus theta and a prestimulus 4–14Hz band were used to control stimulus application. All three selective-stimulation conditions showed nearly identical ISIs. If the amplitude gains compared to control EPs were due to the longer ISIs, it would be expected that all three conditions would depict similar amplitude increases. This was not the case at all (see above). We repeatedly observed that the selective stimulation procedure seemed to bring about a higher correlation between single sweeps (Rahn and Başar, 1990). This is in good accord with the results of McDonald (1964) who claimed that the main reason for variability in single sweeps was differences in the initial EEG conditions. His data refer to brainstem acoustic evoked responses in cats, the contingent time period being in the range of about 50 ms. The increase in correlation coefficients between individual records and averaged evoked response also matches the work of Sayers et al. (1974) who stated that effective stimuli act by synchronising the phases of spectral components of the spontaneous EEG activity already present. According to our working hypothesis, an enhancement of the theta response following low prestimulus alpha, and vice versa, is not to be expected. Nevertheless, especially for the vertex-dependent selective stimulation, a marked enhancement in the alpha band was observed, not only for low prestimulus alpha as the stimulation criterion, but also for low prestimulus theta activity. For the latter condition, the amplitude-frequency characteristics of grand average theta-dependent EPs revealed a resonant maximum at 7Hz reaching to the 8-Hz range. We interpret the enhancement of the alpha band as an interaction of the slow alpha with the fast theta component, especially since the filter

Time and the brain

336

characteristics included a broader frequency range. The discrete frequencies are superimposed; thus the broader range cannot be separated into two distinguishable bands. In addition, the filter characteristics partly included border frequencies, so that a 7–9 Hz range was evaluated in both conditions. We suppose that the most prominent effect could be reached by adapting the frequency range for rms evaluation to the range of maximal responsiveness of the evoked potential (“prestimulus adaptive filtering”). This approach would extend the adaptive filtering method introduced by Başar (1980) for EPs to the prestimulus EEG segment. Moreover, the length of prestimulus epochs (Is) is not necessarily the optimal one: shorter epochs might be more relevant for the poststimulus epoch. Why is there no additional effect on amplitude increase if both frequency ranges are used as stimulation criteria? The alpha and theta band do not increase or decrease synchronously. If both are evaluated together, it is plausible that low activity in one band is coupled with high activity in the other or that both show mean levels of synchronization. Therefore, the trigger threshold was adjusted to a rather high level, which caused a poorer separation between stages of low and high activity. 4.3. Frequency Content of EPs from Different Locations: Major Operating Rhythms (MORs) Başar et al. (1992) assumed that it was impossible to design a pure sensory or pure cognitive paradigm in EP research. It is to be expected that during the standard EP, besides sensory processing, various cognitive processes also come into play. Recent studies of Posner and Petersen (1990) emphasized the topographical characteristics of cognitive processing. Goldman-Rakic (1988) showed in a neuroanatomical study the parallel distributed networks in primate association cortex. Taken together, these works suggest a distributed sensory-cognitive parallel processing system in the brain. In such a system the primary sensory processes and various associative or cognitive functions might be coactivated in different brain structures during the perception of a physical stimulus. This type of distributed parallel processing could be responsible for the differences of frequency content of responses obtained in different locations. In this context we want to introduce the expression of the “major operating rhythm” (MOR) which characterizes the major activity of a given brain structure. For example would it be adequate to state that 3–6 Hz frequency is the MOR of the frontal cortex in contrast to 10 Hz, which is seemingly the MOR of the occipital cortex?

The relation between EEG and evoked potentials

337

4.3.1. MOR of the frontal lobe: theta? Rémond and Lesèvre (1957) reported on a predominance of the theta-rhythm in the frontal central region, whereas Mundy-Castle (1951) described more pronounced theta activity in temporal regions. Recent results of Westphal et al. (1990) showed that the theta amplitude is highest over the anterior midline (Fz- and Cz-locations) which is in accordance with mapping findings (Walter et al., 1984; Mizuki et al., 1983). The review of Miller (1991) concludes that theta activity recorded from the hippocampus has been difficult to find in humans because of the difficulties of human central electrophysiology. Some recent evidence that the midline prefrontal region of the cortex can generate 5 Hz-theta activity was reported by Mizuki et al. (1980) in certain cognitive states. Spectral analysis of frontal EEG showed that theta frequencies were increased during motor or verbal learning tasks (Lang et al., 1987; Westphal et al., 1990). In Miller’s (1991) view, the data of groups of Lang and of Mizuki are compatible with the hypothesis that theta activity in frontal regions is associated with a theta activity in the hippocampus (see Başar, 1998a,b). Başar-Eroğlu et al. (1991a,b) have shown that the significant theta response at the CA3-pyramidal cell layer cannot be recorded in analogous fashion in the cortex by volume conduction. Therefore, the significant cognitive theta enhancements (see also Miller, 1991; Mizuki et al., 1980; Lang et al., 1987) in frontal and parietal recordings might occur by mechanisms of Hebbian cooperation among the neuronal populations of the frontal cortex, the parietal cortex and the hippocampus. The resonant theta response of the hippocampus to auditory and visual stimuli was explained in detail in an earlier report concerning a component analysis of hippocampal evoked potentials (Başar and Ungan, 1973). Recently, the concept of theta resonance has been analyzed extensively by Miller (1991) who describes the cortico-hippocampal interaction as a basic resonance phenomenon in the theta frequency range. According to the anatomical and physiological evidence, Miller takes the viewpoint that theta modulated signals are likely to influence limbic and prefrontal areas, and also—directly or indirectly—other areas of (mainly association) cortex. Tentative interpretations of previous results have led us to postulate the existence of a “diffuse alpha response system” and a “diffuse theta response system” in the brain. We tentatively assumed that the theta component of the evoked potentials and/or slower responses might reflect the responsiveness of various brain areas involved with global associative-cognitive performance (Başar et al., 1992; for the “diffuse gamma response system”, see: Başar and Demiralp, 1995; Schürmann et al., 1997). Theta increases during time-prediction tasks were evident especially in the frontal and parietal recording sites (Demiralp and Başar, 1992). These results suggest an association between the theta frequency components of transient evoked responses, the association areas of the brain and the cognitive performance. A series of results from experiments with freely moving cats by using a passive P300paradigm led to the assumption that P300-like potentials have multiple cortical and subcortical generator sites including the reticular formation of the brain stem, the

Time and the brain

338

hippocampus and the auditory cortex. The P300 potential is most significant, stable and has the largest amplitudes in CA3-layer of the hippocampus of the intact cat brain. The hippocampal P300 manifests an enhancement of the theta activity of the field potentials and/or a type of resonance phenomenon in the theta frequency range (Başar-Eroglu et al., 1991a,b). Together with Miller’s hypothesis (1991), these data give support to the role of the diffuse theta system in cognitive components. These reports underline that some special anatomical structures preferably responding in a specific frequency range are more involved for generation of theta resonances. 4.4. MOR of Occiput and Central Region (Vertex) Brandt and Jansen (1991; see also Brandt et al., 1991) studied the relationship between levels of prestimulus alpha amplitude and the N1 to P2 peak-to-peak amplitude of the parieto-occipital visual EP obtained upon brief photo flashes, in a manner similar to that used by Başar (1980). Root-mean-square amplitude derived from power spectral measures in the alpha band of the 1 s prestimulus EEG were related to the peak-to-peak amplitude of the N1 and P2 components of the visual evoked potentials. They report a highly significant correlation between prestimulus alpha amplitude and N1-P2 amplitude, a general inverse relationship between visual EP enhancement and prestimulus alpha amplitude. The alpha rhythm is known to be preferentially produced over the occiput. Visual stimulation elicits a marked “alpha response” there, i.e. an enhancement of alpha activity in the first 300 ms following stimulation. Brandt and Jansen (1991) also found an inverse relation between the amplitudes of the pre- and poststimulus alpha. Their findings are thus in ageement with the following of our own concepts: (a) there exists an inverse relationship between pre- and poststimulus rhythmicities; (b) the major operating rhythm of a brain structure or region controls or dominates the amplitude of evoked potentials. At the vertex, auditory and visual evoked potentials can be described as compound alpha and theta responses. At the centre of the head, neither alpha nor theta activity can be denoted as single MOR, rather both frequencies are present. In fact, we found a nearly equal dependence of auditory EPs and visual EPs on prestimulus alpha and theta components. In the light of all these findings, it can be concluded that the effectiveness of the proposed algorithm depends on the MOR of the brain region investigated. 4.5. Comparison with Results of Other Laboratories on EEG and EP/ERP Relationships An increasing number of studies hint that fixed patterns of interaction exist between both evoked and event-related potentials and the background EEG activity. Nearly 30 years ago, McDonald (1964) applied stimulation in relation to the spontaneously occuring background activity and found a reduced variability within single trials. Jones and Armington (1977) separated visual evoked potentials that occurred in conditions of high and low alpha for different stimulus intensities. With increases in stimulus luminance, visual EPs averaged when alpha had low amplitude increased monotonically; those

The relation between EEG and evoked potentials

339

averaged in the presence of high alpha were more variable. In addition, a dependency between the phase angle of the alpha EEG at the moment of presentation of a visual stimulus and the visual EP has been demonstrated (Rémond and Lesèvre, 1957, 1967; Trimble and Potts, 1975). Recently, Jansen and Brandt (1991) reported for the occipital N1 that alpha activity present prior to the stimulus continues into the N1 latency range (entrained alpha), and that, for several defined phase angles, the alpha amplitude increases upon stimulation. In another visual EP study, Brandt et al. (1991) reported that, for a majority of subjects, a significant increase in the averaged N1-P2 amplitudes occurred as a function of increasing prestimulus relative alpha power for a POz lead; for increasing relative delta-theta power, they describe a decrease in the N1-P2 amplitude. This means that a proportional relationship exists for the alpha range, but there is an inverse relationship for the delta/theta range. On the other hand, using single trial analysis, the same group replicated for the alpha band the results of Başar (1980) concerning an inverse relationship between visual EP enhancement and prestimulus alpha amplitude in all subjects (Brandt and Jansen 1991). Romani et al. (1988) found high power in the theta and delta band, and low power in the subsequent auditory vertex EP. Our results seem to correspond partly to these reports (prestimulus theta), but partly they are at odds (prestimulus alpha). A direct comparison is not possible due to restrictions imposed by the many methodological differences. Romani et al. (1988) indeed used auditory stimuli and vertex derivations, but stimulation parameters differ, subjects were instructed to count the tones silently, and two different channels were used for the data of pre- and poststimulus epochs. The study of Brandt et al. (1991) refers to the visual modality and an electrode site between Pz and Oz. Moreover both studies are related to relative prestimulus power computations, whereas we referred to an absolute quantity, the prestimulus rms value evoked responses. As to the alpha band and occipital recordings, Brandt’s finding of an increase of the occipital visual EP directly proportional to that of alpha activity is contradictory to our hypothesis. Jasiukaitis and Hakarem (1988) showed that increased prestimulus alpha amplitude was a predictor of increased P3 amplitude but did not correlate with N1. On the other hand, our study revealed no significant amplitude increase at occipital electrode sites (i.e. near the cortical termination of the visual modality), as was the case for most other electrode sites. This may be a hint of a functional difference between the occipital alpha rhythm and the more anterior ones, possibly in the sense of a scanning process in the visual cortex as proposed by Shevelev et al. (1991). As Pfurtscheller and Klimesch (1992) state, event-related synchronization (ERS) and event-related desynchronization (ERD) of alpha frequency components can be observed within the same time interval at different locations of the scalp. In particular, occipital and central areas can behave quite differently, in terms of the generation of alpha band activity, at the same moment of time. The ERD concept implies in addition that the ERD is dependent on the amplitude of alpha band activity before the event (stimulus): without alpha band activity at all, no ERD can be measured. ERD can also be negative, meaning that alpha band activity can be provoked (synchronization effect). Until now, there is no widely accepted view concerning the functions of the alpha rhythms and their relation to visually evoked responses. Might the visual EPs and the

Time and the brain

340

“long latency” potentials that can be observed between 50–300 ms represent a stimulusinduced brain rhythm? Or do visual EP waves occur with a marked alpha band rhythmicity, even though they apparently represent the sequential activation of separate neuronal generators (Mangun, 1992)? In that context, the consistent results concerning the inverse relation between visual evoked potential enhancement and the prestimulus alpha rhythm (Başar, 1980; Brandt and Jansen, 1991) are especially noteworthy. There is also sufficient correspondence among the studies mentioned above to encourage further examination of these effects to get information about the neurophysiological mechanisms responsible for evoked potential generation. 4.6. Functional Significance of the EEG-EP Interrelation Different EEG segments—defined by stable map configurations of varying duration— might manifest basic steps of brain information processing (Lehmann, 1990). Galambos (1992) studied steady-state evoked responses elicited by the repetitive, rapid presentation of stimuli in several modalities. Consistently, the response amplitude cycles up and down spontaneously every minute or so (“minute rhythm”). This accounts for the dynamic influences on sensory information: The rhythms might reveal “microshifts” in alertness or attention, an issue also discussed recently by Polich (1997). Romani et al. (1988) came to the conclusion that vigilance fluctuations (as measured by a vigilance-related index of delta/theta power) have a strong effect on stimulus processing. Sayers et al. (1974, 1979) stated that effective stimuli act by synchronizing the phases of spectral components of the spontaneous EEG activity already present. It has been shown repeatedly in studies of changes in the brain’s cognitive responses (e.g. Başar et al., 1989a,b), that EPs and ERPs reflect a transition to coherent stages of already-existing information channels: there are no new frequencies in the responses, but a kind of tuning of the existing resonance properties. Furthermore, the signals emanating from the brain generators as a result of sensory stimulation take into account the dynamic changes that have occurred as a result of the preceding stimuli. The effects of incoming sensory information are modulated by physiological activities endogenous to the nervous system—this being a common aspect in most studies mentioned above. In this framework, one could speculate about the physiological function of the inverse relation between pre- and poststimulus EEG rhythmicities: possibly information processing is more effective following stages of poor synchronization in defined frequency bands, which could be measurable in reaction time tasks. The new concept may also lead to a better understanding of cognitive processing: The synchronization of the prestimulus EEG should be considered as an active component in evoked responses (Başar, 1980). If external stimuli are applied during phases of highly synchronized activity, it may not be possible to elicit further enhancement and frequency stabilization. Earlier results on the development of “preparation rhythms” (Başar and Stampfer, 1985; Başar et al., 1989a,b) have shown that it is possible to measure almost reproducible EEG patterns in subjects expecting defined sensory stimuli (targets). Evidence was presented that pretarget activity interacts with evoked potentials: Pretarget alpha effected a reduction, or sometimes almost complete absence of the response,

The relation between EEG and evoked potentials

341

namely of the N1 component (Başar et al., 1989a,b).

5. CONCLUSION A common observation in all studies using the selective stimulation algorithm (Rahn and Başar, 1993a,b) is the general dependence of the EP upon the prestimulus EEG activities. The synopsis of the results strongly support the hypothesis of an inverse relationship between prestimulus EEG states and evoked potentials and suggests that different states of EEG rhythmicities contribute directly to differences in amplitudes of the cortical evoked responses. Consequently, we conclude that: 1. the prestimulus EEG states should be taken into consideration before interpreting variations in evoked potentials; 2. an alternative method for averaging that may be more appropriate than the conventional one is introduced; and 3. the algorithm partly eliminates the variability induced by spontaneous activity and, thus, produces more consistent recordings. The vertex seems to be the most adequate input channel as an input reference for selective stimulation that is EEG-sensitive. Compared to the F4 lead, the vertex has two advantages: the recorded activity has a global nature and alpha and theta frequency components make a comparable contribution to the generated EP. Different cortical areas depict different major operating rhythms (MOR). It is evident that the MOR of the frontal cortex significantly influences the evoked potentials; the theta activity in frontal lobes controls the frontal EPs directly. In other words, i.e. the behavioral theta states should play a major role in the genesis of frontal EPs.

REFERENCES Arieli, A., Sterkin, S., Grinvald, A., and Aertsen, A. (1996) Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science, New York, 272, 1868–1871. Başar, E. (1980) EEG-brain dynamics. Relation between EEG and brain evoked potentials. Elsevier, Amsterdam. Başar, E. (1998a) Brain Function and Oscillations I. Brain Oscillations: Principles and Approaches. Berlin, Heidelberg, New York: Springer. Başar, E. (1998b) Brain Function and Oscillations II. Integrative Brain Function: Neurophysiology and Cognitive Processes. Berlin Heidelberg New York: Springer. Başar, E., and Demiralp, T. (1995) Fast rhythms in the hippocampus are a part of the diffuse gamma-response system. Hippocampus, 5, 240–241. Başar, E., and Ungan, P. (1973) A component analysis and principles derived for the understanding of evoked potentials of the brain: studies in the hippocampus. Kybernetik, 12, 133–140. Başar, E., and Stampfer, H.G. (1985) Important relation between EEG-Dynamics, eventrelated potentials, short-term memory and learning. International Journal of Neuro

Time and the brain

342

science, 26, 161–180. Başar, E., Başar-Eroglu, C., Röschke, J., and Schütt, A. (1989a) The EEG is a quasideterministic signal anticipating sensory-cognitive tasks. In: E.Başar and T.H.Bullock (eds) Brain Dynamics. Springer, Berlin Heidelberg New York, pp. 43–71. Başar, E., Başar-Eroglu, C., Röschke, J., and Schult, J. (1989b) Chaos- and alpha reparation in brain function. In: R.Cotteril (ed.) Models of brain function. Cambridge University Press, pp. 365–395. Başar, E., Başar-Eroglu, C., Parnefjord, R., Rahn, E., Schürmann, M. (1992) Evoked potentials—ensembles of brain induced rhythmicities in the alpha, theta and gamma ranges. In: Başar, E., Bullock TH (eds) Induced Rhythms in the Brain, Boston: Birkhäuser pp. 155–181. Başar-Eroglu, C., Başar, E., and Schmielau, F. (1991a) P300 in freely moving cats with intracranial electrodes. International Journal of Neuroscience, 60, 215–226. Başar-Eroglu, C., Schmielau, F., Schramm, U., and Schult, J. (1991b) P300 response of hippocampus analyzed by means of multielectrodes in cats. International Journal of Neuroscience, 60, 239–248. Başar-Eroglu, C., Başar, E., Demiralp, T., and Schürmann, M. (1992) P300-response: possible psychophysiological correlates in delta and theta frequency channels. A review. International Journal of Psychophysiology, 13, 161–179. Başar, E., Rahn, E., Demiralp, T., and Schürmann, M. (1998) Spontaneous EEG theta activity controls frontal visual evoked potential amplitudes. Electroencephalography and Clinical Neurophysiology, 108: 191–109. Brandt, M.E., and Jansen, B.H. (1991) The relationship between prestimulus alpha amplitude and visual evoked potential amplitude. International Journal of Neuroscience, 61, 261–268. Brandt, M.E., Jansen, B.H., and Carbonari, J.P. (1991) Pre-stimulus spectral EEG patterns and the visual evoked response. Electroencephalography and Clinical Neurophysiology, 80, 16–20. Davis, H., Mast, T., Yoshie, N., and Zerlin, S. (1966) The slow response of the human vertex to auditory stimuli: recovery process. Electroencephalography and Clinical Neurophysiology, 21, 105–113. Demiralp, T., and Başar, E. (1992) Theta rhythmicities following expected visual and auditory targets. International Journal of Psychophysiology, 13, 147–160. Fruhstorfer, H., and Bergström, R.M. (1969) Human vigilance and auditory evoked respones. Electroencephalography and Clinical Neurophysiology, 27, 346–355. Galambos, R. (1992) A comparison of certain gamma band (40-Hz) brain rhythms in cat and man. In: E. Başar and T.H. Bullock (eds) Induced Rhythms in the Brain. Birkhäuser, Boston, pp. 201–216. Goldman-Rakic, P.S. (1988) Topography of cognition: Parallel distributed networks in primate association cortex. Annual Review of Neuroscience, 11, 137–156. Hari, R., Kaila, K., Katila, T., Tuomisto, T., and Varpula, T. (1982) Interstimulus interval dependence of the auditory vertex response and its magnetic counterpart: implications for their neural generation. Electroencephalography and Clinical Neurophysiology, 54, 561–569. Jansen, B.H., and Brandt, M.E. (1991) The effect of the phase of prestimulus alpha activity on the averaged visual evoked response. Electroencephalography and Clinical Neurophysiology, 80, 241–250. Jasiukaitis, P., and Hakarem, G. (1988) The effect of prestimulus alpha activity on the P300. Psychophysiology, 25, 157–165.

The relation between EEG and evoked potentials

343

Jones, K.G., and Armington, J.C. (1977) The removal of alpha from the VECP by means of selective averaging. Vision Research, 17, 949–956. Lang, M., Lang, W., Diekmann, V., and Kornhuber, H.H. (1987) The frontal theta rhythm indicating motor and cognitive learning. In: R. Johnson, J.W. Rohrbaugh and R, Parasuraman (eds) Current Trends in Event-Related Potential Research (EEG Suppl. 40). Elsevier, Amsterdam, pp. 322–327. Lehmann, D. (1990) Brain electric microstates and cognition: the atoms of thought. In: E.R. John, (ed.) Machinery of the mind. Boston, Birkhäuser, pp. 209–223. Mangun, G.R. (1992) Human visual evoked potentials: Induced rhythms or separable components? In: E. Başar and T.H. Bullock (eds) Induced Rhythms in the Brain. Birkhäuser, Boston, pp. 217–231. Maras, L., Palejev, G., and Radil, T. (1990) Dynamics of evoked responses according to defined patterns of EEG activity. In: Abstracts: IBRO International Symposium: Mathematical Approaches to Brain Functioning Diagnostics , Prague. McDonald, M. (1964) A system for stabilizing evoked potentials obtained in the brain stem of the cat. Medical Electronics Biological Engineering, 2, 417–423. Miller, R. (1991) Cortico-hippocampal interplay and the representation of contexts in the brain. Springer, Berlin, Heidelberg, New York. Mizuki, Y., Masotoshi, T., Isozaki, H., Nishijima, H., and Inanaga, K. (1980): Periodic appearance of theta rhythm in the frontal midline area during performance of a mental task. Electroencephalography and Clinical Neurophysiology, 49, 345–351. Mizuki, Y., Takii, O., Nishijima, H., and Inanaga, K. (1983): The relationship between the appearence of frontal midline theta activity (Fm) and memory function. Electroencephalography and Clinical Neurophysiology, 56, 56. Mundy-Castle, A.C. (1951): Theta and beta rhythm in the electroencephalograms of normal adults. Electroencephalography and Clinical Neurophysiology, 3, 477–486. Pfurtscheller, G., and Klimesch, W. (1992): Event-related synchronization and desynchronization of alpha and beta waves in a cognitive task. In: E.Başar and T.H.Bullock (eds) Induced rhythms in the brain. Birkhäuser, Boston, pp. 117–128. Polich, J. (1997) On the relationship between EEG and P300: individual differences, aging and ultradian rhythms. International Journal of Psychophysiology, 26, 299–317 (Special Issue: E.Başar, F.H.Lopes da Silva, R.Hari and M.Schürmann [eds] Brain Alpha Activity—New Aspects and Functional Correlates). Posner, M.I., and Petersen, S.E. (1990) The attention system of the human brain. Annual Review of Neuroscience, 13, 25–42. Pritchard, W.S., Brandt, M.E., Shapell, S.A., O’Dell, T.J., and Barratt, E.S. (1985) P300 amplitude/prestimulus EEG power relationships. Psychophysiology, 22, 609–610. Rahn, E., and Başar, E. (1990) Standard prestimulus EEG conditions allow measurements of auditory evoked potentials with replicable patterns. German EEG Society, 35th annual meeting, Bonn. Rahn, E., and Başar, E. (1993a): Prestimulus EEG-Activity strongly influences the auditory evoked vertex responses: A new method for selective averaging. International Journal of Neuroscience, 69, 207–220. Rahn, E., and Başar, E. (1993b): Enhancement of visual evoked potentials by stimulation during low prestimulus EEG stages. International Journal of Neuroscience, 72, 123– 136. Rémond, A., and Lesèvre, N. (1957) Remarques sur l’activité cérébrale des sujets normaux. In Fischgold, H., and Gastaut, H. (eds.) Conditionnement et réactivité en electroencephalographie. Electroencephalography and Clinical Neurophysiology,

Time and the brain

344

Supplement, 6, 235–255. Rémond, A., and Lesèvre, N. (1967) Variations in averaged evoked potential as a function of alpha rhythm phase. Electroencephalography and Clinical Neurophysiology, Supplement, 26, 42–52. Romani, A., Callieco, R., and Cosi, V. (1988) Prestimulus spectral patterns and the evoked auditory vertex response. Electroencephalography and Clinical Neurophysiology, 70, 270–272. Sayers, B.McA., Beagley, H.A., and Henshall, W.R. (1974): The mechanism of auditory evoked EEG responses. Nature, London, 247, 481–483. Sayers, B.McA., Beagley, H.A., and Riha, J. (1979): Pattern analysis of auditory evoked EEG potentials. Audiology, 18, 1–16. Schürmann, M., Başar-Eroglu, C., and Başar, E. (1997): Gamma responses in the EEG: elementary signals with multiple functional correlates. NeuroReport, 8, 531–534 (erratum: 8, 1793–1796). Shevelev, I.A., Kostelianetz, N.B., Kamenkovich, V.C., and Sharaev, G.A. (1991) EEG alpha-wave in the visual cortex: check of the hypothesis of the scanning process. International Journal of Psychophysiology, 11, 195–201. Trimble, J.L., and Potts, A.M. (1975) Ongoing occipital rhythms and the VER. I. Stimulation at peaks of the alpha-rhythm. Investigative Ophthalmology, 15, 537–546. Walter, D.O., Etevenon, P., Pidoux, B., Tortrat, D., and Guillou, S. (1984) Computerized topo-EEG spectral maps: difficulties and perspectives. Neuropsychobiology, 11, 264– 272. Westphal, K.P., Grözinger, B., Diekmann, V., Scherb, W., Reeß, J., Leibing, U., and Kornhuber, H.H. (1990) Slower theta activity over the midfrontal cortex in schizophrenic patients. Acta Psychiatrica Scandinavica, 81, 132–138.

12 Coherence and Phase Relations between EEG Traces Recorded from Different Locations Peter Rappelsberger1, S. Weiss1 and Baerbel Schack2 1

Brain Research Institute, University of Vienna, Spitalgasse 4, A-1090 Vienna, Austria Tel: (0043) 1–4277–62850; FAX: (0043) 1–4277–62849 e-mail: [email protected] 2 Institute for Medical Statistics, Informatics and Documentation, Friedrich Schiller University, John Strasse 3, D-07740 Jena, Germany

This contribution deals with EEG coherence and phase analyses using two different techniques. First, Fourier transform was applied to compute cross-spectra between pairs of electrode signals of EEG trials of constant length. The cross-spectra of a number of trials were averaged, yielding coherence and phase spectra representing the mean properties within the length of the trials, usually 1 or 2 seconds. Second, an adaptive autoregressive-moving-average (ARMA) model was used to compute instantaneous coherence and phase values with a time resolution in the millisecond range. In the first experiment EEG was recorded during auditory presentation of concrete and abstract nouns. 19 right-handed female native German speakers participated. The main coherence results using the Fourier approach relate to the Alpha-1 band (8–10 Hz) and the Beta-1 band (13–18 Hz). In the Alpha-1 band both word classes revealed about the same changes during word processing, suggesting that this band reflects processes common to both word classes. In contrast, in the Beta-1 band, clear differences were found. These differences concern mainly the involvement of visual and frontal association areas, probably due to visual images evoked by the concrete nouns. The second experiment was conducted with 25 right-handed females. Concrete and abstract nouns were presented auditorily but also visually. The main coherence results using the Fourier approach revealed that the Alpha-1 band is sensitive to the modality of stimulus presentation, but does not distinguish between the memorization of abstract and concrete nouns. In contrast, coherence differences independent of stimulus modality were found in the Delta, Theta and Beta-1 bands. Measurement of time relations, i.e. studies of the direction of information transfer, was made using both the Fourier approach for time intervals and the ARMA approach at time instants during the memorisation of nouns. Due to the highly dynamic process, with changing directions of information transfer, the Fourier approach, based on 1 s trials after stimulus onset, yielded only very coarse estimations of the time relations during word processing. The short-lasting properties during word processing

Coherence and phase relations between EEG

347

were smeared and to a great extent extinguished. In single subject studies, the ARMA approach clearly demonstrated an occipital-frontal information transfer in the Beta-1 band for the visual stimulus presentation. During auditory stimulus presentation temporal sites tended to lead occipital sites but there was also a trend for occipital sites to lead central and left frontal sites. KEYWORDS: EEG coherence; Phase relations; Fourier spectra; Adaptive ARMA model; Word processing; Concrete and abstract nouns; Visual and auditory word presentation

1. INTRODUCTION The EEG has been used for decades as a valuable tool for diagnostic purposes in neurology and psychiatry. However, it contains much more information than can be extracted by pure visual examination. Many efforts have been made, and highly sophisticated mathematical procedures and computer techniques have been applied, to extract that information. Scalp-recorded EEG coherence turned out to be an essential large scale measure of functional relationships between pairs of neocortical regions (Sklar et al., 1972; Busk and Galbraith, 1975; Shaw et al., 1976). Coherence is defined as the normalized cross-power spectrum per frequency band, or the correlation coefficient per frequency band. It is computed between two EEG signals recorded simultaneously from different sites of the scalp. In studying functional relationships, coherence analyses yield important new aspects of brain activities which complement the data obtained by power spectral analyses. This was the reason why some authors employed coherence analyses in studying cognitive processes (Beaumont et al., 1978; Colter and Shaw, 1982; French and Beaumont, 1984; Tucker et al., 1985, 1986; Thatcher et al., 1986; Rappelsberger andPetsche, 1988). In recent years, coherence has been used increasingly in different research areas. In our laboratory, coherence has been applied to study cognitive processes like reading, visual imagery, pain perception, sensory-motor task processing, language processing, music perception (Petsche et al., 1992; von Stein et al., 1993; Chen and Rappelsberger, 1994; Rescher and Rappelsberger, 1996; Weiss and Rappelsberger, 1996, 1997; Sarnthein et al., 1997). Furthermore, coherence turned out to be very sensitive in describing periodicities in sleep EEG (Scheuler et al., 1990), to compare Down Syndrome patients and healthy subjects (Schmid et al., 1992), and for differentiating between psychiatric disorders (Rappelsberger et al., 1994; Tauscher et al., 1998). A review of the recent literature reveals that, for estimation of coherence, Fourier spectra were predominantly used. Only a few authors have used other techniques. Although coherence and phase are very closely connected, as outlined in the Appendix, results of phase analysis are very rare in the literature. Mainly applications are described in epilepsy, in order to locate epileptic foci, and to determine the direction of spike and seizure propagation (Gotman, 1983, 1987; Rappelsberger et al., 1987; Duckrow and Spencer, 1992). Gath et al. (1992) stated that estimation of autospectra, and coherence

Time and the brain

348

and phase spectra of seizure EEG using Fourier technique causes “smearing” of the rapid dynamic changes which occur during the seizure. This is inherent to Fourier spectral estimation, due to the averaging process which is necessary in order to get consistent spectral estimates (see Appendix). The authors suggested multivariate autoregressive modelling of the multichannel seizure EEG. Kaminski and Blinowska (1991) developed the “directed transfer function” (DTF). It builds upon earlier attempts to define a directed coherence function. The DTF is calculated directly from the transfer function of a fitted autoregressive model. This calculation utilizes phase information implicit in the transfer function to construct measures of the capacity of each channel to “drive” each other channel. In order to study the EEG dynamics during self-paced finger movements using Fourier technique Leocani et al. (1997) computed event-related coherence and phase, with a sliding 500 ms time window. Averaging was done over trials. Phase at 10 Hz showed a lead of anterior regions to posterior regions, hinting at the implication that frontal lobes are in control of movement planning and execution. Wolff and Thatcher (1990) showed coherence and phase probability maps of 79 deaf children and a matched control group of hearing children. Error probabilities were obtained by multivariate analysis of covariance. Unfortunately, absolute values of coherence and phase, as well as phase directions, were not given by the authors. Deaf children manifested higher coherence and lower phase in certain left hemispheric areas, suggesting less neural differentiation, but lower coherence and higher phase in certain right hemispheric areas, suggesting greater differentiation. Deaf children had a higher power in bilateral frontal cortex than hearing children. The data also suggested compensatory functioning in the visual cortex of the deaf subjects. The pattern of results varied somewhat in relation to the cause of deafness. These findings support the hypothesis that prelingual deafness results in a partial reorganization of the cerebral cortex. The authors did not apply Fourier techniques, but used small band-pass filters to compute auto- and cross-spectra for coherence and phase estimation. Data were based on EEG sections varying between 16 to 60 s. Measurement of dynamic changes in the magnitude of neural network coupling require high temporal resolution. The discrete Wigner-Ville spectral distribution is one method that has been used to obtain high temporal resolution of changes in EEG covariance and phase (Gevins et al., 1994a). However, the technique is computationally demanding, and often difficult to interpret. A simple and easily implemented method to compute hightemporal-resolution EEG coherence and phase is to use the method of complex demodulation. This method is referred to as “instantaneous coherence and phase” or “event related coherence and phase” (Thatcher et al., 1994; Thatcher, 1995) since coherence and phase are explicitly evaluated in the time domain. This technique is distinctly different from frequency domain analyses associated with the Fourier transform. Thatcher et al. (1994) presented instantaneous coherence and phase delays of dipole sources, reconstructed from data obtained during voluntary finger movements. Our approach to explore ultra short-term cognitive processes is based on general adaptive principles and the adaptive fit of a bivariate linear autoregressive moving average (ARMA) model with time varying parameters (Schack and Krause, 1995; Schack et al., 1995; Schack, 1997). A short mathematical description is given in the Appendix.

Coherence and phase relations between EEG

349

This model allows the continuous estimation of power, coherence and phase for certain frequencies or frequency bands. In the section about Technical Aspects of this chapter a definition of coherence is given and computation is discussed shortly both for Fourier transform-based coherence and instantaneous coherence based on an adaptive ARMA model. For the interpretation of coherence a simple model of two neuronal assemblies of a large scale network is introduced. Finally, problems concerning EEG recordings and computation of coherence are discussed. The section about EEG Coherence and Word Processing deals with results of experiments studying the processing of concrete and abstract nouns, presented either visually or in the auditory modality. Coherence data of this section are based on the Fourier technique. The last section, about Phase Analyses, contains coherence and phase data of a group study, and data of a single-subject study obtained with the ARMA model.

2. TECHNICAL ASPECTS 2.1 Definition of Coherence Mathematically, coherence is defined as the squared normalized cross-power spectrum, and represents a correlation function in the frequency domain, i.e. correlation coefficient per frequency or frequency band. Coherence values can run from 0 to 1. Coherence of “1” means that the corresponding frequency components of the two time dependent signals x1,t and x2,t are identical and the only differences which can exist are in amplitude and a constant time relation (phase delay). Coherence of “0” means that the corresponding frequency components of both signals are not correlated. For the transition from signals in the time domain, x1,t and x2,t, to spectra in the frequency domain, S12(f), S11(f) and S22(f), the Fourier transform is usually applied. The most common Fourier transform algorithm is the Fast-Fourier-Transform (FFT) introduced by Cooley and Tukey (1965). For practical reasons the Fourier transform can only be applied to an EEG signal of limited length. This results in an estimation, the periodogram, of the true spectrum. Unfortunately, periodograms have bad statistical properties, i.e. their variance is independent of the epoch length (Jenkins and Watts, 1968). In order to improve these properties, smoothing techniques have to be applied. In EEG analysis long records are usually subdivided into a number of epochs of equal length, and spectra are estimated by averaging the periodograms over epochs. In event-related studies the spectra of the individual trials are averaged. The epoch length usually determines the resolution in the frequency domain. For example, an epoch length of T=2 s results in a frequency resolution of ƒ=1/T=0.5 Hz. A mathematical description of the computational procedure of coherence and phase estimation using Fourier transform is given in the Appendix. In addition to multivariate spectral analysis methods based on the Fourier transform, the fit of linear multivariate models and the parametric calculation of the spectral density

Time and the brain

350

matrix and other spectral parameters can be used. One advantage is the enormous data reduction. However, the real advantage is the possibility of proceeding from a purely static estimation of spectral parameters to a dynamic procedure. Since biological signals very frequently are non-stationary, linear modelling is often criticized. To account for the non-stationary EEG signals Isaksson et al. (1981) recommended the transition to nonlinear models, or the use of linear models with time-dependent parameters. Following these recommendations, an adaptive fitting algorithm of bivariate autoregressive moving average (ARMA) models, with time-varying parameters was constructed, which combines the least mean square algorithm and general adaptive estimation procedures (Schack, 1997). With such an adaptive model the dynamic estimation of the spectral density matrix allows continuous investigation of all bivariate spectral functions with high frequency resolution at every time point. Thus, an adaptation of coherence and phase estimation according to the structural changes of the EEG is achieved. The static coherence and phase estimation based on fixed time epochs, and yielding results which may hide inherent properties (due to the necessary averaging procedures) is complemented by a dynamic estimation at every sampling point. This technique enables the observation of the time course of coherence and phase function for every EEG channel pair, and for every frequency or frequency band. Thus,

Coherence and phase relations between EEG

351

Figure 1. Coherence and phase changes between EEG signals recorded from electrodes O1 and T5 after visual presentation of a German concrete noun. The left part of the figure shows coherence results. In the left upper picture dependence of coherence on frequency (ordinate scale from 0 to 40 Hz) and time (scale in ms) within the first second after word presentation is demonstrated. Coherence values are colour coded. The graphs below show the time-dependent coherence changes in the alpha-1 (8–10 Hz) and beta-1 (13–18 Hz) bands. The right part presents the corresponding results of phase analyses. In the three dimensional phase plot phase scale is in [degree], and in the graphs below phase is converted to milliseconds according to the formula

with fm the centre frequency of the

frequency band.

single trial analysis of various EEG events can be performed. This is demonstrated in Figure 1. A mathematical description of the coherence and phase estimation procedure is given in the Appendix. Usually the variance of single trial analyses is quite large. In order to reduce variance in the examples concerning word processing presented in this chapter, instantaneous coherence and instantaneous phase were averaged over trials time-point by time-point.

Time and the brain

352

2.2 Interpretation of Coherence Since coherence is a correlation coefficient per frequency or frequency band it may be used to describe the relationship or coupling between signals at that frequency or in that frequency band. In EEG analysis it may be (for example) the Alpha peak frequency or the Beta-1 frequency band. Figure 2 will help to explain the idea of coupling between two neuronal assemblies generating the EEG signals x1,t and x2,t. A mathematical description is given in the Appendix. According to the model presented in Figure 2, coherence measures the degree of functional connectivity between neuronal assemblies. In reality, EEG signals represent electrical fields spreading through the tissue underlying the recording electrodes. Unfortunately, the measured signals do not allow estimation of the respective portions of functional connectivity and pure electrical volume conduction. However, by experience, the problem of volume conduction seems to play a minor role, especially when electrode distances are large, as compared with the problem dealt with in the next section. 2.3 The Problem with the EEG Reference EEG signals are recorded differentially. This means that a recording electrode is connected to the positive input of a differential amplifier and either a reference electrode is connected to, or a computed reference signal is fed into, the negative input. The difference between the electrode signal and the reference signal appears at the output of the

Figure 2. Schematic model for the explanation of coherence. For a specific

Coherence and phase relations between EEG

353

task, two neuronal assemblies of a large scale network become functionally connected, which is represented by a common signal portion st. However, not all neurones of one assembly are necessarily involved in that task but may be involved in other procedures generating EEG signal portions n1,t and n2,t, respectively, which are not correlated with the coherent signal 5, and not correlated with each other, i.e. they are noise portions. EEG signals generated by both cell assemblies will result in x1,t and x2,t,. “a” and “b” are simple scaling factors for the coherent signal portion st. Fourier transform and computation of coherence results in equation (20) of the Appendix demonstrating that coherence can be expressed by noise-to-signal ratios. If all cells of both assemblies are involved in generating the coherent signal portion within a specific frequency band, i.e. complete synchronization, the noise terms vanish and coherence will result in K2=1. In contrast, if there is no coherent activity within the specific frequency band between both cell assemblies, K2=0 according to st=0.

amplifier. For mathematical reasons, it is not possible to determine the amount of the reference signal. Its influence on the recorded EEG signals, and on computed coherence, can only be estimated. Basically, three recording techniques have to be considered for subsequent estimation of coherence: reference recording against a reference electrode, common average reference recording, and source derivation. Reference recording has the disadvantage of being an unknown signal picked up with an electrode from a selected position. The properties of the reference signal may have a tremendous influence on coherence estimations, especially when the reference electrode contains a portion of a coherent signal. This was shown by some authors and summarized in a recent systematic study (Essl and Rappelsberger, 1998). Ideally the reference signal should approach zero amplitude. However, this can hardly ever be achieved and controlled. An acceptable compromise turned out to be to use the average from both ear lobe signals as a reference. The background for this approach is that ear lobe signals are in most cases not (or only negligibly) influenced by electrical activity from the temporal lobes. The reason for this is the poor conduction properties of the tissue and bone between temporal brain areas and ear lobes. Only in case of focal temporal processes, like temporal epilepsy, can a signficant signal amplitude be recorded from leads positioned at the ear lobes. Secondly, it can be assumed with high probability, that the signals at both ear lobes are independent. The average of independent signals results in a signal which is lower than either of the original signals. Common average reference recording (Goldman, 1950; Offner, 1950) was introduced to EEG methodology with the concept that the average of a number of independent activities approaches zero, this being an ideal reference. However, this is hardly the case in real EEG recordings with synchronized activities over large brain areas. Due to the use of differential amplifiers, in clinical EEG records frontal alpha activity is frequently found as a projection of occipital and parietal alpha by the common-average reference signal at the negative input of the amplifiers. Since the common average reference signal does not usually approach zero amplitude, the recorded signals contain signal

Time and the brain

354

components from all recording sites. In many cases, this poses a serious problem for proper interpretation, as could be demonstrated in simulation studies (Rappelsberger, 1989). In conclusion, common average reference recordings has undoubted advantages in some specific EEG examinations, but its use for coherence estimations is problematic. The concept of source derivation (Hjorth et al., 1975) has proven to be a valuable tool to ascribe local EEG events (like spikes) to the underlying cortical structures. Mathematically this technique is closely related to the Laplace difference equation, and according to this equation, the recorded physical parameter is not the potential difference, but is proportional to current source density. A major advantage of this method is its independence of a reference electrode. However, due to the differential amplifiers for EEG recording a reference signal is needed. The mathematical procedure of the Laplace derivation produces a reference signal as the weighted average of the potentials occurring at all electrodes surrounding one specific electrode. Therefore, each electrode has its own reference signal. This technique can be applied successfully when closely spaced electrodes are used. For instance, Gevins et al. (1994b) made 124-channel recordings in their experiments. As Nunez (1995, 1998) pointed out serious problems may arise when the Laplace derivation method is applied to a relatively coarse electrode grid as given by the basic 10/20 positions (Jasper, 1958). Gevins and others, who have been working with the Laplacian, produce arguments that the influence of volume conduction was minimized (Le and Gevins, 1993). However, there are no systematic investigations about the amount of volume conduction. Only Nunez (1995) used simulation studies with a head model and showed that volume conduction may diminish at electrode distances of 7 cm and more. However, the results of such simulations are dependent strongly on the basic assumptions about conductivity properties, which are not specified by the author. In our opinion, the influence of volume conduction seems to be overestimated, especially when large electrode distances are used. The reference signal, as the weighted average of surrounding electrode signals, consists of a mixture of many unknown signal components, and subsequent computation of EEG coherence would be influenced by those unknown inputs. Therefore, interpretation of the results of EEG coherence analysis obtained with this method would hardly be possible, as was demonstrated with simulated signals with known coherence properties (Rappelsberger, 1989). To summarise the above arguments, there are three possibile ways of using a reference technique for EEG recording and estimation of coherence: first, there is source derivation (Laplace). Here it is impossible to estimate the error occurring with this technique when using large electrode distances, as are given using the basic positions of the 10/20 system, or to check the influence of numerous electrode signals used for computation of the reference signals. Moreover, one has to be aware, that physical dimensions change the current source densities. Second, one can use common average reference recordings, with the impossibility of estimating the influence of the other 17 electrode signals when computing coherence between two of the 19 electrodes placed. Third, one can use reference recording against the average of two electrodes placed at both ear lobes . The uncertainty in this case remains restricted to two signals, and therefore the error can be estimated relatively simply (Essl and Rappelsberger, 1998).

Coherence and phase relations between EEG

355

3. EEG COHERENCE AND WORD PROCESSING 3.1. Introduction Language is a multi-level system consisting of phonological, morphological, syntactic, semantic, and pragmatic levels. Knowledge about these language levels and categories is based mostly on theoretical findings in philosophy and linguistics, and on empirical research in psycholinguistics. However, little is known about the physiological reality of such linguistic entities. It is not known, for example, if subclasses of nouns (e.g. “concrete” vs. “abstract”), or even if grammatical categories (e.g. “verb” or “noun”) are artificial classes of linguistics, or if they refer to real cognitive categories which were established during the phylogenetic development of cognition. From a neurophysiological perspective, the questions arise how the underlying neural substrate is organized, and if there exists a biological reality to these linguistic categories. A possible relation could exist, for example, between the linguistic category “noun” and the cognitive concept “object”, or the category “verb” and the cognitive concept “action”. A similar uncertainty predominates in the discussion about the above mentioned linguistic levels of language. Investigation of the physiological basis of language processing, and of the usage of mental concepts in humans is one of the most challenging topics in cognitive neuroscience (Gazzaniga, 1995). However, because of its complexity, little is known about the neurophysiology of language processing. An adequate and non-invasive method to get physiological information about the underlying neuronal processes is to record the electrical activity of the human cortex, the electroencephalogram (EEG) during language processing. Up to now, most of the electrophysiological studies concerning language processing have generally dealt with the analysis of event-related potentials (ERP; e.g. Kutas and Van Petten, 1994; Müller et al., 1997). Despite all the advantages of this method, ERP analysis provides neither frequency information, nor information about the cortical interplay or co-operation between different parts of the brain during cognitive processing. Since the interaction of distributed neuronal systems is generally considered to be a substrate for the representation of certain higher cognitive brain functions, the question is raised how these systems are integrated to a coherent functional unit. In other words, during cognitive processing many different neuronal systems are necessary which have to act together. During processing of spoken language for example, the neural substrates for analysing prosodic and phonological aspects of language have to be integrated. Moreover, during reading of sentences, processes in the auditory and the visual modality have to be bound together, and elements have to be combined across time. A possible mechanism of binding together distributed systems is the temporal synchronization of the participating substrates (von der Malsburg, 1981). Time seems to be the most crucial parameter the brain uses to perform efficient information processing. As an example, Figure 1 demonstrates the varying time relations between the EEG signals at two recording positions during processing of a single word. Recent neurobiological findings

Time and the brain

356

revealed, in extensive studies of the visual cortex in cats and monkeys, that neurones which are likely to encode properties of the same object synchronize their discharges, whereas neurones which respond to features of another object do not (Eckhorn et al., 1988; Gray et al., 1989; Singer, 1993, 1994; chapter by Eckhorn, this volume). This was shown for the frequencies around 40 Hz both in humans and in animals (Bressler, 1990; Başar-Eroglu, 1996; Llinás and Paré, 1996). Moreover, the strength of synchronization seems to vary according to task demands, which implies another coding parameter for information processing (König et al., 1995). Therefore, during such a highly developed cognitive skill as human language processing, one may assume increasing and decreasing synchronization of distributed neuronal systems over large cortical distances to play an important role (Weiss et al., 1997a; compare the explanation to the model in Figure 2). As for language processing, increasing and decreasing synchronization takes place within certain frequency ranges that seem to be more important than others (Weiss and Rappelsberger, 1996). This can be explained by resonance characteristics of the underlying neural networks (Lopes da Silva, 1991). An adequate parameter for studying frequency band-related “synchronization” during language processing on a macro scale is the computation of coherence between EEG signals recorded from different sites of the scalp (Petsche et al., 1993; Weiss and Rappelsberger, 1996; Weiss, 1997).

Coherence and phase relations between EEG

357

3.2. Concrete and Abstract Noun Processing Concrete nouns (or “reality-nouns”) are linguistically characterized by high reference and high intention. They represent individual entities of the world with constant features, which can be referred to in the three-dimensional space. In contrast, abstract nouns or “thought-names” have no spatiotemporal concreteness and their individuality is low. The lexeme “rabbit”, for instance, refers to an object that can be seen, heard, smelled, felt or tasted, whereas the lexeme “truth” cannot be experienced through our senses, and is represented mainly verbally within our brain. In psycholinguistic experiments, different processing of concepts for concrete and abstract nouns can be observed within normal participants, who show a remarkable concreteness effect while processing nouns (Bleasdale, 1987; Eviatar et al., 1990). This effect can be explained by two facts, the high individuality and the multimodal representation of concrete nouns. An attribute which is shared by a large number of things is not a very effective retrieval cue, since it does not accurately pick out a particular memory representation. As a consequence, the representation of abstract nouns is more difficult to access and the processing of abstract nouns is easier to disturb (Weiss et al., 1999). This is also supported by various neuropsychological studies, which show that certain patients have a selective difficulty in processing abstract nouns, although they still are able to cope with concrete nouns (Coltheart, 1987; Tyler et al., 1995). These selective deficits have also been shown for grammatical word classes (Hillis and Caramazza, 1995), and for distinct categories of concrete nouns, e.g. “animals” and “tools” (Damasio et al., 1996). To show how the processing of abstract nouns can be disturbed more easily by a lesion than the processing of concrete nouns Hinton and Shallice (1991) used a neural network simulation. Furthermore, Abdullaev and Bechtereva (1993) performed intracortical recordings, and found neurones in the left prefrontal cortex which selectively responded to abstract nouns and not to concrete nouns. 3.2.1. Experiment I According to the above findings a highly complex system of word representations within the brain has to be assumed, but the exact categories reflecting representations of word classes and their subclasses is not known yet. A representational difference of concrete and abstract nouns should be expressed by different patterns of increasing and decreasing synchronization in the brain. These changes in synchronization should be observable within certain frequency bands of the EEG by comparing the stimulus-related activity with the corresponding parameters of the resting EEG. Thus, different patterns of coherence change should be expected for the processing of abstract and concrete nouns. 19 right-handed female native German speakers participated in this experiment (age 24.2±2.6). Two lists of 25 German nouns were selected, separated into concrete and abstract nouns by their imageability, and equated for frequency of occurrence in the German language (Meier, 1967). The nouns had a mean word length of 0.9 s±0.08, and were presented in the auditory modality, with an interstimulus interval of 3 s. The

Time and the brain

358

participants were requested to memorise the nouns, and within 1 minute after the presentation of each list they had to recall the remembered nouns. The EEG was recorded with 19 gold-disc electrodes according to the 10/20 system (Jasper, 1958) against the averaged signals (A1+A2)/2 of both ear lobe electrodes. Filter settings were 0.5–35 Hz, and sampling frequency was 128 Hz. During the recording, eyes were open, and to reduce eye movements the subjects were asked to fixate a dot on the wall in front of them. The beginning of the presentation of each word was marked by a trigger, and the following 2 s EEG epochs containing the word were selected and Fourier-transformed. Epochs withG artefacts were eliminated from further processing. According to equation (3) of the Appendix averaged power and cross-power spectra were computed. 2 s artefact-free epochs of a resting EEG with eyes opened were also Fourier-transformed, and the corresponding spectral parameters served as “baseline” for the task condition. Data reduction was performed by averaging adjacent spectral lines to obtain parameters for broad frequency bands. The final step was the computation of 19 mean amplitudes (square root of power) and coherence between 171 possible electrode pairs for each frequency band. Due to earlier findings (Weiss, 1994) examinations were focused on the alpha-1 band (8–10 Hz) and the beta-1 band (13–18 Hz). For the evaluation of significant coherence differences between the two tasks, or between one task and the EEG at rest, paired Wilcoxon-tests were applied. The obtained rank sums were converted to error probabilities, which were presented in probability maps (Figure 3). For statistical reasons these evaluations are only of descriptive nature (Abt, 1988). Concerning amplitude and coherence changes with respect to the EEG at rest, in the alpha-1 band, both word classes revealed about the same results: there were significant intra- and interhemispheric coherence increases between temporo-central and temporoparietal electrodes and frontal coherence decreases. Amplitudes decreased at all sites. This implies that two different functional processes have to be considered. First, temporocentral and temporo-parietal coherence increases whereas amplitudes decrease, and second, frontal coherence decreases whereas amplitudes decrease as well. Concerning absolute coherence and amplitude values no significant difference was found between the two word classes using the paired Wilcoxon-test (p < 0.05). The results of this study suggest that alpha-1 band coherence changes (compared to the EEG at rest) reflect processes that are common to both word types, as for example the sensory processing of the stimuli, motor processes caused by inner rehearsal of the words (Weiss and Rappelsberger, 1998) and memory processes. These results are supported by other studies that indicate that the alpha band reflects sensory processing of stimuli (Schürmann and Başar, 1994; Başar et al., 1997) and memory processes (Klimesch et al., 1993). Since the multimodal frontal lobe is considered as a neuronal substrate for working memory, the decoupling of functional units within the frontal lobe observed in our study might indicate the use of the working memory (Weiss and Rappelsberger, 1996). In our opinion the dissociation into smaller functional units within the frontal lobes enables a more efficient co-operation of the frontal cortex with other brain regions. The temporal and temporo-central increase of coherence in the alpha-1 band compared to the EEG at rest could be based on the auditory analysis of the nouns: like every acoustic event a single word leads to a short-lived auditory trace. After listening, the word stays available in a so-called phonological loop for at least 1 s after the end of its presentation.

Coherence and phase relations between EEG

359

Therefore, it is assumed that coherence increase between temporal and temporo-central regions during the 2 second epoch containing the word presentation could be correlated with the auditory analysis and search for lexical entries. Consequently the frontal coherence decrease in the alpha-1 band could be interpreted as long lasting frontal desynchronization (amplitude and coherence decrease) during mnemonic processes and the temporo-central increase as short lasting coupling (amplitude decrease and coherence increase) triggered by the word perception and primary processing. In contrast to alpha-1, completely different results were obtained in the beta-1 band (13–18 Hz). Figure 3 shows significant intra- and interhemispheric coherence increase and decrease during the processing of concrete (C) and abstract (A) nouns with respect to EEG at rest. During the memorization of concrete nouns the pattern of coherence changes within the left (C1) and right hemisphere (Cr) were rather similar. In the left hemisphere coherence increased between T5 and occipital, parietal, central and frontal electrodes indicating the left posterior-temporal area below T5 as the “main processing node” (Weiss and Rappelsberger, 1996). This term is used when most of the significant coherence changes are found between this recording site and other regions. Coherence decreased within the frontal lobe and between the “main processing node” F1 (=Fp1) and central and posterior locations. Coherence changes within the right hemisphere showed a similar pattern. The few interhemispheric coherence changes (Ci) hint at an increased coupling between left occipital and right frontal regions and right occipital and left frontal regions. Interhemispheric coherence decrease was found between F2(=Fp2) and C3 and between F1(=Fp1) and P4, and additionally between F3 and F4. It may be argued that coherence increase between different electrode sites may be due to an increase of spectral amplitudes. However, in the beta-1 band no significant amplitude increase was found. In contrast, amplitudes decreased at Fz, F4, F8, Cz and C4, which is indicated in Figure 3 by grey shaded circles. Despite amplitude decrease, such as at Cz, coherence between T5 and Cz increased. This clearly demonstrates that coherence is independent of amplitude behaviour, and is a completely different measure for the characterization of brain dynamics. Coupling and de-coupling of neuronal assemblies may or may not be accompanied by amplitude changes. According to our model in Figure 2, and equations (20) and (21) of the Appendix, coherence is dependent on noise-to-signal ratios, i.e. on the relations between incoherent and coherent signal portions of neuronal assemblies. In Figure 3 the abstract nouns (A) show a completely different pattern of coherence increase and decrease. In the left hemisphere the “main processing node” of abstract nouns seems to be in the anterior temporal region (T3) with an increased coupling with frontal (F3) and posterior association areas. Within the right hemisphere coherence increase was found between posterior sites and between posterior electrodes and mid-line electrodes. Coherence decrease appeared between posterior electrodes Pz and T6 and the right frontal electrodes F2(=Fp2) and F8, respectively. Additionally, a decrease in short distance coherence (F4-F8) appeared. Concerning interhemispheric coherence changes, abstract nouns (A) elicited a strong interhemispheric coupling between central and parietal sites. Comparing the processing of both word classes, in the beta-1 band abstract and concrete nouns elicited different coherence changes both intra- and interhemispherically.

Time and the brain

360

During the processing of concrete nouns an increased bilateral coupling between occipital and parieto-temporal regions was found. Since there is considerable evidence that visual association areas are engaged during the processing of visual images (e.g. Farah, 1989;

Coherence and phase relations between EEG

361

Figure 3. Significant decreases and increases of coherence in the beta-1 band (13–18 Hz) during processing of concrete (C) and abstract (A) nouns, compared with the EEG at rest. Left hemispheric (C1, A1), right hemispheric (Cr, Ar) and interhemispheric (Ci, Ai) coherence changes are mapped separately. The thickness of the line is inversely proportional to the obtained error probability, p≤.01, p≤.02 and p

Time and the brain

362

≤.05, respectively. Coherence increase is indicated by full lines, coherence decrease by broken lines. The grey shaded circles indicate significant decreases of spectral amplitude (p≤.05). No significant amplitude increases were observed.

Weiss et al., 1995) and most of the participants of this study reported visual images during the processing of concrete nouns, it can be assumed that visual images which were automatically evoked by the concrete nouns mediated the increased coupling between the posterior regions. In another experiment we could show that the memorization of concrete nouns affects not only the visual association areas, but also the frontal association areas (Weiss and Rappelsberger, 1998). Abstract nouns (A1) elicit fewer coherence changes than concrete nouns (C1) within the left hemisphere. In comparison to concrete nouns, the processing of abstract nouns seems to be shifted to mid-temporal regions, which may support our assumption that concrete and abstract nouns have a distinct lexical organization within the brain. In addition, this study shows a massive engagement of the right hemisphere in abstract word processing, and it seems that the right posterior part is strongly coupled with the left hemisphere. In summary, concrete and abstract nouns seem to be represented within the brain as different patterns of networks, which seem to be based on different “main processing nodes” and connections that may link separate regions. During the processing of concrete nouns the functional units which become coupled are more widely distributed within distinct brain regions than during processing of abstract nouns. As mentioned before some patients show greater difficulties in the processing of abstract nouns than of concrete ones. Consequently, one can speculate that, in spite of a lesion of the cortical network, the concrete nouns can be accessed via different pathways and the disturbance remains mild. The retrieval of (or access to) the abstract representations is disturbed more easily, because fewer brain regions can be used to compensate the defect of the network. 3.2.2. Experiment II One of the main results of experiment I was that the alpha-1 band reflects processes that are similar for both concrete and abstract nouns, such as sensory or mnemonic processes, whereas the beta-1 activities seem to be closely related to super-ordinate semantic processes and more complex associative functions. Experiment II was designed to find out first, whether some frequency bands show amplitude and coherence changes due only to the modality of stimulus presentation—either auditory or visual—and second, whether other frequency bands show modality-independent effects which reflect real cognitive differences between different word classes—either concrete and abstract nouns (Weiss and Rappelsberger, 1998). The experiment was conducted on 25 right-handed females. Subjects had to meet the same conditions as in experiment I. The experimental set up was similar except that the number of words presented in the auditory modality was doubled, and in addition words were presented visually on a screen 120 cm in front of the subjects. Character height was

Coherence and phase relations between EEG

363

1.5 cm and stimulus duration was 0.5 s. Participants were requested to memorize the nouns presented in auditory and visual modality, and had to recall them after the presentation. As in experiment I, EEG was recorded with 19 electrodes under the same recording conditions, that is during memorizing the different lists of nouns, and during four resting periods lasting one minute each, with eyes opened. In contrast to experiment I only 1 s epochs after onset of the presentation were selected for further processing, in order to exclude additional cognitive processes, such as the intentional generation of mental images, which is known to last at least 1 to 2 seconds. Coherence and amplitude were computed for the frequency bands delta (1–4 Hz), theta (5–7 Hz), alpha-1 (8–10 Hz), alpha–2 (11–12 Hz), beta–1 (13–18 Hz) and beta–2 (19–31 Hz). As in experiment I, statistical analysis between selected conditions was performed using paired Wilcoxon-tests. The resulting error probabilities were presented in topographic maps. However, it must be emphasized again, that, from the statistical point of view, due to the numerous parallel comparisons, the results can be used only for exploration and description. In other words, they give hints at the potential differences existing over the many comparisons, but cannot be used to reject or accept a null hypotheses. Concerning EEG data, according to our study aim, two main findings were obtained. First, the alpha-1 band indicates modality-specific EEG correlates, and second, the delta, theta and beta-1 bands show left frontal EEG properties independent of the modality of stimulus presentation. Figure 4 presents the coherence and amplitude probability maps of comparing word processing and EEG at rest in the alpha-1 band for both, the auditory presentation and the visual presentation. The alpha-1 band does not indicate an essential difference between concrete and abstract noun processing. However, there is a clear difference between the auditory and visual mode of stimulus presentation. During memorizing of nouns presented in the auditory mode, decreases in alpha-1 amplitudes with respect to the EEG at rest were seen only at electrodes T3 and T4, with increases at O2. The desynchronization at T3 and T4 may be caused by an increase of activity in underlying brain regions accompanying auditory processing. During memorizing of visually presented nouns in most electrodes alpha-1 desynchronization is observed over wide areas, probably involved in word processing during the first second after the stimulus. The different amplitude findings may be explained by postulating that auditory word processing is a sequential process lasting until the end of the word presentation, on the average 760 ms after stimulus onset in our experiment. In contrast, visual word processing is performed within the first 100 to 200 ms after stimulus perception. These differences may be responsible for the different alpha-1 desynchronization as demonstrated in Figure 4. Memorizing of words presented in auditory and visual modes leads to different patterns of coherence changes. While the auditory modality exhibits numerous coherence increases between different electrode pairs, visual processing shows only a few coherence changes. Comparison of the results of abstract and concrete word processing within one of the two stimulus modalities exhibits minor differences, as compared with the findings between modalities.

Time and the brain

364

Modality-independent effects, i.e. significant EEG coherence differences reflecting the different processing procedures for concrete and abstract nouns, independent of stimulus presentation, were found in the delta, theta and beta-1 bands. Results are summarized in Figure 5 and exhibit higher coherence between left frontal electrodes during concrete word processing, compared with abstract word processing. The connections between electrode F3 (“main processing node”) and the others seem to be most prominent.

Figure 4. Error probability maps of the alpha-1 band (8–10 Hz) to demonstrate amplitude and coherence changes during word processing, of either concrete or abstract nouns, compared to the EEG at rest. Empty electrodes indicate amplitude decreases, full electrodes represent amplitude increases (error probability p≤0.05). At grey shaded electrodes no significant amplitude change was observed. Significant coherence change is indicated by a line between both electrodes. Full lines exhibit coherence increases and broken lines coherence decreases. Thick lines represent an error probability of p≤0.01, thin lines of p≤0.05. Amplitude as well as coherence changes within the alpha-1 band reflect the modality of stimulus presentation, auditory or visual. Almost no differences can be observed concerning the processing of the two different word classes. During auditory presentation amplitude decreased only at T3 and T4 (arrows) probably due to the activation of the underlying brain regions. During memorising of visually-presented nouns, wider brain areas are

Coherence and phase relations between EEG

365

activated. During auditory presentation only coherence increases are found in both word classes, but between numerous electrode pairs. In contrast, during visual presentation only a few coherence changes can be observed, coherence decreasing (mainly) between frontal electrodes and coherence increasing (mainly) between posterior electrodes.

Higher coherence during processing of concrete nouns may be due to a higher synchronization of distributed neuronal systems within the left frontal cortex compared with processing of abstract nouns. This higher synchronization or co-operation probably reflects the differences in semantic encoding of concrete and abstract nouns. Semantic encoding differs for concrete and abstract nouns since concrete material can be categorized and organized more easily because of multiple sources, such as visual images, which support the process.

Figure 5. Error probability maps of modality-independent coherence differences between concrete and abstract nouns (p≤0.05). Memorising of concrete nouns induces significant higher coherence at left frontal electrodes for both the auditory and visual modality.

Activity of the left frontal region, especially the left inferior prefrontal lobe is often correlated with semantic processing of words (McCarthy et al., 1993; Petersen et al., 1988). Furthermore, Abdullaev and Bechtereva (1993) reported about a patient with implanted intracortical electrodes who performed a lexical decision task with concrete and abstract nouns. Neurones in the left prefrontal cortex (area 10 and 46) responded only during the presentation of abstract nouns and not during the presentation of concrete nouns. Modality-independent coherence differences between concrete and abstract nouns at left frontal electrodes were found in delta, theta and beta-1 bands. According to Klimesch et al. (1996) episodic memory processes are reflected in the theta band, whereas semantic long term memory processes are reflected in the upper alpha band, or higher frequencies

Time and the brain

366

(see also chapters by Klimesch, this volume). Concerning the delta band, there are hints (Başar et al., 1994) that it reflects global mechanisms like focused attention and signal detection. In summary, our EEG coherence studies revealed that the alpha-1 band is sensitive to the modality of stimulus presentation, but does not distinguish between the memorization of abstract and concrete nouns. This was also described in a similar study by Weiss et al. (1997) and for sentence processing by Müller et al. (1997). Delta, theta and beta-1 activities seem to be sensitive to other cognitive processes, especially the differences in the semantic encoding of distinct word classes.

4. PHASE ANALYSES An issue of special significance for topographic studies is the detection and measurement of time delays between different recording sites. The presence of a time delay is indicated in the phase spectrum (equation 6 of the Appendix). For a single frequency ƒ the time delay τ [ms] is determined from the estimated phase φ (ƒ) given in [rad] by:

In a frequency band the existence of a time delay is indicated by a linear relationship between phase and frequency across the band. The magnitude of delay is proportional to the slope of the line (Rappelsberger et al., 1987). A measured time delay is indicated if the signal recorded at one site leads or lags behind the signal recorded at another site, and thus hints at the direction of information transfer within the given frequency band. 4.1. Group Study Figure 6 shows the results of phase analyses of the group of 25 females described for experiment II in the previous section. 25 concrete and 25 abstract words were presented visually, and averaged Fourier cross-spectra were computed for phase estimations. Because the presentation was visual and the visual system was involved, interest was concentrated on phase relations between both occipital electrodes and other sites of the corresponding hemisphere. Frequency bands alpha-1 and beta-1 were examined. The absolute time delays obtained ranged from approximately 0 ms between closely spaced electrodes to 17 ms between occipital and frontopolar leads. There was a clear correlation between absolute time delay and electrode distance. Comparing the results in the two frequency bands, some differences can be seen concerning the direction of information transfer. The differences are larger for abstract words than for concrete words. Comparing the results between the two stimulus conditions, in the alpha-1 band almost identical pictures are obtained. In contrast, in the beta-1 band clear differences appear between concrete and abstract word processing with respect to the direction of information transfer between occipital and frontal (O1-F3, O2-

Coherence and phase relations between EEG

367

F4) and occipital and frontopolar (O1-Fp1, O2-Fp2) leads. These results confirm the coherence findings (section 3) which showed that activity in the beta-1 reveal differences in cognitive processing and that the alpha-1 band is sensitive to the modality of stimulus presentation. Word processing is not a static but a highly dynamic process. Both phase and coherence analysis using an epoch length of 1 s as above can yield only a global and smeared image of the process. From the technical point of view an interpretation of phase relations is only sensible if coherence between both signals is high enough. The reason is that variance of phase estimation is inversely proportional to coherence (Rappelsberger, 1977; Bendat and Piersol, 1986; Kelly et al., 1997). Low coherence means high phase variance and consequently broad confidence intervals. In fact, low coherence values of about 0.10 were obtained for wide electrode distances from occipital to frontal. Such values can also easily occur for uncorrelated signals, if the averaging process for a reliable estimation is insufficient. In our case threefold averaging was performed: averaging over 3 (alpha-1) or 5 (beta-1) adjacent spectral lines, averaging over 25 trials per person, and averaging over 25 subjects. Thus the obtained

Figure 6. Grand mean time delays of 25 female volunteers between O1 and O2, respectively, and all other intrahemispheric electrode positions during visual presentation of concrete and abstract words. Results in

Time and the brain

368

frequency bands alpha-1 and beta-1 are depicted. Solid lines indicate time delays with O1 and O2, respectively, leading, broken lines indicate the opposite, i.e. information transfer towards occipital electrodes. The absolute time delays range from about 0 ms to 17 ms. Absolute time delay increases with increasing electrode distance. Only time relations exceeding 1ms absolute are presented. Note the almost identical pictures in the alpha-1 band and the differences between concrete and abstract word processing concerning direction of information transfer between occipital and frontal and occipital and fronto-polar leads in beta-1.

number of degrees of freedom is high, yielding small confidence intervals both for coherence and phase. Despite low coherence values of 0.1 or less, the results can be considered as reliable.

Coherence and phase relations between EEG

369

4.2. Single Subject Study Time-dependent coherence and phase analyses of single trials were performed using an adaptive autoregressive moving average (ARMA) model, as outlined in the technical part and the Appendix (equations 7 to 15). An example of a single trial instantaneous coherence and instantaneous phase analysis between O1-T5 is presented in Figure 1 for two frequency bands. Time scale of the graphs is 1 s, starting with the presentation of a visual stimulus. Both coherence and phase show considerable fluctuations during word processing. For instance, alpha-1 coherence ranges from approximately 0 at about the 250-millisecond point to almost 1 around the 500-millisecond point. Alpha-1 phase at around the 150-millisecond point is about -10 ms, which may be interpreted as an information transfer from T5 to O1. In contrast, phase around the 800-millisecond point has changed to about +10 ms, indicating an information transfer in the opposite direction. Despite the considerable change of coherence and phase over time, short sections in the range of 100 ms and longer with approximately constant coherence or phase can also be observed. In order to reduce trial-to-trial variance, and to obtain reliable results, in the following examples instantaneous coherence and instantaneous phase were averaged over trials, time-point by time-point. Figure 7 presents the time relations between occipital electrodes and the other electrodes of the same hemisphere. Frequency bands alpha-1 and beta-1 are depicted for concrete visual word presentation. The first three columns relate to time points 200, 400 and 800 ms after stimulus onset. Only those phase relations were taken into consideration which were approximately stable within a time window of about 100 ms around the chosen time-points. The visual stimulus lasted 500 ms. For comparison, the right column

Figure 7. Time relations in a single subject between occipital electrodes and all

Time and the brain

370

other intrahemispheric positions in the alpha-1 and the beta-1 bands during concrete word processing after visual presentation. The first three columns present results at different times after stimulus onset. Stimulus lasted about 500 ms. Estimation was performed with an adaptive ARMA model. The right column presents the results estimated from Fourier spectra with an epoch length of 1 s following stimulus onset. Solid lines indicate positive time delays, i.e. the occipital electrodes are leading, and broken lines indicate negative time delays, i.e. information transfer towards the occipital electrodes. In the first three columns only stable results within an interval of about 100 ms around the given time point are taken into consideration. In the right column only absolute values higher than 1ms are considered.

presents the phase results estimated from Fourier spectra with an epoch length of 1 s starting with stimulus onset. Comparing the maps at 200 ms and 400 ms in both frequency bands no significant difference, (i.e. change of direction of information transfer) is observed. However, in the alpha-1 band at the 800-millisecond time-point, direction had changed between O1-F7, O1-F3, O2-F4 and O2-C4. This effect is much more pronounced in the beta-1 band, for which direction of information transfer changed between almost all electrode pairs. This may be caused by the altered information processing or cognitive state after the end of the stimulus at the 500-millisecond time-point. Figure 8 demonstrates coherence and phase values in the beta-1 band at the 400millisecond time-point after stimulus onset (compare Figure 7). The bars show decreasing coherence, from about 0.8 between occipito-parieto-temporal leads to about 0.4 between occipito-frontal leads. Phase values increase from a few milliseconds between closely spaced electrodes to about 15 ms for wide distances. There seems to be a trend to smaller time delays in the right than in the left hemisphere. Abstract word processing revealed the opposite, i.e. lower time delays in the left than in the right hemisphere. However, whether this is a real finding or falls within the expected limits of statistical deviations needs further examination. Until now, the data of only 3 subjects are available, but all 3 show this effect. Figure 9 shows coherence and phase values obtained from Fourier spectra. Whereas in Figure 8 only a short time window of about 100 ms, at about 400 ms after stimulus onset, was considered, Fourier spectra are based on an epoch length of 1 s after stimulus onset. Because of the reduction in time resolution the short lasting properties during word processing are smeared, and to a great extent extinguished. The trends towards decreasing coherence and increasing absolute phase are preserved, but the interpretation of phase must be done with care since variance caused by low and almost zero coherence is high (Rappelsberger, 1977; Bendat and Piersol, 1986; Kelly et al., 1997). In a single subject the averaging processes over adjacent spectral lines (and over trials) could not yield enough degrees of freedom to compensate this disadvantage, as in the grand mean study (compare Figure 6). A central issue in cognitive neuroscience concerns the question of whether the functions of the cerebral cortex are localized in circumscribed areas, or are equally

Coherence and phase relations between EEG

371

represented throughout the entire cortex. Solution of this controversy came about with the realization that cortical areas do perform unique elementary functions, but that complex functions require the integrated action of many areas distributed throughout both cerebral hemispheres. Therefore, a complex function may be considered to be a system of interrelated processes directed toward the performance of a particular task, which is implemented neuronally by a network of functionally related areas. Large scale networks of cortical areas are seen as essential for high-level functions underlying cognition, in the special case of word processing. Of course, since all cortical areas have extensive subcortical connections, large scale cortical networks operate as part of even larger, whole-brain systems (Bressler, 1995). Cortical areas may not only become co-active during word processing, but also may become functionally connected. Thus fragmentary sensory, memory, motor and associative processes, occurring in individual cortical areas may become integrated by the functional connection of those areas in large-scale networks.

Time and the brain

372

Figure 8. Mean coherence and phase values within a time window of about 100 ms round 400 ms after onset of visual presentation of 25 concrete nouns. For coherence and phase estimation an ARMA model was used. Coherence values in the beta-1 band decrease from about 0.8 between closely spaced electrodes to about 0.4 between widely spaced electrodes. In contrast, phase values increase from a few milliseconds to almost 15 ms. The missing bar for phase O1-Fp1 corresponds to a missing phase stability within the observation window (compare Figure 7).

In the visual system, parallel pathways project from the retina through the lateral geniculate nucleus and primary visual cortex, and widespread divergence and

Coherence and phase relations between EEG

373

convergence of projections exist among extrastriate areas. Functionally distinct, parallel visual subsystems in the temporal and parietal regions receive projections from multiple extrastriate areas. Moreover, neurones in multiple visual areas show temporally-sustained and

Figure 9. Mean coherence and phase values in the beta-1 band estimated with Fourier spectra based on 1 s epochs after visual stimulus onset. The results are due to presentation of 25 concrete nouns. Compare right lower phase map of Figure 7. Due to the reduction of time-resolution, results are smeared values and generally lower than those presented in Figure 8.

Time and the brain

374

overlapping responses to patterned visual stimuli. These findings suggest that the visual cortex is best viewed as a large scale network with parallel processing capability (Bressler, 1995). Visual word stimulation activates first of all the visual system, resulting in information transfer from the primary visual cortex to higher level visual areas and extrastriate areas. During the processing, functional connections within these large scale net-works can change, as can also the direction of information transfer. This may explain the results demonstrated in Figure 6. Due to the meaning of different frequency bands (and their possible correlation with different simultaneously-acting large scale networks) relations in (for example) the alpha-1 and beta-1 bands can show opposite behaviour. Non-visual areas are also interconnected in large scale networks. Much is known about subdivisions of the posterior parietal cortex, which do not simply relay sensory information to motor areas, but are very strongly (and uniquely) interconnected with multiple areas of sensory, limbic and frontal cortex. Auditory areas can be assumed to be similarly organized and interconnected in large scale networks. Figure 10 demonstrates phase relations at different time points after the onset of auditory presentation of concrete nouns. Because the stimulation was in the auditory modality, electrodes T3 and T4 were taken as reference for phase estimations using adaptive ARMA models. Just as for the data in Figure 7, a time window of about 100 ms was used to find out stable phase relations. The right column of Figure 10 depicts the corresponding results of Fourier analysis.

Figure 10. Time relations in a single subject between temporal electrodes and all other intrahemispheric positions in the alpha-1 and the beta-1 band during concrete word processing after acoustic presentation. The first three columns present results at different times after stimulus onset,

Coherence and phase relations between EEG

375

which lasted 760 ms on the average. Estimation was performed with an adaptive ARMA model. The right column presents the results estimated from Fourier spectra with an epoch length of 1 s following the stimulus onset. Solid lines indicate positive time delays, i.e. the temporal electrodes are leading, and broken lines indicate negative time delays, i.e. information transfer towards the temporal electrodes. In the first three columns only stable results within an interval of about 100 ms around the given time point are taken into consideration. In the fourth column only absolute time delays higher than 1 ms are considered.

In the alpha-1 band there is a trend towards a temporo-parieto-occipital information transfer whereas the tranfer is in the opposite direction between frontal and temporal electrodes. These properties can also be observed in the right column of figure 10 despite the smearing effect of the Fourier spectra. In the beta-1 band, relations change from timepoint to time-point within the processing interval, hinting at a reorganisation of the large scale networks necessary for word processing. Comparison of phase relations for the two modalities of stimulus presentation is shown in Figure 11. The results relate to the beta-1 band at a time-point 400 ms after stimulus onset. Phase relations are different, depending on the modality of stimulus presentation. During visual presentation there is a clear positive phase relation between the “specific” occipital electrodes and the other intrahemispheric sites, but also from the “non-specific” temporal electrodes to frontal sites as depicted in the right upper map. During auditory stimulation the relations are not so clear. The “specific” electrodes, T3 and T4, tend to lead occipital sites but there is also an indication that “non-specific” occipital electrodes lead central and left frontal electrodes. When two more subjects, analysed with the adaptive ARMA model, were included, the results demonstrated high

Time and the brain

376

Figure 11. Comparison of phase relations between visual and acoustic presentation of concrete nouns 400 ms after stimulus onset in the beta-1 band. Compare Figure 7 with the left upper map and Figure 10 with the lower right map. Left and right columns relate to occipital and temporal reference electrodes for phase estimations. Note the different phase relations when stimulated visually and acoustically.

stability in the beta-1 band when stimulated visually. In contrast, in the same frequency band, auditory stimulation showed relatively divergent phase results. A possible explanation for these findings (based however on only three subjects until now) could be the sequential acoustic information processing during hearing of the words, leading to a greater temporal dispersion of the connections within the sequential large scale networks. The concept of large-scale networks provides a reasonable framework for integrating results of different studies on distributed functioning of the cerebral cortex. According to this description, elementary functions are localized in discrete cortical areas, whereas complex functions like word processing are processed in parallel in widespread cortical networks. Coherence and phase analyses may help to understand the nature and dynamics of different cognitive processes, as attempted in this contribution.

Coherence and phase relations between EEG

377

5. CONCLUSION Modern computer aided analyses of bioelectric signals offer excellent possibilities for the investigation of complex cognitive processes. In particular, the non-invasive recording of electrical activity of the human brain, the electroencephalogram (EEG), a technique which has a 70 years history, has strongly influenced our present knowledge about normal and disturbed human brain functioning. In contrast to sophisticated neuroimaging techniques the relatively inexpensive EEG yields high temporal resolution in the measurements of electrophysiological correlates of cognitive processes, which is especially important for the investigation of highly complex processes such as language comprehension. Up to now, most of the EEG-studies concerning language processing have used eventrelated potentials (ERP) which are time-locked to a particular sensory stimulus, and which are extracted from the background EEG by signal-averaging techniques (e.g. Kutas and Van Petten, 1994; Müller et al., 1997). Despite all the advantages of this method, ERP analysis provides neither frequency information nor information about the cortical interplay or co-operation between different parts of the brain during cognitive processing. However, knowledge about activities within certain frequency bands, as well as information about cortical co-operation, seems to be necessary for an adequate characterization of brain functioning. Hence, spectral analysis of the EEG, especially coherence analysis during cognitive processing, seems to be one of the most suitable methods for the analysis of higher cognitive processes. Coherence between two EEG signals is the correlation coefficient in the frequency domain, and can be interpreted as a measure of the functional co-operation or information transfer between the underlying generators of the EEG signals. High coherence between EEG signals recorded at different sites of the scalp may hint at an increased functional interplay between the underlying neuronal networks. In studying functional relationships, coherence analyses yield important new aspects of brain activities, which complement the data obtained by power spectral analyses. In the light of recent neurobiological findings coherence analysis of the human EEG has gained additional importance. EEG coherence analysis is even more suited for the investigation of higher cognitive processes since current neurobiological findings report about changes of synchronisation within narrow frequency bands (Singer, 1994) as correlates of information processing. As demonstrated in this chapter computation of coherence between EEG signals of electrodes placed on different sites of the scalp turns out to be an adequate parameter to study frequency-band-related “synchronization” during language processing on a macro scale. In the thesis of Weiss (1994) the first hints were given that coherence in the alpha-1 frequency band (8–10 Hz) reflects changes which are similar for the processing of concrete as well as abstract nouns, whereas coherence in the beta-1 band (13–18 Hz) reflects real cognitive-linguistic differences between these word types. Further experiments indicated that alpha-1 is engaged mainly in the auditory processing of stimuli (Weiss and Rappelsberger, 1996; Weiss, 1997; Weiss et al., 1997, 1999). In the

Time and the brain

378

beta-1 band auditorily presented concrete nouns elicited a higher number of coherences between distributed brain regions than did abstract nouns. From these results it was hypothesized (Weiss and Rappelsberger, 1996) that due to the multimodal nature of concrete nouns more distributed brain areas should co-operate and should be functionally synchronized. Further experiments and coherence analyses revealed that the alpha-1 band reflects the modality of stimulus presentation, and therefore differentiates between the auditory and the visual modality but does not differentiate between concrete and abstract nouns. All other frequency bands show modality specific EEG differences between memorizing concrete and abstract nouns. The delta, theta and beta-1 bands demonstrate that memorizing of concrete nouns induces higher coherence than abstract nouns in the left frontal areas independent of the modality of stimulus presentation. This modalityindependent higher synchronization probably reflects encoding strategies which differ between concrete and abstract nouns. For the estimation of time relations, i.e. the direction of information transfer, between different recording sites during memorizing of nouns, two approaches were followed. In group studies Fourier transform was applied to compute cross-spectra between pairs of electrode signals of EEG trials of constant length. The cross-spectra of a number of trials were averaged, yielding coherence and phase spectra representing the mean properties within the length of the trials, usually 1 or 2 seconds. In single subject studies an adaptive autoregressive moving average (ARMA) model was used to compute instantaneous coherence and phase values with a time resolution in the millisecond range. Due to the highly dynamic process with changing directions of information transfer the Fourier approach based on 1 s trials after stimulus onset yields only very coarse estimations of the time relations during word processing. The short-lasting properties during word processing are smeared and to a great extent extinguished. In single subject studies, the ARMA approach clearly demonstrates an occipital-frontal information transfer in the beta-1 band for the visual stimulus presentation. During auditory stimulus presentation temporal sites tend to lead occipital sites but there is also a trend of occipital sites to lead central and left frontal sites. These analyses are completely new, and need more data in order to find appropriate interpretations of the findings made so far. In any case, phase relations during word processing are important new parameters in studying cognitive processes.

Coherence and phase relations between EEG

379

ACKNOWLEDGEMENT The presented studies were supported by the Austrian Science Foundation-FWF (P111572-MED and P13578-MED).

APPENDIX Definition and Computation of Coherence Mathematically, coherence is defined as the squared normalized cross-power spectrum and may be written:

(1) S12(f) is the cross-power spectrum and S11(f) and S22(f) are the auto-spectra of two EEG signals x1,t and x2,t, respectively. Fourier transform of EEG signals x1,t and x2,t yields and Usually, X1(f) and X2(f) are complex functions and periodograms are obtained according to

(2) and are the conjugate complex functions of X1(f) and X2(ƒ), respectively. Periodograms have bad statistical properties, their variance being independent of the epoch length. To improve those properties smoothing techniques have to be applied. Averaging over periodograms yields smoothed spectra

(3)

‘E’ denotes expectation or average which may be written for autospectrum

(4) with “n” being the number of epochs. Beside the application of the above averaging technique, in EEG analysis subdivision of EEG records into epochs has the advantage

Time and the brain

380

that artefact-contaminated or distorted epochs can easily be rejected from further analysis. Replacing the spectra in equation (1) by the smoothed spectra, squared smoothed coherence can be written:

(5) For the complete description of a complex function like the cross-power spectrum not only the absolute value but also the phase angle is needed. The phase angle indicates if signal x1,t is leading or lagging x2,t with respect to a certain frequency component and may be expressed:

(6) Re and Im denote the real and imaginary part of the cross power spectrum, respectively. Beside multivariate spectral analysis methods based on the Fourier transform, the fit of linear multivariate models, and the parametric calculation of the spectral density matrix and other spectral parameters can be used. In the following section, a brief explanation will be given of the adaptive bivariate autoregressive moving average (ARMA) method to estimate instantaneous coherence and phase (Schack, 1997). Let

(7) be the sample values of two channels of an EEG record. The two-dimensional EEG signal is fitted by a two-dimensional linear model, which is able to react to structural changes of the signals. This is achieved by an autoregressive moving average (ARMA) model with time-varying parameters:

(8) p and q are the model orders, z1 is a two-dimensional noise process, y1 is the twodimensional model process, and and are 2*2 matrices of the autoregressive and moving average parameters, respectively. The parameter matrices are estimated at every sample point by minimising the prediction error of the model. Estimation is realized by the following adaptive estimation procedure:

(9)

Coherence and phase relations between EEG

381

where is the estimated vector sequence of the prediction error and {c1}t = 1,2,... is a control sequence determining the speed of adaptation (Schack et al., 1995). The orders p and q are determined using usual estimation algorithms (Hannan and Kavalieris, 1984). The instantaneous transfer function of the fitted ARMA model may be calculated by the formula

(10) with the instantaneous matrices

(11)

may be The instantaneous covariance matrix Ct of the bivariate prediction error estimated adaptively (Schack et al., 1995) and is used to compute the spectral density matrix at every sample point:

(12) denotes the complex conjugate and transpose of Ĥ1(f). The adaptively estimated spectral density matrix

(13) is a function of frequency and time. Finally, the instantaneous squared coherence function is yielded:

(14) Correspondingly, the instantaneous cross-phase spectrum may be calculated:

(15) Time-dependent coherence and phase functions can be estimated for every EEG

Time and the brain

382

channelpair and for every frequency or frequency band. Thus single trial analysis of various EEG events can be performed. This is demonstrated in Figure 1. Model orders for the examples presented in this paper were fixed with p=15 for the autoregressive, and q=5 for the moving average process. Due to the recursive adaptive algorithm a time delay occurs in the estimation, which may be controlled by the sequence . Furthermore, there is a delay in detecting an oscillatory frequency of about a half wave, since at least that time is necessary to recognize an oscillation, i.e. about 55 ms for alpha-1 activity and about 33 ms for beta-1 activity (see Figure 1). Interpretation of Coherence Figure 2 demonstrates the coupling of two neuronal assemblies generating the EEG signals x1, t and x2,t, and also shows how the estimated coherence can be used to describe this coupling. Assume both signals are composed of a coherent part, representing the cooperation of both cell assemblies, and an incoherent part or noise. The noise portion n1,t and n2,t, respectively, may be due to the EEG activity produced by nerve cells within one assembly which are not involved in the generation of the coherent portion st. Mathematically, this model can be expressed:

(16) “a” and “b” are simple scaling parameters to take into account different amplitudes of the coherent portions within both signals. If all neurones within one assembly are busy only in generating the coherent EEG signal, the noise portion will vanish. In contrast, if no coupling exists between both assemblies, the coherent portion will converge towards zero. Fourier transform of x1,t and x2,t, yields:

(17) In the following, capital letters are used to express functions in the frequency domain. For simplification the frequency-dependence is omitted in the equations below. According to equations (2) periodograms result in:

(18) Because of the assumed independence of the EEG signal s1 and the noise signals nl,t and n2,t and therefore the independence of the Fourier transform S, N1 and N2, by sufficient smoothing the mixed terms in S11(f), S22(f) and S12(f) will converge towards zero and the smoothed spectra result in:

Coherence and phase relations between EEG

383

(19)

Finally, according to equation (5), smoothed squared coherence is obtained:

(20) with

(21) K1 and K2 are the noise-to-signal ratios of the two signals X1,t and x2,t in the frequency domain. Therefore, equation (20) demonstrates that coherence can be expressed by signal-to-noise and noise-to-signal ratios, respectively. If both neuronal assemblies are completely synchronized, i.e. both noise terms nl,t and n2,t in equation (16) vanish, then In contrast, if there is no coherent activity between both cell assemblies, corresponding to st=0. REFERENCES Abdullaev, Y.G., and Bechtereva, N.P. (1993) Neuronal correlate of the higher-order semantic code in human prefrontal cortex in language tasks. International Journal of Psychophysiology, 14, 167–177. Abt, K. (1988) Descriptive data analysis (DDA) in quantitative EEG studies. In: D.Samson-Dollfus (ed.) Statistics and Topography in Quantitative EEG. Amsterdam: Elsevier, pp. 150–160. Başar, E., Schürmann, M., Başar-Eroglu, C., and Demiralp, T. (1994) Theta and delta responses in cognitive event-related potential paradigms and their possible psychophysiological correlates. In: H-J.Heinze, T.F.Münte and G.R.Mangun (eds) Cognitive Electrophysiology. Birkhäuser, Boston, pp. 335-367. Başar, E., Schürmann, M., Başar-Eroglu, C., and Karakas, S. (1997) Alpha oscillations in brain functioning: an integrative theory. International Journal of Psychophysiology, 26, 5–29. Başar-Eroglu, C., Strüber, D., Schürmann, M., Stadler, M., and Başar, E. (1996) Gammaband responses in the brain: a short review of psychophysiological correlates and functional significance. International Journal of Psychophysiology, 24, 101–112. Beaumont, J.G., Mayer, A.R., and Rugg, M.D. (1978) Asymmetry in EEG alpha coherence and power: effects of task and sex. Electroencephalography and Clinical Neurophysiology, 45, 393–401. Bendat, J.S., and Piersol, A.G. (1986) Random Data. Analysis and Measurement Procedures, second edition. New York: Wiley.

Time and the brain

384

Bleasdale, F.A. (1987) Concreteness-Dependent Associative Priming: Separate Lexical Organization for Concrete and Abstract Words. Journal of Experimental Psychology: Learning, Memory and Cognition, 13, 582–594. Bressler, St.L. (1990) The gamma wave: a cortical information carrier? Trends in Neurosciences, 13, 161–162. Bressler, St.L. (1995) Large scale cortical networks and cognition. Brain Research Reviews, 20, 288–304. Busk, J., and Galbraith, G.C. (1975) EEG correlates of visual motor practice in man. Electroencephalography and Clinical Neurophysiology, 38, 415–422. Chen, A., and Rappelsberger, P. (1994) Brain and Human Pain: Topographic Amplitude and Coherence Mapping. Brain Topography, 7, 129–140. Colter, N., and Shaw, J.C. (1982) EEG coherence analysis and field dependence. Biological Psychology, 15, 215–228. Coltheart, M., Patterson, K.E., and Marshall, J.C. (1987) Deep dyslexia, second edition. London: Routledge and Kegan Paul. Cooley, J.W., and Tukey, J.W. (1965) An algorithm for machine calculation of complex Fourier series. Mathematical Computation, 19, 297–301. Damasio, H., Grabowski, T.J., Tranel, D., Hichwa, R.D., and Damasio, A.R. (1996) A neural basis for lexical retrieval. Nature, London, 380, 499–505. Duckrow, R.B., and Spencer, S.S. (1992) Regional coherence and the transfer of ictal activity during seizure onset in the medial temporal lobe. Electroencephalography and Clinical Neurophysiology, 82, 415–422. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H.J. (1988) Coherent oscillations: A mechanism for feature linking in the visual cortex? Biological Cybernetics, 60, 121–130. Essl, M., and Rappelsberger, P. (1998) EEG coherence and reference: experimental results and mathematical explanations. Journal of Medical & Biological Engineering & Computing, in press. Eviatar, Z., Menn, L., and Zaidel, E. (1990) Concreteness: nouns, verbs, and hemispheres. Cortex, 26, 611–624. Farah, M.J. (1989) The neural basis of mental imagery. Trends in Neurosciences, 12, 395–399. French, C.Ch., and Beaumont, J.G. (1984) A critical review of EEG coherence studies of hemispheric function. International Journal of Psychophysiology, 1, 241–254. Gath, I., Feuerstein, C, Pham, D.T., and Rondouin, G. (1992) On the tracking of rapid dynamic changes in seizure EEG. IEEE Transactions in Biomedical Engineering, 39, 952–958. Gazzaniga, M.S. (1995) The Cognitive Neurosciences. Cambridge: MIT-Press. Gevins, A., Cutillo, B., and DuRousseau, D. (1994a) High-resolution evoked potential technology for imaging neural networks of cognition. In: R.W.Thatcher, M.Hallet and T.Zeffiro (eds) Functional neuroimaging: technical foundations. Orlando FL: Academic, pp. 223–232. Gevins, A., Le, J., Martin, N.K., Brickett, P., Desmond, J., and Reutter, B. (1994b) High resolution EEG: 124-channel recording, spatial deblurring and MRI integration methods. Electroencephalography and Clinical Neurophysiology, 90, 337–358. Goldmann, D. (1950) The clinical use of the average recording in monopolar recording. Electroencephalography and Clinical Neurophysiology, 2, 209. Gotman, J. (1983) Measurement of small time differences between EEG channels: method and application to epileptic seizure propagation. Electroencephalography and

Coherence and phase relations between EEG

385

Clinical Neurophysiology, 56, 501–514. Gotman, J. (1987) Interhemispheric interactions in seizures of focal onset: data from human intracranial recordings. Electroencephalography and Clinical Neurophysiology, 67, 120–133. Gray, C.M., König, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, London, 338, 334–337. Hannan, E.J., and Kavalieris, L. (1984) Multivariate Linear Time Series Models. Biometrika, 69, 81–94. Hillis, A.E., and Caramazza, A. (1995) Representation of grammatical categories of words in the brain. Journal of Cognitive Neuroscience, 7, 396–407. Hinton, G.E., and Shallice, T. (1991) Lesioning an attract network: investigations of acquired dyslexia. Psychological Review, 98, 74–95. Hjorth, B. (1975) An on-line transformation of EEG scalp potentials into orthogonal source derivations. Electroencephalography and Clinical Neurophysiology, 39, 526– 530. Isaksson, W., Wennberg, A., and Zetterberg, L.H. (1981) Computer analysis of EEG signals with parametric models, Proceedings IEEE, 69, 451–461. Jasper, H.H. (1958) The Ten Twenty Electrode System of the International Federation, Electroencephalography and Clinical Neurophysiology, 10, 371–375. Jenkins, G.M., and Watts, D.G. (1968) Spectral analysis and its application. San Francisco: Holden Day. Kaminski, K.J., and Blinowska, K.J. (1991) A new method of the description of the information flow in the brain structures. Biological Cybernetics, 65, 203. Kelly, E.F., Lenz, J.E., Franaszczuk, P.J., and Truong, Y.K. (1997) A general statistical framework for frequency domain analysis of EEG topographic structure. Computers and Biomedical Research, 30, 129–164. Klimesch, W., Schimke, H., and Pfurtscheller, G. (1993) Alpha frequency, cognitive load and memory performance, Brain Topography, 5, 241–251. Klimesch, W., Doppelmayr, M., Russegger, H., and Pachinger, T. (1996) Theta band power in the human scalp EEG and the encoding of new information. NeuroReport, 7, 1235–1240. König, P., Engel, A.K., and Singer, W. (1995) Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A., 92, 290–294. Kutas, M., and Van Petten, C.K. (1994) Psycholinguistics Electrified. Event-related Brain Potential Investigations. In: M.A. Gernsbacher (ed) Handbook of Psycholinguistics. San Diego: Academic Press. Le, J., and Gevins, A. (1993) Method to reduce blur distortion from EEGs using a realistic head model. IEEE Trans actions of Biomedical Engineering, 40, 517–528. Leocani, L., Toro, C., Manganotti, P., Zhuang, P., and Hallet, M. (1997) Event related coherence and event-related de-synchronization/synchronization in the 10 Hz and 20 Hz EEG during self-paced movement. Electroencephalography and Clinical Neurophysiology, 104, 199–206. Llinás, R., and Paré, D. (1996) The brain as a closed system modulated by the senses. In: R.Llinás and P.S.Churchland (eds) The Mind-Brain Continuum. Cambridge: MIT Press. Lopes da Silva, F. (1991) Neural mechanisms underlying brain waves: from neural membranes to networks. Electroencephalography and Clinical Neurophysiology, 79,

Time and the brain

386

81–93. Malsburg, C.v.d. (1981) The correlation theory of brain function. Internal report, MaxPlanck-Institut für Biophysikalische Chemie, Göttingen, Germany. McCarthy, G., Blamire, A.M., Rothman, D.L., Gruetter, R., and Shulman, R.G. (1993) Echo-planar magnetic resonance imaging studies of frontal cortex activation during word generation in humans. Proceedings of the National Academy of Sciences, U.S.A., 90, 4952–4956. Meier, H. (1967) Deutsche Sprachstatistik, Hildesheim: Olms. Müller, H.M., King, J.W., and Kutas, M. (1997) Event related potentials elicited by spoken relative clauses. Cognitive Brain Research, 5, 193–203. Müller, H.M., Weiss, S., and Rappelsberger, P. (1997) EEG coherence analysis of auditory sentence processing. In: H.Witte, U.Zwiener, B.Schack, and A.Doering (eds) Quantitative and Topological EEG and MEG Analysis. Druckhaus Mayer Verlag, Jena, pp. 440–442. Nunez, P.L. (1995) Neocortical Dynamics and Human EEG Rhythms. New York: Oxford University Press. Nunez, P.L. (1997) EEG Coherence Measures in Medical and Cognitive Science: A general Overview of experimental methods, computer algorithms, and Accuracy. In: H.Witte, U.Zwiener, B.Schack, and A.Doering , (eds) Quantitative and Topological EEG and MEG Analysis. Druckhaus Mayer Verlag, Jena, pp. 385–392. Offner, F.F. (1950) The EEG as potential mapping: The value of the average monopolar reference. Electroencephalography and Clinical Neurophysiology, 2, pp. 213. Petersen, S.P., Fox, P.T., Posner, M.I., Mintun, M., and Raichle, M.E. (1988) Positron emission tomographic studies of the cortical anatomy of single-word processing. Nature, London, 331, 585–589. Petsche, H., Lacroix, D., Lindner, K., Rappelsberger, P., and Schmidt-Henrich, E. (1992) Thinking with images or thinking with language: a pilot EEG probability mapping study. International Journal of Psychophysiology, 12, 31–39. Petsche, H., Etlinger, S.C., and Filz, O. (1993) Brain electrical mechanisms of bilingual speech management: an initial investigation. Electroencephalography and Clinical Neurophysiology, 86, 385–394. Rappelsberger, P. (1977) Spektrale Kurzzeitanalyse stochastischer Signale zur Auswertung von Elektroenzephalogrammen. Doctoral Thesis, University of Technology, Vienna. Rappelsberger, P., Pockberger, H., and Petsche, H. (1987) Evaluation of relationships between seizure potentials. In: H.G.Wieser and Ch.Elger, Methods of presurgical evaluations of epileptic patients. Springer, Heidelberg, pp. 215–222. Rappelsberger, P and Petsche, H. (1988) Probability mapping: power and coherence analyses of cognitive processes. Brain Topography, 1, 46–54. Rappelsberger, P. (1989) The reference problem and mapping of coherence: a simulation study. Brain Topography, 2, 63–72. Rappelsberger, P., Lacroix, D., Steinberger, K., and Thau, K. (1994) EEG-Amplitudenund Kohärenzanalysen bei medikamentenfreien Schizophrenen. Zeitschrift für Electroencephalographie und Electromyographie, 25, 144–150. Rescher, B., and Rappelsberger, P. (1996) EEG-changes in amplitude and coherence during a tactile task in females and in males. Journal of Psychophysiology, 10, 161– 172. Sarnthein, J., vonStein, A., Rappelsberger, P., Petsche, H., Rauscher, F., and Shaw, G. (1997) Persistent patterns of brain activity: an EEG coherence study of the positive

Coherence and phase relations between EEG

387

effect of music on spatial-temporal reasoning. Neurological Research, 19, 107–116. Schack, B., Grieszbach, G., Arnold, M., and Bolten, J. (1995) Dynamic cross-spectral analysis of biological signals by means of bivariate ARMA processes with timedependent coefficients. Journal of Medical & Biological Engineering & Computing, 33, 605–610. Schack, B., and Krause, W. (1995) Dynamic power and coherence analysis of ultra shortterm cognitive processes—a methodical study. Brain Topography, 8, 127–136. Schack, B. (1997) Adaptive Verfahren zur Spektralanalyse instationärer mehrdimensionaler biologischer Signale. Habilitation Thesis, Technical University Ilmenau, Germany. Schümann, M., and Başar, E. (1994) Topography of alpha and theta oscillatory responses upon auditory and visual stimuli in humans. Biological Cybernetics, 72, 161–174. Scheuler, W., Rappelsberger, P., Schmatz, F., Petsche, H., and Kubicki, St. (1990) Periodicity analysis of sleep-EEG: abnormal arousal periodicities. Electroencephalography and Clinical Neurophysiology, 76, 222–234. Schmid, R.G., Tirsch, W.S., Rappelsberger, P., Weinmann, H.M., and Pöppl, S.J. (1992) Comparative coherence studies in healthy volunteers and Down Syndrom patients from childhood to adult age. Electroencephalography and Clinical Neurophysiology, 83, 112–123. Shaw, J.C., O, Connor, K.P., and Ongley, C.O., (1976) The EEG as a measure of cerebral functional organization. British Journal of Psychiatry, 130, 260–264. Singer, W. (1993) Synchronization of cortical activity and its putative role in information processing and learning. Annual Review of Physiology, 55, 349–374. Singer, W. (1994) Coherence as an organizing principle of cortical functions. International Review of Neurobiology, 37, 153–183. Sklar, R.L., Hanley, J., and Simmons, W.W. (1972) An EEG experiment aimed toward identifying dyslexic children. Nature, London, 240, 414–416. Tauscher, J., Fischer, P., Neumeister, A., Rappelsberger, P., and Kasper, S. (1998) Low frontal EEG coherence in neuroleptic-free schizophrenic patients. Biological Psychiatry, Vol. 44, No. 6, 438–447. Thatcher, R.W., Krause, P.J., and Hrybyk, M. (1986) Cortico-cortical associations and EEG coherence: a two-compartmental model. Electroencephalography and Clinical Neurophysiology, 64, 123–143. Thatcher, R.W., Toro, C., Pflieger, M.E., and Hallet, M. (1994) Human neural network dynamics using multi-modal registration of EEG, PET and MRI. In: R.W.Thatcher, M.Hallet and T.Zeffiro (eds) Functional neuroimaging: technical foundations. Orlando FL: Academic Press, pp. 269–278. Thatcher, R.W. (1995) Tomographic Electroencephalography/Magnetoencephalography: dynamics of human neural network switching. Journal of Neuroimaging, 5, 35–5. Tucker, D.M., Dawson, S.L., Roth, D.L., and Penland, J.G. (1985) Regional changes in EEG power and coherence during cognition: intensive study of two individuals. Behavioral Neuroscience, 99, 564–577. Tucker, D.M., Roth, D.L., and Bait, T.B. (1986) Functional connections among cortical regions: topography of EEG coherence. Electroencephalography and Clinical Neurophysiology, 63, 242–250. Tyler, L.K., Moss, H.E., and Jennings, F. (1995) Abstract word deficits in Aphasia: Evidence from semantic priming. Neuropsychology, 9, 354–363. Von Stein, A., Rappelsberger, P., Filz, O., and Petsche, H. (1993) EEG Korrelate bildlicher Vorstellung: Eine Amplituden und Kohärenzuntersuchung, Zeitschrift EEG-

Time and the brain

388

MEG, 24, 217–224. Weiss, S. (1994) EEG als Korrelat mentaler Prozesse: Spektralanalyse des Spontan-EEG in Ruhe und während sprachlicher Aufgaben. Doctoral Thesis, University of Vienna. Weiss, S., Rappelsberger, P., and Petsche, H. (1995) Amplitude and coherence analysis of visual imagery and visual perception. In (ed.) M.Eiselt, U.Zwiener and H. Witte, Quantitative and Topological EEG and MEG Analysis. Universitätsverlag, Jena., pp. 181–186. Weiss, S., and Rappelsberger, P. (1996) EEG coherence within the 13–18 Hz band as correlates of a distinct lexical organization of concrete and abstract nouns in humans. Neuroscience Letters, 209, 17–20. Weiss, S., and Rappelsberger, P. (1997) Modality specific and semantic processes during language comprehension are reflected within different frequency bands: a topographic EEG coherence study. Biomedizinische Technik, 42, 174–177. Weiss, S. (1997) EEG-Kohärenz und Sprachverarbeitung: Die funktionelle Verkopplung von Gehirnregionen während der Verarbeitung unterschiedlicher Nomina. In: G.Rickheit (ed.) Studien zur klinischen Linguistik: Modelle, Methoden, Interventionen. Opladen: Westdeutscher Verlag. Weiss, S., Müller, H.M., and Rappelsberger, P. (1999) Processing concepts and scenarios: electrophysiologial findings on language representation. In: A.Riegler and M.Peschl (eds). Understanding Representation in the Cognitive Sciences, Plenum Press, pp. 237–246. Weiss, S., Schack, B., and Rappelsberger, P. (1997b) Lexical processing within the brain: evidence from EEG spectral analysis and dynamic topographic coherence analysis. In: H.Witte, U.Zwiener, B.Schack and A. Doering (eds.). Quantitative and topological EEG and MEG analysis. Jena: Druckhaus Mayer, pp. 403–405. Weiss, S., and Rappelsberger, P. (1998) Left frontal EEG coherence reflects modality independent language processes. Brain Topography, 11, 33–42. Wollf, A.B., and Thatcher, R.W. (1990) Cortical reorganisation in deaf children. Journal of Clinical and Experimental Neurophysiology, 12, 209–221.

13 Temporal Structure of Neural Activity and Models of Information Processing in the Brain Galina N.Borisyuk, Roman M.Borisyuk and Yakov B.Kazanovich Institute of Mathematical Problems in Biology, Russian Academy of Sciences, Pushchino, Moscow Region, 142292 Russia [email protected]; [email protected]

This chapter considers the main models of neural networks based on the temporal structure of neural activity, which have been developed in recent years. The models aim to reproduce neurophysiological experimental data on spatiotemporal patterns registered in different brain structures, and to check hypotheses about the role of temporal and phase relations in information processing. Three sections are devoted, respectively, to the models of temporal structure of neuronal spike trains, to the models that are used to study phase relations in oscillatory activity of neural assemblies, and to synchronization-based models of binding and attention. KEYWORDS: Oscillatory neural networks; Binding; Attention.

1. INTRODUCTION Theoretical study of the temporal structure of neural activity has attracted much attention in recent years. Despite the great complexity and variability of electrical activity in the brain, constantly increasing experimental data reveal consistent temporal relations in the activities of single neurones, neural assemblies and brain structures. Without a proper theoretical background, it is very difficult to guess how these relations manifest themselves, and what their role could be in information processing. This is especially important in the situation when detailed knowledge of the mechanisms of neural activity and neural interactions have not led to significant progress in understanding how information in the brain is coded, processed, and stored. What are the general principles of information processing in the brain? How can they be discovered through analyses of electrical activity of neural structures? Which parts of the available experimental data reflect these general principles, and which is related to the peculiarities of biological implementation of these principles in different brain structures? Are the observed temporal relations in neural activity related to information coding, or they are the artefacts generated by special experimental conditions? These questions still await their

Time and the brain

390

answer in future theoretical studies. Computational neuroscience is one of the promising directions in developing brain theory. Mathematical and computer models provide the following possibilities: to form general concepts and to apply them to the analysis of experimental data; to extract essential variables and parameters of neural systems which determine their information processing capabilities; to analyze the role of different mechanisms (biophysical, biochemical, etc.) in neural system functioning; to propose new hypotheses and to check their validity by comparing the modelling results with experimental data; to make suggestions about the further progress of neuroscience; and to formulate the main ideas of new experiments and possible drawbacks. In this chapter, we consider several hypotheses which have been put forward to explain the role of temporal structure of neural activity for information processing. We describe neural networks that have been developed in support of these hypotheses, and whose analysis reveals what kind of model neurones or neuronal assemblies are suitable, and how their interaction should be organized to implement different types of information coding and processing. Corresponding to the main types of models, the chapter is divided into three parts, which mostly take into account the following characteristics of neural activity: 1) the temporal structure of spike trains; 2) phase relations in the average activity of neural assemblies; and 3) synchronization of neural activity. In section 2, we consider the problem of information encoding by spike trains. Two approaches to this problem are presented: encoding by the rate of spikes in the trains, and encoding by the fine temporal structure of spike trains generated by individual neurones. The latter possibility has found experimental support in recent years (see for instance Villa, this volume). Modelling is used to verify its theoretical background and encoding capabilities. In section 3, we focus on the dynamics of neural activity in a system of interacting neural populations. We describe the conditions in which oscillatory activity takes place in such a system, and we determine the phase relations that may appear, depending on the type of interaction between populations and external stimulation. We also discuss the possible role of phase lags in information processing. Section 4 is devoted to the synchronization principle and its application in modelling preattentional processes (feature binding) and attention. We show that the models of binding can be conveniently classified, according to the formalization of the problem, and the architecture of the synchronizing and desynchronizing connections used in the network. We also present a short discussion of a model of attention, which combines the synchronization principle with the idea of a central element of the attention system. The model is based on a new regime of partial synchronization, which was recently found and analyzed. The focus of attention is formed as a result of the partial synchronization of the central element with a group of cortical oscillators. Section 5 discusses the significance of the results obtained and proposes directions for future work.

Temporal structure of neural activity and models of information

391

2. TEMPORAL STRUCTURE OF SPIKE TRAINS In this section, we consider the hypothesis that information in the brain can be coded by the fine temporal structure of spike trains. We describe some models where the possibility of such coding is discussed and conditions are formulated to implement this coding. Traditionally, a neurone is considered as a device that transforms a changing sequence of input spikes into discrete action potentials that are transmitted through the axon to the synapses of other neurones. What are the properties of the neural spike-train that provide the possibility of carrying information or of taking part in information processing? Until recently, the most popular hypothesis stated that this property is the rate of spikes in the train. This assumption has been supported by evidence that neurones change their firing rate in response to proper stimulation. Moreover, it has been shown that the firing rate is correlated with stimulus intensity (e.g. Henry et al., 1973). Rate coding explains how the presentation and intensity of the stimulus can influence neural activity, but this coding neglects the temporal organization of spike trains. Experiments show that temporal patterns of neural activity can be very complex, and it is natural to admit that there should be some information encoded by the moments of spike generation. For example, different stimuli or tasks can elicit different patterns of activity that have the same firing rate. Experimental data obtained in recent years show that, in slowly changing surroundings, the rate code might be useful, but its efficiency drops abruptly if stimulation conditions change quickly. In the latter case, the fine temporal structure of spike trains should play a much more important role (Mainen and Sejnowski, 1995; Bair and Koch, 1996; de Ruyter van Steveninck et al., 1997). Theoretical and experimental results related to this problem are under intensive discussion now (see, e.g. Rieke et al., 1997; Fujii et al., 1996; Ferster and Spurston, 1995; Sejnowski, 1995). If we agree that the temporal pattern of activity carries information about the stimulus, which features of this pattern are important? The popular hypothesis is that the stimulusdriven oscillatory activity in a neurone is a code of a significant stimulus (Borisyuk et al., 1990; more recent discussion of this problem can be found in Singer, 1994). The essence of oscillatory coding can be reduced to two basic ideas (Fujii et al., 1996): place coding (the stimulus is coded by the location of a neurone that shows the oscillatory activity) and binding (the integration of local stimulus representations is realized through synchrony of action potentials). Models related to oscillatory coding and synchronization will be considered in the following sections. The other approach takes into account the fine temporal structure of spike trains. The approach is based on the evidence that under some conditions of multiple stimulus presentation a neurone can respond, reproducing the moments of spikes with precision of 1 msec (Mainen and Sejnowski, 1995). Note that the general pattern of activity is far from being regular in these experiments. Since there is no direct experimental confirmation of the informational significance of these patterns, it would be interesting to study the validity of the theoretical background

Time and the brain

392

to this approach, using a plausible model of neurones and their interaction. The big diversity in the temporal patterns of neural activity is well known (Softky and Koch, 1993). For example, during task implementation by an animal, the activity of a cortical cell can demonstrate interspike intervals with a coefficient of variation (CV) close to one. This means that such activity is purely random and can be described by a Poisson process. On the other hand, the temporal activity of a motor neurone can be highly regular, with a CV between 0.05 and 0.1. Thus, the following questions arise: what should be the design of a model neurone for it to be able to reproduce this variability? In particular, how it is possible to get spike trains whose irregularity is not reduced to the presence of a noise component in the signal. The theoretical analysis of Softky and Koch (1993) shows that a conventional leaky integrate-and-fire neurone can generate only highly regular spike trains in response to a random sequence of input impulses. Hence, this model is useless for implementation of temporal coding. To explain how irregular patterns could appear, the authors consider active dendrites with fast Na+ and K+ conductances, which accelerate the decay of synaptic potential, and this in turn results in suppression of time-averaging of the input signals. In addition, it is assumed that a large enough number of synaptic inputs is activated simultaneously and a spike is generated due to fast crossing the threshold. The main drawback of the model is that no such conductances have been observed in experiments. Another idea for obtaining irregular spike trains has been suggested by Shadlen and Newsome (1994). In this model, irregular spike trains appear due to a specially-chosen balance between excitatory and inhibitory input activities. In other words, the irregularity is a result of neurone interactions in the network. The paper by Bugman et al. (1997) shows that, under some conditions, even a simple leaky integrate-and fire neurone model can generate irregular high-frequency spike trains. The authors obtain this result by introducing a partial reset into the leaky integrator model: At the moment t of spike generation, the membrane potential V(t) jumps to the value βVth, where Vth is a threshold, and β is a reset parameter (0≤β≤1). Changing β, it is possible to get spike trains with different values of the CV. Note that Softky and Koch (1993) consider the case of total reset which corresponds to β=0. In this case the model neurone generates a nearly-regular spike train in response to a random input sequence of impulses and, therefore, the CV of the output spike train is near to zero. If β increases, the CV of the output spike train increases too (for example, β=0.91 corresponds to CV 1, β=0.98 corresponds to CV 1.6). In principle, there is one more possibility to get an irregular spike train, using an integrate-and-fire neurone model, with a time-dependent threshold (see, for example: Wilbur and Rinzel, 1983; Borisyuk et al., 1986). Bugman et al. (1997) have shown that this model is equivalent to the model with partial reset.

3. PHASE RELATIONS OF NEURAL ACTIVITY This section is primarily concerned with the average activity of neural assembles, without paying attention to spike trains of single neurones. This approach is very popular in

Temporal structure of neural activity and models of information

393

computational neuroscience, and there are several types of models that use such simplified descriptions for the dynamics of neural activity. We consider the case when the dynamics of neural activity is of an oscillatory type. This case is important for modelling, since many EEG recordings show various rhythms (alpha, beta, gamma, etc.) during background activity (i.e. without presentation of an external stimulus) and in some more active functional states. For example, the importance of oscillatory activity and corresponding phase relations for information processing has been demonstrated by O’Keefe and Recce (1993). They have found specific phase relationships between the activity of hippocampal place cells and the EEG theta-rhythm. The firing of a place cell begins from a particular phase as the rat enters the field of the maze related to this place cell, but then the phase shifts in a systematic way during field crossing, moving forward on each theta cycle. Thus, one can assume that the phase of place cell firing (with respect to the EEG theta-rhythm) contains information about the relative position of the rat. To perform mathematical analyses of oscillatory processes in neural systems, a theory of oscillatory neural networks has been developed which concentrates on the dynamical behaviour of interacting oscillators. Usually an oscillator is formed by two interacting populations of excitatory and inhibitory neurones. For simplicity, a population can be approximated by a single excitatory or inhibitory element, which represents the average activity of a neural population (the term neurone being retained to denote this element as well). A typical example is the Wilson-Cowan oscillator (Wilson and Cowan, 1972), modifications of which are used most frequently in oscillatory neural networks. If the input signal is absent or small, the oscillator will maintain a low stationary level of activity. If an oscillator receives a strong enough input signal, the activity of its excitatory and inhibitory components starts to oscillate. Since they are connected to each other in a network, the oscillators are capable of running with the same frequency. Thus, one or several assemblies of in-phase running oscillators are formed with different phase-shifts between the assemblies. The values of the shifts depend mostly on the constraints put on the oscillators, and their coupling. The question about frequency and phase relations in a system of coupled oscillators has been discussed in many papers. In particular, the case of weak coupling of two oscillators has been investigated in detail, because it is analytically tractable. Suppose that each of two identical oscillators is represented by a system of differential equations, and that a stable limit cycle exists in the phase-space of the system. Then the dynamics of a single oscillator can be described by just one variable, which is the current phase of movement around the limit cycle. The dynamics of the system of two weakly-coupled oscillators can be described by their phase difference ∆ . There are three possible types of stable steady states for ∆ : •∆ •∆ •∆

(In-phase oscillations); where T/2 is the oscillation period (anti-phase oscillations); and is neither zero nor T/2 (out-of-phase oscillations).

To study the stability of ∆ steady states, a so called H-function is constructed (Ermentrout and Kopell, 1991; Cymbalyuk et al., 1994). If the H-function vanishes at some point, this corresponds to a steady state, the stability being determined by the sign

Time and the brain

394

of the derivative of the H-function at this point. A typical example of such a study of two weakly-coupled Hodgkin-Huxley model neurones is given in Hansel et al. (1993). The main result of this paper is that in the case of excitatory coupling between two neurones, bistability exists for some parameter values. This means that in-phase and anti-phase oscillations coexist, and the resulting dynamics depends upon the initial point only. We omit other papers on this subject, since a complete presentation of weakly-coupled neural oscillators is contained in the recently published book by Hoppensteadt and Izhikevich (1997), where a reader can find all necessary details. In the following text, we address the case where connections between oscillators are strong enough. Usually, if connection strengths are not small, it is impossible to describe phase relations in terms of a mathematical theorem. The main tool kit for such investigations includes computer simulations and numerical methods for bifurcation analysis and parameter continuation. Complete bifurcation analysis of the system of two coupled neural oscillators of a Wilson-Cowan type is given by Borisyuk et al. (1992, 1995) and Khibnik et al. (1992). In these papers, the authors study how the type and the strength of connections affect the dynamics of a neural network. All different connection architectures are investigated separately from each other. In the case of weak connections, the connections from excitatory to inhibitory neurones and from inhibitory to excitatory neurones (synchronizing connections) lead to periodic in-phase oscillations, while the connections between neurones of the same type (from excitatory to excitatory and from inhibitory to inhibitory) lead to periodic anti-phase oscillations (desynchronizing connections). For intermediate connection strengths, the network can enter quasiperiodic or chaotic regimes, and can also exhibit multistability. More generally, the analysis highlights the great diversity of neural network dynamics resulting from the variation of network architecture and connection strengths. Phase shifts of oscillations in a system of two electrically coupled Fitzhugh’s oscillators were studied by Kawato et al. (1979). A stable regime of anti-phase oscillations and bistability (coexisting in-phase and anti-phase oscillations) was found. Similar results were obtained by Cymbalyuk et al. (1994) for two electrically coupled model neurones described by Hindmarsh-Rose equations. It is shown that the system demonstrates one of the five possible dynamical regimes, depending on the value of the external polarizing current: 1) in-phase oscillations with zero phase shift; 2) anti-phase oscillations with half-period phase shift; 3) oscillations with an arbitrarily fixed phase shift depending on the value of the current; 4) both in-phase and anti-phase oscillations for the same current value, where the oscillation type depends on initial conditions; 5) both in-phase and quasiperiodic oscillations for the same current value. In the paper of Kopell and Somers (1995), a system of two relaxational oscillators with excitatory connections is considered. It is shown that the system demonstrates both antiphase oscillations and bistability for appropriate parameter values. Bistability means coexistence of in-phase and anti-phase periodic solutions. Which one of these solutions is observed depends on the initial condition.

Temporal structure of neural activity and models of information

395

Ermentrout and Kopell (1994) present a learning algorithm to memorize the phase shift between oscillations in a system of two identical Wilson-Cowan oscillators. The case where the first oscillator influences the second through the connection between the excitatory neurones is considered. Some functional is introduced to describe the synchronization of two oscillators. The functional is used to modify the connection strength between the oscillators to increase the synchrony. The learning rule is implemented in the form of a differential equation for the connection strength. The steady state of the equation coincides with the desirable phase shift. It was shown by Borisyuk et al. (1995) that, in the case of excitatory connections, the oscillators can run with the same period but some phase shift (out-of-phase oscillations). The phase shift value varies in a broad range, depending on the coupling strength and other parameters of oscillators. The learning rule introduced by Ermentrout and Kopell allows step-by-step adaptation of the connection strength to move the phase shift to an assigned value. There is no general theory of oscillatory neural networks in the case of more than two interacting oscillators and an arbitrary architecture of connections. Most results are related to the following two important types of architectures: all-to-all (global) connections and local connections. In both cases non-trivial relations between oscillation phases can appear. For example, in a network of identical oscillators with global connections, a so-called splay-phase state can exist, when there is a constant time-lag in the phase dynamics of oscillators (see, e.g. Swift et al., 1992). The case of local connections is investigated in more detail when oscillators form a chain, with nearest-neighbour coupling. A rather complete set of results on the dynamics of oscillator chains is given by Kopell and Ermentrout (1990). In particular, it was shown that such coupling is sufficient to produce travelling waves with constant time-lags. This result is important for modelling lamprey swimming. Local coupling on the plane can give complex distributions of oscillator phases even for networks of identical oscillators. In particular, introducing non-homogeneity by adjusting initial conditions, one can obtain rotating waves of phases (Sakaguchi et al., 1988). Mathematical studies of phase relations in different oscillatory networks form the bases for the models that are directed to using phase lags as a tool for information processing. Strong (1993) considers phase logic as a biologically-relevant logic, and applies it to some complex problems of artificial intelligence. Burgess et al. (1994) describe a model of the hippocampus, where the position of a rat is encoded by the phase shift of corresponding place cell activity with respect to the theta-rhythm. If a place cell starts to fire during the initial stage of the theta-wave, this means that the angle between the direction of the rat’s head and the cue/goal is small. If the angle increases, this results in a bigger phase shift. Traditionally, phase relations are applied to model the control of locomotion and types of gait. The important results of Collins and Stewart (1993) show remarkable parallels between the generalities of coupled non-linear oscillators and the observed symmetries of gaits. Each gait is characterized by a specific set of phase lags during leg movements. (For example, the trot is characterized by the following phase relations: diagonal legs, i.e. left front/right back, move together and in phase; right front and left back legs move together a half-period behind the other pair). The authors state that transition from one gate to another can be described by a symmetry-breaking bifurcation in a system of

Time and the brain

396

coupled oscillators. The study of oscillatory neural network dynamics for the case of hexapodal gaits gives a complete classification of the gaits, related symmetries, and bifurcations leading to transitions between the gates. This approach leads to natural hierarchies of gaits, ordered by symmetry, and to natural sequences of gait bifurcations.

4. SYNCHRONIZATION-BASED MODELS OF BINDING AND ATTENTION According to modern concepts, information in the brain is processed on two relatively independent levels. A low level (associated with preattentional processing) is responsible for extracting features from input stimuli and for providing simple combinations of features. At this level, the brain structures operate in parallel, without preselection of input components. A high level (associated with attention) is responsible for forming complex representations of reality. At this level, the information fragments supplied by sensory modalities, memory, and motor components are bound into meaningful patterns that are recognized and memorized. A characteristic feature of this level is its serial form of processing. At any moment attention is focused on a portion of information, that is analyzed more carefully and in greater detail than the other information available (this portion of information is said to be in the focus of attention). The focus of attention then moves from one object to another, with a preference for new, strong, and important stimuli. The features of an object differ in their origin. They can be related to geometrical or spectral characteristics of an image, or even belong to different (e.g. visual or auditory) modalities. Such features are processed in separate parts of the cortex (Damasio, 1989; Zeki and Shipp, 1988). Therefore, one should explain how features are later bound into a complex representation of an object. This is called the binding problem. The problem is not trivial, and there is no possibility of obtaining all of the potentially necessary combinations of features through a suitable organization of connections in a multilayer neural network, since the number of connections in this network would grow exponentially with the number of features used. A possible solution to the binding problem is to label the features of an object through coherent activity of neural populations that code these features (Malsburg, 1981; Crick and Koch, 1990). This labelling hypothesis was confirmed indirectly by experiments where stimulation induced synchronous oscillations in stimulus-specific cortical regions (Gray et al., 1989; Eckhorn et al., 1988). The concept that there should be common principles of grouping information, at both levels of processing—led to the application of the synchronization hypothesis to explain also how the focus of attention is formed (Crick and Koch, 1990; Kryukov, 1991). The following difference was assumed to exist between the two levels: on the low level the synchronization appears as a result of interaction between neural assemblies in the primary cortical areas, while on the higher level the synchronization is controlled by some special brain structures (the thalamus, the hippocampus, the prefrontal cortex), which participate in selective synchronization of cortical areas that should be included in

Temporal structure of neural activity and models of information

397

the attention focus. Thus, this point of view suggests a plausible and general mechanism of parallel processing on the low level, and of sequential processing on the higher level. In what follows, we consider some approaches, where the synchronization principle is applied to develop mathematical models of binding and attention. Let us start from the models of feature binding. Suppose that a sensory input of a network is simultaneously activated by several objects. The network model of feature binding should satisfy the following condition: the features of each object should be represented in the network by a synchronously working assembly of oscillators, while there should be no synchronization between assemblies representing the features of different objects (synchronous assemblies may appear simultaneously or in some sequence). Below we shall focus on the case when objects are represented by several features. The models of single feature binding, simulating the data of Singer, Eckhorn and their collaborators, are well known (see, e.g. Abarbanel et al., 1996), and will be omitted here. Consider a network model, which is supposed to solve the binding problem. An evident idea is to synchronize the elements of the network by properly-chosen local synchronizing connections. Unfortunately, this idea has the following drawback. If connections are weak, it is impossible to obtain reliable synchronization between the oscillators coding one object. If connections are strong, this can lead to synchronization between the groups of oscillators representing different objects. To overcome this difficulty, two suggestions have been made. The first one is to use connections with adaptive strengths (Sporns et al., 1991). If oscillators work synchronously, the strengths of their mutual connections gradually increases. The nonsynchronous work of oscillators weakens these connections. The other suggestion is to use (simultaneously) synchronizing connections for nearby neighbours and desynchronizing connections for more distant neighbours and for the oscillators representing qualitatively different features (Schillen and König, 1994). Both suggestions lead to the same result: if synchronous assemblies of oscillators form isolated clusters there will be no synchronization between them. More detailed development of these ideas depends on how precise the model should be in reflecting experimental conditions, in particular, how the input information is coded. Sporns et al. (1991) try to reach better agreement with experiment. They developed a network that operates with moving objects whose contours are approximated by short bars. The elements of the network are integrate-and-fire neurones, sensitive to short bars moving in a given direction. Oscillators are implemented as interacting assemblies of excitatory and inhibitory neurones. The model of Schillen and König is more abstract. It adapts some earlier ideas of Shimizu et al. (1985) to demonstrate general principles of binding-implementation and their possible application in technical devices. The network developed consists of several interacting modules. Each module is a three-dimensional arrangement of Wilson-Cowan oscillators: a two-dimensional topographic map of the visual field versus a one dimensional representation of a feature domain such as disparity, orientation or colour. This architecture allows the network to operate in the case when the images of objects overlap. If the overlapping images have different values of a feature, correct binding can be obtained because in the module corresponding to this feature the images will be

Time and the brain

398

represented by separate clusters of oscillators, with synchronizing connections in the cluster and desynchronizing connections between the clusters. Note that in the papers considered, binding was a one stage procedure for both simple and complex (compound) features. This seems to contradict the experimental evidence, because, in the known examples of binding, synchronization has been observed only in the case of simple features like a moving bar. The solution to this problem can be found if one supposes that binding is organized hierarchically: simple features of one type are bound in the primary cortex, and combinations of different features are obtained in the higher cortical areas. Such a hierarchical model of binding has been suggested by Borisyuk et al. (1999). The main idea of this approach is to use multi-frequency (in particular, two-frequency) envelope oscillations. Simple features are bound at a high frequency, while low frequency synchronization is used to bind the compound features of a complex stimulus. To implement this idea, the following network has been developed. The network consists of two layers. Each layer is represented by a chain of Wilson-Cowan oscillators. The connections in a layer are local. The feedforward and feedback connections between the layers are convergent, so that each oscillator is influenced by several oscillators from the local part of the other layer. There is an external input to the excitatory neurone of each oscillator in the first layer. The presentation of a stimulus increases the input signals of selected oscillators in the first layer. This induces oscillatory activity in the corresponding network elements. A simple stimulus elicits activity in a locally-connected group of oscillators. A complex stimulus, on the other hand, elicits the activity in separate groups of oscillators which are not locally connected. The important peculiarity of the model is its ability to generate two-frequency envelope oscillations (the modulation frequency is one order of magnitude lower than the main high frequency). The following example shows how the network operates in the case of a complex stimulus (Figure 1). Let the stimulus elicit oscillations in the elements 3,4,5, and 7,8,9 in the first layer (there is no input to the other elements of the network). Denote the activity of an excitatory neurone, where i is its layer number and j is its number in a layer. It can be seen from Figure 1 that and are synchronous at a high frequency, because the oscillators are located in the same excited region. There is no synchronization at a high frequency between these oscillations and, say, which belongs to the other excited region. Nevertheless, there is synchronization of all oscillators in both excited regions at a low frequency. (Figure 1 shows synchronous low frequency oscillations in The synchrony at both low and high frequencies between the oscillators in the same excited region appears due to interlayer connections. The synchrony at a low frequency between the oscillators of different excited regions results from the influence of the second layer. This synchronization is interpreted as the formation of a pattern relevant to feature-binding of a complex stimulus. In the models of binding considered above, synchronous assemblies of oscillators representing different objects coexist in the network simultaneously. We already mentioned that there is another possibility, when synchronous assemblies appear in the network oneby-one. In other words, the input objects are represented by the network in a random or determined sequence. This kind of a model is given in the papers (Ritz et al., 1994a, b). The model adapts Hopfield’s associative memory (Hopfield, 1982) to

Temporal structure of neural activity and models of information

399

oscillatory neural networks. As usual, an element of the network is an oscillator formed by a pair of neurones coupled by excitatory and inhibitory connections, respectively. The scheme of connections between the oscillators in the network is all-to-all. Connection strengths are fixed according to a given set of training patterns and a learning rule similar to the one for Hopfield’s networks. Such a choice of connection strengths realizes the following tendency: the oscillators belonging to the same pattern are coupled by synchronizing connections, and the oscillators belonging to different patterns are coupled by desynchronizing connections. If a combination of several patterns from the training set is provided to the input of the network, a special type of oscillatory activity arises in the network: the synchronous activity of the neurones representing one of the patterns will be changed after some time by the synchronous activity of the neurones representing another pattern, etc. Moreover, the model combines binding with associative capabilities. If a pattern is presented with some distortions, the network will reconstruct the correct pattern which has been learned before.

Time and the brain

400

Figure 1. Example of a two-layer oscillatory neural network stimulated by a complex stimulus. P denotes external inputs to the oscillators with numbers 3, 4, 5 (the first excited area) and oscillators with numbers 7, 8, 9 (the second excited area) of the first layer. Shaded circles denote activated oscillators. Activities of excitatory populations (t) and (t) are shown (these oscillators belong to the first excited area). Also, the activity of excitatory population (t) is shown (this oscillator belongs to the second excited area). Note the envelope profile and synchronization between all shown oscillators at a low frequency. Zoom shows that (t) and (t) run in-phase at a high frequency but (t) runs in anti-phase relative to (t).

Temporal structure of neural activity and models of information

401

Desynchronizing connections appear in some form or other in many models of feature binding. In the model just discussed, the network contained multiple, long-range, desynchronizing connections. The complex structure of desynchronizing connections can be avoided if an inhibitory neurone connected to all oscillators of the network is used as a common source of desynchronization. Sometimes a small number of such inhibitory neurones can be used, each neurone being connected to its own group of oscillators. This approach is represented by the works of Horn, von der Malsburg, Wang and their co workers. The paper of Horn et al. (1991) is one of the first in this direction. The main idea of this work is to label the features of an object by noise. It is assumed that, due to different locations of objects, the noise is identical for the features of a particular object, but different objects are labelled by different noise. Desynchronization between the populations of oscillators representing different objects is performed by an inhibitory neurone, which sequentially synchronizes with each of the populations. Noise is used as a source of synchronization for the features of an object. A similar mechanism of desynchronization has been described by Malsburg and Buhman (1992). Study of a network with two Wilson-Cowan oscillators, coupled by excitatory connections and interacting with a common inhibitory neurone, showed that the oscillators can generate anti-phase oscillations if the influence of the inhibitory neurone is strong enough. A model of binding has been developed by generalizing this result to large three-dimensional networks. In these networks, the objects are represented by non-overlapping populations of oscillators (as usual, different layers representing different features). In the case of two objects, a global inhibitory neurone causes the antiphase activity of populations, while the synchrony inside a population is the result of horizontal and vertical excitatory connections between oscillators. This model was improved further by Wang and Terman (1995). This paper presents a network with a relatively small number of elements and connections which can be used to solve the binding problem for real pictures, containing the images of several objects. The network is called LEGION (Locally Excitatory Globally Inhibitory Oscillator Network). LEGION is a two-dimensional network of oscillators coupled by local excitatory connections. There is also a global inhibitory neurone that receives inputs from the entire network, and feeds back with inhibition to produce desynchronization of the oscillator groups representing different objects. Relaxation oscillators are used to simplify the analysis of network dynamics and to accelerate computations. Another peculiarity is to make connection strengths dependent on the activity of oscillators. If an oscillator is more active, its influence on the neighbouring oscillators increases. This gives a reliable synchronization of oscillators representing a particular object. Consider the case where the input image contains several objects, which induce corresponding patterns of activity in the network. A pattern is synchronized by local connections and competes with other patterns via the general inhibitor. The parameters in the network are chosen so that the pattern whose activity increases faster than the activity of other patterns cannot be immediately suppressed by the general inhibitor. This pattern keeps a high level of activity for some time, while the activity of other patterns will be inhibited. When the activity of the pattern drops down, this results in decreasing the input signal to the global inhibitory neurone, and therefore its inhibitory influence on the

Time and the brain

402

network is stopped, and the network is ready to activate another pattern. The sequence in which patterns are activated is random. The choice of an active pattern is determined by the noise component in the input signal. Hopeful results of LEGION application in object segmentation of real medical images have been reported by Wang and Terman (1997). Note that the functioning of an oscillatory neural network with a central element can be interpreted not only in terms of binding but also as a model of attention. Such a possibility of different interpretations can already be found in the work of Malsburg and Schneider (1986). It is well known that one of the main tasks performed by attention is to choose from the given sensory information that part which is relevant to a prescribed object (Treisman and Gelade, 1980). Hence, attention can be described as binding applied to the features of an object which is currently in the focus of attention. This point of view is underlined in the paper of Kryukov (1991), where a model of attention is formulated in terms of an oscillatory neural network with a central element. The network differs from the one of Wang-Terman by the choice of a central element. In Kryukov’s model the central element is not an inhibitory neurone, but an oscillator (the so-called central oscillator (CO)), which is coupled with other oscillators (the so-called peripheral oscillators (PO)) by feedforward and feedback connections. Such network construction facilitates the analysis of network dynamics and the interpretation of network elements in terms of brain structures. It is presumed that the septo-hippocampal region plays the role of the CO, while the POs are represented by cortical columns sensible to particular features. This concept is in line with Damasio’s hypotheses that the hippocampus is the vertex of a convergent zone pyramid (Damasio, 1989) and with the ideas of Miller (1991) who formulated a theory of representation of information in the brain based on cortico-hippocampal interplay. Attention is realized in the network in the form of synchronization of the CO with some POs. Those POs that work synchronously with the CO are supposed to be included in the focus of attention (here synchronization implies nearly equal frequencies). The parameters of the network that control formation of the focus of attention are coupling strengths between oscillators and natural frequencies of oscillators (the frequency becomes natural if all connections of an oscillator are cut off). Let the set of POs be divided into two groups, namely A and B, each being activated by one of two stimuli, simultaneously-presented to the attention system. The following types of dynamics of the network are interesting for attention modelling: (a) global synchronization of the network (this mode is attributed to the case when the attention focus includes both stimuli); (b) partial synchronization of the CO and a group of POs (this mode is attributed to the case when the attention focus includes one of two competing stimuli); and (c) no-synchronization mode (this mode is attributed to the case when the attention focus is not formed). For mathematical analysis, the model has been specified as a network of phase oscillators (a phase oscillator is described by a single variable, the oscillation phase). The study has been restricted to the case when natural frequencies of oscillators representing the features of a stimulus are similar to each other. The results of the study give complete information about the conditions when each of

Temporal structure of neural activity and models of information

403

the above-mentioned types of dynamics takes place, and describe possible scenarios of transition from one mode to another under the variation of some parameters (Kazanovich and Borisyuk, 1994; Borisyuk et al., 1999). In particular, the model shows that switching the focus of attention from one stimulus to the other goes through an intermediate state when either the focus of attention is absent or both stimuli are included in the focus of attention. Another result of modelling is the formulation of conditions, when decreasing the interaction of the CO with the oscillators representing one of two stimuli that form the focus of attention may lead not to focusing attention on the other stimulus, but to destruction of the attention focus. For some parameter values, it is found that the CO is capable of synchronizing alternately with one or the other of two groups of POs. This can be interpreted as a spontaneous switching of the attention focus, which is observed in some psychological experiments.

Figure 2. Example of chaotic coherent oscillations in integrate-and-fire model with 100 inhibitory and 100 excitatory neurones with all-to-all connections. A. Example of a spike train of a single inhibitory neurone (neurone # 151). B. Example of a spike train of a single excitatory neurone (neurone # 50). C. Total PSP of 100 inhibitory neurones (2 msec averaging). D. Total PSP of 100 excitatory neurones. E.Spikes of 100 inhibitory neruones versus time. F.Spikes of 100 excitatory neurones versus time. Time scale for each frame is 0–1000 msec.

The models of binding and attention considered above are based on synchronization of

Time and the brain

404

regular oscillations. The following example shows that regularity is not an obligatory condition to obtain synchronous oscillations of neural activity. In addition, synchronized chaotic oscillations can be generated by the networks of integrate-and-fire neurones. This gives the possibility of using chaotic oscillations in binding and attention models. An example of a neural network that combines chaotic dynamics with synchronization is presented by Borisyuk and Borisyuk (1997). The authors have developed a neural network of excitatory and inhibitory integrate-and-fire neurones with global connections that can show spatially coherent chaotic oscillations (Figure 2). A specific property of this regime is that the periods of neurone bursting activity alternate with irregular periods of silence. Moreover, the number of spikes in burst and interburst intervals varies over a broad range. Despite the irregular, chaotic dynamics of a single neurone, the global activity of the network looks very coherent. Almost all neurones of the network fire nearly simultaneously in some short time intervals.

5. CONCLUSION The purpose of this chapter was twofold. First, we aimed to show that temporal structures appearing in dynamic activity of neural network models are rich enough to reflect properly the basic neurophysiological data. Second, we aimed to show that dynamic models are helpful for checking the validity of hypotheses about the principles of information processing in the brain. In particular, the models can be used to elucidate the possible significance of temporal relations in neural activity. Our arguments were based on consideration of those models which concentrate on different features of neural activity, from the temporal structure of spike trains to phase relations appearing in average activity of neural assemblies. We summarized the main ideas used in the synchronization-based models of feature binding and attention, and demonstrated that these models are compatible with experimental data and are promising for parallel implementation of information processing. In comparison with traditional connectionist theories, the theory of oscillatory neural networks has its own advantages and disadvantages. This theory tries to reach a better agreement with neurophysiological evidence, but this results in more complicated analysis of the models. Further progress of the theory may be achieved in the following directions.

Temporal structure of neural activity and models of information

405

5.1. Oscillatory Networks with Varying Input Signals Most of the models developed until now are restricted to considering the case when the input signals of oscillators are fixed. It would be much more important to study neural networks which are influenced by a time-dependent stimulus (in mathematical terms, this results in the study of non-autonomous dynamical system). For example, Hoppenstaedt and Izhikevich (1997) suggested an interesting idea that the information encoding in the brain may be similar to FM radio, where frequency modulation is used to encode incoming signals. A variable input makes the analysis of an oscillatory network a very difficult problem, which becomes even more difficult if one takes into account the variation of network parameters related to learning and memory formation (modification of connection strengths and time delays, tuning thresholds, etc.). The following preliminary steps to the solution could be: • to study the dynamics of possible different input signals and their phase relations. Another idea for obtaining irregular spike trains has been suggested by Shadlen and Newsome (1994). In this model, irregular spike trains appear due to a specially-chosen balance between excitatory and inhibitory input activities. In other words, the irregularity is a results of neurone interactions in the network; • to study network dynamics under a small amplitude of variable input signals; and • to study the changes in network dynamics after a short presentation of a constant stimulus. 5.2. Hierarchical Oscillatory Neural Networks Multilayer oscillatory neural networks with different architectures should be developed and analyzed. It seems reasonable to use, in the earlier stages of information processing, oscillatory neural networks with local connections which could provide the interaction of small neural assemblies, similar to the interaction of pyramidal neurones in cortical columns. Convergent forward and backward connections can be used for parallel transmission of information between the layers. For later stages of information processing, networks with a central element should be used to provide the intensive information exchange between arbitrary parts of the network with a relatively small number of long-range connections. In particular, such networks are relevant to modelling the interaction between the hippocampus or the thalamus and the cortex. 5.3 Networks with Multifrequency Oscillations Envelope (multifrequency) oscillations have not received much attention yet. We believe that envelope oscillations may be very helpful for information encoding. It is known that frequency encoding of stimuli is impeded by insufficiency of information capacity. Indeed, the range of admissible frequencies is not large, and due to relatively low

Time and the brain

406

resolution in the frequency domain, it may not be easy to distinguish between different frequencies. Therefore increasing the number of admissible frequencies will be helpful for weakening the limitations of frequency encoding. For example, double-frequency oscillations make it possible to extend frequency encoding, since the second frequency can play the role of a second encoding variable. Therefore, two co-ordinates could be used for encoding instead of one. Besides the theory of information processing in the brain, there is another important field of applications of oscillatory neural networks. We refer to the theory of artificial neural networks. After a period of intensive development, this theory seems to suffer from a reduction in the flow of new ideas. Neuroscience is a reliable and inexhaustible source of such ideas. In the near future we can expect significant progress in neurocomputing in relation to improved understanding of the principles of information processing in the brain. The dream of many researchers involved in neurobiological modeling is that some day their findings will result in development of artificial neural systems with a broad spectrum of intellectual capabilities, comparable with those of the living systems. This dream may soon come true. An important step in this direction would be to develop a computer system for a combined solution of the problems of binding, attention, recognition and memorization. This project is quite real now. All the necessary components are already known. The current task is just to scan through the set of existing models in order to choose those which are most efficient and most compatible with modern concepts of neuroscience, and then to find a proper interaction of these models in a unified system. Many details of this interaction are known already, others should be discovered by further experimental investigations and related mathematical and computer modelling.

ACKNOWLEDGEMENT This work was suppported by grant 99–04–49112 from the Russian Foundation of Basic Research.

REFERENCES Abarbanel, H.D.I., Rabinovich, M.I., Selverston, A., Bazhenov, M.V., Huerta, R., Sushchik, M.M., and Rubchinskii, L.L. (1996) Synchronization in neural assemblies. Uspekhi—Fizicheskikh Nauk, 166, 363–390. Bair, W., and Koch, C. (1996) Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkeys. Neural Computation, 8, 1185–1202. Borisyuk, R., and Borisyuk, G. (1997) Information coding on the basis of synchronization of neuronal activity. BioSystems, 40, 3–10. Borisyuk, G.N., Borisyuk, R.M., Kazanovich, Ya.B., and Strong, G. (1999) Oscillatory neural networks: Modeling binding and attention by synchronization of neural activity. In: D.S.Levine, V.R.Brown, V.T.Shirey (eds.) Oscillations in Neural Systems. Lawrence Erlbaum Associates, Inc, Hillsdale, N.J., 261–284.

Temporal structure of neural activity and models of information

407

Borisyuk, G.N., Borisyuk, R.M., and Khibnik, A.I. (1992) Analysis of oscillatory regimes of a coupled neural oscillator system with application to visual cortex modeling. In: J.G.Taylor, E.R.Caianiello, R.M.G. Cotterill and J.W.Clark (eds) Neural Network Dynamics, Proceedings of the Workshop on Complex Dynamics in Neural Networks, Perspectives in Neural Computing. Springer-Verlag, Heidelberg, pp. 208– 225. Borisyuk, G., Borisyuk, R., Khibnik, A., and Roose, D. (1995) Dynamics and bifurcations of two coupled neural oscillators with different connection types. Bulletin of Mathematical Biology, 57, 809–840. Borisyuk, G.N., Borisyuk, R.M., Kirillov, A.B., Kovalenko, E.I., and Kryukov, V.I. (1986) New statistical methods for analisys of neuron activity. Pushchino: ONTI NCBI, 200 pp. Borisyuk, G.N., Borisyuk, R.M., Kirillov, A.B., Kryukov, V.I., and Singer, W. (1990) Modelling of oscillatory activity of neuron assemblies of the visual cortex. In Proceedings of the International Joint Conference on Neural Networks—1990, SanDiego, 2, 431–434. Bugmann, G., Christodoulou, C., and Taylor, J.G. (1997) Role of temporal integration and fluctuation detection in the highly irregular firing of a leaky integrator neuron model with partial reset. Neural Computation, 9, 985–1000. Burgess, N., Recce, M., and O’Keefe, J. (1994) A model of hippocampal function. Neural Networks, 7, 1065–1081. Collins, J.J., and Stewart, I.N. (1993) Coupled nonlinear oscillators and the symmetries of animal gaits. Journal of Nonlinear Science, 3, 349–392. Crick, F., and Koch, C. (1990) Some reflections on visual awareness. In Cold Spring Harbor Symposium on Quantitative Biology, 55, 953–962. Cymbalyuk, G., Nikolaev, E., and Borisyuk, R. (1994) In-phase and antiphase selfoscillations in a model of two electrically coupled pacemakers. Biological Cybernetics, 71, 153–160. Damasio, A. (1989) The brain binds entities and events by multiregional activation from convergent zones. Neural Computation, 1, 123–132. De Ruyter van Steveninck, R.R., Lewen, G.D., Strong, S.P., Koberle, R., and Bialek, W. (1997) Reproducibility and variability in neural spike trains. Science, New York, 275, 1805–1808. Eckhorn, R., Bauer, R, Jordan, W, Brosch, M, Kruse, W, Munk, M., and Reitboeck, H. (1988) Coherent oscillations: A mechanism of feature linking in the visual cortex? Biological Cybernetics, 60, 121–130. Ermentrout, B., and Kopell, N. (1991) Multiple pulse interactions and averaging in systems of coupled neural oscillators. Journal of Mathematical Biology, 29, 195–217. Ermentrout, B., and Kopell, N. (1994) Learning of phase lags in coupled neural oscillators. Neural Computation, 6, 225–241. Ferster, D., and Spurston, N. (1995) Cracking the neuronal code. Science, New York, 270, 756–757. Fujii, H., Ito, H, Aihara, K., Ichinose, N., and Tsukada, M. (1996) Dynamical cell assembly hypothesis—theoretical possibility of spatio-temporal coding in the cortex. Neural Networks, 9, 1303–1350. Gray, C.M., König, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, London, 338, 334–337. Hansel, D., Mato, G., and Meurnier, C. (1993) Phase dynamics for weakly coupled

Time and the brain

408

Hodgkin-Huxley neurons. Europhysics Letters, 25, 367–372. Henry, G.H., Bishop, P.O., Tupper, R.M., and Dreher, B. (1973) Orientation specificity and response variability of cells in the striate cortex. Vision. Research, 13, 1771–1779. Hopfield, J. (1982) Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Science, U.S.A., 79, 2554–2558. Hoppensteadt, F.C., and Izhikevich, E.M. (1997) Weakly Connected Neural Networks. Springer Verlag, Heidelberg. Horn, D, Sagi, D., and Usher, M. (1991) Segmentation, binding and illusory conjunctions. Neural Computation, 3, 510–525. Kawato, M., Sokabe, M., and Suzuki, R. (1979) Synergism and antagonism of neurons caused by an electrical synapse. Biological Cybernetics, 34, 81–89. Kazanovich, Y.B., and Borisyuk, R.M. (1994) Synchronization in a neural network of phase oscillators with the central element. Biological Cybernetics, 71, 177–185. Khibnik, A.I., Borisyuk, R.M., and Roose, D. (1992) Numerical bifurcation analysis of a model of coupled neural oscillators. In International Series of Numerical Mathematics, 104, 215–228, Basel, Birkhauser Verlag. Kopell, N., and Ermentrout, B. (1990) Phase transitions and other phenomena in chains of coupled oscillators. SIAM Journal of Applied Mathematics, 50, 623–660. Koppel, N., and Sommers, D. (1995) Anti-phase solutions in relaxation oscillators coupled through excitatory interactions. Journal of Mathematical Biology, 33, 261– 280. Kryukov, V.I. (1991) An attention model based on the principle of dominanta In: A.V.Holden and V.I.Kryukov (eds) Neurocomputers and Attention I. Neurobiology, Synchronization and Chaos. Manchester, Manchester University Press, pp. 319–352. Mainen, Z.F., and Sejnowski, T.J. (1995) Reliability of spike timing in neocortical neurons. Science, New York, 268, 1503–1506. Malsburg, C.von der (1981) The correlation theory of brain function. Internal report, Max-Planck Institute for Biophysical Chemistry, Goettingen RFA. Malsburg, C.von der, and Buhman, J. (1992) Sensory segmentation with coupled neural oscillators. Biological Cybernetics, 67, 233–242. Malsburg, C.von der, and Schneider, W. (1986) A neural cocktail-party processor. Biological Cybernetics, 54, 29–40. Miller, R. (1991) Cortico-hippocampal interplay and the representation of contexts in the brain. Berlin, Springer-Verlag. O’Keefe, J., and Recce, M. (1993) Phase relationship between hippocampal place units and EEG theta rhythm. Hippocampus, 3, 317–330. Rieke, F., Warland, D, de Ruyter van Steveninck, R.R., and Bialek, W. (1997) Spikes: exploring the neural code. Cambridge MA, MIT Press. Ritz, R., Gerstner, W., Fuentes, U., and van Hemmen, J.L. (1994a) A biologically motivated and analytically soluble model of collective oscillations in the cortex. Biological Cybernetics, 71, 349–358. Ritz, R., Gerstner, W., and van Hemmen, J.L. (1994b) Associative binding and segregation in a network of spiking neurons. In: E.Domany, J.L.van Hemmen, K.Schulten (eds) Models of Neural Networks II. New York: Springer-Verlag, pp. 175– 219. Sakaguchi, H., Shinomoto, S., and Kuramoto, Y. (1988) Mutual entrainment in oscillator lattices with non-variational type interaction. Progress in Theoretical Physics, 79, 1069–1079.

Temporal structure of neural activity and models of information

409

Schillen, T., and König, P. (1994) Binding by temporal structure in multiple feature domains of an oscillatory neuronal network. Biological Cybernetics, 70, 397–405. Sejnowski, T.J. (1995) Time for a new neural code? Nature, London, 376, 21–22. Shadlen, M.N., and Newsome, W.T. (1994) Noise, neural codes and cortical organization. Current Oppinion in Neurobiology, 4, 569–579. Shimizu, H., Yamaguchi, Y., Tsuda, I., and Yano, M. (1985) Pattern recognition based on holonic information dynamics: towards synergetic computers. In (ed.) by H.Haken, Complex Systems—Operational Approach, Berlin:Springer-Verlag. Singer, W. (1994) Putative functions of temporal correlations in neocortical processing. In: C.Koch and J.L.Davis (eds) Large scale neuronal theories of the brain. Cambridge MA: MIT Press. Softky, W.R., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSP. Journal of Neuroscience, 13, 334–350. Sporns, O., Tononi, G., and Edelman, G.M. (1991) Modeling perceptual grouping and figure-ground segregation by means of active reentrant connections. Proceedings of the National Academy of Science, U.S.A., 88, 129–133. Strong, G. (1993) Phase logic is biologically relevant logic. Behavioral and Brain Science, 168, 472–473. Swift, J.W., Strogatz, S.H., and Wiesenfeld, K. (1992) Averaging in globally coupled oscillators. Physica D, 15, 239–250. Treisman, A., and Gelade, G. (1980) A feature-integration theory of attention. Cognitive Psychology, 12, 97–136. Wang, D-L., and Terman, D. (1995) Locally excitatory globally inhibitory oscillator network. IEEE Transactions on Neural Networks, 6, 283–286. Wang, D-L., and Terman, D. (1997) Image segmentation based on oscillatory correlation. Neural Computations, 9, 805–836. Wilbur, A.J., and Rinzel, J. (1983) A theoretical basis for large coefficient of variation and bimodality in neuronal interspike distribution. Journal of Theoretical Biology, 105, 345–368. Wilson, H.R., and Cowan, J.D. (1972) Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal, 12, 1–24. Zeki, S., and Shipp, S. (1988) The functional logic of cortical connections, Nature, London, 335, 311–317.

Discussion Section Note from editor: This chapter arose from two sorts of intervention by the editor. When chapters had been submitted, I wrote two brief “discussion statements”, one concerning temporal structure in multispike trains, the other concerning a possible unifying principle for some of the EEG frequency bands. In addition, in individual letters to chapter authors, I posed a number of other questions raised by their chapters, and invited authors’ comments. With regard to the two discussion statements (and in section 9 [below] on corticohippocampal interplay), the editor is certainly not unbiased, but a protagonist in the debate. I have therefore indicated contributions of this sort as “Miller”. With regard to the other parts of this discussion chapter, arising from the individual letters, I have no particular viewpoint to promote, but, rather, am probing in a neutral fashion, to see where the discussion leads. Here I have indicated my contribution as “Editor”.

1. THE ROLE OF AXONAL DELAY LINES Miller: The chapter by Swadlow shows that axonal conduction delays are as long as several tens of milliseconds for some cortico-cortical axons. In animals with large brains and correspondingly longer axonal trajectories, such as humans, conduction times may be much longer, even 100–200 msec (Swadlow et al., 1979). Axonal conduction time is generally a rather stable feature of an axon, provided one is talking about specific branchpathways of a single axon. Admittedly, as Swadlow mentions, in the slow-conducting axons, substantial changes in conduction time may occur when two impulses are conducted within a few milliseconds of each other. However, Swadlow also mentions that in the cortex, such slow-conducting axons usually have very low levels of spontaneous firing, and when they respond to sensory stimuli, only a minority can sustain an elevated rate of firing (Swadlow, 1990). Therefore only rarely (if ever) would the slow conducting axons convey impulses so close together in time that conduction time was significantly modified. Given this conclusion, axonal conduction time appears to be a precise and stable feature of axonal pathways. The question is, does it play any part in neural representation? The chapters in this volume by Villa, Nowak and Bullier, and Ghazanfar and Nicolelis all give evidence about temporal structure in spike trains. This temporal structure may be revealed by cross-correlation methodology, when sometimes (but not often) sharp peaks are seen in the cross-correlation histograms, betraying precise temporal relationships between the firing of two neurones (Nowak and Bullier). Precise temporal structure may also be revealed in the temporal patterning of neuronal discharge following a sensory stimulus (Ghazanfar and Nicolelis), or in the temporal patterns seen when spike trains of several neurones are considered together (Villa, Ghazanfar and Nicolelis).

Discussion section

411

Putting these two sets of data together, an important question is raised: could axonal conduction time be one of the methods used by the brain in determining the precise temporal structure seen in the spike trains of single neurones? With regard to the analysis of mutiple spike trains, an argument has been used against the idea that conduction time in single axons plays a part in determining temporal structure: Abeles (1982) makes two assumptions: first axonal conduction times are quite short (at most a few milliseconds); second, to fire any cortical pyramidal cell requires the coincidence of very many afferent synaptic influences (i.e. thirty or more). Combining these two assumptions Abeles concludes that the brain dynamic processes which are responsible for the observed temporal structure involve many stages of synaptic relay, and at each stage many neurones need to be activated together in concert to ensure the security of transmission. According to this scheme, axonal conduction time plays little or no part. Nowak and Bullier (this volume) also discuss this question. They note the fact that cross-correlation histograms for cortical neurones are only very rarely displaced significantly from the time origin, and, like Abeles, they conclude that axonal conduction time plays little part in determining temporal structure, and that the cross-correlation histogram method tells us little about anatomical connectivity. This makes sense if neuronal firing is always a result of coincident activation by very many EPSPs. However, in view of Swadlow’s results that axonal conduction time may be many tens of milliseconds, there are perhaps reasons for reopening this subject. In this context two other relevant findings should be mentioned: (i) In cross-correlation histograms between cortical and thalamic neurones, two studies (Tsumoto et al., 1978; Johnson and Alloway, 1994) find peaks displaced quite a long way from the time origin. This correlates with the fact, revealed in many of Swadlow’s papers, that cortico-thalamic projections are very slow-conducting axons (see also Miller, 1996). (ii) There is a evidence that, under some circumstances, the membrane potential of cortical (and other) neurones may be held stably just below the threshold for firing, so that only a few EPSPs are needed to produce an action potential from this potential level. This may be the case for the thalamic neurones mentioned above (Hirsh et al., 1983; see also Miller, 1996) and apparently also can apply to cortical and striatal neurones (Stern et al., 1997). Full details of the mechanism for raising membrane potential to this just-subthreshold level, and holding it at that level are unresolved. For the striatal (and probably also the cortical) neurones Stern et al., suggest that coincident activity in many excitatory inputs to a neurone is needed to achieve the transition to the “up” state. Nevertheless, since neurones can be held for some time in this just-subthreshold state without firing, there may be some dissociation between the processes need to reach the up-state, and those few additional synaptic inputs then required to actually fire the neurone. Whatever the detail, the membrane potential could sometimes be held, poised slightly below threshold, allowing the neurone to produce an action potential with coincidence of very few afferent synaptic activations. Thus, provided the membrane potential of the cortical pyramidal cells involved were poised just below threshold, only a few critical synaptic inputs would be needed to define the moments of firing in cortical pyramidal cells. In this case, axonal conduction times in

Discussion section

412

these inputs could be an important determinant of the precise temporal structure seen in multiple spike trains. Likewise, the cross-correlation histogram method could tell one about the anatomical connectivity of those neurones at the times they are held just below threshold (which may be very important times for information processing). Even so, the chances of recording from two neurones which are directly connected may be quite low, especially for long cortico-cortical links. This may contribute to the difficulty of demonstrating cross-correlation histogram peaks displaced from the time-origin for recordings in cortical sites. Other mechanisms then come into play to produce peaks centred on the time origin, as discussed by Nowak and Bullier. The chapter by Ghazanfar and Nicolelis documents the fact that neurones in cortical sensory regions respond to complex stimuli with replicable temporal patterns of discharge. (This has also been reported earlier by Johannesma et al. [1981] and Johannesma and Aertsen [1982].) When the complex stimuli are replaced by simpler ones, such as might be used for investigating a “receptive field”, the responses are specific not only to spatial features (or spectral ones in the auditory sense) but also to specific times in relation to the stimulus. By combining several simple stimuli, the temporal pattern of firing produced by the combination is predictable from the temporal patterns of response to individual simple stimuli. Nicolelis does not offer detailed explanations of his results at the cellular level. However, one possible explanation is that every stimulus element has one (or several) “labelled lines” connecting it to each cortical neurone. These lines are defined not only spatially (i.e. what connects to what) but also temporally (i.e. what is the delay imposed by the connection). This time/space inseparability is exactly what would be expected if the labelled lines were individual slowly-conducting axonal delay lines. Obviously in something as complex as the cortex, it would be foolish to rule out alternative mechanisms. However, the case is put that, at least under some circumstances, the precision of conduction time in axonal delay lines can play a part in determining the precise temporal structure in cross-correlation histograms, in multiple spike trains, and in sensory responsiveness of single cortical neurones. Nowak and Bullier: The question of the role of axon conduction times in generating precise temporal patterns in spike trains appears to require a very stable membrane potential poised just below the threshold level. In intracellular recording performed in vivo, such a steady level is very rarely observed, at least in neurones that can be impaled and recorded in an anaesthetized cat preparation. Whether a different situation exists in the awake animal appears difficult to test at the moment. Another way to produce precise timing between spikes is through the intrinsic properties of the neurones: some neurones can generate a similar temporal pattern of spikes under the same circumstances. For example, within bursts of action potentials, very little variability of interspike interval is observed. However, it is not clear whether single spikes can also be produced with a faithful temporal precision by intrinsic mechanisms. In most cases, small bursts of a variable numbers of spikes are generated. This method of producing temporal patterns would not require precise arrangements of axon conduction times of afferents. Miller: In the Up-state in lamina V cortical neurones, in anaesthetized animals, membrane potential may be poised stably, just below firing threshold. It is likely that the

Discussion section

413

Up-state prevails more commonly in the awake and alert animal than in the anaesthetized animal. Matsumura et al. (1996) have provided data in waking cats, showing that 1 mV of depolarization corresponds to about 0.3 above-chance impulses in cross correlograms, implying that, in the waking state, cells are maintained within a few mV of firing threshold. With regard to the possibility of intrinsic neuronal properties such as burst firing determining temporal structure, the sorted rasters of spike trains in the data of Villa show precise temporal structure, amongst neurones which show no burst firing. Swadlow: The view of axonal conduction times serving as “delay lines” is attractive, and has proved useful in some very fast-conducting systems (e.g. Bennett, 1971). Several issues need to be addressed in order to assess the utility of this idea in accounting for the various types of temporal structure reported in this volume: 1. The temporal structures described in the chapters by Villa, Nowak and Bullier, and Ghazanfar and Nicolelis were either stimulus-elicited, or the result of “spontaneous” impulse activity. Indeed, silent cells generate very poor cross-correlograms! Yet, the great majority of neurones with very slowly conducting (<1 m/s) neocortical axons have very little or no spontaneous activity, and, even in the awake state and in granular sensory cortex, few respond with spikes to peripheral stimulation (Swadlow, 1988, 1990). Many such cells do have subthreshold receptive fields (Swadlow and Hicks, 1997), but these will not help in the generation of cross-correlograms. In the forelimb representation of the motor cortex, such cells do not become active even when subjects are running and jumping over barriers (Beloozerova, Sirota and Swadlow, unpublished observations). 2. One can suppose that a small sub-population of neurones with very slowly conducting axons is active and responsive to peripheral stimulation, and can therefore be engaged in the ensemble activities described in this volume. However, are the axonal conduction times of such neurones sufficiently stable to mediate the observed temporal structure? As reviewed in Chapter 4, many slowly conducting cortical axonal systems (including the primate corpus callosum) demonstrate marked variations in conduction velocity, following prior impulse activity. A “supernormal period” of increased conduction velocity follows the relative refractory period, and this is followed by a “subnormal” period of decreased conduction velocity. Although the supernormal period usually lasts for <100 ms, the magnitude and duration of the subnormal period cumulates with the number and frequency of prior impulses, and can last for several minutes (cf., Figures. 5–7 of Swadlow et al., 1978). Variations of ±10% in conduction velocity (often corresponding to several ms) are not uncommon during these periods. Miller correctly points out that most such neurones have very low levels of spontaneous activity and, in sensory neocortex, they are difficult to drive from the receptor periphery. However, if such neurones do have a role in producing the temporal patterns seen in ensemble behaviour, presumably they must be active to do so, and if they are active, then predictable variations in conduction times will ensue. 3. Is the synaptic drive generated by very slowly conducting non-myelinated cortical axons sufficient to generate the kinds of temporal structure described? There are few studies concerning the influence of very slowly conducting cortical axons on postsynaptic targets. Central synapses show considerable variability in strength and

Discussion section

414

reliability (e.g.Walmsley et al., 1998), and one might expect a single non-myelinated axon of 0.1–0.3 µm diameter to support fewer terminals, and/or have weaker postsynaptic effects than a single, large-diameter myelinated axon. The terminal arbors of fine-diameter cortico-geniculate axons, for example, are more widely distributed, and make far fewer synapses (<200) within the dorsal LGN than do retinal ganglion cell axons (Robson, 1984), and corticocortical synapses in rat barrel cortex (many of which are mediated by non-myelinated axons) have a lower innervation density and lower release probability than do thalamocortical axons (which are myelinated, Amitai et al., 1998). 4. Compounding this potential problem is the temporal dispersion of post-synaptic effects that are mediated by non-myelinated axons. Antidromic conduction times of corticocortical and layer-6 corticothalamic axons are not only very long, but they are very broadly distributed (Figure 2 of Chapter 4, this volume; Miller, 1996). Thus, if a single stimulus or event brings to threshold many neurones of a non-myelinated axonal system, the resulting barrage of impulses will suffer considerable temporal dispersion before converging on target structures, and this will considerably weaken the postsynaptic impact. The above considerations place restraints on the proposed model of slowly conducting cortical axons serving as delay lines in generating precise spatiotemporal neural patterns. However, they are not necessarily incompatible with this proposal. Miller’s idea is certainly intriguing, and makes for an elegant and parsimonious use of a large (perhaps 20–30% of cortical neurones) and rather mysterious subpopulation of neocortical elements. Miller: Concerning point 1: I presume that most of the neurones whose spike trains are used to show spatiotemporal structure are lamina V cells, which have relatively steady, and high rates of spontaneous activity. They also have rapidly-conducting axons, but do not provide long corticocortical axons and so their axons are unlikely to provide a basis for temporal structure in multi-spike trains. Many of the cells which provide long corticocortical axons (with axons having slow conduction velocity), are the mainly “silent” cells referred to by Swadlow, located predominantly in laminae II and III. This could be thought a difficulty for the hypothesis that long conduction delays play a major part in determining temporal structure in multiple spike trains. However, inputs to lamina V cells (rather than their own axons) may be the ones with long conduction delays. For instance, long corticocortical axons may form afferent synapses on the dendrites of lamina V cells. Each of these input cells may itself receive inputs from lamaina V cells. The intervening cell in the superficial laminae would have very low levels of sponteous activity, but each lamina V cell is likely to receive inputs from very many corticocortical cells. Thus, temporal structure in multispike trains recorded from several lamina V cells could be determined by axonal conduction, even though each of the axons involved carry a very low level of impulse traffic. (Alternatively corticothalamic cells [which also have very slow-conducting axons], may determine firing in lamina V cells [after a shortlatency synaptic relay in ascending thalamocortical cells—which are rapidly conducting].) Concerning point 2: I also have seen the variations of latency in responses in cells with slow-conducting axons, in the aftermath of repetititve firing in those axons. However, I

Discussion section

415

would be surprised if the variations occurring naturally were sufficiently large to disrupt temporal patterning seen amongst spike trains. Swadlow suggests that variations of a few milliseconds may be the cumulative result of many prior impulses, which may seldom occur physiologically. Even this may not be enough to disrupt temporal patterning, bearing in mind that the neuronal integration time (approximate duration of the EPSP) is also several milliseconds in duration. Concerning point 3: We do not know what is the relation (if any) between axonal calibre and the synaptic “weight” of the synapses borne by the axon. It is by no means clear that small calibre axons would always have lower synaptic weights. CCH methodology has been used to investigate interaction of thalamic relay neurones both with cortical neurones sending descending projections to the thalamus, and with neurones lower in the brain stem with ascending projections to the thalamus. Although the former axons are far finer in calibre than the latter, there is little difference in efficacy of impulses in the two types of axons afferent to thalamic relay cells: Alloway et al. (1994) find, in anaesthetized rats, that cuneothalamic unitary connection strength was 0.03, on average, while corticothalamic connection strength in the same somatosensory nucleus in anaesthetized cats was 0.024 (Johnson and Alloway, 1994), that is, in the same range. Concerning point 4: Temporal dispersion in a population of slow conducting axons may mean temporal divergence of a signal originating at a point in time from a “transmitting” locus. However, temporal dispersion also permits temporal convergence of signals arriving at a point in time at a “receiving” locus. Thus the accurately-timed impulses, seen in the sorted raster displays of Villa may represent times when impulses in several (not many) afferents to a neurone, relayed along axons with different delays, happen to coincide on the postsynaptic neurones. For this to be plausible, the spatiotemporal connectivity of the network must be sufficiently dense. There is a serious defficiency in quantitative data on the statistics of spatial and temporal aspects of connectivity of long range corticocortical axons. The plausibility of the scheme proposed cannot be confirmed until such quantitative data are available. Villa: There is evidence from several laboratories that detectable and significant spatiotemporal firing patterns may be formed by events separated by intervals lasting several hundreds of milliseconds. Theoretical models of converging/diverging chains of neurones may sustain precisely-timed activity across many synapses, and keep the jitter to a very low level, compatible with the precision observed experimentally. However, I do not see any fundamental reason to reject the idea of axonal delay lines playing a role in these complex patterns of firing. Miller, and Nowak and Bullier argue that this hypothesis requires some mechanism to poise the membrane potential just below the threshold level, in a very stable way. Indeed, a number of so-called unspecific projections might globally affect the level of excitability of cortical and/ or thalamic cells. For example, Shulz et al. (1997) have demonstrated that cholinergic agonists affect primarily the excitability of cortical neurones, rather than the strength of cortical connections. Indirect cholinergic activation of the rat thalamus by intracerebroventricular administration of nerve growth factor (Villa et al., 1996) has also shown that excitability at the thalamic level is deeply influenced by this modulatory input. It is therefore likely that the combination of a global increase of excitability, and the activation of specific axonal delay lines may produce reliable patterns of activity. Then, these patterns would

Discussion section

416

be not only input-specific, but also state-specific, where the state corresponds to a state of activation determined by the modulatory inputs. In this situation, the modulatory activation would elevate the excitability and sustain the putative “binding” played by coherent firing in distributed neuronal networks, where the actual “contextual” message is transmitted through slow-conducting axons. It is important to notice that this would happen without the need for small bursts, or intrinsic properties of selected classes of neurones. The variations in conduction velocity that have been emphasized by Swadlow might still fall within the limits of jitter compatible with a reliable transmission of temporally organized information. I found very attractive the suggestion raised by Miller of events belonging to spatiotemporal firing patterns representing coincidences of impulses in several afferents to a neurone, relayed along axons with different delays. This hypothesis is very interesting for those patterns related to associative processes. One might assume that the input coming from an area travels through several axonal lines at different speeds. The “association” between neurones of different areas would be established only by those coincidences falling within a finite delay, in a kind of winner-takes-all mechanism, where one neurone happens to be the “winner”, at a certain time, because of its timed inputs. Neighbouring neurones are likely to share the afferent inputs from remote areas, to an extent which, on the basis of available studies, is still hard to determine quantitatively. Then it is possible to postulate that changes in delay, due to slight variations in the input pattern or neuronal death of a neurone in a specific “chain” of the network, among many possible causes, would produce a different “winner”. Nevertheless, the associative process would be formed by elements very close to the previous cell assembly, and it is likely that the output pattern of the process would be very similar to the previous one. However this process introduces some variability, and this might correspond to the variability observed experimentally at cortical and thalamic levels when even simple stimuli are repeated in a sequence. Moreover, the possibility of having a flexible “winner” for a process which, overall, remains the same, would be more robust in the face of sudden interruptions in the rings of the neuronal chains, and also would permit the establishment of new links, through oriented shifts of the winner in a particular direction. These shifts could be due to variations in the weight of some inputs related to “noncontextual” learning processes. However, when an areal problem happens, e.g. neuronal death due to a stroke, the possibility of keeping active certain processes by shifts of the winner would become impossible, and some processes, distributed in nature, but not completely holographic in essence, would be lost for ever. In general terms I would say that there is no reason to search for only one mechanism responsible for precise spatiotemporal patterns of activity. Patterns exist, but they can be of very different origin. The difficulty of detecting this type of activity, even with large sets of microelectrodes recording simultaneous spike trains, does not allow one to speculate on the fine structure of these patterns on a statistical basis. I believe that we will need to accumulate thousands of patterns, recorded in comparable situations (similar brain areas and similar states of anaesthesia) before allowing any speculative hypothesis to be based on solid ground. If preferred intervals within long lasting patterns are detected, and it is shown that these intervals are related to specific axonal delays, then those patterns will have a very specific significance. On the contrary, corticofugal axonal

Discussion section

417

delays of few milliseconds (Ahlsen et al., 1982) may sustain loops of activity responsible for shorter reverberations, that could play the role of dynamical funnelling or separation of activity among parallel thalamocortical modules.

2. INTERPRETATION OF CROSS-CORRELATION HISTOGRAMS Editor: Does the “common input” mechanism of synchronization of neuronal firing require monosynaptic common input? In the chapter by Bullier/Nowak, the authors suggest that perhaps it does not. If this is the case, and polysynaptic common inputs will suffice, the connectivity allowing this might be very common. This could be another reason why cross-correlation histogram peaks centred on the time origin are far more common than displaced peaks. Nowak and Bullier: Probably a monosynaptic common input is not required, although this is a difficult aspect to demonstrate directly. What we can offer is our experimental results of the effects of cortical and callosal lesions on the different types of crosscorrelation histogram peaks (Munk et al., 1995), which we observed between areas 17 of the two hemispheres. We found that removing the cortical areas that provide monosynaptic feed-back inputs to the recorded sites reduces the probability of observing intermediate (C) and broad (H) peak types. The results we obtained when we cut the posterior half of the corpus callosum were interesting, because they showed different effects on the incidence of C and H peaks: while C peaks were completely suppressed, H peaks still remained abundant. When the anterior part of the corpus callosum was severed as well, the incidence of H peaks was dramatically reduced. This suggests that at least some of the axons that are involved in generating H peaks circulate through the anterior part of the corpus callosum. There are no cortical areas that project directly upon area 17 of the two hemispheres via the anterior part of the corpus callosum. Interactions that take place through these pathways must take place between areas other than areas 17, which in turn would project on areas that have a monosynaptic projection on area 17 (hence reconciling our two sets of observations). The implication is that this mechanism of generation of (some) H peaks involves polysynaptic pathways. Although the evidence is limited, it appears likely that sharp peaks (T) are mediated by monosynaptic inputs, whereas wide peaks (H) involve some polysynaptic pathways. The situation concerning the intermediate peaks (C) is less clear, but could involve monosynaptic (interhemispheric) as well as polysynaptic pathways (for example in the case of spindle activity, where intermediate peaks are observed in all cortical layers). Villa: Cross-correlation histograms are a statistical means of assessing the functional interaction between two spike trains recorded simultaneously. I agree with Nowak and Bullier when they say those sharp and wide peaks in the “common input” curves are likely to represent evidence of different mechanisms of synchronization. The evidence in favour of a monosynaptic common input is tiny, if any. Because of the statistical nature of the correlation histograms, and because of the widely accepted pattern of multiple divergence/convergence connectivity at cortical level, it is likely that the “common input”

Discussion section

418

is in fact the result of the influence of global excitation and inhibition on the interactions of pairs of cells within a subnetwork. In a set of simulation studies, the cross-correlation histograms indicated that, when a balance of excitation and inhibition exists, the contribution of the PSPs to the membrane potential, and the integration time determined by the membrane time-constant may play a key role in forming spatiotemporal representations (Hill and Villa, 1997).

3. PSYCHOPHYSICS OF TIME PERCEPTION Editor: Moving on from the single unit level of analysis, one may ask, for psychophysical estimates of duration by humans (such as described in the chapter by Rammsayer), what cellular components of the brain, or what physiological process, might determine the observed capacity for duration discrimination? Could axonal conduction time play a part in this? Rammsayer: As outlined in Chapter 5 by Rammsayer and Grondin temporal processing of brief durations below approximately 500 ms appears to be beyond cognitive control, being based on processes which are predicted to be automatic, and, most likely, located at a subcortical level (Michon, 1985; Mitrani et al., 1977; Rammsayer, 1992b, 1994a,b; Rammsayer and Lima, 1991). Therefore, temporal processing of intervals in the range of milliseconds is explained by the assumption of a hypothetical internal clock. Although the internal clock metaphor does not suggest any specific biological substrate underlying time perception, it proved to be a useful heuristic for elucidation of biological mechanisms involved in processing of brief durations. Neuropharmacological studies in animals provided experimental evidence that the clock rate is positively related to dopamine (DA) D2 receptor activity (Church, 1984; Maricq and Church, 1983; Meck, 1996, 1986). Similarly, after pretreatment with the D2 receptor antagonist haloperidol, human subjects showed impaired performance on temporal discrimination of intervals in the range of milliseconds (Rammsayer, 1989, 1993, 1994a, 1997b). This finding is consistent with the assumption of a DA-induced decrease in pacemaker speed. Furthermore, results of pharmacopsychological studies in healthy subjects (Rammsayer, 1997b, 1993) as well as abnormal findings in temporal processing of extremely brief intervals observed with patients suffering from Parkinson’s disease (Artieda et al., 1992; Pastor and Artieda, 1996; Rammsayer and Classen, 1997) suggest that pacemaker speed may be modulated by dopaminergic activity in the basal ganglia. Unlike temporal processing of time intervals in the range of milliseconds, temporal processing of longer intervals is cognitively mediated. In a psychophysical study, Rammsayer and Lima (1991) provided the first experimental evidence that timing of auditory intervals in the range of seconds is dependent on memory processes, but timing of intervals in the range of milliseconds is not. In subsequent pharmacopsychological studies, the benzodiazepine midazolam (Rammsayer, 1992b) and the atypical neuroleptic drug remoxipride (Rammsayer, 1997b) were shown to produce a very pronounced decrease in temporal processing of intervals in the range of seconds, while processing of intervals in the range of milliseconds was not affected. These findings suggest that any

Discussion section

419

treatment that interferes with active processing of information in working memory results in a performance decrement in processing of durations longer than approximately 500 ms. Therefore, it seems to be highly unlikely that one highly specific cellular or physiological process in the brain may account for performance in timing of longer durations. Furthermore, it is interesting to note that time psychophysics provides similar patterns of results for animals and humans. For example, Fetterman and Killeen (1992) yielded very similar data for temporal discriminations by pigeons and humans, especially for durations below 100 ms. Both were well described by the generalized Weber function. In other words, despite the involvement of complex cognitive processes, human timing may be based on temporal mechanisms similar to those of animals. This may indicate that rhythms inherent to animal cells provide the first level of temporal adjustment to the environment. Editor: Is the great temporal precision of some musicians in perceiving and generating rhythms a reflection, at the molar level, of the precision of axonal conduction time at the molecular level? Rammsayer: I don’t have any own data on outstanding musicians. However, in a previous paper (Rammsayer, 1994d), 24 blind and 24 matched normal-sighted subjects were compared with regard to their performance on auditory temporal discrimination of intervals in the range of seconds and milliseconds. Investigations on auditory processing in congenital or early-acquired blindness showed that there are no differences between blind and sighted subjects, as long as basic auditory functions located at a lower level of the central nervous system (such as interaural time differences for directional hearing, or acoustic reflex thresholds) are concerned (e.g. Starlinger and Niemeyer, 1981). If temporal processing of filled intervals in the range of milliseconds is based on mechanisms located at a lower level of the CNS, or at the cellular level, no differences between blind and sighted subjects are to be expected. Furthermore, Musicant and Butler (1980) found that internal representation of auditory space in sighted subjects markedly improved with practice. This points to the conclusion that practice-dependent enhancement of cognitive representations of perceived auditory information accounts for better performance on auditory localization normally observed in the blind (Rice, 1970). Since duration discrimination of intervals in the range of seconds is largely dependent on cognitive representations of the intervals to be compared, more efficient, and thus more accurate cognitive representations will result in less variability in processing of durations in the range of seconds. Therefore, one would expect a higher level of performance in blind as compared to sighted subjects. Superior performance of the blind subjects was found with intervals in the range of seconds. No significant difference was found with intervals in the range of milliseconds. This pattern of results provides at least indirect evidence for the notion that the great temporal precision of some musicians in perceiving and generating rhythms may represent a reflection, at the molar level, of the precision of axonal conduction time at the molecular level. Finally, Rammsayer (1994c) found that performance on duration discrimination with auditory intervals in the range of milliseconds does not improve over 20 testing sessions. This lack of practice effects also points to the existence of a very basic biological timing

Discussion section

420

mechanism that, unlike many higher cortical functions, is not susceptible to experience and training. These data may also support the notion of a timing mechanism, located at the molecular level, for processing very brief durations.

4. THE DEGREE OF UNIT SYNCHRONY NEEDED TO PRODUCE AN EEG WAVE Editor: Eckhorn mentions evidence that Fast Cortical Oscillations (FCOs) are more replicable across stimulus repetitions, and more easily correlated between loci, if local field potentials rather than multiunit records are used. Can one conclude from this that anything which is detectable in an EEG or electrocorticogram trace represents some degree of synchrony? For instance, when recording “low voltage fast” (“desynchronized” activity), do the fast irregular low amplitude waveforms still correspond to some degree of coherence within local cell groups, rather than complete desynchronization (i.e. complete lack of co-ordination between neurones)? Eckhorn: My answer is yes, and my arguments are derived from our multiple microelectrode recordings in the visual cortex of anaesthetized cats and awake monkeys (Eckhorn, this volume). The argument is based on the rather narrow cortical extent of coherence among intracortically recorded fast oscillations in relation to the broad range averaged in EEG-recordings. There are seemingly contradictions between observations of scientists measuring EEG (or MEG) and those recording local field potentials (LFP, 0 to 150 Hz) intracortically with micro-electrodes. While EEG signals show low frequency, high amplitude, synchronized signals during many stages of sleep and unattentive waking states they become low amplitude, high frequency, desynchronized when the brain performs cognitive tasks (e.g. Nunez, 1995). In accordance with these observations, intracortical recordings with microelectrodes also show synchronization of high amplitudes in the low frequency range during inattentive states and drowsiness, and reduction of low frequency signals when the brain changes its state to attentiveness (e.g. Lopez da Silva, 1991). However, during attentive waking states intracortical microelectrodes found high frequency (30 to 80 Hz) synchronized oscillations of high amplitudes in areas of the visual (review in Gray 1994) and motor system (Sanes and Donoghue, 1993; Murthy and Fetz 1994). These findings seem contradictory: why do EEG (and MEG) recordings show low voltage fast desynchronized signals during these same brain states, and fast synchronized oscillations can barely be detected (e.g. Tallon-Baudry et al., 1997)? Our microelectrode recordings may answer this question, at least for the visual cortex. We found the coherence of fast oscillations (FCOs, 35 to 80 Hz) declining with cortical distance (spatial constant about 1.5 mm; Juergens et al., 1996; see Eckhorn, this volume) corresponding to a circular effective area of coherence of about 10l mm2. This means that neighbouring areas of equal extent are only weakly synchronized, their signals, including FCOs, are statistically largely independent. In comparison, an EEG electrode has an effective input range of about 103 to 104 mm2, and therefore averages across 102 to 103 cortical patches of locally coherent, but globally independent FCOs. The superposition of

Discussion section

421

102 to 103 nearly-independent FCO sources superimpose to low voltage, even if signal transmission from cortical circuits to scalp EEG were without amplitude reduction. However, the many locally-coherent sources do not completely cancel each other in the EEG, because their number is not very large, and each can contribute large amplitudes, leading to detectable low voltage fast random signals. If even in the small (10 mm2) patches the neurones would contribute completely independent signals to the LFP (and hence to the EEG) they would superimpose in the scalp recording to undetectable levels. My arguments may be expanded to other frequency bands. If EEG and MEG signals can be detected above noise, this is probably due to the fact that thousands of neurones are partially synchronized in their activities. As LFP and EEG reflect mainly the extracellular average of postsynaptic potentials (Creutzfeldt et al., 1966; Mitzdorf, 1987) we may conclude that the many independent components of stochastic, broad-band, low amplitude EEGs are generated by local groups of partially synchronized neurones. Two effects will generally cause larger EEG amplitudes: higher amplitudes of neural activation and, in particular, broader cortical ranges of synchronization. In high voltage EEG states, the low frequency components are present in extensive neural networks, but in the gamma range (of FCOs) synchronization is restricted to small patches so that single patches contribute only very small amplitudes to the scalp signal.

5. THEORY OF CORTICAL OSCILLATIONS AND SIZE OF CORTICAL REGIONS INVOLVED Editor: Wennekers and Palm were asked if the arguments presented in section 2.2 of their chapter depend on the assumption that “higher associative areas are usually small”? Wennekers/Palm: In short, no. Theoreticians prefer simple models. In particular they like limiting cases. Those often provide insights into typical dynamical or functional aspects of considerably more complicated systems. With respect to cortical areas, two limiting cases have been investigated in the literature. The first are models that describe the cortex as a topographically organized (potentially infinitely extended) structure with local connectivity. Such models enable the study of spatial excitation patterns on a large scale. The second type of model targets the local dynamics of neural networks. These neglect the spatial extent of an area, but assume that the investigated phenomena take place in a sufficiently small piece of cortical tissue with dense lateral connectivity. The models described in sections 2.1 and 2.2 of our chapter are both intended to provide a description of local cortical processes. Processes possibly going on in extended cortical structures are briefly and only qualitatively discussed in sections 3 and especially 4. Inasmuch as section 2 deals with local networks and section 4 already considers the interactions between several areas, we realize that we should have added a section discussing a single topographically organized area. We do this in what follows:

Discussion section

422

Figure 1. Activation in a topographically organized area in response to the static “grating”-stimulus in (a). In (b) 25 single unit spike trains (SUA), their sum (MUA) and the averaged membrane potential (LFP) of a 5 times 5 square around unit (30,37) are shown. Some cells fire regularly, others lock randomly into the population spikes. Amplitudes and individual period durations are highly irregular due to this random entrainment of cells. (c) Spike patterns of excitatory cells at different times (coarsely 1.5 periods) in top-down view onto the model area. Synchronization is restricted to connectivity regions. Each plot re-bins spikes from 6 ms.

Figure 1 reveals simulation results for a topographically organized area. The network consists of 64 times 64 excitatory spiking neurones, which are locally connected within neighbourhoods of size 11 times 11. Furthermore the local activity within the network is controlled by inhibitory interneurones which receive input from the excitatory cells in a neighbourhood of 7 times 7 cells and inhibit these cells accordingly. Input to the network is the grating-pattern in Figure 1a; excitatory cells along the lines of the grating receive some external input current, that raises them above threshold. Further details of the model can be found in Wennekers and Palm (1997). For the current discussion it suffices to note, that the local connectivity is similar to that in the models described in section 2 of our chapter: excitatory cells are recurrently connected and controlled by inhibitory cells that measure the local firing activity. It turns out that not only the structure, but also the local dynamics is very similar in these models. This can be seen in Figure 1b, where 25 spike trains (single unit activity: SUA), their sum (multi-unit activity: MUA) and the locally averaged membrane

Discussion section

423

potentials (LFP) within a neigbourhood of 5 times 5 cells around position (30,37) of the cell layer are shown. The figure should be compared with Figure 2 in our chapter. What happens is that the local signals are oscillatory, and within a single period some excitatory cells start firing. Those also raise sub- (but near-) threshold cells in their neighbourhood above threshold. Finally, the local and somewhat delayed inhibition switches the activity off, thereby avoiding an uncontrolled state of persistent firing of all cells at high rates, and enabling a new response cycle after roughly ten milliseconds. This aspect of the local dynamics is common in the local networks in sections 2.1 and 2.2 and the topographically ordered model in Figure 1. Figure 1c, however, reveals that the ongoing processes in the topographic model are nonetheless somewhat more complex. Spikes of the 64 times 64 excitatory cells (in topdown view onto the extended model area) are shown for 6 consecutive times, binned over 6 ms. It should be noted that cells fire, synchronized only over certain regions, which roughly correspond with the regions of physical connectivity. For longer distances, correlations decay to zero, i.e. distant sites are relatively isolated from each other (correlograms not shown). The hypothesis drawn from this observation, that the range of anatomical connectivity and that of significant synchronization are related, seems also to be supported experimentally (e.g. Eckhorn et al., 1988; König et al., 1995b). The observation that the local dynamical processes in the topographically organized model are similar to the processes in the models described in section 2 supports a modularized view of extended cortical areas. Although such modules need not be defined in a strict anatomical sense, it seems reasonable to model local cortical regions by associative networks as we have done in section 2.2. Inasmuch as higher cortical areas are often relatively small, a single associative network may suffice to describe the main processes going on in such an area. Inasmuch as higher areas can also be quite large, the topography of interactions in these areas may become important. In some cases such interactions might be restricted only to some neighbourhoods, leading to a relative independence of the processing at distant sites comparable to the simulation in Figure 1. If, however, long range collaterals within the area are expected to play an important role, then many local modules may interact. This situation may be described by an associatively-connected network of local associative subnetworks. Such networks are similar to those discussed in relation to the concept of “global assemblies” in section 4 where we qualitatively described some of their properties. In particular when propagation delays between distant submodules are large, we do not expect that those modules are necessarily synchronized. Whether systematic phase shifts occur—i.e. ones which are in some sense “topographically” organized—when the local modules reveal an oscillatory behaviour is a complex question, discussed below.

6. TOWARDS A UNIFIED VIEW OF DIFFERENT EEG FREQUENCIES Miller: In Eckhorn’s chapter, evidence is presented that, as gamma frequency falls, the area of cortex over which gamma is coherent increases. In Klimesch’s chapter on alphaband activity, the area for which alpha is coherent is greater than that for gamma; and

Discussion section

424

within the alpha band, the area of coherence increases as frequency falls. Thus, over a wide frequency range from 8–80 Hz the area of cortex over which coherent oscillations occur appears to vary inversely with the frequency. This generalization can also be supported by data in Klimesch’s chapter on alpha band activity: Alpha frequency appears to be inversely related to brain size. In addition, the higher the frequency, and the more local the oscillations, the more specific the information processing appears to be. This generalization appears to apply to the three bands of alpha whose functions were analysed by Klimesch. It also applies within the gamma band (see chapter by Eckhorn). Are these two generalization valid? Rappelsberger: I can support the statement that oscillation frequency is inversely proportional to volume of brain tissue involved. This is based on extensive studies on the synchronization mechanisms of epileptic seizures. However, the formulations you used in the first two sentences: “…as gamma frequency falls, the area of cortex…increases”, and “…the area of coherence increases as frequency falls.” implies that frequency determines the amount of neuronal tissue. In contrast, I would say: “…as the involved area of cortex increases, gamma frequency falls”, and “…frequency falls, as the area of coherence increases.” Our investigations concerning the generation mechanisms of the EEG and synchronization mechanisms of epileptic seizures (Petsche et al., 1984) were made with closely spaced epicortical and/or intracortical electrodes in the rabbit’s cortex. These experiments revealed clearly that after local cortical application of small amounts of penicillin, seizures started very locally, with high frequency activity, in the range of 30– 40 Hz. Due to the involvement of more and more of the surrounding tissue, frequency rapidly decreased and amplitude increased (Petsche et al., 1987). Klimesch: In the above paragraph Miller correctly points out that EEG alpha activity shows a similar inverse relationship (between the frequency and the size of the activated area) as do gamma oscillations (see the chapter by Eckhorn). The most basic findings are that larger brains tend to have a slower alpha frequency than do smaller brains, and that activity in the lower alpha band is topographically more widespread than activity in the (faster) upper band. However, if we consider the finding that (i) larger brains tend to be more intelligent (Andreasen et al., 1993; Bigler et al., 1995; Harvey et al., 1994; Wickett et al., 1994; Willerman et al., 1991) and that (ii) more intelligent people (Anokhin and Vogel, 1996) as well as people with better memory perfomance (see the data reviewed in Klimesch’s chapter) have a significantly higher alpha frequency, we arrive at a clear contradiction. The following two hypotheses provide possible solutions to this contradiction: (i) Due to more intensely myelinated axons, intelligent brains are bigger and faster. In evaluating this hypothesis, it should be noted that the white matter makes up a large proportion of brain volume, and that (in a statistical sense) the number of neurones (particularly in the cortex) is highly correlated with the amount of white matter. Thus, the larger volume of more intelligent brains may very well be due to a larger number

Discussion section

425

of neurones and not to more intensely myelinated axons per se. Swadlow emphasizes that the cost in brain volume of rapid communication is high, and that such biologically expensive processes are limited to special situations. (ii) Although having a larger brain, more intelligent people activate smaller cortical areas more selectively, while less intelligent people show a more global cortical activation pattern. This assumption is based on the “selective activation hypothesis”, which holds that intelligence depends on the degree of specialized functions in the brain. Higher intelligence is characterized by a higher functional differentiation of brain areas and by a higher selectivity of activation, of only those areas which are specialized for the task in hand (e.g. Ibatoullina et al., 1994). Studies measuring glucose metabolic rates (Berent et al., 1988; Parks et al., 1988; Haier et al., 1988) support this hypothesis. When one accepts the selective activation hypothesis, additional evidence for a general validity of an inverse relationship between frequency and the size of the activated brain area is provided. The behaviour of individual alpha frequency (IAF) with respect to task difficulty also agrees well with predictions derived from the selective activation hypothesis. As an example, in Klimesch et al. (1993a; cf. Figure 5) we found that with increasing task difficulty, IAF decreases for poor but not for good memory performers. Thus, we may assume that poor memory performers activate larger cortical areas than good performers. This assumption is supported further by the fact that, during the most difficult parts of the task (during encoding and recognition), band power changes (in the lower alpha band) are much more widespread for poor as compared to good performers (Klimesch et al., 1993a; cf. Figure 10). Wennekers/Palm: Origin of Oscillations. Oscillation frequencies in different EEGranges are probably due to different mechanisms. Rhythms in the same frequency range may even have different physiological origins. They are likely to be generated in local networks where conduction times are negligible (relative to the oscillation period). Although EEG-rhythms in different frequency ranges have been studied for many years there is no general agreement about their physiological origin. This is particularly true for fast rhythms (beta, gamma), whereas things seem to be simpler with respect to slower rhythms (alpha, theta, delta). It is commonly believed that theta is brought up to the hippocampus (archicortex) from intrinsically oscillating pools of pacemaker cells in the basal forebrain (septum, see Lee et al., 1994; Buzsaki and Chrobak, 1995, [and references therein]). The main projection system from the septal region to the hippocampus is inhibitory. Similarly a common view on alpha frequency components in (neo-)cortex is that they are generated by networks of intrinsically oscillating inhibitory cells in the reticular nucleus, from where they are transferred to cortex via thalamic relay nuclei (e.g. Steriade et al., 1991, 1993, 1996). Although, in both cases, sources intrinsic to the cortex (for theta- and alpha-oscillations) cannot be ruled out, the main sources seem to be extra-cortical. In contrast it is likely that gamma- (and beta-) oscillations are generated by mechanisms intrinsic to the cortex, although there is no agreement about their detailed physiological origin. Several possible mechanism have been discussed in the literature: Many simulation studies assume that collective oscillations arise from coupled networks of pyramidal cells whose firing is intrinsically periodic, so that they

Discussion section

426

synchronize their spikes in time. In section 2.1 of our chapter we have argued against this possibility because most cortical cells do not really fire periodically, particularly not all at the same frequency in the gamma-range (see also Wennekers and Palm, 1997). Instead, cells reveal a broad distribution of firing frequencies, and most cells couple only loosely into the collective local oscillation (cf. Wennekers and Palm, 1999). Gray and McCormick (1996), however, reported the existence of certain cells (“chattering cells”) located in layers II/III of cat striate cortex, which generate high frequency bursts at repetition frequencies from 20 to 70 Hz in reponse to supra-threshold depolarization. Such cells have axon collaterals that extend horizontally, and ramify in layers I, II, III, and V. Even if they are not very numerous they may act as pacemaker cells, recruiting large ensembles of neurones into coherent oscillations. A recruitment of cells may be furthermore supported by sub-threshold membrane oscillations intrinsic to single cells, that have been observed in the beta- and lower gamma-range in several studies (Llinás et al., 1991; Steriade et al., 1991; and others). Buzsaki and Chrobak (1995) propose a different origin for gammaoscillations (in the forebrain). They suggest that those arise from neuronal “supernetworks” of GABAergic cells that synchronize by themselves and cooperatively entrain large populations of principal cells. The basic idea is that such synchronized oscillations in interneuronal networks keep the membrane potentials of principal cells near, but below, threshold, thereby maintaining them in “readiness” and providing a precise timing of action potentials of the principal cells. Buzsaki and Chrobak furthermore suggest that interneuronal networks may act similarly for oscillations in different frequency ranges (4–12 Hz, 40– 100 Hz and around 200 Hz [“ripple-oscillations”]). Synchronization in networks of inhibitory cells has, for example, been shown for thalamic reticular neurones, which can oscillate intrinsically in the alpha-range (e.g. Steriade et al., 1993), and in slices of rat hippocampus and cortex (at frequencies in the gamma range: Whittington et al., 1995). Cobb et al. (1995) further demonstrated that even individual GABAergic interneurones in the hippocampus are able to synchronize neuronal activity in their postsynaptic excitatory target cells. The impact of synchronized ensembles of inhibitory cells might therefore be even more pronounced. An alternative view on the origin of gamma oscillations is that they are not generated by individual cells (either excitatory or inhibitory) which already fire periodically and simply align their spikes in time when they are coupled with other cells, but that the rhythm is a collective effect within pools of cells which are not necessarily firing periodically. This view is also adopted in our simulations, where the oscillations arise from two interacting pools of excitatory and inhibitory cells. In our chapter we have shown that collective oscillations in such networks have many properties in common with experimentally-derived recordings, particularly if the oscillations are very noisy and the network operates near the instability between stationary and oscillatory behaviour

Discussion section

427

(section 2.1., cf. also Wennekers, 1999). As a further tentative source for cortical activity in the gamma-band we should also mention that there is evidence that gamma is not exclusively generated intracortically, but that it is at least modulated by subcortical structures. Ghose and Freeman (1992), for example, have argued that cortical gamma might be driven by gamma-oscillations in the LGN. Other studies also prove subcortical influences on the generation and synchronization of cortical gamma-oscillations (Steriade et al., 1991, 1993, 1996; Molotchnikoff and Shumkhina, 1996; Barth and MacDonald, 1996; and others). The above brief review shows, that there is no agreement on the origin of cortical gammaoscillations. The situation seems to be more consistent with respect to alpha and theta, although the detailed physiological mechanisms for their generation are different, and cortical sources for these rhythms also cannot be excluded. (It is quite probable that such sources exist at least in the alpha-range.) It is furthermore not unlikely that different mechanisms may generate rhythms in the same frequency range under different physiological conditions or in different cortical structures (cf. also Gray, 1994). Therefore we cannot agree with the suggestion of a “unified view of different EEG frequencies”. Miller: If the above generalizations are valid (even just within the fast frequencies of the EEG) it immediately raises questions about the underlying mechanism. From Swadlow’s chapter, it is clear that axonal conduction time in some corticocortical axons may be several tens of milliseconds, and may even be 100–200 millseconds for axonal pathways in the human hemisphere (Swadlow et al., 1979). An important question is therefore raised: what part, if any, do these conduction delays play in determining degree of coherence, area of coherent oscillations, and phase relations of oscillations recorded at different cortical loci? One suggestion is that oscillation frequency is a function of the “round trip” time (Pulvermüller’s succinct phrase). With such a model, the wider the area of coherent oscillation, the longer the conduction times involved, and therefore the slower the collective frequency. This seems to fit the facts, or at least some of them. However, given the model based on “round trip time”, one might expect to see regular phase differences between waveforms recorded at different loci, and the phase difference should increase as the spatial separation of the loci increases. This has not often been reported, but Rappelsberger’s chapter does show such a relation (see also Thatcher et al., 1986). However, such a relationship is not obligatory: rhythms recorded at different loci may be well-synchronized, even though, if the “round trip time” model were to be relevant, the likely conduction time between loci would be enough to give a detectable phase delay (see Eckhorn’s chapter). Several chapter authors (Nowak and Bullier, Eckhorn, Borisyuk et al.) have mentioned theoretical models which show that such synchrony of oscillations over space can occur commonly even when there are substantial conduction delays. In fact, neither of the theoretical chapters in this book (viz: those by Wennekers and Palm, and by Borisyuk et al.) use the concept of axonal delay lines in developing their models. As a result of these considerations, two critical questions arise. These are addressed to the authors of several of the empirical papers, but especially to the authors of the two theoretical papers: (i) under what conditions, if any, could conduction time between loci

Discussion section

428

determine the frequency of coherent oscillations? (ii) under what conditions, if any, could conduction time between loci determine phase differences between correlated oscillations at different loci? (The answer to these two questions may not be the same: conduction time for “round trip” pathways might be inversely related to frequency, even though the coherent oscillations so produced are nevertheless synchronized across space.) Eckhorn: The arguments for a “unified theory of the EEG” over frequencies from 8 to 80 Hz proposed by Miller in this Volume are mainly generalized from facts reported by other contributions in this volume (Klimesch; Swadlow; Eckhorn). It is argued that fast locally-restricted synchrony of high frequency signals denotes states of high information processing, while slow synchronized rhythms are indicative of less information processing. In terms of information theory (Shannon, 1948) and nonlinear dynamics this statement seems acceptable: the highest amount of information can theoretically be processed if neural signals are as independent and as fast as possible. On the other hand, information processing would be minimal and completely absent if all neurones of the brain engage in a synchronized oscillation. Hence, parallel processing in many local assemblies at high signal rates may therefore be indicative of intense information processing. However, restricted local processing, without medium and global range interactions, would make no sense for the tasks a brain has to solve. For bi-directional corticocortical processing across large distances (a common task), axonal transmission delays will play a considerable role. If synchronization among remote areas is required, this would cause particular problems. Gerstner and co-workers showed in a neural network model that synchronization at zero correlation delay is possible if the axonal transmission times between separate neural populations are shorter than about a third of the oscillation’s period (Gerstner et al., 1993). If associative processing among remote positions really requires mutual synchronization of the local oscillations then two simple arguments can be developed for the “unified” requirements of operation in different frequency bands. Assuming the same spike velocity in all lateral cortical connections, the cortical range of associations will increase linearly with the duration of the oscillations’ period (definition of synchronization or association field AF: Eckhorn et al., 1990; Eckhorn, 1998). As local circuits synchronize at high frequencies, remote interactions should occur at lower frequencies, either at subharmonics or, with non-linear phase-coupling, at lower frequencies. Such nonlinear couplings over larger distances have not been confirmed to date in EEG (possibly because the methods of analysis which have been applied were not adequate). However, phase-coupling among low, medium and high frequency ranges has been found in local microelectrode recordings from cat and monkey visual cortex (Schanze and Eckhorn, 1997). In this work it is proposed that local linking of a visual object’s features may be supported by high frequency oscillations, while their global relations are supported by phase-coupled rhythms at lower frequencies (see also Eckhorn, this volume). As is argued above, the lower frequencies can cover larger areas of cortex, where they can engage in synchronous oscillations and therefore appear at higher probability at detectable amplitudes in an EEG trace. In a realistic cortical model, however, there is not a single axonal velocity, but lateral and other bi-directional corticocortical and corticothalamic connections are generally formed by fibre bundles with a variety of different velocities. In general there are many

Discussion section

429

slowly conducting axons, fewer of medium velocity and very few of high velocity (see Swadlow, this volume). It seems therefore possible that synchronization, even at high frequencies, is mediated by the few very fast axons, while the huge amount of detailed information is transmitted via the more slowly conducting fibres. If this is really an important mode of processing by the brain, synchronized fast oscillations (FCOs) should be found among all cortical positions that are monosynaptically connected (and include very fast fibre connections). This is obviously not the case, because then coherence of FCOs should be present across larger distances than those found with intracortical recordings (Eckhorn, this volume). Observing coherence at low frequencies over large cortical distances does not mean that this effect is due to the same type of circuit extending its possible spatial range when frequencies decrease. Even if such a mechanism is operational, one has to bear in mind that a variety of cortical and subcortical loops exist, having very different properties and circuit diagrams. If any such circuits are activated it is expected that the probability of becoming phase-coupled over larger distances is higher, the lower their frequencies are. In conclusion, coupled dynamic systems in which oscillatory activities play a role for information processing, and in which considerable transmission delays are present will synchronize over larger distances the lower are the frequencies of the oscillations that should be synchronized, and the faster the conduction velocity of neurones that couple these processes. However, for me it is not clear whether such considerations can be unified to a theory of the EEG. Rappelsberger: Generally, from the technical point of view, the generation of an oscillation needs a feedback loop. This means part of the output of a system (e.g. of a neuronal ensemble) must be fed back to the input. For the development of an oscillation, a pre-condition is the time or phase relation between input and output. Therefore, oscillation frequency may be considered as a function of that time relation. Feedback loops are well known in the brain. They are described for instance between a few neurones within one specific structure via interneurones and axonal collaterals, or between different structures, such as between the visual cortex and the geniculate nucleus of the thalamus. Already in the seventies the thalamocortical feedback loop was a matter of discussion for the generation of cortical alpha rhythm (Lopes da Silva et al., 1974, 1976). The question about what part conduction delays play in determining the degree of coherence is not easy to answer. I would distinguish between very local activities in the gamma range, and coherent activities between distant areas in the theta, alpha and lower beta range. As far as gamma activity is concerned, very fast local feedback loops between only a relatively small number of neurones seem to be involved. Due to the short distances I think that axonal conduction time plays only a minor role compared with the synaptic delays, and the necessary time relations of the feedback loops are mainly determined by these synaptic delays. In the second case, of coherent activities recorded at distant brain regions, I would like to draw attention to the following model. In Figure 2 of our chapter we tried to explain coherence in terms of the output signals of neuronal ensembles. Both neuronal ensembles generate coherent signal portions. The frequency of the coherent portions of the signal may be determined by the necessary relatively local feed-back loops, either

Discussion section

430

corticocortical or corticothalamic. However, synchronization of both ensembles is obtained via the connections between both ensembles, and phase relations between the oscillations of both ensembles may be strongly influenced by conduction times. Phase delay is determined by the conduction time between distant loci, and oscillation frequency is determined by the local feed-back conditions, which however may be strongly coupled via the different pathways. It is my belief that the frequency of coherent oscillations is determined by more-or-less local feed-back loops (i.e. by the resonance frequency of neural tissue). The time conditions for the generation of an oscillation may be influenced by local conduction times and synaptic delays. From the technical point of view, conduction time between distant loci can determine the frequency of oscillation only if the conducting pathway is included in the feedback loop. This means the distant neuronal ensembles are parts of a global feedback system. However, it may happen that one neuronal ensemble is triggered by the other in such a way. Editor: Could phase differences for EEG traces recorded in different locations be determined in part by axonal conduction time? In support of this, I notice that phase delays in the chapter of Rappelsberger increase with distance between recording sites, an observation also made by Thatcher et al. (1986). However, I also notice that in the paper of Thatcher et al. (1986), phase delay, measured in milliseconds was greater for the slower than the faster rhythms. This might be difficult to reconcile with the idea that phase delays in the EEG are a direct reflection of axonal conduction, unless populations of axons with different conduction times are involved in transmitting oscillations of different frequencies. Could you comment on this? You do not give data on phase delays comparing between alpha and beta frequencies. Is that data easily available? If so, does it support what Thatcher et al. (1986) found? Rappelsberger: Coherent activities recorded from distant brain regions can be considered as phase-locked, since coherence may be understood as a measure of phase stability (Nunez et al., 1997). Synchronization of cell ensembles is maintained via axonal connections, and phase relations may be strongly influenced by conduction times. From the above mentioned paper of Thatcher posterior-anterior phase delays of about 30 to 35 ms for distances above 20 cm in the alpha range (7–13 Hz) and 10 to 15 ms in the beta range (13–22 Hz) can be read. Concerning our own data, phase analyses between electrodes O1-Fp1 during visual presentation of concrete words in (so far) 22 subjects (see experiment II of our contribution) yielded: on the average 31 ms in the alpha-1 range (8–10 Hz) and 20 ms in the beta-1 range (13–18 Hz). Admittedly, both the conditions of EEG recording and the methods for phase estimation are difficult to compare, since Thatcher used spontaneous EEG of children and band pass filtering, and our results are based on EEG of adults during word processing and the Fourier approach. Nevertheless, the results are in essential agreement. Comparing phase analyses with the data presented by Swadlow on axonal conduction time (up to 30 ms dependent on structure, cell types, etc.) one cannot deny the strong correlation between both. The problem of differences of phase delays (e.g. between alpha-1 and beta-1 activities) may have two answers. First, since oscillation frequency is inversely proportional to the volume of brain tissue involved, the necessary local feedback loops for generating fast oscillations usually are faster. This may result in shorter conduction and phase delays.

Discussion section

431

The second point concerns the computation of phase itself and needs some mathematical explanations. Equation (6) of the Appendix of our contribution demonstrates the computation of phase from the real and imaginary part of the crossspectrum. The range of values of this arg function is from −180° to + 180°. The equation in the legend to Figure 1 shows the computation of phase in [ms] taking into account the oscillation frequency. Thus, a 10 Hz (Alpha range) oscillation may show phase delays between −50 [ms] and +50 [ms], a 20 Hz (Beta range) activity may show phase delays between −25 [ms] and +25 [ms]. In the case of Beta, this means that phase delays lower than −25 [ms] and higher than +25 [ms] cannot be obtained. An actual phase delay of +30 [ms] will result, by computation, in −20 [ms], an actual phase delay of+50 [ms] will result in 0 [ms]. Such ambiguities are difficult to recognise. For some time we have been trying to find appropriate strategies to overcome these difficulties. A general solution is not yet available. Whether the dispersion of phase delays for slower and faster activities is due to feedback properties, or is based on a mathematical artefact is difficult to decide. Truth seems to me somewhere in the middle. Başar: In my opinion, a unified view of the EEG should be only descriptive and not theoretical. Local generator mechanisms, as they are described now, are insufficient to explain the processes in the entire brain. I think that, at the present time, only global rules for the role of EEG and ERPS in communication, or ways to approach their function can help towards a general theory. My view would be that we have to consider the EEG frequencies as elementary building blocks, or universal codes for brain communication and functioning. This is similar to the problem in elementary particle physics at the turn of the century. We have a long way to go. Wennekers/Palm: We argue against the “round-trip time” hypothesis. Our main proposal is that the size of a synchronized region, together with conduction times, probably do not determine the oscillations frequencies, but in contrast, oscillation frequencies and conduction times are likely to determine the range of synchronized regions. Range of Synchronization: Oscillation frequencies and the round-trip time model. With respect to cortical ranges of synchronization, it is our impression that the “round-trip time” model exchanges cause and effect. As far as we understand it, it suggests that the region over which a cortical rhythm is synchronized, and the velocity of cortical signals between distant sites, determine the frequency of the respective cortical rhythm. In our opinion the situation is quite the opposite of this suggestion: The frequency of a cortical oscillation, together with propagation velocities, determine the range of synchrony. Moreover, if cortical activity is non-oscillatory, then the propagation velocities (plus local noise) determine the distances over which significant correlations can be found. The round-trip time model seems to come out of an analogy between the cortex and physical models of extended structures that are capable of resonating. Such models, for example, describe musical instruments like drums or strings, but also the propagation of electro-magnetic fields in a vacuum. These models, as well as their physical realizations, do not reveal intrinsic oscillation frequencies per se. Those are selected by boundary conditions, like the size of the drum or the length of a string. The round-trip time model for cortical rhythms assumes that somewhat similar

Discussion section

432

processes are occurring in both musical instruments and cortical structures: sensory inputs bang the cortical drum, waves spread over the cortex, and some boundary conditions select certain frequencies. Low frequencies may result from constraints on a large scale, high frequencies from more local constraints. These constraints—the boundary conditions—might have an anatomical origin, like the range of intrinsic cortical connectivity, or the fixed distance between two synchronized areas. It is assumed that the typical value of the respective (anatomically defined) length-scale together with the typical propagation velocity of signals determine the oscillation frequency. The model seems to be consistent, but it faces (at least) two serious problems. The first can possibly be resolved in some way: this problem is that cortical rhythms do not have a fixed frequency; the range of frequencies typically extending over more than an octave in any EEG band. In contrast, the round-trip time model suggests a fixed frequency. Varying frequencies require either varying constraints (analogous to picking notes on a violin) or varying propagation velocities. Whereas mechanisms for the first are hard to imagine (though they are perhaps not impossible), the second may, for example, result from a selection of subsets of neurones with different axonal velocity distributions in different cortical states. This is quite likely to happen for oscillations in the alpha-range. The contribution of Klimesch in this book assigns different cognitive phenomena to distinct frequency ranges in the alpha band. So, the origin of these distinct frequency ranges might be different, underlying neuronal networks with longer or shorter conduction times. However, with respect to gamma oscillations (30–90 Hz) we do not think that the same argument is applicable. Even an identical stimulus can induce oscillations with a wide scattering of frequencies in subsequent trials (cf. e.g. Ghose and Freeman, 1992). There is no obvious reason to believe that the underlying networks are different in these trials. (See Pulvermüller, below, for further comment.) The second more serious problem for the round-trip time model has already been introduced above. The overwhelming majority of evidence speaks for generation of cortical rhythms in local networks of cells, or even in individual neurones that mutually synchronize with each other. Axonal propagation delays in the locally oscillating networks can be reasonably neglected. They are short in comparison with the oscillation periods, as long as the local generating networks are not larger than, say, a millimeter (for gamma-oscillations; in lower frequency ranges they can be considerably larger). Thus, oscillation frequencies are determined by intrinsic single cell and/or local network properties. Corticocortical round-trip times seem to play no obvious part in the basic mechanisms that generate the oscillations. They may, however, influence the oscillations and their properties in indirect ways (compare e.g. Wennekers et al., 1994). Range of synchronization in a single cortical area: What sets the range of synchronized cortical regions? For gamma-oscillations we have addressed this question by means of

Discussion section

433

computer simulations of topographically organized extended cortical areas (preferably interpreted as a primary visual area). Figure 1 (see above) shows a simulation run and is explained in detail elsewhere. The neural activity in Figure 1 is characterized by local oscillations along the continuous input lines. The network is homogeneous, although it contains some randomness in connectivities. Therefore each of the local oscillators may lead, relative to its neighbours, in a given cycle. The leading oscillator initiates a wave of activity that spreads along the input line. We will explain below how phase fluctuations can lead to phase zero correlations between distant sites, but first, we ask how far apart correlated sites can be. The answer depends on three things: i) the propagation speed v of the wave of activity (which is related to, but not equal to axonal velocities, see Wennekers, 1996); ii) the local noise level; and iii) the oscillation frequency 1/T. Without any noise sources in the network, or irregularities in the input (that means, constant contrast along the input lines) one expects, in the simplest case, that plane regular waves travel along the excited regions. In this respect, the dynamics is somewhat comparable to the round-trip time model, but the dynamic processes are completely different, because the oscillations (and their frequencies) are determined locally. Plane waves, however, imply that correlation functions oscillate with distance (beside their oscillations in time); the spatial length constant, i.e. the distance between maximally positively-correlated sites, is determined by the signal propagation speed v, and the oscillation period T. It is given by . Further peaks exist for any integer multiple of d. Thus the noise-free model implies infinitely extended long-range correlations. In addition, between two maxima in space, this model predicts regions (e.g. at d/2) where sites oscillate in anti-correlated fashion. Both effects, long-range and anti-correlations are typically not observed in experiments. Therefore regular plane waves probably do not propagate over long distances, along extended stimulus edges in cortex. Experimental data suggest that activity can travel only between sites that can be reached within a fraction of the period duration, and in particular within the excitatory phase of the period. Then, spikes emitted at one site can contribute to the excitatory amplification process going on at the other site. If the same spikes reach the target site too late, they fall into the inhibited phase of the oscillation at the second site. Then they are not expected to have a large impact on either the dynamics or on the local information processing at that site. The excitatory phase of the period lasts for roughly one fourth to one third of the period duration. Assuming a propagation speed of 1 m/s and a period of 20 ms, one estimates that signals can propagate roughly 5 to 7 mm. Similar values have been found for the spatial decay constant of correlations in primary visual cortex (Eckhorn et al., 1988). Note also that higher oscillation frequencies lead to smaller length constants. We already mentioned that plane waves imply spatially oscillatory correlations. Taking the above values, v=1 m/s and T=20 ms, the spatial distance between two maxima is 2 cm. In experiments, sites separated by more than 8 or 10 mm within a single cortical area seldom reveal correlations; the average spatial decay constant in cat visual cortex has been found to be roughly 7 mm (Eckhorn et al., 1988). Therefore, distant sites seem to be effectively decoupled. A natural explanation for this finite correlation length is local noise, for example due to spontaneous background activity, spike noise in neural signals or randomness in synaptic

Discussion section

434

connections. Noise disturbs the propagation of activity and leads to phase fluctuations which accumulate with distance. Therefore, long-time correlograms between distant sites, or correlograms averaged over many trials will decay on a spatial scale inversely related to the noise-level. The noise, of course, also leads to a decay of correlations in the temporal domain. The fact that many experimentally-determined temporal correlation functions reveal only a single (or a few) weak oscillatory side peak(s) suggests that the cortical noise level during gamma-oscillations is indeed very high (compare the discussion in section 2.1 of our chapter). If the noise is strong enough it can even perturb the propagation within the above estimated range of 5–7 mm. Experimental results, however, suggest that, at least for smooth stationary stimuli, decay constants are roughly in the stated range. Therefore, spatial anti-correlations (e.g. at d/2=1 cm) and long-range correlations (at nd, where n is an integer) are suppressed. It might be possible that correlograms determined over short time windows show longrange peaks, and perhaps also anti-correlations. This would give some evidence for propagating waves on the spatial scale of millimeters. The long-range phase relations, however, must fluctuate strongly in that case. Otherwise they would not average out in long-time correlograms. We should further mention that distant sites can be correlated even if the local activity is not oscillatory. In that case, noise may similarly set the length scale of spreading activity and therefore that of significant correlations. The above arguments imply a relation between signal velocity, region of synchronization and period duration: The region is roughly proportional to the velocity times one quarter of the period duration. In other words, lower frequencies correspond with potentially larger synchronized regions. This relation should also be valid in other EEG frequency ranges than gamma. The “round-trip time” model suggests the same relationship, but one should note that it relies on a completely different explanation. In our opinion the frequency is a fundamental property of local circuits, and, together with signal propagation speeds, determines the range of synchronization. The round-trip time model assumes that propagation velocities and synchronization regions are fundamental and determine the oscillation frequency. Regions of synchrony and synaptic connectivity: Earlier we have stated that the range of synchronization is determined by the physical range of excitatory interactions. In the above arguments we assign it basically to the distance signals can spread during a fraction of the period duration. The two statements are not necessarily compatible, but it is interesting to note that, for gamma-oscillations in a single cortical area, they are consistent. The first statement was drawn from the spatial decay of correlograms computed for the simulated data shown in Figure 1. Thus, the finding that the decay constant roughly equals the spatial extent of excitatory connections is of an empirical nature. The second statement relies on theoretical reasoning and implies that the decay constant is also roughly the distance signals spread during a gamma period. Interestingly both length constants are similar in the context of gamma-oscillations in primary visual areas: Collaterals of pyramidal cells can extend over a few millimeters, and—as estimated above—the distance of spread is also a few millimeters. One may ask if this coincidence is only accidental? Recent computer studies by Mirko Saam from Reinhard

Discussion section

435

Eckhorn’s lab in Marburg suggest that both length constants might be related, due to cortical learning processes (Saam and Eckhorn, 1998). Synaptic plasticity based on the near-coincidence of spikes (a modified Hebbian learning rule) explains the coincidence of the above two length scales, because the range of synchronization determines the range of connectivity during learning. Correlograms computed between distant sites in the simulations in Figure 1 do not show phase shifts. They have peaks centred near zero as long as the activity at the two sites is correlated at all (data not shown). This result is somewhat surprising, since one could expect that neural signals need some time to travel from one site to the other. These delays might show up in cross-correlograms in the form of shifted peaks. The reason why shifts are not present in the simulated data is related to the homogeneity of the network along the lines of the grating. Assume that “electrodes” are fixed at two distant sites along one of the lines in Figure 1 (different lines of the grating are so far away from each other that we never observed any correlations between cells falling on them). It turns out that the activity at both electrodes is oscillatory, but due to different sources of randomness the local oscillations are not very regular (Figure 1b). This has the effect that at some time the first of the local networks leads the oscillation and at another time the second. This alone cannot yet explain the absence of phase shifts, but could still lead to 2 peaks shifted into the positive and negative direction, each by an amount given by the delay between the two sites. However, in between the two sites along the input line the local networks also oscillate and any of these sites may lead the oscillation as well. In that case the local network at electrode one does not entrain that at electrode two, but the actually leading site in between starts a spreading wave of acticity that entrains sites in its neighbourhood and finally also the local activity at the recording electrodes. If the site that is actually leading is located right between the two electrodes the “delay” observed between site 1 and 2 will be zero; if the leading site is nearer to recording site 1 this site will fire somewhat earlier than the other. The situation is obviously symmetric for the two electrodes and the maximum delay between the recording sites is the time the spreading activation needs from one recording site to the other. (This time is different from axonal propagation delays between sites one and two, because membrane charging times are involved. Those typically slow down the propagation, but not very much because cells along the line receive external input and are therefore near threshold.) Now, since the probability that one of the recording sites leads (or lags) relative to the other is a symmetric function, the average observable lag must be tightly centred around zero. The above argument implies two things: First, the picture of two interacting pools isolated around the recording electrodes is not appropriate; the many locally oscillating subnetworks between the recording sites are important as well, at least in the simulations and, we believe, also in biological experiments utilizing continuous bars (or gratings) as stimuli. As a consequence of this, the synchronization between the two hemispheres may be based on a different mechanism. Second, the explanation for phase zero correlations relies on an averaging argument. The longer the signals last that are taken for the computation of the correlations the better will be the sampling of the distribution of lags, and therefore the closer will the central peak be centred around its average value zero. In contrast, taking correlograms only over

Discussion section

436

short times, those may reveal shifts; in the worst case up to the delay between the recording sites. Furthermore the peak-width may systematically increase with distance, because the maximal possible delays increase when the electrodes are farther away from each other. Pulvermüller: I should defend the reverberation model, or “round-trip time” model: It posits that there are cell assemblies with strong corticocortical excitatory connections in which neuronal activity can circulate or reverberate thereby producing complex spatiotemporal patterns of activity (see Abeles et al., 1993). These assemblies form due to associative learning caused, for example, by correlations between stimuli. If associations are between stimuli of different modalities, assemblies storing these associations must be dispersed over the respective cortical areas (widely distributed transcortical assemblies). Presentation of a stimulus for which there exists a corresponding assembly will activate it, thus leading to reverberations of synchronous or near-synchronous waves of activity in the many reverberatory assembly-internal loops. It is believed that loop length determines reverberation frequencies of these (near-) synchronous waves. Dominant reverberation frequencies can be estimated from the velocities of action potentials in cortico-cortical axons which can, in turn, be estimated from axon diameters, and, importantly, from the frequency with which axons of certain diameters occur in the cortex (see tables in Pulvermüller, this volume). Wennekers and Palm believe that the reverberation model cannot explain variable frequencies induced by the same stimuli. This is incorrect. By assumption, there are many reverberatory loops in a transcortical assembly determining reverberations. Consider the case of an assembly distributed over various cortical areas. Cortical areas over which the assembly is distributed can be aroused to different degrees, that is, they can exhibit varying degrees of background or baseline activity. If the entire set of areas of an assembly is aroused, long distance reverberations are likely, but if only one area is aroused (for example, if attention focusses on one sensory channel) there will be a bias towards local reverberations. Local reverberations have higher frequencies than those in long-distance loops. Thus, frequecies may vary between experimental conditions and even trials. In conclusion, the “round trip time” model is consistent with variable oscillation frequency defined as the dominant reverberation frequency. A second objection Wennekers and Palm raise against the reverberation model relates to what they consider the main stream view of the generation of high-frequency cortical activity. They favour the view that oscillations are generated locally, in excitatoryinhibitory cortical loops. However, there are several competing views regarding this. High-frequency activity has been related to cortical cells which are spontaneously active at high frequencies (Gray and McCormick, 1996), subcortical generators (Llinás and Ribary, 1993) cortical reverberations between excitatory and inhibitory neurones (Schuster and Wagner, 1990), cortical reverberations in the cortical inhibitory network (Traub et al., 1996), and they can as well be related to Hebbian reverberations in purely excitatory corticocortical loops (as suggested by Hebb’s neuropsychological theory, see Pulvermüller, this volume). There is, as to my view, no general agreement. Rather, it appears that there are different types of high frequency activity related to different causes (Pulvermüller et al., 1997). There is no evidence against excitatory corticocortical reverberations. Most importantly, the postulate of neuronal assemblies implies the

Discussion section

437

possibility of reverberations, because strong bi-directional connections between many cortical neurones are assumed. If properties of corticocortical fibres are considered, it becomes likely that high-frequency reverberations dominate over slower ones (Pulvermüller, this volume). In essence, wherever the many different oscillations in cortex come from, there must be at least one type related to reverberations in excitatory loops of cortical cells. On the other hand, Wennekers and Palm’s “cortical resonance theory” provides another plausible mechanism for the generation of oscillations. Considering a cortex-like network in which cell assemblies have formed, it is clear that excitatory-inhibitory loops as well as purely excitatory loops can lead to reverberations and thus generate oscillations. Many experimental data show that coherent pictures, words or other stimuli lead to stronger high frequency activity (compared to physically similar but incoherent stimuli)(see Pulvermüller et al., 1997, for an overview). This can be explained based by both approaches. Both views would predict that cell assemblies ignite only if coherent stimuli are being processed. The “round trip time” model suggests that it is the reverberatory loops in the cell assembly themselves which generate strong high frequency responses. The cortical resonance theory proposes that mass action between cell assembly neurones and their many neighbouring inhibitory cells cause stronger HF activity after ignition of an assembly. Here, the enhancement of high frequency activity would go back to an activity increase due to the strong connections in the assembly. In conclusion, it appears premature at the moment to rule out one of the two possibilities. It may well turn out that both mechanisms play a role.

Discussion section

438

7. THE PARADOX OF THE ALPHA FREQUENCY BAND Editor: Whatever the role of conduction delays in determining frequency and phase relations of EEG rhythms, much of the above discussion was based on the fact that cortico-cortical connections were of prime importance. As far as the faster frequencies go, this may not be very contentious. After all, Nunez bases a lot of his theory of the EEG on cortico-cortical interaction. Slow rhythms such as theta clearly have their origin at a subcortical level. However, the alpha frequency band is a source of controversy. In past times, a long line of research has been developed suggesting that alpha oscillations are imposed on the cortex by thalamic pacemakers, rather than being derived from corticocortical interaction. While this is the traditional belief, there is now an increasing body of evidence for the latter. For instance, in Klimesch’s chapter on alpha activity, in a word-recognition task, individual alpha frequency (IAF) correlates with reaction time only in the left occipital cortex and Pz. The left location is to be expected for a linguistic task, but the fact that the correlation is in left occipital cortex, rather than in the language zones is more interesting. It seems to imply that alpha rhythms in the occipital region are generated by interaction with the more anterior regions, where word-recognition is more likely to occur initially. This implies that alpha in occipital regions is determined largely by corticocortical interaction. In addition, in the chapter by Rappelsberger, evidence is presented that auditory presentation of verbal stimuli is accompanied by widespread increases in alpha-1 coherence, with little change in amplitude. In contrast, for visual words, there is much less influence upon coherence, and a widespead fall in amplitude. The decrease in amplitude on visual stimulation is to be expected from much past work. However, the increase in coherence with auditory stimuli is more interesting, and suggests that corticocortical connections can be very important determinants of dynamics in the alpha frequency range. A “unified theory” of the EEG over frequencies from 8–80 Hz is attractive. However it is most difficult in so far as it applies to alpha. Klimesch’s chapter on alpha activity still seems to imply that alpha (at least “global alpha”) is an inactive state, and desynchronization of alpha is an activation process, occurring in association with information processing. However, in the faster frequency bands (gamma, beta) the evidence suggests that the active state is one of maximum rhythmicity, and the oscillations decline in amplitude when less information processing is going on. Can this inconsistency be resolved? Could it be, for instance, that the lowest-frequency, most widespread oscillations (“global alpha”) actually correspond to some very general sort of information processing which has not yet been properly defined because it is not very specific? Klimesch: The more we learn from microelectrode recordings the more paradoxical does alpha activity (as measured from macroelectrodes) appear. The alpha rhythm is (to my knowledge) the only brain oscillation which desynchronizes (becomes suppressed) during actual task performance. As Miller mentions, global alpha activity might corresponds “to some very general sort of information processing which has not yet been properly defined because it is not very specific”. I think that this conjecture is correct. In trying to provide

Discussion section

439

a possible explanation of the “paradoxical” behaviour of alpha activity, I proceed from the following two basic assumptions: (i) Complex cognitive tasks (such as thinking, memorizing, problem solving, etc.) require a complex and very selective activation pattern, (ii) Based on theoretical reasoning derived from memory networks (cf. Klimesch, 1996) and on empirical evidence described nicely in Eckhorn’s chapter, it may be assumed that selective activation (or encoding) generally means that different and highly selective cell assemblies oscillate with different frequencies. From (i) and (ii) it follows that a state of global synchronous rhythmic activity (in which very large cell assemblies, or even the entire cortex, are involved) destroys a selective pattern of “local oscillations with different frequencies”. The reason is that in such a state, no information can be transmitted that is encoded in terms of different frequencies. Thus, a global oscillation such as alpha may serve to suppress selective cortical activation and encoding processes. If this conclusion is true, the frequency of alpha should be in a range such that large cortical areas can be functionally integrated by feedback or reentrant loops, that are set into a state of oscillation, possibly via pace-makers in the thalamus. Because large networks (with long axons) are involved, the frequency of such an oscillation should be rather slow. Based on these considerations we can easily understand why the slower alpha frequencies in particular (which reflect the more global aspect of alpha activity) may play a crucial role in attention: if slow alpha oscillations are selectively induced in parts of the cortex, information processing in these regions is inhibited, and activity in other brain areas may become more focused. Evidence for the suggested interpretation comes from a variety of findings which are based on the ERD-method, and which indicate that not only alpha desynchronization but also alpha synchronization may occur “locally” and—in the latter case—over task irrelevant regions. As an example, in a task where subjects had to judge visuallypresented items and to give a motor response, we have found that, during visual stimulation, alpha desynchronizes over the occipital recording sites, but synchronizes over the motor cortex. During the motor response, the opposite result was obtained: regions over the motor cortex desynchronized and occipital areas synchronized. Further and more specific evidence comes from a recent study (Klimesch et al., 1999) in which a specially designed memory search paradigm was used to maximize episodic short-term memory (STM), and to minimize semantic long-term memory (LTM) demands. The results show that the upper alpha band synchronizes selectively in those conditions and time intervals where episodic STM demands are maximal. This finding of a selective alpha synchronization occuring only in the upper alpha band and during the highest task demands is surprising, because it is well known that alpha usually desynchronizes during mental activity. Because experiments from our laboratory indicate that desynchronization in the upper alpha band is related to semantic LTM processes, the present finding suggests that a selective synchronization in this frequency band reflects inhibition of semantic LTM. It can be assumed that, once the capacity limits of STM are reached or exceeded, processing resources are no longer distributed, and it can also be assumed that potentially interfering, task irrelevant, brain areas or processing systems are

Discussion section

440

inhibited. Type 1 and type 2 synchronization: an attempt to resolve the ‘alpha paradox: For a better understanding of the fact that oscillations within the gamma band synchronize during mental activity, whereas the opposite is true for alpha, it might be helpful to distinguish between two types of oscillations (cf. Klimesch, 1999). The synchronous activity of large cortical areas reflecting mental inactivity is termed type 1 synchronization, whereas the regular synchronous oscillatory discharge pattern of selected and comparatively small cortical areas is termed type 2 synchronization. Type 1 synchronization, reflecting the summed activity of a large number of cell assemblies, is a strong signal that can easily be recorded by macro-electrodes from the scalp. In contrast, the synchronous discharge of a small number of cell assemblies is a rather weak signal for the human scalp EEG. We assume that during alpha desynchronization different neuronal networks reflect type 2 synchronization, because they start to oscillate at different frequencies. If a large number of different networks shows type 2 synchronization, the large scale type 1 synchronization disappears. The behaviour of the alpha rhythm can thus be explained by type 1 synchronization during mental inactivity and type 2 synchronization during mental activity. Gamma oscillations, on the other hand, can be described as type 2 synchronization. Thus, the general conclusion is that regular type 2 synchronization is that oscillatory mode in all of the frequency bands that reflect actual information processing in the brain. Rappelsberger: Our experiments presented in this book demonstrate that changes in the alpha-1 band (8–10 Hz) are about the same, independent of the presentation of concrete and abstract words. This means that this band reflects processes common to both word classes. However, comparison of visual presentation and auditory presentation of either concrete words or abstract words revealed that the alpha-1 band is sensitive to the modality of stimulus presentation. During auditory presentation, amplitude decrease occurred only at T3 and T4 whereas during visual presentation amplitude decreased over wide areas. In the beta-1 band (13–18 Hz) clear differences between both word classes were found. Amplitudes decreased at some locations during presentation of concrete nouns, but not during abstract nouns. I cannot agree with the general statement that alpha is an inactive state. I think the assumption (Miller, above) that alpha corresponds to a general form of information processing which has not yet been properly defined is more realistic. Furthermore, I am not happy about the expression “global alpha” (Klimesch). This implies that there is only one single generator producing that activity. In contrast, numerous alpha generators should be considered localised at different brain areas, frequently showing differences of their resonant frequencies of more than 1 Hz. This can easily be verified with simple EEG recordings. However, all alpha generators may be considered as more or less synchronized. Editor: In the chapter of Başar, the larger post-stimulus EEG in the alpha band when the prestimulus alpha activity is low suggests that the brain is more sensitive when the EEG is “desynchronized”. However, this has two rather contradictory implications: that the state of preparedness for information processing involves alpha desynchronization, but the actual information processing involves synchronized alpha activity. This is related to the issue raised above, about whether it is alpha activation or alpha desynchronization

Discussion section

441

that is related to active information processing. Başar was invited to comment on this matter. Başar: In a number of papers, but particularly in the special issue on alpha activity (Başar, 1997) we have shown evidence that “alphas” may appear in different locations, related to multiple, or several different functions. They can be enhanced, phase-locked, induced, or desynchronized. There exists a selectively-distributed alpha system of the brain. This view, based on empirical findings, explains the existence of the contradictions. I also have the same hypothesis related to the multifunctional, selectively distributed gamma system. Your own work (Miller, 1991) supports the same view for the theta system. A number of groups have also demonstrated alpha phase-locking at the cellular level, upon visual or auditory stimulation.

8. MECHANISM OF EP/EEG INTERACTIONS Editor: Başar finds that, when prestimulus activity in theta or alpha band is low, there is an increase in the amplitude of the evoked response. He also produces evidence that this is in part due to better phase-locking of the EEG rhythm contributing to the evoked response. What could be the mechanism for these effects? One might explain the greater amplitude when prestimulus EEG activity is low, by supposing that inhibition in the cortex can only come about as a result of prior excitation, and that EEG activity involves excitation. Thus low EEG activity implies a low level of subsequent inhibition, and therefore larger EPs. Possibly the better phase-locking of the alpha and theta components of the EP when prestimulus activity is low could arise simply because the baseline from which the EP arises is less variable. Başar was invited to comment. Başar: The explanation is probably the fact that EEG and EPs share some common generators. If EEG generators are quiet, they will be activated upon stimulation. This is the most straightforward explanation. We cannot demonstrate the existence of an inhibitory mechanism. At least the generators are usually in an excited state prior to stimulation. I have introduced the expression “internal evoked potentials”, coming from hidden sources for high amplitude EEG states. This phenomenen is also described in a detailed manner in my monographs (Başar, 1998a,b).

9. THEORY OF CORTICO-HIPPOCAMPAL INTERPLAY Miller: As I understand it, the interactions revealed in Figure 5 of the chapter by Wennekers and Palm, between cortex and hippocampus are actually not dependent on delay-lines between cortex and hippocampus for which the “round-trip time” is similar to the theta period. If so it is a point of significant difference from the original formulation of cortico-hippocampal interplay (Miller, 1991). Indeed it may be more accurate than the original version, because the activated phase of a theta cycle (say, the peak third or quarter) may itself be as long as 50 msec, and therefore not precise enough to specify any

Discussion section

442

particular set of delay lines as being critical. Wennekers/Palm: The purpose of the simulations shown in Figure 5 in our article was to demonstrate how detailed information about some complex situation can be bound or integrated into a reduced contextual representation in some higher area, which could be the hip-pocampus. A second aspect addressed by the simulation concerns cortical consolidation processes which may be supported by such highly integrated chunks, inasmuch as they can serve as a permanent source of coherent input to the involved subassemblies that will, in general, reside in different and perhaps only weakly-connected cortical areas. Such coherent input increases the probability for synchronous firing events in and between those areas, and thereby the chance for the Hebbian strengthening of synapses between the sub-assemblies (memory consolidation; cf. Palm [1993]). Miller’s original proposal for the formation of cortico-hippocampal phase-locked loops assumes that the round-trip time through the loops is the period of the theta rhythm (Miller, 1991). Formation of loops is envisaged to be a process that selectively enhances connections with matching cycle-times, from a reservoir of cortico-hippocampal and hippocampo-cortical projections with a broad distribution of delays. Only loops “resonant” with the current theta frequency can be strengthened. Although we did not emphasize the fact, the simulations shown in Figure 5 contain delays between cortical and hippocampal areas, and back. If T is the period length of the theta cycle these delays are fixed at T/2. (The theta oscillation is introduced into the model by an external rhythmic input into the hippocampal cells, representing septal influences.) This means that we did not include broad delay distributions, but only the possibility for temporal resonance and phase-locking. The simulations in some sense reveal the formation of phase-locked loops intended by R.Miller, but there are also important differences, the main difference being that the cortical areas develop intrinsic rhythms by themselves, which are independent of each other, and of the septal theta pacemaker. The intrinsic rhythms are in the gamma-range and dominate the firing activity in the cortical areas. (Here, we should again stress the point that the time-scale in Figure 5 is somewhat compressed to enhance simulation speed: a single theta cycle lasts for 80 steps, whereas the intrinsic oscillations have a duration of roughly 35 steps. We identify these oscillations with gamma-oscillations because they are due to the same mechanisms, and have the same properties as the gamma-oscillations shown in Figures 2 and 3 of our article.) Thus, in our simulations, cortex and hippocampus oscillate at incommensurable frequencies. Cortical oscillations at the theta frequency with a fixed phase between different areas, as one would expect in Miller’s model (1991) are only briefly observable in Figure 5 during the initial learning phase, as long as synapses in the cortical areas and between cortex and hippocampus are still weak. Some hippocampal cells fire around steps 420, 500, 580, and so on (trace “Central” in Figure 5). After a transmission time of 40 steps—the delay between hippocampus and cortex—these firings evoke an excess of almost synchronous spikes clearly visible in the cortical areas A1 and A2. These spikes—again after 40 steps— support the firing of the next hippocampal cells. During this initial learning phase, synapses are weak and the gamma-rhythm is not visible. It appears soon after a sufficient amount of synapses have been enhanced in the cortical areas. Then it quickly dominates the cortical processes. During these gamma-dominated phases, the impact of the

Discussion section

443

hippocampal theta-rhythm is only occasionally visible in the form of spike-doublets or apparently irregular shifts in the cortical firing times. An interesting modification of the simulation would be to switch off the learning within the associative areas A1 and A2, while leaving that within the hippocampus as well as between A1/A2 and the hippocampus (and back) active. We have not performed such simulations, but it is very likely that, in this case, autonomous oscillations in the cortical areas do not occur, because those rely on sufficiently strong recurrent synapses within A1 and A2. Instead, a dynamical state as seen during the initial part of the simulations in Figure 5 will probably persist, where all areas reveal a theta rhythm, but the cortical areas are phase-shifted by 180 degrees. Whereas the cells firing during each theta cycle in the hippocampal area appear to be random in the initial phase, it seems likely that repeated activation leads to the strengthening of synapses between active cells in A1 and A2, and those cells excited most strongly in the hippocampal area. The latter cells are determined by randomly initialized projections from the cortical areas to the central one. In this way “phase-locked loops” more in accord with Miller’s original proposal may build up, although the scenario still does not take very broad delay distributions into account. Those distributions essentially pose additional temporal constraints on cells that become recruited into the hippocampal assemblies, beside the spatial constraints imposed by the initial random connectivity from cortex to hippocampus and back in our model. The main difference between the two scenarios described above is that, in the first case (the simulation in our Figure 5), the cortex behaves autonomously, whereas in the second the hippocampus can entrain cortical cells in the theta-range. More precisely we should say that entrainment is also possible in the first scenario (for example occasionally observable in the form of spike doublets), but by far the most spike activity within A1 and A2 is independent of the hippocampal theta activity, at least as soon as the cortical assemblies start to consolidate. Thus theta activity can hardly be seen within A1 and A2 (which seems to be in accordance with the observation that EEG activity in the theta range is rare in cortical areas). It cannot be ruled out that under some physiological conditions the hippocampus may strongly entrain the cortex in the theta range, such that loops according to the second scenario may build up. Nonetheless, we envisage the case of autonomous or almost autonomous cortical activity as the more generic one. This activity must not necessarily be dominated by oscillations in the gamma-range, as in the simulations; other frequency components may be equally important as discussed throughout the book. Furthermore the firing dynamics may also be asynchronous. In any case, if the cortical processes essentially follow their own laws, incommensurable with theta oscillations, it is hard to imagine how precise time-locked loops can come out of the interaction between cortex and hippocampus. What still seems possible are relatively precise forward projections from cortex to hippocampus (or the other way round) that can excite relatively specific target cells, perhaps including various delay lines. But how (and why) such pathways should become cyclically closed—i.e. forming phase-locked loops—is harder to imagine. Miller: These comments suggest a critical experiment. In the chapter by Villa, behavioural correlates of temporal patterns in multispike trains are described. These behavioural correlates are reminiscent of the behavioural correlates of theta activity (see

Discussion section

444

Miller, 1991). Therefore, if the original concept of phase-locked loops is correct, one might expect to see these temporal patterns related not just to theta activity in a general way, but to precislyrepeatable phases of the the concurrent theta cycle. If, on the other hand, cortico-hippocampal interplay comes about by a mechanism such as that analysed by Wennekers and Palm (where precise phase-locking is of little importance), such phase-locking of temporal patterns in multispike trains to the theta cycle should not be found.

10. LATERALITY QUESTIONS Editor: The data presented in Rappelsberger’s chapter suggest that phase delays might be greater in the left hemisphere than in the right. Thatcher et al. (1986) report the same thing. Could these two studies both be pointing towards axonal conduction being on average slower in the left than in the right hemisphere? Rappelsberger: In Figure 8 we report about a trend of smaller time delays in the right than in the left hemisphere for concrete word processing and visual presentation. However, abstract word processing revealed the opposite, i.e. lower time delays in the left than in the right hemisphere. Maybe there are differences between hemispheres in axonal conduction time within the hemisphere, dependent also on the task to be performed and the systems involved. However, data presented in our contribution are based on only three subjects. Computations were made using Schack’s autoregressive moving average (ARMA) technique trial-by-trial. This is very time consuming. We have been continuing with the analyses but unfortunately, until now, we are not able to present further material to support or to reject the above statements.

11. THE RELATION BETWEEN SYNCHRONY AT THE SINGLE UNIT LEVEL AND AT THE LEVEL OF EEG RHYTHMS Editor: To conclude the discussion, one may ask a general question: What is the relation between the precise synchrony at the neuronal level, such as demonstrated by Abeles, Villa, etc., and the synchrony underlying EEG waveforms. From the chapter by Wennekers and Palm, it is apparently only the fastest (gamma) rhythms where synchrony in the EEG could depend on synchrony of inputs arriving at single neurones. This is because the neuronal integration time (10 msec or less) can correspond to the most activated phase of an EEG rhythms, only for rhythms with a period in the gamma range (around 25 msec) Wennekers/Palm: In short, there is no particular relation. Abeles, Villa, etc. actually do not investigate synchronized firings of cells, they merely observe replicating spikepatterns (doublets, triplets, quadruplets, etc.) with precisely defined interspike times of up to several hundred milliseconds. From the occurrence of those patterns they only infer the existence of synchronously firing pools of cells that excite each other iteratively in linear order—the classical synfire chain model.

Discussion section

445

The interesting point with the observed spike-patterns is that they appear to exist in simultaneous recordings of several cells which, when inspected visually, look as if they were uncorrelated Poisson spike trains (though perhaps in part rate-modulated). The work of Abeles and others at first shows that, although the spike trains look random, they nonetheless can contain a rich (and, to some degree, deterministic) spatiotemporal structure in the form of precise replicating patterns. When discussed in relation to EEGsignals, several properties of such patterns should be highlighted: a single pattern appears only a few times even in data-sets of several minutes length. A single data set, however, may contain spikes belonging to many patterns. The time of occurences of patterns seems to be quite arbitrary, say for example, unrelated to phases of elevated firing rates. Nonetheless, patterns can be loosely coupled in time to behavioural events. In some recorded neurones all spikes within a window of several hundred milliseconds around some behavioural event belong to patterns, whereas no spike outside that window belongs to any pattern (Abeles et al., 1993b). The firing activity of such cells, however, when inspected visually, hardly reveals any recognizable differences inside and outside the respective window, and standard statistical tests (e.g. based on low order moments) would probably also fail to prove a difference. Thus the findings by Abeles and others represent a rather subtle kind of spatiotemporal correlations. The synfire chain model is a simple and not implausible model for those surprising patterns. EEG-rhythms reflect the orchestrated firing of millions of cells. The properties of synfirepatterns seem not to be consistent with such a widely-orchestrated mode of activity. Spike trains derived for pattern analysis do not show any clear temporal structure: they seem to be random. As mentioned, the occurrence of patterns seems not to be related to phases of elevated firing rates, and phases during which patterns occur are almost indistinguishable from phases where they do not appear (in fact, they are distinguishable only by an extensive search for patterns that involves the whole data set). For these reasons we do not see any obvious relation between synfire patterns and EEG signals. We should, however, mention results by Lestienne and Strehler (1987) which may be interpreted as giving some evidence for such a relation. Lestienne and Strehler searched for patterns in data from the visual system of monkeys. They found very high numbers of patterns, even though the bin size of their data was only 0.25 ms. In addition they plotted interval distributions between the replication of a pattern (Figure 4 in their paper). Those distributions show a slight excess of repetition times in the range of 50 to 70 ms. This may indicate that, at least in some of their data, oscillations in the beta range were present. In our opinion this does not imply a relation between synfire activity and the beta rhythm; it only shows that collective oscillations and individual spike patterns are no contradiction per se but may coexist in the same network. In the same sense one may interpret computer simulations by Abeles et al. (1993), which show oscillations in a network of two mutually connected synfire chain models. In simulations of a heteroassociative synfire-chain model (Wennekers, in preparation; see also Wennekers, 1998) we have induced oscillations by a global inhibitory activity control. Here, the patterned synfire activity, and the collective oscillation are two independent behavioural modes of the network which can be excited simultaneously.

Discussion section

446

REFERENCES Abeles, M. (1982b) Local cortical circuits. Studies in Brain Function No 6. Springer, Heidelberg. Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993) Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology, 70, 1629–1638. Ahlsen, G., Grant, K., and Lindstrom, S. (1982) Monosynaptic excitation of principal cells in the lateral geniculate nucleus by corticofugal fibers. Brain Research, 234, 454– 458. Alloway, K.D., Wallace, M.B., and Johnson, M.J. (1994) Cross-correlation analysis of cuneothalamic interactions in the rat somatosensory system: influence of receptive field topography and comparison with thalamocortical interactions. Journal of Neurophysiology, 72, 1949–1972. Amitai, Y., Connors, B.W., and Gil, Z. (1998) Properties of unitary thalamocortical and intracortical synaptic events of barrel cortex. Society for Neuroscience, Abstracts, 24, 248.1, 1998. Andreasen, N.C., Flaum, M., Swayze, V., II, O’Leary, D.S., Alliger, R., Cohen, G., Ehrhardt, J., and Yuh, W.T.C. (1993) Intelligence and brain structure in normal individuals. American Journal of Psychiatry, 150, 130–134. Anokhin, A., and Vogel, F. (1996) EEG alpha rhythm frequency and intelligence in normal adults. Intelligence, 23, 1–14. Artieda, J., Pastor, A., Lacruz, F., and Obeso, J.A. (1992) Temporal discrimination is abnormal in Parkinson’s disease. Brain, 115, 199–210. Earth, D.S., and MacDonald, K.D. (1996) Thalamic modulation of high-frequency oscillating potentials in auditory cortex. Nature, London, 383, 78–81. Başar, E., Hari, R., Lopes da Silva, F.H., and Schürmann, M. (1997) Brain alpha activity—new aspects and functional correlates. International Journal of Psychophysiology, 26, 1–482. Başar, E. (1998a) Brain function and oscillations. I.Brain oscillations: Principles and Approaches. Berlin, Heidelberg, Springer. Başar, E. (1998b) Brain function and oscillations. II. Integrative Brain Function. Neurophysiology and Cognitive Processes. Berlin, Heidelberg, Springer. Belair, J., Campbell, S.A., and van den Driesche, P. (1996) Frustration, stability, and delay-induced oscillations in a neural network model. SIAM Journal of Applied Mathematics, 56, 245–255. Bennett, M.V.L. (1971) Electric organs. In: J.S.Hoar and D.J.Randall (eds.), Fish Physiology. Volume 5, pp. 347–491, Academic Press, New York. Berent, S., Giorgani, B., and Lehtinen, S. (1988) Positron emission tomographic scan investigations of Huntington’s disease: Cerebral metabolic correlates of cognitive function. Annals of Neurology, 23, 541–546. Bigler, E.D., Johnson, S.C., Jackson, C, and Blatter, D.D. (1995) Aging, brain size, and IQ. Intelligence, 21, 109–119. Bullier, J., Munk, H.J., and Nowak, L.G. (1993) Corticocortical connections sustain interarea synchronization. Concepts in Neuroscience, 4, 159–174. Buzsaki, G., and Chrobak, J.J. (1995) Temporal structure in spatially organized neuronal ensembles: a role for interneuronal networks. Current Opinion in Neurobiology, 5,

Discussion section

447

504–510. Church, R.M. (1984) Properties of the internal clock. In: J.Gibbon and L.G.Allan (eds) Timing and Time Perception. New York: New York Academy of Sciences, pp. 566– 582. Cobb, S.R., Buhl, E.H., Halasy, K., Paulsen, O., and Somogyi, P. (1995) Synchronization of neuronal activity in hippocampus by individual GABAergic interneurons. Nature, London, 378, 75–78. Creutzfeldt, O.D., Watanabe, S., and Lux, H.D. (1966) Relation between EEGphenomena and potentials of single cells. Part I and II. Electroencephalography and Clinical Neurophysiology, 20, 1–37. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H.J. (1988) Coherent oscillations: A mechanism of feature linking in the visual cortex? Biological Cybernetics, 60, 121–130. Eckhorn, R., Reitboeck, H.J., Arndt, M., and Dicke, P. (1990) Feature linking among distributed assemblies: Simulations and results from cat visual cortex. Neural Computation, 2, 293–306. Fetterman, J.G., and Killeen, P.R. (1992) Time discrimination in Columbia livia and Homo sapiens. Journal of Experimental Psychology: Animal Behavior Processes, 18, 80–94. Gerstner, W., Ritz, R., and van Hemmen, J.L. (1993) A biologically motivated and analytically soluble model of collective oscillations in the cortex. I. Theory of weak locking. Biological Cybernetics, 68, 363–374. Gerstner, W. (1996) Rapid phase locking in systems of pulse-coupled oscillators with delays. Physical Review Letters, 76, 1755–1758. Ghose, G.M., and Freeman, R.D. (1992) Oscillatory discharge in the visual system: does it have a functional role? Journal of Neurophysiology, 68, 1558–1574. Gray, C.M. (1994) Synchronous oscillations in neural systems: Mechanisms and functions. Journal of Computational Neuroscience, 1, 11–38. Gray, C.M., and McCormick, D.A. (1996) Chattering Cells: Superficial Pyramidal Neurons Contributing to the Generation of Synchronous Oscillations in the Visual Cortex. Science, New York, 274, 109–113. Haier, R.J., Seigel, B.V. Jr., and Nuechterlein, K.H. (1988) Cortical glucose metabolic rate correlates of abstract reasoning and attention studied with positron emission tomography. Intelligence, 12, 199–217. Harvey, I., Persaud, R., Ron, M.A., Baker, G., and Murray, R.M. (1994) Volumetric MRI measurements in biopolars compared with schizophrenics and healthy controls. Psychological Medicine, 24, 689–699. Hill, S.L., and Villa, A.E.P. (1997) Dynamic transitions in global network activity influenced by the balance of excitation and inhibition. Network: Computation in Neural Systems, 8, 165–184. Hirsch, J.C., Fourment, A., and Marc, M.E. (1983) Sleep-related variations of membrane potential in the lateral geniculate body relay neurons of the cat. Brain Research, 259, 308–312. Ibatoullina, A.A., Vardaris, R.M., and Thompson, L. (1994) Genetic and environmental influences on the coherence of background and orienting response EEG in children. Intelligence, 19, 65–78. Johannesma, P., and Aertsen, A. (1982) Statistical and dimensional analysis of the neural representation of the acoustic biotope of the frog. Journal of Medical Systems, 6, 399– 421.

Discussion section

448

Johnson, M.J., and Alloway, K.D. (1994) Sensory modulation of synchronous thalamocortical interactions in the somatosensory system of the cat. Experimental Brain Research, 102, 181–197. Juergens, E., Eckhorn, R., Frien, A., and Woelbern, T. (1996) Restricted coupling range of fast oscillations in striate cortex of awake monkey. Elsner, N., Schnitzler, H.-U. (eds) Proceedings of the Goettingen Neurobiology Conference, 1996, Thieme, Stuttgart New York, p. 418. Klimesch, W. (1996) Memory processes, brain oscillations and EEG sychronization. International Journal of Psychophysiology, 24, 61–100. Klimesch, W. (1999) EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Research Reviews, in press. Klimesch, W., Doppelmayr, M., Schwaiger, J., Auinger, P., and Winkler, Th. (1999) Paradoxical alpha synchronization in a memory task. Cognitive Brain Research, in press. Johannesma, P., Aertsen, A.D., Cranen, B.v, and Van Erning, L. (1981) The phonochrome: a coherent spectrotemporal representation of sound. Hearing Research, 5, 123–145. König, P., Engel, A.K., and Singer, W. (1995) Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A., 92, 290–294. Kreiter, A.K., and Singer, W. (1996) Stimulus-Dependent Synchronization of Neuronal Responses in th Visual Cortex of the Awake Macaque Monkey. Journal of Neuroscience, 16, 2381–2396. Kruse, W., and Eckhorn, R. (1996) Inhibition of sustained gamma oscillations (35–80 Hz) by fast transient responses in cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A., 93, 6112–6117. Kuramoto, Y. (1991) Collective Synchronization of pulse-coupled oscillators and excitable units. Physica D., 50, 15–30. Lee, M.-G., Chrobak, J.J., Sik, A. Wiley, R.G., and Buzsaki, G. (1994) Hippocampal theta activity following selective lesion of the septal cholinergic system. Neuroscience, 62, 1033–1044. Lestienne, R., and Strehler, B.L. (1987) Time structure and stimulus dependence of precisely replicating patterns presents in monkey cortical neuronal spike trains. Brain Research, 437, 214–238. Llinás, R.R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in the mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10–to 50-Hz frequency range. Proceedings of the National Academy of Sciences, U.S.A., 88, 897–901. Llinás, R.R., and Ribary, U. (1993) Coherent 40–Hz oscillation characterizes dream state in humans. Proceedings of the National Academy of Sciences, U.S.A., 90, 2078–2081. Lopes da Silva, P., Hoeks, A., Smits, H., and Zetterberg, L.H. (1974) Model of Brain Rhythmic Activity: The Alpha-Rhythm of the Thalamus. Kybernetik, 15, 27–37. Lopes da Silva, F.H., van Rotterdam, V., Barrts, P., van Heusden, E., and Burr, W. (1976) Model of Neural Populations: The Basic Mechanisms of Rhythmicity. In: M.A.Coner and D.F.Swaab, (eds) Progress in Brain Research, Vol. 45: Perspectives in Brain Research, Elsevier, Amsterdam, pp. 281–308. Lopez da Silva, F. (1991) Neural mechanisms underlying brain waves: from neural membranes to networks. Electroencephalography and Clinical Neurophysiology, 79, 81–93. Maricq, A.V., and Church, R.M. (1983) The differential effects of haloperidol and

Discussion section

449

methamphetamine on time estimation in the rat. Psychopharmacology, 79, 10–15. Matsumura, M., Chen, D., Sawaguchi, T., Kubota, K., and Fetz, E.E. (1996) Synaptic interactions between primate precentral cortex neurons revealed by spike-triggered averaging of intracellular membrane potentials in vivo. Journal of Neuroscience, 16, 7757–67. Meck, W.H. (1986) Affinity for the dopamine D2 receptor predicts neuroleptic potency in decreasing the speed of an internal clock. Pharmacology, Biochemistry and Behavior, 25, 1185–1189. Meck, W.H. (1996) Neuropharmacology of timing and time perception. Cognitive Brain Research, 3, 227–242. Michon, J.A. (1985) The compleat time experiencer. In: J.A.Michon and J.L.Jackson (eds) Time, Mind, and Behavior. Berlin: Springer, pp. 21–52. Miller, R. (1991) Corticohippocampal interplay and the representation of contexts in the brain. Berlin, Heidelberg, Springer Verlag. Miller, R. (1996) Cortico-thalamic interplay and the security of operation of neural assemblies and temporal chains in the cerebral cortex. Biological Cybernetics, 75, 263–275. Mitrani, L., Shekerdjiiski, S., Gourevitch, A., and Yanev, S. (1977) Identification of short time intervals under LSD25 and mescaline. Activitas Nervosa Superior, 19, 103–104. Mitzdorf, U. (1987) Properties of the evoked potential generators: current source-density analysis of visually evoked potentials in the cat cortex. International Journal of Neuroscience, 33, 33–59. Molotchnikoff, S., and Shumkhina, S. (1996) The Lateral Posterior-Pulvinar Complex Modulation of Stimulus-dependent Oscillations in the Cat Visual Cortex. Vision Research, 36, 2037–2046. Munk, M.H.J., Nowak, L.G., Nelson, J.L, and Bullier, J. (1995) The structural basis of cortical synchronization. II. Localization of the common input. Journal of Neurophysiology, 74, 2401–2414. Murthy, V.N., and Fetz, E.E. (1994) Effects of input synchrony on the firing rate of a 3conductance cortical neuron model. Neural Computation, 6, 1111–1126. Musicant, A.D., and Butler, R.A. (1980) Monoaural localization: an analysis of practice effects. Perception and Psychophysics, 28, 236–240. Neuenschwander, S., and Singer, W. (1996) Long-range synchronization of oscillatory light reponses in the cat retina and lateral geniculate nucleus. Nature, London, 379, 728–733. Nunez, P.L. (1995) Neocortical Dynamics and Human EEG Rhythms. New York: Oxford University Press. Nunez, P.L., Srinivasan, R., Wijesinghe, R.S., Westdorp, A.F., Tucker, D.M., Silberstein, R.B., and Cadusch, P.J. (1997) EEG Coherency I: Statistics, Reference Electrode, Volume Conduction, Laplacians, Cortical Imaging, and Interpretation at Multiple Scales. Electroencephalography and Clinical Neurophysiology, 103, 499–515. Parks, R.W., Crockett, D.J., and Tuokko, H. (1988) Neuropsychological “systems efficiency” and positron emission tomography. Journal of Neuropsychiatry, 1, 269– 282. Pastor, M.A., and Artieda, J. (1996) Involvement of the basal ganglia in timing perceptual and motor tasks. In M.A.Pastor and J.Artieda (eds.). Time, internal clocks and movement Amsterdam: Elsevier. (pp. 235–255). Petsche, H., Pockberger, H., and Rappelsberger, P. (1984) On the search for the sources of the EEG. Neuroscience, 11, 1–27.

Discussion section

450

Petsche, H., Pockberger, H., and Rappelsberger, P. (1987) Mechanisms leading to the propagation of self-sustained seizure activities. In: H.G.Wieser, E.-J. Speckmann and J.Engel (eds.): The Epileptic Focus. John Libbey and Company Ltd., pp. 59–81. Pulvermüller, F., Birbaumer, N., Lutzenberger, W., and Mohr, B. (1997) High-frequency brain activity: its possible role in attention, perception and language processing. Progress in Neurobiology, 52, 427–445. Rammsayer, T. (1989) Is there a common dopaminergic basis of time perception and reaction time? Neuropsychobiology, 21, 37–42. Rammsayer, T. (1992b) Effects of benzodiazepine-induced sedation on temporal processing. Human Psychopharmacology, 7, 311–318. Rammsayer, T.H. (1993) On dopaminergic modulation of temporal information processing. Biological Psychology, 36, 209–222. Rammsayer, T. (1994a) A cognitive-neuroscience approach for elucidation of mechanisms underlying temporal information processing. International Journal of Neuroscience, 77, 61–76. Rammsayer, T.H. (1994b) Temporal information processing and memory. In: L.M.Ward, Fechner Day 94. Proceedings of the Tenth Annual Meeting of the International Society for Psychophysics. Vancouver, Canada : The International Society for Psychophysics, pp. 48–53. Rammsayer, T. (1994c) Effects of practice and signal energy on duration discrimination of brief auditory intervals. Perception and Psychophysics, 55, 454–464. Rammsayer, T.H. (1994d) Temporal information processing in blind and sighted subjects. In: L.M.Ward (Ed.), Fechner Day 94. Proceedings of the Tenth Annual Meeting of the International Society for Psychophysics, pp. 179–184. Vancouver, Canada: The International Society for Psychophysics. Rammsayer, T.H. (1997b) Are there dissociable roles of the mesostriatal and mesolimbocortical dopamine systems on temporal information processing in humans? Neuropsychobiology, 35, 36–45. Rammsayer, T.H., and Lima, S.D. (1991) Duration discrimination of filled and empty auditory intervals: Cognitive and perceptual factors. Perception and Psychophysics, 50, 565–574. Rammsayer, T.H., and Classen, W. (1997) Impaired temporal discrimination in Parkinson’s disease: temporal processing of brief durations as an indicator of degeneration of dopaminergic neurons in the basal ganglia. International Journal of Neuroscience, 91, 45–55. Rice, C.E. (1970) Early blindness, early experience and perceptual enhancement. American Foundation for the Blind Research Bulletin, 22, 565–574. Roelfsma, P.R., Engel, A.K., König, P., and Singer, W. (1997) Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature, London, 385, 157–161. Robson, J.A. (1984) Reconstructions of corticogeniculate axons in the cat. Journal of Comparative Neurology, 225, 193–200. Saam, M., and Eckhorn, R. (1998) Width of association fields in visual cortex is influenced during Hebbian learning by signal correlation among inputs and lateral spike velocity: a network model with spiking neurons. In: H.Lachnit, A.Jacobs and F.Roesler (eds), Experimented Psychologie. Pabst Science Publishers, Berlin, p. 298. Saam, M., and Eckhorn, R. (1998) Width of association field in topographic neural map depends on lateral spike velocity and temporal dispersion of correlated input: A learning model with spiking neurons. In N.Elsnerand R.Wehner, (eds) Proceedings of

Discussion section

451

the 26th Göttingen Neurobiology Conference, volume 2, p. 767. Georg Thieme Verlag. Salin, P.A., and Bullier, J. (1995) Corticocortical Connections and the Visual System: Structure and Function. Physiological Reviews, 75, 107–154. Sanes, J.N., and Donoghue, J.P. (1993) Oscillations in local field-potentials of the primate motor cortex during voluntary movement. Proceedings of the National Academy of Sciences, U.S.A., 90, 4470–4474. Schanze, T., and Eckhorn, R. (1997) Phase correlation among rhythms present at different frequencies: Spectral methods, application to microelectrode recordings from visual cortex and functional implications. International Journal of Psychophysiology, 26, 171–189. Schuster, H.G., and Wagner, P. (1990) A model for neuronal oscillations in the visual cortex: 1. Mean-field theory and derivation of the phase equations. Biological Cybernetics, 64, 77–82. Shulz, D.E., Cohen, S., Haidarliu, S., and Ahissar, E. (1997) Differential effects of acetylcholine on neuronal activity and interactions in the auditory cortex of the guineapig. European Journal of Neuroscience, 9, 396–409. Shannon, C.E. (1948) A mathematical theory of communication. Bell Systems Technical Journal, 27, 379–423, 623–653. Sillito, A.M., Jones, H.E., Gerstein, G.L., and West, D.C. (1994) Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex, Nature, London, 369, 479–482. Starlinger, J., and Niemeyer, W. (1981) Do the blind hear better? Investigations of auditory processing in congenital or early acquired blindness: I. Peripheral functions. Audiology, 20, 503–509. Steriade, M., Dossi, C., Paré, D., and Oakson, G. (1991) Fast oscillations (20–40 Hz) in thalamo-cortical systems and their potentiation by mesopontine cholinergic nuclei in the cat. Proceedings of the National Academy of Sciences, U.S.A., 88, 4396–4400. Steriade, M., McCormick, D.A., and Sejnowski, T.J. (1993) Thalamo-cortical Oscillations in the Sleeping and Aroused Brain. Science, New York, 262, 679–685. Steriade, M., Amzica, F., and Contreras, D. (1996) Synchronization of fast (30–40 Hz) spontaneous cortical rhythms during brain activation. Journal of Neuroscience, 16, 392–417. Stern, E.A., Kincaid, A.E., and Wilson, C.J. (1997) Spontaneous subthreshold membrane potential fluctuations and action potential variability of rat corticostriatal and striatal neurons in vivo. Journal of Neurophysiology, 77, 1697–1715. Swadlow, H.A. (1988) Efferent neurons and suspected interneurons in binocular visual cortex of the awake rabbit: receptive fields and binocular properties. Journal of Neurophysiology, 59, 1162–1187. Swadlow, H.A. (1990) Efferent neurons and suspected interneurons in S-1 forelimb representation of the awake rabbit: receptive fields and axonal properties. Journal of Neurophysiology, 63, 1477–1498. Swadlow, H.A., Rosene, D.L., and Waxman, S.G. (1978) Characteristics of interhemispheric impulse conduction between prelunate gyri of the rhesus monkey. Experimental Brain Research, 33, 455–467, 1978. Swadlow, H.A., Geschwind, N., and Waxman, S.G. (1979) Commissural transmission in humans. Science, New York, 204, 530–531. Swadlow, H.A., and Hicks, T.P. (1997) Subthreshold receptive fields and baseline excitability of “silent” SI callosal neurons in awake rabbits: contributions of AMPA/kainate and NMDA receptors. Experimental Brain Research, 115, 403–409.

Discussion section

452

Tallon-Baudry, C., Bertrand, O., Wienbruch, C., Ross, B., and Pantev, C. (1997) Combined EEG and MEG recordings of visual 40 Hz responses to illusory triangles in human. Neuroreport, 8, 1103–1107. Thatcher, R.W., Krause, P.J., and Hrybyk, M. (1986) Cortico-cortical associations and EEG coherence: a two-compartmental model. Electroencephalography and Clinical Neurophysiology, 64, 123–143. Traub, R.D., Whittington, M.A., Stanford, I.M., and Jeffreys, J.G.R. (1996) A mechanism for generation of long-range synchronous fast oscillations in the cortex. Nature, London, 383, 621–624. Tsumoto, T, Creutzfeldt, O.D., and Legendy, C.R. (1978) Functional organization of the corticofugal system from visual cortex to lateral geniculate nucleus in the cat (with an appendix on geniculo-cortical monosynaptic connections). Experimental Brain Research, 32, 345–364. Villa, A.E., Bajo Lorenzana, V.M., and Vantini, G. (1996) Nerve growth factor modulates information processing in the auditory thalamus. Brain Research Bulletin, 39, 139–147. Walmsley, B., Alvarez, F.J., and Fyffe, R.E. (1988) Diversity of structure and function at mammalian central synapses. Trends in Neuroscience, 21, 81–88, 1998. Wennekers, T., Erb, M., Eckhorn, R., and Palm, G. (1994) Dependence of oscillation frequency on stimulus parameters in primary visual cortex of monkeys: simulations and experimental results. European Journal of Neuroscience, Supplement , 7, 66.03. Wennekers, T., and Palm, G. (1996) Controlling the Speed of Synfire Chains. In: C.von der Malsburg et al., (ed.), Proceedings of the International Conference on Artificial Neural Networks ICANN96, pp. 451–456. Berlin, Springer. Wennekers, T., and Palm, G. (1997) On the relation between neural modelling and experimental neuroscience. Theory in Biosciences, 116, 273–289. Wennekers, T., and Palm, G. (1999) How Imprecise is Neuronal Synchronization? Neurocomputing (in press). Whittington, M.A., Traub, R.D., and Jeffreys, J.G.R. (1995) Synchronized oscillations in interneuron networks driven by metabotropic receptor activation. Nature, London, 373, 612–615. Wickett, J.C., Vernon, P.A., and Lee, D.H. (1994) In vivo brain size, head perimeter, and intelligence in a sample of healthy adult females. Personality and Individual Differences, 16, 831–838. Willerman, L., Schultz, R., Rutledge, J.N., and Bigler, E.D. (1991) In vivo brain size and intelligence. Intelligence, 15, 223–228.

POSTLUDE THE NEUROANATOMY OF TIME Valentino Braitenberg Max Planck Institute for Biological Cybernetics, Spemann Strasse 38, D7400, Tübingen, Germany.

1. THE STRUCTURE OF THE BRAIN AS FROZEN HISTORY The neuroanatomist sinking his knife in the rosy substance cuts through the brain in more than one way. If he is prone to meditation, as many of his craft are, he is well aware that the thing on his tray is but a momentary cross-section through a temporal process, the material leftover from a number of intersecting histories, the focal point of a complexity which, to be fully understood, ought to be fathomed in its near-infinite temporal as well as spatial extension. The structure of the brain cannot be explained from the properties and dynamics of its components. Explanation of every structural detail requires a review of at least three kinds of development, each involving conditions that lie outside, sometimes far outside the object which we observe here and now. First, Darwinian evolution which in the course of a few billion years, adding an average of one or two neurones a year to the primordial unicellular organism, gave us our recent brain with its few billion neurones. Of course, it is not only size and number of neurones that evolutionary pressure enforced, but more important, structure. Fibre bundles, patterns of connections within the grey matter, shapes of dendritic and axonal arborizations are to a large extent genetically determined. They impose constraints on the functional states of the brain; and, if evolution was right, these constraints mimick those imposed by the physical nature of things on the events that are likely to occur. Thus a large part of the structure of the brain is knowledge acquired in the arduous process of surviving among the myriads of would-be ancestors who did not survive, encoded in a form that remains generally valid whatever individual experiences are made later. Second, there is learning in the course of a lifetime. Individual experience must, of necessity, involve changes in the synaptic structure of the brain, for if it did not, how could we possibly explain the rerouting of signals that makes behavioural responses after learning different from the ones before? The anatomical changes which make the experienced brain different from the naive one are surprisingly elusive, and the debate about their localisation is very active today. Most likely, the traces of individual experience are perfectly visible in electron micrographs and perhaps also at the level of ordinary light microscopy. We do not recognize them because the representation of a particular thing or learned event is so diffuse that it is practically impossible to isolate its disparate anatomical traces from those of the other representations with which it

Postlude

455

interlaces. Third, the miraculous unfolding of multicellular organs from their unicellular origin repeats itself in an abbreviated version in every individual. The average rate at which neurones are built into the human brain during the 300 days of its gestation is about one or two thousand neurones per second. In the process of development the feed-forward messages from the genes interact with the feed-back influences that developing cells exert on each other. These complicated processes not only mediate the information that Darwinian evolution meant to incorporate into the brain to make it more efficient, but also follow their own rules. Quite possibly some details of brain anatomy cannot be explained by invoking functional optimization, but must be understood as leftovers from the construction site. Similarly, pathology may strike the brain and leave traces in its structure. This is not learning, for it does not improve function, nor is it what evolution had forseen. Rather, it is a later chapter in a story which begins with embryology and ends with death, a story which evolves quite independently from the information-handling feats of the brain, and reflects the wider biological context in which brains are immersed.

2. THE REPRESENTATION OF TIME IN THE BRAIN The most powerful metaphor in brain theory is not the calculating machine, but the simulation of reality on something resembling a computer or, as some put it, the brain as a model of the world. This brings order into our thoughts about brain structure and function, but also raises some weighty questions of a philosophical, informationtheoretical and technical nature. To what extent are the things and events of the world imaged in discrete ensembles of active neurones, and what are the rules that govern the coding? More fundamental still, how are relevant things selected out of the continuum of sensory stimuli reaching the organism, or: what is figure and what is background, what is signal and what is noise? Are the spatial dimensions of the environment mapped directly on some spatial dimensions in the geometry of the brain? Is it objective space that is mapped there, or an egocentric system of coordinates anchored on the axes of the animal’s body? There are partial answers to these questions well grounded in physiology, but the question most relevant in the present context is still quite unsettling: How is time represented in the brain? No doubt we have a “sense of time”. We are usually able to say which of two events happened before the other, at time scales involving years, days or minutes, with a precision that decreases with the time elapsed. This works even with very small intervals of time, down to events separated by a few milliseconds or hundredths of seconds where, both in acoustic and in visual perception, confusion may arise. We are also able to produce motor activity with astonishing temporal precision: catching a ball in flight, playing a piece of music with a temporal scatter not exceeding a few tens of seconds in the course of minutes, pronouncing different syllables distinguished by differences of no more than 20 milliseconds in the “voice onset time” of certain phonemes. This wide range of temporal order cannot be the responsibility of a single clock. It is

Postlude

456

especially in language that different timing devices become apparent, distinguished not only by the range of their operation but also by the different rules they obey. 2.1. Syllables These are episodes of about 250 ms which constitute the fundamental segmentation of speech in all languages. They cannot be interrupted, nor can they be stretched or speeded up beyond certain limits (except in the very artificial situation of singing). Very slow speech is produced mainly by introducing pauses between syllables, much less by lengthening the duration of the individual syllable. The concept of a syllable can easily be extended to motor control of the extremities. Most so-called fast movements have a duration of 200 to 300 ms. The sequence of activations of the agonist and antagonist muscles involved in an elementary movement (e.g. extension or flexion of the forearm) follows a strict temporal order and is preprogrammed, not dependent on peripheral feedback, since the sequence of muscular activations takes its course even when the movement is mechanically blocked soon after its initiation (Wadman et al., 1979, 1980). At the cortical level, the phenomenon which corresponds to the syllable is the synfire chain, theoretically postulated and experimentally documented by Abeles (Abeles 1982, 1991). Spike activity in sets of cortical neurones reveals patterns of sequential activation which keep strict timing over periods of several hundred milliseconds, exactly what one would expect as the material counterpart of the syllable. Like syllables, these chains of active sets of neurones apparently follow their intrinsic dynamics undisturbed by external influences. Quite possibly synfire chains are also involved in a different form of temporal order, the periodic repetition of movements in identical form, one immediately following the other. Repetitive limb movements are one example, repetition of syllables in emotion laden utterances another. In the performance of music, repetitive patterns abound, and obviously also in movements such as walking, scratching, etc. If we allow the neuronal patterns which make up the synfire chain to be arranged in circular order, periodic patterns are easily explained. Besides the cerebral cortex, the cerebellum has also been suspected of being a depository for patterns of temporal order in the millisecond range (Braitenberg et al., 1997). The interplay of cerebrum and cerebellum in motor control is all but clear, but there is evidence that in cerebellar patients both the distinction of syllables involving strict temporal order and the production of such syllables is disturbed (Lisker and Abramson, 1964, 1967; Ackermann et al., 1997) 2.2. Sequences of Syllables Some syllables are carriers of meaning, others become such only when they combine to form polysyllabic words. The sequence of syllables in a word is kept together by different ties from those which bind phonemes within the syllable. The utterance of a polysyllabic

Postlude

457

word can easily be interrupted at the boundaries between syllables, and in some cases (“compound words”) the syllables maintain a certain degree of independence of meaning. This leads to the kind of temporal order known as grammar. Quite likely, in the early stages of language acquisition, correct sequences of words are learned wholesale by the infant, much as compound words are. From then on different rules obtain. Words can be substituted for the original ones, and entire phrases can be inserted at some crucial point in the sequence (e.g. relative clauses). This indicates that the order of words in a sentence is layed down in a flexible way, in the form of sequences that can be interrupted in their course and then resumed at will. The preprogrammed feed-forward synfire chains cannot do this. The most likely mechanism is that of chains of Hebbian cell-assemblies. These are groups of neurones, with strong reciprocal excitatory connections which keep them active once they have been excited. One cell assembly may excite the next to which it is connected, and thus again we have the possibility of activity travelling along fixed routes in sequences of cell assemblies. Different from synfire chains, such a sequence can be interrupted without losing its activity, which is stored in the last active assembly by virtue of its internal excitatory connections. If we want to apply this mechanism to language, we must postulate additional mechanisms which can activate the cell assemblies of a chain one after another at the correct pace, and stop or resume the sequence when some higher order requires it. 2.3. Inverted Temporal Order A peculiar phenomenon of grammar is the inverted order of elements in part of a sequence, with respect to associated elements in another part of it. Embedded (nested) phrases are closed in the inverse order in which they are opened. This poses special requirements on the neuronal mechanisms of language (“push down memory”), which are, however, easily met by invoking the well known phenomenon of adaptation as applied to cell assemblies. If active assemblies are the markers for embedding sentence, first order embedded phrase, second order embedded phrase, etc., and if the activity in the assemblies slowly decreases in time by adaptation, we only have to suppose that the phrases are closed in the order of the strength of activity in the corresponding markers: the assembly last excited will be the strongest and will be closed first, the next to last second, etc. (Braitenberg and Pulvermüller, 1992) 2.4. Present, Past and Future Both the past and the future are involved in the momentary experience which we call the present. The past, since nothing is felt or decided without implicit or explicit reference to previous experience. The future in the form of the goals we intend to achieve, or the perils we want to avoid, or simply as an extrapolation of movements perceived or intended. Apart from its influence on our experience of the present, the past can also be actively

Postlude

458

recalled. It is possible not only to evoke pictures from the past, but also to traverse stretches of past time by letting pictures evolve in sequences reflecting more or less faithfully the sequence of their actual occurrence. (It is much more difficult and less natural to traverse past memories in the opposite direction, a feat that is probably achieved by devious routes and not without the aid of internal language.) Thus, the model of the world in the brain relies on a dynamic as well as a structural representation. This is obvious at a microscopic scale: In visual perception it is not still pictures that are primarily perceived (and remembered), but pictures composed of figural elements that may be moving. Movement detectors are found at a very basic level in perception, along with detectors of colour, contrast, edges and the like. Similarly, in acoustic perception we do not perceive spectrograms, but something like sonograms which have a temporal besides a frequency dimension: upsweeps, downsweeps, bangs, noises. This is not only inference from psychophysics: neurones responsible for movement detection have been observed both in the visual and in the auditory system (Hubel and Wiesel, 1977; Newman 1978; Newman and Symmes, 1979; Whitfield and Evans, 1965) However, this small-scale dynamics does not explain temporal coherence of memories at a larger scale. If memory traces pale with time, there may be a rule by which the pale and indistinct memory leads to ever more distinct memories, and this may be sufficient to reconstruct past history. More likely, however, the functional states of the brain which represent things and events (probably cell assemblies), as well as their internal connections which give them their individuality, are endowed with connections leading to other states (other cell assemblies) that have often followed them. So the model, even without further input, can run through sequences that have occurred before, or are likely to occur. This makes a predictor out of the brain. Starting from the present activity, on the basis of dynamic rules acquired through past experience, the future can be predicted. In real time, this makes it possible for the brain to compare the predicted future to that which actually occurs, and thus distinguish the likely from the unlikely, the expectation from the surprise. Even more important would be the action of the predictor if it could speed up its prediction with respect to objective time. The action could then be planned before it is too late, the flying rock avoided before it hits. Unfortunately, there is no experimental evidence for either real time or speeded up prediction, except that in traffic most of the time we do avoid collisions. (for Almut Schüz on her birthday)

REFERENCES Abeles, M. (1982) Local cortical circuits. Springer, Berlin, Heidelberg, New York. Abeles, M. (1991) Corticonics: Neural circuits of the cerebral cortex. Cambridge University Press. Ackermann, H., Gräber, S., Hertrich, I., and Daum, I. (1997) Categorical speech perception in cerebellar disorders. Brain and Language, 60, 323–331. Braitenberg, V., and Pulvermüller, F. (1992) Entwurf einer neurologischen Theorie der Sprache. Naturwissenschaften, 79, 103–117.

Postlude

459

Braitenberg, V., Heck, D., and Sultan, F. (1997) The detection and generation of sequences as a key to cerebellar function: Experiments and theory. Behavioural and Brain Sciences, 20, 229–277. Hubel, D.H., and Wiesel, T.N. (1997) Functional architecture of macaque monkey visual cortex. Proceedings of the Royal Society of London, Series B, 198, 1–59. Lisker, L., and Abramson, A.S. (1964) A cross-language study of voicing in initial stops: Acoustical measurements. Word, 20, 384–422. Lisker, L., and Abramson, A.S. (1967) Some effects of context on voice onset time in English stops. Language and Speech, 10, 1–28. Newman, J.D. (1978) Perception of sounds used in species-specific communication: The auditory cortex and beyond. Journal of Medical Primatology, 7, 98–105. Newman, J.D., and Symmes, D. (1979) Feature detection by single units in squirrel monkey auditory cortex. Experimental Brain Research, Supplement, 2, 140–145. Wadman, W.J., Denier van der Gon, J.J., Geuze, R.H., and Mol, C.R. (1979) Control of fast goal-directed arm movements. Journal of Human Movement Studies, 5, 3–17. Wadman, W.J., Denier van der Gon, J.J., and Derksen, R.J.A. (1980) Muscle activation patterns for fast goal-directed arm movements. Journal of Human Movement Studies, 6, 19–37. Whitfield, I.C., and Evans, E.F. (1965) Responses of auditory cortical neurones to stimuli of changing frequency, Journal of Neurophysiology, 28, 655–672.

Author Index

Abarbanel, H.D.I. 404 Abbott, L.F. 9, 316 Abdullaev, Y.G. 365, 373 Abel, S.M. 194 Abeles, M. 3, 7, 11, 13, 14, 16, 28, 54, 69, 77, 83, 93, 97, 119, 172, 203, 289, 312, 313, 317, 322, 418, 451, 452, 464 Aboitiz, F. 165, 291, 292F Abramson, A.S. 464 Abt, K. 366 Ackermann, H. 464 Addison, P.S. 40 Adey, W.R. 73, 85 Adrian, E.D. 2, 72, 89, 91, 98, 131, 214 Aertsen, A. 11, 68, 75, 79, 84, 86, 92, 93, 97, 100, 203, 313, 419 Agmon-Snir, H. 202, 203, 214 Ahissar, E. 80 Ahissar, M. 77, 93, 139 Ahlsen, G. 424 Aiple, F. 86, 171 Albus, S. 156 Allan, L.G. 187, 190, 192, 316 Allman, J. 103 Alloway, K.D. 418, 422 Alonso, J.-M. 74, 124, 171, 174 Amit, D.J. 44, 45, 49, 55, 316 Amitai, Y. 222, 421 Amizca, F. 92 Andreasen, N.C. 431 Andronov, A. 50 Anokhin, A. 432 Antal, M. 283 Aranibar, A. 251, 275 Arduini, A. 275 Arieli, A. 329 Armington, J.C. 329 Armstrong-James, M. 127 Arnoldi, H.M. 97 Arnolds, D.E.A.T. 276 Artieda, J. 194, 425 Atick, J.J. 122

Index

461

Babloyantz, A. 41 Baddeley, A. 276 Bair, W. 5, 398 Baker, J. 168 Baker, S.N. 90, 101, 214 Bal, T. 33, 91 Baranyi, A. 80, 97, 169 Barlow, H.B. 117, 132 Barr, H.L. 193 Barrie, J.M. 89 Barth, D.S. 89, 214, 434 Başar, E. 242, 261, 329, 331F, 332, 333F, 335, 336F, 337F, 338, 341F, 342, 343, 344, 345, 346, 357F, 349, 366, 372, 448 Başar-Eroğlu, C. 448, 345, 346, 363 Beaumont, J.G. 356 Bechtereva, N.P. 365, 373 Beck, P.D. 128 Beidenbach, M.A. 168 Bendat, J.S. 168, 377 Bennett, M.V.L. 156, 166, 170, 420 Berent, S. 432 Berger, H. 246 Bergström, R.M. 330 Berlucchi, G. 176 Bernander, O. 203 Berry, M. 165 Bester, H. 157 Bialek, W. 3, 9 Bibbig, A. 304N, 316, 318, 321 Bienenstock, E. 313, 315, 317, 321 Bigler, E.D. 248, 431 Billings, R.J. 431 Bindra, D. 186 Birch, B. 193 Bishop, G.H. 155 Bishop, P.O. 157, 158 Bland, B.H. 270 Bleasdale, F.A. 365 Blinowska, K.J. 357 Block, R.A. 187, 193 Boltz, M.G. 187 Bolz, J. 86 Bonhoeffer, T. 80 Borisyuk, G.N. 398, 399, 401, 405, 410, 411 Borisyuk, R.M. 305, 410, 411 Botte, M.-C. 192 Boudreau, J.P. 192

Index

462

Bouyer, J.J. 84, 90, 96 Boven, K.-H. 75 Bower, J.M. 95 Boyapati, J. 157, 169 Boyd, I.A. 156 Bragin, A. 214 Braitenberg, V.B. 3, 194, 203, 223, 304, 321, 464, 465 Brandt, M.E. 330, 347 Brauer, W. 97 Brenner, R.P. 243, 283 Breslau, J. 243 Bressler, S.L. 288, 364, 378, 381 Brett, B. 89 Brillinger, D.R. 3, 50, 56 Bringuier, V. 96 Brody, C. 82N Brosch, M. 83, 89, 93, 207, 211 Brown, A.G. 174 Brown, S.W. 187 Brunel, N. 45, 49, 55 Bryant, H.L. 71 Bugman, G. 399 Buhman, J. 408 Bullier, J. 74, 158, 167, 171, 173, 175, 207 Bullock, T.H. 3, 255, 261 Buonomano, D.V. 194 Burgess, N. 402 Burns, B. 73 Burton, H. 128 Burton, R.M. 41 Bush, P.C. 95, 176, 223 Bushnell, E.W. 223 Busk, J. 356 Butler, R.A. 426 Buzsaki, G. 95, 270, 272, 433 Caramazza, A. 365 Cardoso de Oliveira, S. 83, 85, 86, 100 Carlson, V.R. 187 Castro, R. 43 Catsman-Berrevoets, C.E. 166 Celebrini, S. 5 Celletti, A. 43 Chagnac Amitai, Y. 97 Chang, H.S. 316 Chapin, J.K. 125, 126, 127, 145 Chen, A. 356 Christian, W. 243, 283

Index Chrobak, J.J. 433 Church, R.M. 192, 425 Classen, W. 194, 425 Clausen, J. 186 Clausen, V.R. 187 Cleland, B.C. 74, 170 Cobb, S.R. 433 Coben, L.A. 243, 283 Cohen, J. 193 Cole, H.W. 252 Collins, J.J. 402 Colter, N. 356 Coltheart, M. 365 Connors, B.W. 97, 222 Contreras, D. 92, 98 Cooley, J.W. 358 Cope, T.C. 69, 71 Coull, B.M. 157 Cowan, J.D. 4, 400 Cowan, N. 252 Cracco, J.B. 225 Cracco, R.Q. 225 Crawford, H. 243, 253 Creelman, C.D. 192 Creutzfeldt, O.D. 225, 428 Crick, F. 403 Crook, S.M. 224 Curran, H.V. 193 Cutler, C.D. 41 Cymbalyuk, G. 401 Czeiger, D. 175 Damasio, A.R. 318, 321, 403, 409 Damasio, H. 365 Dan, Y. 121, 122 Darian-Smith, I. 117 Davis, H. 344 Daw, N. 202 Dawis, S. 132 Dayhoff, J. 9, 28 de Ruyter van Steveninck, R.R. 3, 398 DeAngelis, G.C. 103, 123, 128 deCharms, R.C. 9, 76, 139 Demiralp, T. 139, 342F, 342F, 346, 357F Deschênes, M. 96, 167, 168, 169, 174 Desimone, R. 103, 137 Destexhe, A. 91 Deuchars, J. 4, 74, 83

463

Index

464

Diamond, I.T. 168 Dickson, J.W. 74, 94 Dinse, H. 123, 124 Doetsch, G.S. 168 Donoghue, J.P. 90, 101, 214, 427 Doob, L.W. 186, 187 Doppelmayr, M. 275, 283 Dostrovsky, J. 283 Douglas, R.J. 4, 95, 213 Drake, C. 192 Duckrow. R.B. 356 Duffy, F.H. 244 Dykes, R.W. 170 Eccles, J.C. 321 Eckhorn, R. 3, 79, 89, 96, 171, 175, 203, 204, 205, 212, 213, 215, 216, 218, 219, 223, 225, 226, 227, 232, 305, 364, 403, 430, 435 Eckmann, J.P. 42 Edmonds, B. 4 Eggermont, J.J. 77, 86, 97 Eisler, A.D. 187 Eisler, H. 187 Engel, A.K. 69, 79, 83, 89, 93, 94, 175, 204, 224 Epping, W.J.M. 69, 77 Epstein, H.T. 243 Erb, M. 97 Ermentrout, B. 401, 402 Essl, M. 362 Evans, E.F. 466 Evarts, E.V. 169 Eviatar, Z. 365 Fabri, M. 128 Faggin, B.M. 129 Fahle, M. 101, 312 Famiglietti, E.V. 174 Farah, M.J. 368 Feinberg, Z. 187 Ferreyra Moyano, H. 158, 170 Ferster, D. 37, 168, 170, 174, 398 Fetterman, J.G. 190, 426 Fetz, E.E. 71, 84, 89, 101, 203, 427 File, S.E. 193 Finlay, B.L. 169 Fischer, R. 193 Fleischhauer, K. 165 Florence, S.L. 144 Fohlmeister, J.F. 222

Index

465

Foote, S.L. 51 Fortin, C. 193 Fortune, E.S. 119 Fox, K. 127, 202 Fox, S.E. 270, 282 Fraisse, P. 193 Frame, A.M. 249 Franks, N.P. 91 Freeman, R.D. 96, 222, 225, 434, 440 Freeman, W.J. 89, 222, 288 Frégnac, Y. 80, 89 French, C.Ch. 356 Freund, T.F. 174, 283 Frien, A. 89, 205, 206, 207, 216, 218F, 222, 224 Fries, P. 100, 216 Friston, K.J. 3 Frost, B.J. 103 Frostig, R.D. 10, 93 Fruhstorfer, H. 330 Fujii, H. 398 Fukuda, Y. 170 Fuller, J.H. 157, 158 Fuller, P.W. 244 Funke, K. 96 Funkenstein, H.H. 119 Furst, G.H. 249 Fuster, J.M. 16, 21, 288, 289, 317, 322 Fyffe, R.E.W. 174 Gabriel, A. 220 Galambos, R. 220 Galanter, E. 220 Galbraith, G.C. 356 Galin, D. 249 Gasser, T. 243 Gath, I. 357 Gawne, T.J. 101 Gazzaniga, M.S. 364 Gelade, G. 409 Georgopoulos, A.P. 5, 37 Gerstein, G. 4, 10, 14, 16, 28, 69, 74, 80, 94, 121, 125, 138 Gerstner, W. 213, 224, 304N, 316, 435 Gescheider, G.A. 435 Getty, D.J. 435, 189, 191 Gevins, A. 357, 362 Ghazanfar, A.A. 125, 127, 128, 135, 142, 145 Ghose, G. 83, 96, 100, 222, 225, 434, 440 Gibbon, J. 190, 316

Index

466

Gilbert, C.D. 167, 204 Gilliland, A.R. 187 Gochin, P.M. 86, 93, 97, 101, 137, 171 Goldman, D. 362 Goldman, L. 156 Goldman-Rakic, P.S. 163, 345 Goldstein, M.H. 119 Golomb, D. 121, 122, 133 Gotman, J. 356 Gottlieb, J.P. 129 Grassberger, P. 41 Gray, C.M. 79, 83, 89, 92, 96, 97, 136, 204, 206, 210, 215, 219, 221, 222, 225, 288, 305, 317, 364, 403, 427, 433, 434, 444 Green, J.D. 444 Greenstein, Y.J. 444 Gribnau, A.A. 168 Griffith, J.S. 74, 172 Grondin. S. 172 191 Guettler, A. 212F Guido, W. 97 Guillery, R.W. 157, 169 Gustafsson, B. 71, 80 Haier, R.J. 432 Hakarem, G. 330 Hannan, E.J. 388 Hansel, D. 95, 97, 103, 224, 401 Harding, G.W. 156, 158, 168 Hari, R. 344 Hartikainen, P. 251 Hartline, H.K. 118 Harvey, A.R. 96, 166, 168, 175 Harvey, I. 431 Hata, Y. 86 Hauser, M.D. 119 Häusser, M. 202 Hazeltine, R.E. 190 Hebb, D.O. 80, 140, 289, 301, 321 Heinemann, U. 97 Henneman, E. 172 Hennessy, M.J. 193 Hénon, M. 45 Henry, G.H. 157, 169, 230, 398 Herrmann, M. 44 Hicks, R.E. 186 Hicks, T.P. 176, 420 Hill, S.L. 7, 41, 52, 425 Hillis, A.E. 365

Index Hirsh, J.C. 418 Hjorth, B. 362 Hoeflinger, B.F. 129 Hoffmann, K.P. 230 Holmes, O. 73, 85 Hoogland, P.V. 169 Hopfield, J. 406 Hoppenstaedt, F.C. 400, 412 Horn, D. 408 Horn, G. 74 Hornstein, A.D. 74 Houchin, J. 73, 85 Hubbard, O. 243, 251 Hubel, D.H. 117, 137, 230, 288, 466 Hudspeth, W.J. 243 Humphrey, A.L. 124, 158, 170 Idiart, M.A.P. 270 Innocenti, G.M. 96, 165, 171, 175 Isaksson, W. 359 Ito, H. 225 Ivry, R.B. 190, 194 Izhikevich, E.M. 401, 412 Jagadeesh, B. 89, 222 Jahnsen, H. 97 James, W. 187, 193 Jansen, B.H. 330, 347 Jasiukaitis, P. 330 Jasper, H.H. 362 Jenkins, G.M. 358 Jenkins, W.M. 135 Jervey, J.P. 317 Jester, J.M. 80 Johannesma, P. 419 John, E.R. 243 Johnson, M.J. 418 Johnson, M.L. 422 Jones, E.G. 168 Jones, K.G. 168 Jones, R.S.G. 97 Joosten, E.A. 168 Jordan, W. 207, 219, 225 Juergens, E. 205, 207, 211F, 215, 219, 229 Kaas, J.H. 143, 144 Kaba, H. 157 Kaernbach, C. 157

467

Index

468

Kahneman, D. 252 Kalcher, J. 255 Kalu, K.U. 156 Kaminski, K.J. 357 Kaplan, D.T. 41 Katada, A. 243 Kaufman, L. 255 Kavalieris, L. 388 Kawato, M. 401 Kazanovich, Y.B. 410 Keele, S.W. 194 Keifer, J.C. 91 Keller, A. 129 Kelly, E.F. 129, 377 Kelly, J.S. 74 Khibnik, A.I. 401 Khodorov, B.I. 174 Killeen, P.R. 174, 189, 190, 426 Kim, U. 97 Kiper, D.C. 101 Kirkwood, P.A. 71, 171 Klemm, W.R. 3 Klimesch, W. 243, 244, 247, 248, 251, 252, 253, 254F, 255, 256, 258F, 261, 269, 272 275, 276, 278, 279, 282, 318, 324, 366, 373, 432, 446, 447 Knierim, J.J. 103 Knox, C.K. 71 Knudsen, E.I. 119 Koch, C. 4, 5, 97, 101, 398, 399, 403 Kolb, B. 250 Kolev, V.N. 243 König, P. 4, 89, 95, 101, 202, 208, 213, 224, 229, 308, 309, 364, 404, 430 Konishi, M. 119 Könönen, M. 243 Kopell, N. 401, 402 Köpruner, V. 244 Koralek, K.-A. 129 Kottmann, M. 216 Krause, W. 358 Kraut, M.A. 225 Kreiter, A.K. 83, 89, 93, 206, 210, 215, 308 Kristofferson, A.B. 192 Kroll, N.E.A. 277 Krüger, J. 86, 171 Krupa, D.J. 129 Kruse, W. 89, 204, 212, 213, 216, 232 Kryukov, V.I. 403, 409 Kuramoto, Y. 213 Kurata, K. 54

Index

469

Kutas, M. 364, 384 LaMantia, A.S. 165, 166, 291, 293 Lamme, V.A.F. 103 Landry, P. 170, 174 Lang, M. 346 Larkman, A. 97 Larson, J. 97, 272 Lashley, K.S. 116 Le, J. 362 Lee, K.H. 294 Lee, M.-G. 432 Leenen, L.P. 168 Legatt, A.D. 204 Legéndy, C.R. 3 Lehmann, D. 3 Lemmon, V. 169 Lennie, P. 232 Leocani, L. 357 Lesèvre, N. 345 Lestienne, R. 317, 452 Leung, L. 452 LeVay, S. 174 Levick, W.R. 68, 74, 76F, 173 Levin, I. 193 Levitan, I.E. 51 Li, C.-L. 72 Lieb, W.R. 91 Lima, S.D. 191, 193, 425 Lindstrom, S. 168, 170 Lisker, L. 464 Lisman, I.E. 270 Little, W.A. 316 Livingstone, M.S. 83, 86, 89, 92 Llinás, R. 84, 96, 97, 221, 288, 364, 433, 444 Longo, V.G. 193 Lopes da Silva, F.H. 83, 89, 261, 270, 282, 364, 427, 436 Luce, R.D. 436 Lumer, E.D. 55, 95, 103 Lytton, W.W. 95 Macar, F. 193 MacDonald, K.D. 434 MacGregor, R.J. 3 Mainen, C.F. 5 Mainen, Z.F. 398 Maldonado, P.E. 138 Malpeli, J.G. 124, 170

Index Mandelbrot, B. 4 Mangun, G.R. 4 Manis, P.B. 157 Manzoni, T. 167 Maras, L. 167 Maren, St. 167, 272 Margoliash, D. 119 Maricq, A.V. 425 Markand, O.N. 243, 249 Markram, H. 6, 9, 74, 80, 83 Marrocco, R.T. 132, 170 Marthis, P. 243 Martignon, L. 3 Martin, K.A.C. 4, 213 Mason, A. 97 Masterton, R.B. 135 Mastronarde, D.N. 93, 97 Matsumoto, G. 156 Matsumura, M. 71, 420 Matthews, B.H.C. 72, 89, 91, 98 Mayer, M. 86 McCarthy, G. 373 McClurkin, J.W. 132, 133 McConchie, R.D. 187 McCormick D.A. 433 McCormick, D.A. 33, 89, 91, 96, 97, 221, 288, 444 McCourt, M.E. 166 McCrea, D. 71 McCulloch, W.S. 3, 322 McDonald, K.D. 214 McDonald, M. 330, 344 McGregor, R.J. 321 McGuire, B.A. 223 McKay, W.A. 214 McNaughton, B.L. 37, 55 Meck, W.H. 425 Mediratta, N.K. 169 Meier, H. 365 Meister, M. 100 Melssen, W.J. 69 Mendell.L.M. 172 Mendoca, A.J. 214 Merzenich, M.M. 9, 77, 139, 194 Michalski, A. 74 Michon, J.A. 193, 425 Middlebrooks, J.C. 135, 139 Mignard, M. 124 Miller, K.D. 103

470

Index

471

Miller, R. 33, 166, 167, 207, 270, 272, 283, 289, 292, 318, 321, 346, 409, 418, 420, 448, 449 Miller, R.F. 222 Milner, B. 222 Milner, P.M. 82, 215 Mishkin, M. 324 Mitrani, L. 193, 425 Mitzdorf, U. 204, 225, 428 Miyashita, Y. 317 Mizuki, Y. 346 Mohr, B. 289 Molina, J.C. 158, 170 Molotchnikoff, S. 434 Montoya, Ch. 434 Moore, G.P. 71, 97, 171 Mountcastle, V.B. 3 Movshon, J.A. 167 Mpitsos, G.J. 41, 54 Mukhametov, L.M. 3 Mulholland, T.B. 252 Müller, H.M. 364, 373, 384 Mundy-Castle, A.C. 346 Munk, M.H.J. 89, 92, 94, 175, 214, 224, 424 Murphy, J.T. 85 Murthy, V. 84, 89, 101, 203, 214, 427 Musicant, A.D. 426 Nafe, J.P. 3 Nakajima, Y. 192 Nelson, J.I. 86, 94, 103, 207, 214, 224 Nelson, M. 202, 203 Neuenschwander, S. 96, 225 Neumann, J.von 322 Neven, H. 100 Newman, J.D. 119, 466 Newsome, W.T. 4, 102, 131, 167, 399 Nichelli, P. 193 Nicolelis, M.A.L. 55, 125, 126, 127, 128, 131, 135, 140, 142, 143, 144, 145, 148 Nicoll, J.A. 169 Niedermeyer, E. 242, 243, 244, 250, 283 Nieman, R.E. 50 Niemeyer, W. 426 Noda, H. 73, 85 Nowak, L.G. 74, 86, 89, 92, 93, 97, 158, 168, 171, 173, 175, 207 Nuñez, A. 96, 288 Nunez, P.L. 248, 249F, 261, 362, 427, 437 O’Boyle, D.J. 194 O’Keefe, J. 194, 400

Index

472

Obermüller, A. 203, 205, 306 Obeso, J.A. 194 Obrist, W.D. 243 Offner, F.F. 362 Ojima, H. 168 Oliveira, L.M.O. 168 Optican, L.M. 133, 146 Osaka, M. 247 Palm, G. 171, 223, 302, 305, 309, 310, 312 313, 315, 316, 317, 318, 321, 430, 432, 449 Palmer, L.A. 176 Papanicolaou, A.C. 249 Paré, D. 364 Parks, R.W. 432 Parnas, I. 174 Partanen, J.V. 243 Pasemann, F. 316 Pasternak, G. 91 Pastor, A. 425 Payne, B.R. 19, 176 Pearlman, A.L. 166, 169 Pentland, A. 169 Perkel, D.H. 3, 11, 67, 71, 73 Peters, A. 174 Petersen, S.E. 345 Petersen, S.P. 373 Petsche, H. 270, 282, 356, 364, 431 Pfeiffer, A.Z. 135 Pfurtscheller, G. 251, 255, 275 Piersol, A.G. 275, 377 Pigeau, R.A. 249 Pinault, D. 96 Pitts, W. 3, 322 Pockberger, H. 97 Polich, J. 97 Pons, T.P. 144 Pontryagin, L. 50 Posner, M.I. 252, 289, 345 Potts, A.M. 345 Preiβl, H. 75 Press, W.H. 16 Pribram, K.H. 243 Prichep, L. 243 Pritchard. W.S. 330 Procaccia, I. 41 Prut, Y. 13, 14 Pulvermüller, F. 288, 289, 297, 443, 465 Purpura, K.P. 134, 146

Index

473

Rabin, A. 186 Raczkowski, D. 168 Radons, G. 168 Rahn, E. 332F, 332, 333F, 335, 337F, 337F, 339, 342F, 342F, 343, 344, 350, 357F Raichle, M.E. 289 Rakic, P. 165, 166, 291, 293 Ralston, D.D. 168 Rammsayer, T.H. 187, 191, 193, 425, 426, 427 Ranck, J.B. 158, 168, 283 Rapp, P.E. 3, 41, 42, 50 Rappelsberger, P. 356, 362, 363, 366, 367, 377, 385 Rauschecker, J.P. 120, 142 Ray, W.J. 252 Raymond, S.A. 170 Recce, M. 170, 400 Redlich, A.N. 122 Reid, R.C. 74, 124 Reitboeck, H.J. 215 Rémond, A. 346 Renaud, L.P. 74 Requin, J. 37 Rescher, B. 356 Reyes, A.D. 102 Ribary, U. 288, 444 Rice, C.E. 426 Richmond, B.J. 101, 133, 146 Riehle, A. 14, 37, 79, 84 Rieke, F. 3, 9, 398 Ringach, D.L. 123 Ringo, J.L. 166, 176 Rinzel, J. 399 Ritz, R. 95, 224, 406 Rizzolatti, G. 176 Robinson, T. 270 Robson, J.A. 169, 421 Robson, J.G. 96 Rockland, K.S. 157, 167 Rodieck, R.W. 96 Roelfsema, P.R. 89, 100 Rolls, E.T. 220 Romani, A. 330 Rosenberg, J.R. 3, 57 Rosenquist, A.L. 176 Roth, A. 202 Rotter, G.S. 202 Rougeul, A. 83, 90, 214 Ruelle. D. 42

Index Rushton, W.A.H. 156 Rutschmann, J. 187 Saam, M. 224, 225, 226 Sacaguchi, H. 402 Sakai, H. 169 Sakmann, B. 5 Sanes, J.N. 90, 101, 214, 427 Sarnthein, J. 356 Sauer, T. 43 Saul, A.B. 124 Sayers, B.McA. 344 Schack, B. 358, 387 Schacter, D. 276 Schanze, T. 206, 214, 219, 436 Schein, S.J. 103 Scheuler, W. 356 Schillen, T.B. 95, 213, 224, 229, 404 Schiller, P.H. 170 Schindler, K.A. 103 Schlag, J. 157, 158 Schmid, R.G. 356 Schneider, J. 203 Schneider, W. 82, 409 Schott, U. 229 Schramm, U. 168 Schreiner, C.E. 119 Schürmann, M. 342F, 342F, 346, 358F, 366 Schuster, H.G. 97, 176, 213, 444 Schüz, A. 3, 203, 222, 223, 304 Schwartz, C. 86 Schwartz, M.L. 163 Schwenker, F. 303, 304, 310 Scoville, W. 310 Sears, T.A. 71, 171, 174 Segal, M. 158, 170 Segev, I. 202, 203, 214 Seggie, J. 165 Segundo, J.P. 3, 50 Sejnowski, T. 4, 5, 95, 176, 223, 398 Shadlen, M.N. 4, 102, 131, 399 Shannon, C.E. 435 Shaw, J.C. 356 Sheridan, P.H. 244 Sherman, S.M. 170 Sherry, C.J. 3 Shevelev, I.A. 3 Shimizu, H. 404

474

Index

475

Shinba, T. 54 Shipp, S. 403 Shulz, D.E. 422 Shumkhina, S. 434 Sillito, A.M. 77, 83, 86, 93, 103, 225 Simmons, P.A. 166 Simons. D.J. 127 Sinclair, B.R. 270 Singer, W. 79, 83, 89, 93, 96, 136, 206, 210, 215, 225, 288, 305, 308, 364, 384 Skaggs, W.E. 55 Sklar, R.L. 356 Skrandies, W. 191 Smale, S. 41 Smith, G.M. 97 Smith, J.M. 155 Softky, W. 4, 102, 399 Softky, W.R. 399 Somers, D. 401 Sommer, F.T. 317, 318, 321 Sompolinski, H. 95, 97, 103, 176, 212, 229 Somsen, R.J.M. 244 Spencer, S.S. 357 Sporns, O. 95, 404 Spruston, N. 37, 398 Squire, L.R. 398 Stagg, D. 171, 174 Stampfer, H.G. 349 Starlinger, J. 426 Stein, R.B. 304N Stemmler, M. 4 Steriade, M. 50, 89, 91, 92, 96, 97, 214, 225, 248, 288, 317, 433 Sterman, M.B. 253 Stern, E.A. 418 Stevens, J.K. 418, 121, 125 Stevens, S.S. 188 Stewart, I.N. 402 Stewart, M. 282 Stoecker, M. 223 Stone, J. 158 Strehler, B.L. 317, 452 Strong, G. 402 Stuart, G.J. 5 Stubbs, D.A. 187 Suda, K. 168, 170, 176 Suga, N. 119 Surwillo, W. 244, 248 Sutter, M.L. 119 Swadlow, H.A. 74, 155, 156, 157, 158, 159, 160F, 163, 164, 165, 166, 167, 170, 171F, 174, 175,

Index 207, 417, 420, 434 Swift, J.W. 402 Symmes, D. 466 Szentagothai, J. 169 Szente, M.B. 80 Takahashi, K. 169 Tallon-Baudry, C. 84, 89, 215, 427 Tamura, H. 86 Tanaka, K. 74 Tang, A.C. 9 Tasaki, I. 156 Tauscher, J. 356 ten Hoopen, G. 191 Terman, D. 408, 409 Tetko, I.V. 7, 15, 19, 28, 33, 41, 46, 48, 49 Thatcher, R.W. 356, 357, 435, 437, 451 Theiler, J. 43 Thom, R. 50 Thomson, A.M. 4, 74, 83 Thorpe, S. 7, 101 Timin, E.N. 174 Tomasch, J. 165 Tononi, G. 100 Torres, F. 251 Tovee, M.J. 220 Towe, A.L. 156, 158, 168 Toyama, K. 74, 166, 167 Tramo, M.J. 167 Traub, R.D. 214, 283, 288, 444 Treisman, A. 252, 409 Treisman, M. 189, 191 Trimble, J.L. 191 Troy, J.B. 96 Troyer, T.W. 103 Ts’o, D.Y. 79, 86, 223 Tsodyks, M.V. 4, 9 Tsukaka, M. 3 Tsumoto, T. 168, 170, 176, 418 Tucker, D.M. 356 Tukey, J.W. 358 Tulving, E. 256, 276 Tyler, L.K. 365 Ungan, P. Usher, M. 345, 4 Vaadia, E. 7, 16, 77, 79, 84, 86, 92, 93, 216

476

Index

477

Van Essen, D.C. 102 van Hemmen, J.L. 304N, 316 Van Petten, C.K. 364, 384 Van Sweden, B. 243 van Vreeswijk, C. 102, 223, 316 Vanderwolf, C. 270 Viana Di Prisco, G. 89, 91 Victor, J.D. 134, 146 Vierordt, K. 146 Villa, A.E.P. 7, 10, 15, 16, 18, 21, 28, 33, 41, 43, 46, 48, 49, 52, 55, 57, 171, 312, 317, 397, 425 Vogel, F. 432 Vogel, W.H. 193 von der Malsburg, Ch. 81, 82, 304, 363, 403 408, 409 von Keyserlingk, D.G. 168 von Seelen, W. 230 von Stein, A. 356 Wadman, W.J. 464 Wagner, P. 176, 213, 444 Waksberg, H. 186 Wallace, M. 186 Walmsley, B. 421 Walter, D.O. 346 Wang Roe, A. 86 Wang, D.-L. 408, 409 Wang, X. 120, 139, 142 Wang, X.-J. 95 Wartenburg, H. 165 Waters, R.S. 168 Watts, D.G. 358 Waxman, S.G. 155, 156, 159, 165, 166, 170, 174, 175 Wearden, J.H. 190 Webb, A.C. 73 Weiss, N.A. 73, 189, 190 Weiss, S. 356, 364, 366, 367, 369, 385 Welker, W.I. 385 Weller, R.E. 158, 170 Wennekers, T. 171, 223, 226, 304N, 304, 310, 315, 316, 317, 318, 323, 430, 432, 433, 440, 452 Wertheimer, M. 223 Westphal, K.P. 346 Weyand, T.G. 97, 164, 169 Whishaw, I.Q. 250 White, E.L. 175 Whitfield, I.C. 466 Whittington, M.A. 95, 433 Wickelgren, W.A. 313, 315, 319, 321 Wickett, J.C. 431 Wieneke, G.H. 249

Index Wiesel, T.N. 117, 137, 230, 466 Wiggin, D.M. 168, 171 Wilbur, A.J. 399 Willennan, L. 248, 431 Willshaw, D. 302 Wilson, H.R. 4, 400 Wilson, M.A. 37, 95 Wilson, P.D. 170 Winter, P. 119 Wise, S.P. 54 Woelbern, T. 208F, 220 Wolff, A.B. 357 Wollberg, Z. 119 Woodward, W.R. 157 Woody, C.D. 169 Wörgotter, F. 96 Wortis, S.B. 135 Xiang, Z. 9, 52 Xing, J. 4 Yordanova, J.Y. 243 Young, M.P. 89, 294 Zakay, D. 186, 187, 193 Zarzecki, P. 166, 168, 171 Zeki, S.M. 167, 403 Zhang, Z.-W. 169 Zurita. P. 3

478

Index

479

Subject Index

acetylcholine 9 action potentials dendritic 5 Adrian, Edgar 2 alertness 252, 427 microshifts in 427 phasic 252 Alzheimer’s disease 244 amblyopia 100 anaesthesia 72, 85, 98, 102, 424 antidromic conduction collision test 158, 167 antidromic latencies 157, 164, 166, 167, 168, 169 for callosal axons 159 for corticocortical axons 163 of thalamocortical axons 159 of cortical efferent neurones 159 sampling biases 157–8, 167 Aplysia. 71 attention 54, 83, 242, 396, 397, 427 parallel vs. serial processing in 403 selective 252 vs preattention 403 attractor 7, 39, 42, 43F, 45, 46F, 53, 316 as description of memory storage 315 chaotic 39, 40, 45, 48 periodic 40 auditory localization in space 426 autocorrelogram 45, 46F, 71, 489, 96, 97, 142, 205F, 227F, 305 autoregressive moving average (ARMA) 355, 357, 374, 376F, 379F, 381F, 385, 387, 451 axon calibre 155, 174, 175, 291, 294T, 443 and spontaneous impulse activity 158, 164, 175 and thresholds to electrical stimulation 157 corticocortical 166 corticothalamic 169 in corpus callosum 159, 166 in humans 288 relation to conduction velocity 156 –7 axon conduction 155 –76 subnormal period 170

Index

480

supernormal period 170 axon conduction times 68, 95, 155, 166, 168, 169, 288, 289, 291, 294T, 295, 417, 418–24, 439 and cortical information processing 175 and phase relations in EEG 434 and spontaneous impulse activity 170 and temporal structure in spike trains 418, 419, 420 –3 between cortex and hippocampus 449 in corpus callosum 166 in preterminal branches 158, 173F inferred from antidromic latencies 157 –8 inferred from synaptic latencies 158 interhemispheric 166 subnormal period 420 supernormal period 420 variability in 417, 420, 423 axon conduction velocity 155, 159, 160F, 163, 164, 165, 166, 168, 169, 226, 294T, 443 and neuronal response properties 164 and spatial extent of synchronization 225 –6 range of 435 variations in 158, 170 axons callosal 156, 159, 160F, 165–6, 171, 175, 176, 291 corticocortical 155, 156, 160F, 163, 166, 167, 176, 421 ipsilateral 171 corticofugal 148, 155, 156, 160F, 168, 424 of layer 5 156, 163–4, 168 of layer 6 156, 164, 168 –9 corticothalamic 160F, 167, 418, 421, 422 fast conducting 176 myelinated 155, 156, 159, 164, 165, 168, 169, 176, 292, 294, 421 and coherence with zero phase lag 435 and intelligence 432 thalamocortical 156, 161F, 164, 170, 174, 421 unmyelinated 155, 157, 158, 159, 164, 168, 176, 292, 294, 420 detectability with light microscope 157, 169 proportion in white matter 165 volume occupied by 175 baboon 84 basal ganglia 80, 194 benzodiazepines 192 Berger, Hans 2, 72 beta rhythm: see: “electroencephalogram: beta activity” binding 66, 84, 100 by synchronization 99–101, 102, 215, 216, 220, 301, 304, 308, 396, 397, 398, 402 –11 network models of 100 of feature detectors 9, 82, 136, 215, 216, 219, 232, 324, 403 –9

Index

481

and attention 409, 411 and object separation 404 by local high frequency oscillations 435 hierarchical 405 binocular rivalry 217 brain as model of the world 463 as a predictor 466 brain state 66 cat 10, 11F, 16F, 18, 72, 79, 84, 86, 89, 93, 96, 99, 121, 134, 135, 155, 165, 167, 168, 169, 201, 203, 204, 206, 214, 215, 220, 223, 291, 305, 363, 420, 422, 427 Caton, Richard 2 cell assemblies 1, 3, 4, 6, 51F, 66, 74, 79, 80, 81, 82, 100, 103, 116, 135, 203, 288, 289–97, 291F, 301–24, 306F, 367, 466 and EEG activity 288 and EEG coherence 288 and high-frequency rhythms 289 and logical operations 322 –3 and long term memory 302 and pattern completion 80 and short term memory 301 and somatosensory peripheral deafferenta tion 143 –4 chains of 465 control by inhibitory’feedback 303 –4 definition of (Hebb) 301 formation of 302 global 317 –23 global vs local 301, 430 ignition of 302 oscillatory activity in 396, 442, 446 phase relations in 397, 399–402, 412, 437 role for information processing 399 reverberation in and axon conduction time 289 spatial extent of 289 thalamocortical 140 centre-surround interactions 103, 121 cerebellum 194 as store of temporal patterns 464 cerebral cortex 4, 29F, 31F, 51, 66, 71, 73, 79, 83–104, 201, 203 association areas 80, 309, 317, 367, 428 auditory 74, 77, 93, 118–, 134, 135, 138, 139 barrel columns in 127, 171, 421 coexistence of different dynamic regimes in 316 –7 cooling of 19–20, 21F, 23F deafferented 73 frontal region 79, 84, 93, 278, 280F, 312, 322, 329, 345, 366 inferotemporal region 21–7, 25F, 27F, 101, 137

Index

482

information processing in 201 –33 language zones 290, 291F layers of 123, 132, 420 phase reversal over 225 modularized model of 430 occipital region 430, 278, 280F parietal region 84, 96, 101, 144, 346, 381 prefrontal region 403 primary motor 33, 35F, 48F, 43F, 48, 74, 84, 101, 155, 156, 159, 160F, 163, 290, 291F somatosensory 73, 84F, 126–7, 128, 140, 142, 143F, 144–7F, 156, 159, 160F, 163, 167 visual 74, 79, 83, 85, 86, 89, 93, 94, 100, 101, 102, 122–4, 133–4, 156, 159, 160F, 163, 166, 169, 201, 205F, 211F, 216–33, 218F, 364, 379, 380 cerebral localisation 462 chaos 39 –53 detection of 41, 44 in spike trains 41, 42, 43 chaotic systems 39, 42, 47 chattering cells 222, 433 classification 233 cognitive compositionality 321 coherence in EEG 355 –89 and semantic judgements 363 –72 and sleep 356 and the problem of reference electrodes 356 –62 brain maps of 357 computation of 386 –8 definition of 356, 358F, 386 –8 derived by Fourier transform 358F derived from autoregressive moving average (ARMA) 358 event-related 357 frequency-specific 359, 435 interpretation of 360, 389 – requirements of temporal resolution 357 single trial analysis 359 use for study of cognitive processes 356 spatial 219 temporal 81, 85 of neuronal origin 71, 73, 77, 86, 93 stimulus locked 69 coincidence detection 4, 82, 101 –2 concrete vs abstract nouns differential representation 364 –92 connectivity in networks potential vs effective 203 corpus callosum 94, 157, 165, 166, 175, 293F regional differences in 165

Index

483

correlated firing between neurones 7, 66 –103 probability of detection of 54 correlation dimension 41, 42, 43F correlation integrals 42F cortical interneurones 9, 156, 170, 171F, 214, 222, 304, 305, 308, 314 corticothalamic pathways 93, 128, 164, 169 cross-correlation histogram: see “cross-correlogram” cross-correlogram 13, 66–103, 138, 171F, 173F, 228F, 308, 417, 418, 420 across hemispheres 78F, 87, 93 and axon conduction velocities 173 –4 and burst firing in 71, 96 and shift predictor method 71 as indicator of connectivity 73, 74, 80, 419 between different cortical areas 85 computation of 66 dependence on membrane potential 75 dependence on stimuli 76 during gamma activity 76, 97 effect of lesions on 94, 424 emergent from network dynamics 95 –6 for monosynaptic excitatory connection 68F, 68 for monosynaptic inhibitory connection 68 in spinal cord 71 in thalamus 74 peak width 66, 85, 86, 87F, 94 mechanisms determining 97 principle of 67 shape of 71 to estimate strength of connection 68, 74, 86 types of 67–8, 97 –9 with broad peak 68, 71F, 85, 98, 424 with centred peak 74, 86, 93, 95, 175, 207, 222, 424 with common input 67F, 68, 71, 73, 74, 93–5, 207, 222, 424, 425 with displaced peak 68, 71, 74, 81, 174, 175, 424, 441 with intermediate peak 71F, 85, 97, 98, 424 with narrow peak 74, 85, 97, 424 Darwinian evolution 462 delayed response task 24F, 33F, 36 dementia 284 desynchronization 214, 408 and arousal 73, 253 and sensory stimuli 72 and wakening 72 event-related 251, 275 phasic vs. tonic 253 deterministic system 39, 41

Index

484

development of brain 463 directed transfer function 356 distribution of cerebral functions 378 dogs 84 Down Syndrome 356 dual intracellular recording 71, 73 dynamical system analysis 40–1, 50, 55 –6 topology of 50F, 51F electroencephalogram 83, 444, 447 7–12 Hz oscillations 142, 143F alpha band and attention 243, 252, 253, 255 and expectancy 252 and semantic memory 252 and sensory-semantic tasks 256, 257F lower 243, 246F, 247, 251, 252, 253, 255, 269 lower-1 242, 253, 255 lower-2 242, 253, 255 meaning of 252–5, 257 upper 242, 246F, 247, 251, 252, 253, 255, 256, 257F, 259F, 269 alpha frequency age related changes in 248, 249F, 260 and brain size 248, 261, 431 and intelligence 432 and head size 249F and memory performance 244, 432 and speed of information processing 244, 248, 260 functional meaning of 244 –9 individually determined 242, 245, 246T, 275, 432 interindividual variability in 244 task-related variability in 244–7, 432 alpha power 243 age related changes in 249 event-related changes in 245F, 249, 254F, 258F functional meaning of 249 –52 hemispheric differences in 249, 259F alpha rhythm 242–62, 329, 343, 366, 437, 445 –8 age-related changes in 243, 244, 248 and cognitive performance 243, 245 and evoked potentials 245, 448 and expectancy 253 –5 and thalamus 432, 437, 446 as travelling waves 261 coherence in 355, 366–7, 374F, 445 dependent on corticocortical interplay 444 desynchronization of 242, 243, 246, 247, 248, 251–2, 253, 254F, 256, 260F, 261, 262, 274F, 275, 276, 445, 446, 447

Index

485

effect of stimulus modality 370, 447 event-related changes in 243, 446 frequency-specific desynchronization of 246 global vs. local 261–2, 446 meaning of 261, 444 peak frequency 244 regional differences in 244, 447 regionally-specific desynchronization of 256, 446 selection of frequency bands for 251 amplitude of inverse relation with evoked potential amplitude 343, 346, 349 beta activity 83, 373, 432, 437, 447, 452 coherence in 355, 367–9, 368F coherence 379F, 380F abstract vs concrete noun processing 367–9, 368F, 371F, 372F, 374F between different locations 355, 360F, 383, 389, 435 frequency-specific effects 384 interhemispheric 367 modality specific effects 369–72, 385 time-dependent 374 coherence spectra 355 common average reference recording 361, 362 continuous estimation of power, coherence and phase 357 degree of unit synchrony required 427, 429 delta activity 275, 355, 373 desynchronization of 214, 246F, 427 frequency-specific 246F diffuse alpha response system 346 diffuse theta response system 346 frequency of and spatial extent of coherence 431 –2 from hippocampus in humans 346 gamma activity 79, 83, 86, 89, 96, 100, 202, 288, 301, 304, 315, 432, 439, 449, 451 and anaesthesia 91 and attention 91 and memory 289 and phase-segregation 312 and sensory stimulation 89, 92, 310F and thalamus 433 generation by inhibitory networks 433 pacemaker 96, 433 precision of 309 relation of spikes to 309 synchronized between areas 89 synchronized between hemispheres 89, 94, 95 higher frequency and “round trip” times 289–91, 294, 295T, 434, 438–41, 442, 444 in hippocampus 270 –2 irregular slow activity 270

Index

486

induced band power 255 lambda waves 255 local field potentials 143F, 203, 205, 211F, 213F, 270, 305, 306F, 427, 429F major operating rhythm 345, 349 of frontal lobe 345 –6 of occipital lobe 346 phase estimation of 358, 437 phase analysis 355, 356, 357, 359F, 373–83, 379F, 380F, 382F, 396, 437 direction of influence 437, 376, 384 during word processing 384, 381 event-related 357 hemispheric asymmetry and 357, 451 in different frequency bands 451, 376F, 377, 24F in single trials 374 modality-specific effects 383 single subject study 374 –83 phase probability maps 357 phase relations in 434 and axon conduction times 434, 435, 436, 437 and spatial separation of electrodes 435 phase spectra 355 possibility of unified theory of 430, 438 –44 reference recording 361, 362 relation to evoked potentials 329 –49 sharp waves 270 source derivation 361, 362 spatial extent of coherence and intelligence 432 spindle waves in 72, 91, 97, 98 theta activity 256 frontal 338 theta frequency 269, 270, 272 –5 and attentional demands 275 and behaviour 270 individual variability of 275 theta phase and behaviour 270 theta power age related changes in 283 and encoding of new information 269 and episodic memory 276, 283 and short-term memory retrieval 269 event-related changes in 275 –6 task-related increase in 269 theta rhythm 269, 278–83, 279F, 280F, 282F, 320, 329, 331F, 335, 336F, 337F, 338, 340, 341F, 342, 343, 344, 345, 346, 349, 355, 357F, 370, 373, 385, 400, 402, 432, 445, 448, 449, 450 and episodic memory 276 and exploratory behavior 270 and memory coding 270 –82

Index

487

and memory retrieval 282 and place cells 400 and resonance between cortex and hippocampus 345 and working memory 272 functional meaning of 269 –72 in experimental animals 270 –2 phase-related unit discharges and 270 “round-trip” time hypothesis 448 separation from alpha 272–5, 273F type-1 270 type-2 270 volume conduction of 270, 362 electron microscopy 156, 157, 166, 168 embedding dimension 41, 42, 43F, 48 ensemble coding 80, 116, 147, 201 and size of classical receptive fields 145 and sound duration 138 as a function of assembly size 138 compared to single unit coding 135, 137 involving multiple brain structures 142 involving multiple cortical areas 144 –7 of auditory information 138 –43 of sound location 138 of stimulus location 145, 146F of visual features 136 –7 redundancy in 138 spatiotemporal 117, 131 –48 epilepsy 356 ethanol 193 event-related potentials 364, 384 P300 346 evoked potentials 329 alpha components in 333, 335 alpha- and theta-dependent 329, 331, 336F, 337F, 338, 340, 344 alpha-dependent 329, 331, 332, 335, 338, 344 amplitudes effect of selective averaging 332 analysed by selective averaging 329 and prestimulus EEG 329, 332–42, 343, 349, 448 in cats 342 topographic aspects of 338 as reorganization of EEG phase 344 as stimulus-induced synchronization 344 auditory 332–5, 334F, 343 beta components in 336 delta components in 336 dependence of alpha phase angle 336 effect of interstimulus interval 331, 332, 340–1, 344 effect of selective averaging on frequency

Index

488

components 334, 336 frequency content of from different locations 345 N1-P2 amplitude 346 N100-P180 339 relation to electroencephalograph 329 –49 selective stimulation paradigm 331, 333F, 337F, 338F, 340, 343, 344 significance of interaction with EEG 344 single trial analyses 344, 333–5 theta components in 333, 335 theta-dependent 332, 335, 337F, 340, 343, 344 vertex recordings 332 visual 335, 336F, 337F, 338, 340, 343, 346 in frontal regions 338 –42 excitation 4 balance with inhibition 4, 96, 223, 226, 229, 288, 399, 425 as origin of synchronized oscillations 222, 400 role in EEG rhythms 308 –9 expectancy 242, 252 exploratory behaviour tactile 142 Favoured Pattern Detection 9, 27 feature detectors 116 features extraction of 232 grouping of 232 field potentials 89 figure-ground segregation 80, 81, 101, 212F, 215 Fourier transform 355, 356, 357, 358 Fritsch, Gustav 2 gait models of 402 Galvani, Luigi 2, 55 gamma rhythm: see “electroencephalogram:gamma activity” Gestalt perception 297 global brain models (associative) 321 Go/NoGo task 28–33, 29F, 31F grammar 465 and coding of inverse temporal order 465 grandmother cell hypothesis 137 Grassberger-Procaccia algorithm 43F Haller, Albrecht von 1 hallucinogens 192 Hebb, Donald 289, 301F

Index

489

Hebbian learning 45, 46F, 201, 226F, 232, 302, 311F, 314 and development of cortical synchronization 225 Hebbian synapses 80, 319 Hénon mapping 45, 47, 48F hippocampo-cortical feedback loops 269 hippocampo-cortical interplay 272, 276, 282, 317, 318–21, 319F, 324, 409, 448 –50 and “phase-locked loops” 450 hippocampus 402, 404, 409, 432, 433 place cells 433 history of electrophysiology 2 Hitzig, Eduard 2 humans 6, 165, 187–93, 346 –9 impulse code: see “temporal code” impulse frequency 288 impulse timing 1, 4, 6, 9, 13, 15, 18, 19, 20 21F, 23F, 28, 36, 39, 45, 48F, 132, 225, 304, 422, 423 and perception 66 in network models 45 –6 precision of 5, 13 variability of 4, 9, 19, 102, 399 inferior colliculus 13 inhibition 4 balance with excitation 4, 222 interneurones: see cortical interneurones intracortical microstimulation and sensory reorganization 138 jitter: see “impulse timing: variability of Joint Triple Histogram 10–, 1OF Kanizsa triangles 84 labelled line hypothesis 103, 140, 419 language processing of 288 lateralization of cerebral functions 451 linguistics relation to cognitive psychology 363 long-term potentiation 269, 272, 282 magnetoencephalograph 288, 427 magnocellular pathway 230, 231, marmoset 120, 139 matching-to-sample task, delayed 20 membrane excitability 50 membrane potential 75, 202, 422 stability of 419 “up” state 418, 420

Index

490

membrane time constants 4, 51, 202, 214 memorization 367, 369, 373F memory 44, 53, 80, 242, 253, 279F, 366 associative 233, 301, 303F, 430 and cell assemblies 282F, 302 and retrieval 302 content-addressable storage 315 iterative retrieval in 310 model of 302 sparse coding in 302, 309 episodic 256, 269, 276, 278, 446 incidental 280 long term 315, 446 and cell assemblies 302 retrieval 269, 318 iterative 309 speed of 310 semantic 310, 279, 280F, 446 short term 53, 301, 315–7, 322, 446 and cell assemblies 301 phonological loop 367 working 366 monkeys 3, 5, 7, 13, 21, 24F, 26F, 32, 33F, 35F, 37F, 43F, 48, 71F, 84, 86, 93, 101, 119, 145, 160F, 165, 168, 201, 203, 204, 206, 212, 216, 220, 223, 230, 291, 305, 322, 364, 427, 452 mouse 165 movement detectors 466 multiunit activity from single electrode 203 multiunit recording 1, 5, 7, 10, 54, 124, 135, 139, 201, 205F, 222 multiunit spike trains 1, 3, 6 nerve cells biophysics of 4 convergence ratio to produce impulse 418, 419, 420 integration time 3, 4, 101, 201, 202, 422, 425 modulation of excitability of 423 spontaneous activity in 420 neural ensemble: see “cell assemblies” neural networks 1, 41, 54, 172F, 173F behaviour of 4, 7 –9 dynamics of 39 LEGION structure 408 mathematical analysis of 396 model of topographically organized area 429 models of 4, 10, 14F, 45–51, 54, 81N, 96, 102, 145, 195, 222, 225, 226, 229F, 229, 231, 304F, 315N, 396–412, 428, 430F for associative retrieval 312 for feature binding 403 –9

Index

491

using continuous time 304 using spiking neurones 303 models of local cortical dynamics 305–9, 306F models of synfire chains 314 oscillatory 55, 222–9, 405, 406F, 411F, 411 and attention 409, 411 hierarchical 412 theory of 400 storage capacity 310 synchronization in 54 neuronal chains 1, 5, 7, 13 neurones as threshold elements 302 object recognition and scene segmentation 215 oddball task 253 orientation selectivity 79, 86, 216F, 218F syncronized oscillations vs rate code 215 orientation tuning 218F, 305, 307 oscillations 54, 72, 226F, 229 and frequency spatial extent of 436 and membrane potential 96 and phase-dependent sensory responses 142, 207 –10 fast cortical 203, 204–16, 427 amplitudes of 206, 212, 226 and desynchronized EEG 214 and impulse frequency 204, 206 and information content 435 and orientation tuning 216 and state transitions 204 arising from chattering cells 222, 433 arising from intrinsic membrane properties 221 arising from network properties 221, 222–9, 439 arising from single neurones 220 –2 arising from spike encoder 221 frequency of 206, 226 in scalp-recorded EEG 214 mechanisms of generation 221 –30 participation of single neurones in 204 phase differences in 207 suppression of by stimulus transients 210–4, 230 synchronized 213 frequency of relation to stimulus properties 206, 207F, 219 dependence on stimulus properties 207F in alpha frequency 242 in cell assemblies 399 in gamma range and inhibition 201, 206

Index

492

in somatosensory system 142 in spike trains 89 pacemaker cells for 433 phase locked 219 relation between frequency and spatial extent 439 –41 resulting from interaction of excitation and inhibition 304, 433, 443 role of feedback loops 436 slow 91, 92, 98 stimulus-dependent 176 suppression of by out-of-phase inputs 212 synchronized 79,F 87, 95, 201, 231 effect of noise on 231 role of inhibition 96 spatial extent of 207, 212,F 215, 225, 427, 428–30, 438 –41 theory of 436 uncorrelated 209 –10 oscillators and frequency modulation 405, 412 assemblies of 406 multi-frequency 405, 412, 435 phase-coupling of 435 oscillators, coupled bifurcation analysis 401 conditions for in-phase vs antiphase oscillations 401 frequency and cortical range of associations 435 model of two interacting pools 442 relation between frequency and phase shift 400 stability of 401 theory of 400 –2 with desynchronizing connections 408 with nearest-neighbour coupling 402 oscillatory coding 398 P300 potential 346 parallel processing 4 parvocellular pathway 230, 231, 232 patch-clamp 9 pathology of brain 463 pattern completion 233, 315 and gamma activity 309 –12 Pattern Detection Algorithm 5,F 14, 15, 27, 32, 43, 45, 47 Pattern Grouping Algorithm 27–8, 32 perception 80 perceptual invariance 233 peri-stimulus time histograms 37,F 68 peripheral deafferentation and plasticity of somatosensory system 143 phase in EEG rhythms 373 –83 phonemes 464

Index

493

place cells 400, 402 place coding 398 point processes 50, 53 population coding: see “ensemble coding” post-stimulus time histogram 3,F 3 post-synaptic potentials 50, 51, 89, 202, 225, 425 excitatory 4, 71, 82, 101, 103, 158, 171, 174, 202, 410,F 418, 422 inhibitory 4, 69, 103 primate (experimental): see “monkeys” psychophysics 186–94, 425, 466 pyramidal tract 157, 168 pyramidal tract neurones 168 rabbit 155, 156, 159–64, 165, 292 raster plot 6,F 16, 19,F 20,F 22,F 25,F 27,F 29,F 34,F 35,F 36, 38,F 46, 47,F 49,F 306, 311,F 314, 319 rat 19, 21,F 23,F 29,F 31,F 73, 124,F 128, 137, 139, 145, 165, 168, 169, 291, 420 rate code 2, 9, 36, 53, 54, 131, 132, 135, 203 215, 398 and location of a sound 134 correlation with behaviour 4, 21–3, 28 noisy 4, 54, 131 receptive fields 230 and axon calibre 157 and axonal conduction velocity 164 and spike synchronization 201 as association fields 218–9, 226,F 230, 231,F 232 classical 118, 122, 125, 204, 218–9, 222, 226, 231 complex 231,F 233 dynamic 117, 127, 128, 148 simple 231,F 233 spatiotemporal 117, 118, 121–2, 123, 124–7,F 140,F 419 and active exploration 419 –131 and corticothalamic feedback 131 and information capacity 133 and sensory deafferentation 128 development of 128 neural circuitry of 124 origin in asynchronous convergence 128 spectrotemporal 119, 139 and natural vocalizations 119 visual complex 230, 232 simple 230, 231 resonance in physical structures 439 reticular formation 92, 214 retina 74, 93, 96, 218, 225 retinal ganglion cells 174, 420

Index

494

reverberatory activity 13, 79, 288, 289, 317 Rhythmic Slow Activity: see “electroencephalogram:theta rhythm” semantic judgements 252, 257F, 363,F 363 –72 and coherence changes 366 and grammatical word classes 364 concrete vs abstract nouns 363–72, 385, 447 sense of time 463 sensory processing 6 sensory systems 116 – sensory-semantic information processing 242 septal nuclei 283, 433 septo-hippocampal projections 282 Sherrington, Sir Charles 3 sleep 66, 72, 85, 91, 92, 95, 98, 99, 102, 215 rapid eye movement 73 slow wave 91, 92, 270 snowflake plot: see “Joint Triple Histogram” space-time inseparability: see “spatio temporal coupling” spatiotemporal coding 201 –33 by multispike trains 116, 131 –47 spatiotemporal coupling 121, 126, 127, 148, 419 spike generation and synchronization of inputs 202 spike synchronization 6, 9, 36, 66, 81, 82, 132, 155, 201, 215 and anaesthesia 72, 91 –2 and attention 83, 101 and axonal conduction times 170–5, 176 and feature binding 136 and reciprocal connections 98, 223 and waking 91 as sign of cell assemblies 66, 74, 84 as sign of cortical connectivity 66, 73, 84 between cortical areas 84, 174–5, 201 historical perspective 72 in cortical interneurones 170 in relation to stimuli 138, 309 interhemispheric 94, 175 interpretation of 66, 72 –4 mechanisms of 74 –98 oscillatory 82, 205F phase lag in 308 –9 relation to behaviour 84, 91, 100 relation to EEG 71, 84, 427 –8 relation to stimuli 71F, 78F, 79, 85, 86, 91, 92–3, 99, 204 role of 66, 99 –102

Index

495

spatial extent of 72, 73, 85, 137, 309 types of 66, 85, 97 –9 spike threshold level 50 spike trains 1, 3, 55, 66–103, 323, 396, 429F burst firing in 15–8, 19, 50, 51, 71, 72, 95, 96–7, 419, 420, 433 information coding by 397 multiunit recording of 1, 3, 28 pattern detection in 1, 4, 6, 9–, 14–5, 54, 55 spatiotemporal patterns in 1, 4, 7, 10–, 15, 16F, 18–32, 19F, 20F, 21F, 24F, 24F, 26F, 31F, 35F, 36, 45, 47, 53, 117, 131, 145, 396, 398, 422, 423, 443, 451, 464 network models 14F, 48F prevalence of 54 relation to EEG 452 relation to reaction time 28, 54 statistical significance of 5F, 10, 15 temporal structure in 1, 4, 50, 53, 312, 397, 398, 399, 417, 420 and axon conduction time 418 behavioural correlates 18, 21–32, 131, 312 information processing by 398 origin within single neurones 419 relation to motor behaviour 36 relation to reaction time 31F relation to stimuli 18, 37F, 417, 419 relation to timing of motor acts 32, 33F spatial extent of 139 state-specific 423 spike-triggered averaging 205F spinal motor neurones 174 strabismus 100 striatum 418 superposition catastrophe 81 syllables 464 sequences of 464 synaptic 5 synaptic depression 9 synaptic effectiveness 68, 74, 76F, 173, 421 and axon calibre 422 synaptic plasticity 53 synchronization 301 and cross-modal integration 364 and feature binding 215, 364 and perceptual processes 215, 304 at unit level vs EEG level 451 –2 between cortical areas 304 by common input 201 fast 201 –33 definition of 202 in gamma range 79, 93, 201 –33

Index

496

in relation to stimuli 79 interhemispheric 79F, 79, 304, 442 non-rhythmic 203, 215 of action potentials: see “spike synchronization” of cortical inhibitory cells 433 role in information processing 201–33, 397 spatial extent of 429 with zero phase lag 176, 201, 204, 223–5, 226, 231, 434, 435, 441, 442 synfire chains 7, 9, 54, 83, 301, 312–7, 323, 464 aggregations of 321 and long-term memory 312 and memory for sequences 32IF, 313 –5 and storage capacity 315 and short-term memory 312, 315 –7 coexistance with Hebbian assemblies 317 tactile stimuli 139 –42 location of 144 –7 temporal “chopping” 219 –20 and oscillation frequency 220 by feedback inhibition 223F, 231 temporal code 1, 3, 4, 7, 9, 18, 37, 39, 51F, 54, 82, 131, 132, 135, 203, 216, 398 and information carrying capacity 133 behavioural correlates of 3 compared to rate code in sensory representation 215 representing sound location 134 –5 representing space 132 –4 temporal convergence 422 temporal dispersion 81, 159, 174, 225, 226F, 420, 422 temporal lobectomy 422 temporal patterns: see “spike trains: spatio-temporal patterns in” temporal processing accuracy of 186 as function of duration 186 –91 temporal segmentation: see “temporal chopping” temporal structure of auditory signals 118 and natural vocalizations 118 of behaviour 116 of brain signals 301 –24 of natural signals 116, 117 and active exploration 117 decoding of 117 of neural activity 396 –412 of visual signals 121 thalamocortical loops 33, 117, 124, 142, 249

Index

497

thalamocortical pathways 74, 93, 116, 117, 128, 155 thalamus 3, 13F, 13, 32, 51, 91, 94, 97, 98, 404, 418 and fast cortical oscillations 225 auditory 13, 17F, 18, 19, 20F, 22F, 23F, 116 intralaminar nuclei 96 motor 163, 164 posterior group 96 reticular nucleus 17F, 18, 19F, 20F, 433 somatosensory 73, 116, 124–6F, 127, 128, 140F, 142, 143, 160F, 163, 164, 170, 171, 422 and cortical inactivation 128 visual 74, 93, 96, 116–22, 123, 132–3, 160F, 164, 218, 225, 379, 420 time delays 9 in EEG 373, 376F, 377 and interelectrode distance 377 lateral differences in 377 time estimation 18 7 – and axon conduction time 425 and benzodiazepines 426 and dopamine antagonists 426 and dopamine receptors 425 and “internal clock” 192, 425 and memory processes 425 and subcortical processes 425, 426 and working memory 193 blind vs normal-sighted subjects 426 by comparison 187 by discrimination 187, 190, 193 by explicit counting 191 by production 187, 190 by reproduction 187 by scaling 187 by verbal estimation 186, 187 cognitive mediation 193 difference threshold 193 in musicians 426 mechanisms of 186 method of limits 186 method of constant stimuli 186 multiple mechanisms of 192 –3 practice effects 427 prospective vs. retrospective methods 187 psychopharmacology of 192 time perception 186, 192, 425 tree shrew 170 trigeminal nuclei 128, 142 vigilance 84 visual perception 66, 203

Index of objects 288 vocalization 119, 134, 139 voice onset time 463 Volta, Alessandro 2 waking state 66, 202 Weber fraction 186, 189, 426 as a function of duration 190 –1 Weber’s law 186, 189 for time estimation 189, 191 whisker twitches 142, 143F white matter 155, 157, 175 word meanings concrete vs abstract 355

498