Time and the Brain
Conceptual Advances in Brain Research A series of books focusing on brain dynamics and information processing systems of the brain. Edited by Robert Miller, Otago Centre for Theoretical Studies in Psychiatry and Neuroscience, New Zealand (Editor-in-chief), Günther Palm, University of Ulm, Germany and Gordon Shaw, University of California at Irvine, USA.
Volume 1 Brain Dynamics and the Striatal Complex edited by R.Miller and J.R.Wickens Volume 2 Complex Brain Functions—Conceptual Advances in Russian Neuroscience edited by R.Miller, A.M.Ivanitsky and P.Balaban Volume 3 Time and the Brain edited by R.Miller Forthcoming Volume Sex Differences in Lateralization in the Animal Brain edited by V.L.Bianki and E.B.Filippova, translated by T.Ganf Volumes in Preparation Cortical Areas: Unity and Diversity The Female Brain Functional Memory and Brain Oscillations This book is part of a series. The publisher will accept continuation orders which may be cancelled at any time and which provide for automatic billing and shipping of each title in the series upon publication. Please write for details.
Time and the Brain Edited by
R.Miller Otago Centre for Theoretical Studies in Psychiatry and Neuroscience New Zealand
harwood academic publishers Australia • Canada • France • Germany • India • Japan Luxembourg • Malaysia • The Netherlands • Russia • Singapore Switzerland
This edition published in the Taylor & Francis e-Library, 2005. "To purchase your own copy of this or any of Taylor & Francis or Routledge's collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk." Copyright © 2000 OPA (Overseas Publishers Association) N.V. Published by license under the Harwood Academic Publishers imprint, part of The Gordon and Breach Publishing Group. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, without permission in writing from the publisher. Printed in Singapore. Amsteldijk 166 1st Floor 1079 LH Amsterdam The Netherlands British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library. ISBN 0-203-30457-8 Master e-book ISBN
ISBN 0-203-34338-7 (Adobe eReader Format) ISBN: 90-5823-060-0 (Print Edition) ISSN: 1029-2136
CONTENTS
Series Preface Preface List of Contributors 1 Empirical Evidence about Temporal Structure in Multi-unit Recordings Alessandro E.P.Villa 2 Cross-correlograms for Neuronal Spike Trains. Different Types of Temporal Correlation in Neocortex, their Origin and Significance Lionel G.Nowak and Jean Bullier 3 The Space-Time Continuum in Mammalian Sensory Pathways Asif A.Ghazanfar and Miguel A.L.Nicolelis 4 Information Flow along Neocortical Axons Harvey A.Swadlow 5 Psychophysics of Human Timing Thomas H.Rammsayer and Simon Grondin 6 Cortical Processing by Fast Synchronization: High Frequency Rhythmic and Non-rhythmic Signals in the Visual Cortex Point to General Principles of Spatiotemporal Coding Reinhard Eckhorn 7 EEG Alpha and Cognitive Processes Wolfgang Klimesch 8 Theta Frequency, Synchronization and Episodic Memory Performance Wolfgang Klimesch 9 Distributed Assemblies, High Frequencies and the Significance of EEG/MEG Recordings Friedemann Pulvermüller 10 Cell Assemblies, Associative Memory and Temporal Structure in Brain Signals Thomas Wennekers and Günther Palm 11 The Relation between EEG and Evoked Potentials Erol Başar , Sirel Karakas , Elke Rahn and Martin Schürmann 12 Coherence and Phase Relations between EEG Traces Recorded from Different Locations Peter Rappelsberger , S.Weiss and Baerbel Schack 13 Temporal Structure of Neural Activity and Models of Information Processing in the Brain Galina N.Borisyuk , Roman M.Borisyuk and Yakov B.Kazanovich
vii ix xi 1 63
112 150 181 196
236 262 281
293 320 346
389
Discussion Section POSTLUDE The Neuroanatomy of Time Valentino Braitenberg Index
410 454 460
SERIES PREFACE The workings of the brain, including the human brain are a source of endless fascination. In the last generation, experimental approaches to brain research have expanded massively, partly as a result of the development of powerful new techniques. However, the development of concepts which integrate and make sense of the wealth of available empirical data has lagged far behind the experimental investigation of the brain. This series of books entitled Conceptual Advances in Brain Research (CABR) is intended to provide a forum in which new and interesting conceptual advances can be presented to a wide readership in a coherent and lucid way. The series will encompass all aspects of the sciences of brain and behaviour, including anatomy, physiology, biochemistry and pharmacology, together with psychological approaches to defining the function of the intact brain. In particular, the series will emphasise modern attempts to forge links between the biological and the psychological levels of describing brain function. It will explore new cybernetic interpretations of the structure of nervous tissue; and it will consider the dynamics of brain activity, integrated across wide areas of the brain and involving vast numbers of nerve cells. These are all subjects which are expanding rapidly at present. Subjects relating to the human nervous system as well as clinical topics related to neurological or psychiatric illnesses will also make important contributions to the series. These volumes will be aimed at a wide readership within the neurosciences. However, brain research impinges on many other areas of knowledge. Therefore, some volumes may appeal to a readership, extending beyond the neurosciences. Books suitable for the series are monographs, edited multiauthor collections or books deriving from conferences, provided they have a clear underlying conceptual theme. In order to make these books widely accessible within the neurosciences and beyond, the style will emphasise broad scholarship comprehensible by readers in many fields, rather than descriptions in which technical detail of a particular speciality is dominant. The next decades promise to provide major new revelations about brain function, with far-reaching impact on the way we view ourselves. These great breakthroughs will require a broad interchange of ideas across many fields. We hope that the CABR series plays a significant part in the exploration of this important frontier of knowledge.
PREFACE Since the days of Galileo, time has been a fundamental variable in scientific attempts to understand the natural world. It has been clear since the first recordings of electrical activity in the brain, by Caton, and later by Danielewski, that electrical signals from the brain consist of very complex temporal patterns. This perspective has continued to the present day. It is now richly demonstrated by recordings of electrical activity at both the single unit level and by the methods of electroencephalography. When recording of the electrical activity from single nerve cells in the brain became possible, the emphasis was initially on events occurring on a more brief time scale than the fastest of psychological processes. However, in more recent years, single unit activity has also been widely studied on a larger time scale, corresponding to many psychological processes. In particular, in the last decade there has been a major shift in the way we view single unit activity in several brain structures: A generation ago, the emphasis was on mapping space (for instance in a sensory receptor surface) against spatial location in the brain. Nowadays we are more concerned with the temporal structure detectable in neuronal impulse trains. Temporal structure in impulse trains may thus be part of the way in which timing of sensory or motor events is represented in the brain; and indeed, it is possible that external events and inner cognitive processes may be represented in the brain in part using temporal coding, even when these events and processes, in themselves, lack detailed temporal structure. Analysis of temporal structure in the activity of large populations of nerve cells in the brain has also undergone major advances in the last ten or fifteen years. Functional brain imaging using metabolic activity gives good spatial resolution of brain processes. In contrast, mapping based on brain activity, based on the EEG or the MEG is superior in revealing the temporal structure of dynamic processes in the brain. For the EEG, the time scale needed to describe the basic rhythms of brain electrical activity corresponds roughly to that of the more rapid of psychological processes. That electrical activity is also known to correlate in many ways with psychological or behavioural events observed at the level of the whole organism. This book, the third in the series Conceptual Advances in Brain Research, explores modern approaches to these temporal aspects of brain electrical activity. The earlier chapters focus mainly on temporal structure revealed from trains of impulses recorded from single nerve cells, or from several such nerve cells recorded at the same time, and their possible relation to behaviour and psychological processes. In the later chapters, the emphasis shifts to temporal structure in the EEG, again with several essays exploring correlations with behaviour and psychology. There are also two chapters of a more theoretical nature. The final chapter forms an edited discussion between contributors, delving further into a variety of issues which arose during editing of the main chapters. In this, a significant question is also touched on—the relation (if any) between temporal structure discernible at the single unit level, and that seen in the EEG.
Study of the electrical activity of the brain by EEG recording and by the methods of single unit electrophysiology are often not well integrated: The two approaches tend to be adopted in different laboratories, and indeed in different countries. By bringing together within a single book modern information acquired using both approaches it is hoped that these two approaches can become better integrated as complementary windows on the information processing achieved by the brain. As editor of this book, I have to say that I am greatly honoured to have received such a fine set of chapters to edit, and I hope that my own contributions to the book are at the same standard as those of the chapter authors. R.Miller.Dunedin. December. 1999
LIST OF CONTRIBUTORS Erol Başar Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Galina N.Borisyuk Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region, 142292 Russia Roman M.Borisyuk Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region, 142292 Russia Valentino Braitenberg Max Planck Institute for Biological Cybernetics Spemann Strasse 38 D-7400 Tübingen Germany Jean Bullier INSERM U371 Cerveau et Vision 18 Avenue du Doyen Lépine 69675 Bron Cedex France Reinhard Eckhorn Department of Physics
Neurophysics Group Philipps-University Renthof 7 D-35032 Marburg Germany Asif A.Ghazanfar Department of Neurobiology Box 3209 Duke University Medical Centre 101 Research Drive Bryan Research Building Durham, NC 27710 USA Simon Grondin Institute of Psychology University of Goettingen Gosslerstrasse 14 D-37073 Goettingen Germany Sirel Karakas TÜBITAK Brain Dynamics Research Unit Ankara Turkey Yakov B.Kazanovich Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region 142292 Russia Wolfgang Klimesch Department of Physiological Psychology Institute of Psychology University of Salzburg Hellbrunnerstrasse 34 A-5020 Salzburg Austria
Miguel A.L.Nicolelis Department of Neurobiology Box 3209 Duke University Medical Centre 101 Research Drive Bryan Research Building Durham, NC 27710 USA Lionel G.Nowak Section of Neurobiology Yale University School of Medicine C303 Sterling Hall of Medicine 333 Cedar Street New Haven, CT 06510 USA Günther Palm Abteilung Informatik Fakultät für Informatik Universität Ulm Oberer Eselberg D-7900 Ulm Germany Friedemann Pulvermüller Department of Psychology Fachgruppe Psychologic Sozialwissenschaftliche Fakultät Universität Konstanz Postfach D23 D-78457 Konstanz Germany Elke Rahn Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Thomas H.Rammsayer Institute of Psychology University of Goettingen
Gosslerstrasse 14 D-37073 Goettingen Germany Peter Rappelsberger Institute for Neurophysiology University of Vienna Waehringer Strasse 17 A-1090 Vienna Austria B.Schack Institute for Medical Statistics, Informatics and Documentation Friedrich Schiller University Jahn Strasse 3 D-07740 Jena Germany Martin Schürmann Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Harvey A.Swadlow Department of Psychology (U-20) The University of Connecticut Storrs, CT-06269 USA Alessandro E.P.Villa Laboratory of Neuro-heuristics Institute of Physiology University of Lausanne 7 Rue du Bugnon CH-1005 Lausanne Switzerland S.Weiss Institute for Neurophysiology University of Vienna Waehringer Strasse 17 A-1090 Vienna
Austria Thomas Wennekers Abteilung Informatik Fakultät für Informatik Universität Ulm Oberer Eselberg D-7900 Ulm Germany
1 Empirical Evidence about Temporal Structure in Multi-unit Recordings Alessandro E.P.Villa Laboratory of Neuro-heuristics, Institute of Physiology, University of Lausanne, 7 Rue du Bugnon, CH-1005 Lausanne, Switzerland Tel: ++41+21–692–5532/X.5516/X.5500; FAX ++41–21–692.5532 e-mail:
[email protected] URL: www-lnh.unil.ch
The brain is a highly interconnected network of neurones, in which the activity in any neurone is necessarily related to the combined activity in the neurones that are afferent to it. Due to the widespread presence of reciprocal connections between brain areas, reentrant activity through chains of neurones is likely to occur. Certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements as a consequence of developmental and/or learning processes. In cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring, ordered, and precise (in the order of few ms) interspike interval relationships are referred to as “spatiotemporal patterns” of discharges. This term encompasses both their precision in time, and the fact that they can occur across different neurones, even recorded from separate electrodes. This chapter introduces the fundamental assumptions and algorithms that lead to the detection of complex patterns of neural discharges and introduces a way of interpreting this activity within the framework of non-linear dynamics. Empirical results of experimental and simulation studies are provided in different sections of the chapter. KEYWORDS: Spatiotemporal firing patterns; Neural dynamics; Brain theory; Time code; Frequency code; Multi-unit recordings; Non-linear dynamics; Sensorimotor association
1. BRIEF HISTORICAL INTRODUCTION In 1753–1755 the physiologist Albrecht von Haller published in Göttingen an historical essay, the “Dissertation on the Irritable and Sensitive Parts of Animals” (original title: De partium corporis humani sensibilius et irritabilus). This work was based on numerous experiments of vivisection, and on stimulation of organs using the new knowledge
Time and the brain
2
offered to physiology by physics, chemistry and natural history. With a rudimentary technique of stimulation, Von Haller classified the parts as irritable, sensible or elastic and noted that the reactions varied between different parts of the brain. The historical importance of the work by Von Haller is not so much related to the results obtained, but rather in systematically developing a transdisciplinary approach to brain research using the new technologies of his time. In 1791 the Italian physician Luigi Galvani started the publication of a remarkable series of studies that demonstrated muscle twitch in a frog by touching its nerves with electrostatically charged metal, and later using two dissimilar uncharged metals. These observations led Galvani to postulate that the circulation of a particular body fluid, that exists naturally in the nerves in a state of disequilibrium, provided the stimuli for the muscle fibres to contract. In addition, normal muscular contraction without a source of electrostatic electricity was, in Galvani’s view, evidence for the existence of an additional and “natural” form of electricity, that he called “animal electricity”. This latter statement set the scene for the famous Galvani-Volta controversy. Alessandro Volta, a friend of Galvani and an Italian physicist who, in 1775, invented the electrophorus—a device to generate static electricity—gave an opposed and experimentally valid explanation of Galvani’s experiment. The electricity did not come from the animal tissue but was generated by the contact of different metals, brass and iron, in a moist environment. The interest in this controversy resides in the confrontation of two basic, irreducible interpretations of the same observation, derived from each scientist’s different background: Galvani saw the frog phenomenon as the work of biological organs, Volta as that of a physical apparatus. The outcome of the controversy was exceptional. On the one hand the challenge of Volta’s opinion led Galvani to perform a new series of experiments that demonstrated muscular contraction by touching the exposed muscle of one frog with the nerve of another frog, thus showing for the first time the existence of bioelectric forces. On the other hand, Volta focused his research efforts upon the study of electric fluids between dissimilar metals, and in 1800 he presented the first electric battery, providing future researchers with a stable source of electricity not dependent on electrostatic forces. With the introduction of currents of “Galvanic fluids” into the brain, a way was opened to making new discoveries in physiology at the beginning of the XIXth century. Electricity was used not just as an experimental intervention, which formed the basis of electrophysiology (the study of the connection between living organisms and electricity), but in addition, the proper characteristics of propagation and generation of this type of energy became the basis of fertile hypotheses. In the last quarter of the XIXth century, Eduard Hitzig and Gustav Fritsch discovered the localization of cortical motor areas in the dog, using electrical stimulation, and Richard Caton was the first to record electrical activity from the brain. Electrophysiology started to develop rapidly, and Edgar D.Adrian published a seminal study suggesting the all-or-none principle in nerve (1912). In the late 1920s Hans Berger in Germany demonstrated the first human electroencephalogram and opened the way to clinical applications of electrophysiology. Nevertheless, the English School led investigations in electrophysiology in the first part of our century, and for his specific research on the function of neurones, Adrian shared the 1932 Nobel Prize for Medicine with Sir Charles Sherrington. Although most remembered for his scientific contributions to neurophysiology, Sherrington’s research focused on spinal reflexes as
Empirical evidence about temporal structure
3
well as on the physiology of perception, reaction and behaviour. His approach was transdisciplinary, in the sense that disciplines were used not only “one next to the other”, but were really intermingled in his protocols, an extraordinary example of a nonreductionist view of neurophysiology. At present, and still based upon the work performed by Adrian (1934), most neurophysiologists make their deductions by observing mean frequencies of nervous discharges (spikes), i.e. whether there are a lot or only a few spikes over a relatively prolonged time interval. Important concepts and findings have been clarified by using the overall mean rate as a measure of neuronal activity, at both peripheral and central levels of the nervous system. In particular it is important to remind ourselves of the influence of the findings of Mountcastle et al. (1963) indicating that the relationship between frequency of firing of selected neurones in the ventrobasal thalamus of deeply anaesthetized monkeys and the angle of extension of the contralateral knee was a perfect power function. Hence, the relationship between log frequency and log angle is virtually a perfect linear function. Although this point will not be developed further in the present chapter, it is important to note that the effect, and the type of anaesthesia determine dramatically the neuronal discharge pattern, especially at thalamic and cortical levels (e.g., see Mukhametov et al., 1970; Zurita et al., 1994). This effect has not always been adequately taken into account by investigators eager to develop theoretical models of “neuromimetic” circuits. The slow integration time of nerve cells, operating in the range of a few milliseconds, roughly a million times slower than presently available supercomputers, and the huge number of connections established by a single neurone (for review, see Braitenberg and Schüz, 1991) has suggested that information in the nervous system might be transmitted by simultaneous discharge of a large set of neurones. Multiple dimensions of stimuli relevant to sensory function and behaviour are processed by thousands of neurones distributed over many areas of the brain. Indeed, the hypothesis that neurones (both individually and jointly) process information over time, and following precise time relationships, has pervaded the neurosciences ever since the nervous system was conceptualized as a set of dynamic networks of interacting neurones (McCulloch and Pitts, 1943; for review, see MacGregor, 1987). That neurones convey a temporal code has been well-known since electrophysiological studies in the 1960s led to the recognition that spike trains—the time series formed by the sequences of time intervals between spikes—were related to meaningful physiological variables (Bullock, 1961; Segundo et al., 1963; Perkel and Bullock, 1965; Segundo et al., 1966; Nafe, 1968). Subsequent work made an important contribution, by recognizing temporal coding in different experimental circumstances and animal species, and by proposing worthwhile quantification procedures (Perkel et al., 1967a,b; Segundo and Perkel, 1969; Legéndy, 1975; Eckhorn et al., 1976; Brillinger et al., 1976; Klemm and Sherry, 1981; Abeles, 1982; Tsukada et al., 1982; Rosenberg et al., 1989; Bialek et al., 1991; Rapp et al., 1994; Friston, 1995; Martignon et al., 1995; Rieke et al., 1995; de Ruyter van Steveninck et al., 1997). This provided new insights into how the association between simultaneously recorded spike trains, in the time and frequency domains, may reflect the degree of coordinated activity within a cell assembly. The integrative capabilities of neurones (for review, see Segundo, 1986) on the one hand, and parallel processing and redundancy
Time and the brain
4
inherent to neural pathways on the other hand, have become more obvious during the past two decades, thanks also to the advances in neuroanatomical methods using retrograde and/or anterograde tracers (e.g. horseradish peroxidase, fluorescent dyes, biocytin, etc.). However, for most functional systems the available electrophysiological data demonstrate merely the presence of temporally organized neural activity, so that during recent years investigations have mainly focused on establishing causal relations between the occurrence of precise temporal relationships and cognitive or motor processes (Fetz, 1997). Therefore, two hundred years after the Galvani-Volta controversy, two main classes of theories explaining information processing in the brain have been proposed: In one, neurones convey a precise temporal code (Abeles, 1982; Abeles, 1991), while the other is based on noisy rate coding (Shadlen and Newsome, 1994). According to the rationale of a noisy rate code, the question has been raised whether the variability of spike intervals carries information. This variability may depend on the role of the decay of the post-synaptic potential (PSP) in determining the prevailing operating mode of network processing (König et al., 1996). Softky and Koch (1993) have shown that, in order to find the same degree of variability seen in vivo, coincidence detection with fast integration of small excitatory PSPs is required in models. Alternatively, Shadlen and Newsome (1994) have proposed that the balance between inhibition and excitation plays a critical role in the variability of network behaviour with membrane time constants of 8–20 ms. If an irregular interspike interval results from integration of excitatory and inhibitory postsynaptic potentials, then the timing of postsynaptic spikes is random and can no longer reflect the timing of presynaptic events. Precise patterns of spikes—their intervals and coincidences—would fail to propagate (Shadlen and Newsome, 1994). König et al., (1996) have provided a summary of these two viewpoints and proposed that the interplay of excitation and inhibition (E-I) effectively reduces the time constant for synaptic integration, thus providing coincidence detection in the cortex, although the actual synaptic decay is relatively long. Indeed, a balance condition of excitatory and inhibitory post-synaptic potentials plays an important role in the activity of many network models (Wilson and Cowan, 1974; Douglas and Martin, 1991; Tsodyks and Sejnowski, 1995; Usher and Stemmler, 1995; Xing and Gerstein, 1996) and has many complex spatial and temporal influences (Thomson and Deuchars, 1994). Changes in cellular excitability modify the time to firing, thus altering the E-I dynamics. Some specific examples of time-varying factors influencing the balance of E-I levels are the cell threshold potential, PSP kinetics, receptor activity such as NMDA, and calcium currents (Edmonds et al., 1995). The inability to preserve information about the time of a spike suggests that the discharge rate cannot be modulated in a time-locked fashion to specific inputs. In a random walk model (Gerstein and Mandelbrot, 1964) this assumption may be wrong because the discharge rate can follow the activity of inputs, but, given a base rate, the time to the next spike would appear as random. Then, the average instantaneous discharge rate through an ensemble of neurones belonging to a functional assembly would be capable of transmitting changes in spike rate with a precision of 10–50 ms (Shadlen and Newsome, 1998). Recordings performed in several cortical areas of behaving monkeys support the hypothesis of a connection between fluctuations in neural discharge rate and behaviour (Georgopoulos et al., 1993; Celebrini and Newsome, 1994). However, the existence of the rate code
Empirical evidence about temporal structure
5
mechanism does not imply that temporal codes do not also exist in the brain. The cerebral cortex is a highly interconnected network of neurones, in which the activity of each cell is necessarily related to the combined activity in the neurones that are afferent to it. Due to the presence of reciprocal connections between cortical areas, reentrant activity occurs, through chains of neurones. Furthermore, certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements, as a consequence of developmental and/or learning processes. According to the rationale of a precise temporal code, in cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring and ordered interspike intervals, with a precision of the order of a few milliseconds, are referred to as “spatiotemporal patterns” of discharges. For true demonstration, such temporal firing patterns must occur to a statistically significant level (see Figure 1). If functional correlates of spatiotemporal neural coding exist, one would expect that whenever the same information is presented, the same temporal pattern of firing would be observed. Several lines of evidence exist showing Spatiotemporal patterns in vivo (Bair and Koch, 1996) and in vitro (Mainen and Sejnowski, 1995). Recent studies on active propagation of action potentials in dendrites have provided additional results supporting the existence of precise neuronal timing (Stuart and Sakmann, 1994). A synaptic response increases if the presynaptic spike precedes the postsynaptic spike, but if the order is reversed, the synaptic response decreases. The window for synaptic
Time and the brain
6
Figure 1 . Outline of the general procedure followed by pattern detection algorithms, (a) Analysis of a set of simultaneously recorded spike trains. Three cells, labelled A, B, and C, participate in patterned activity. Three occurrences of a precise pattern are detected. Each occurrence of the pattern has been labelled by a specific marker in order to help the reader to identify the corresponding spikes, (b) Estimation of the statistical significance of the detected pattern, (c) Display of pattern occurrences as a raster plot aligned on the start of the pattern.
plasticity is 100 ms wide, and a difference in spike timing of only 10 ms near coincidence switches plasticity from potentiation to depression (Markram et al., 1997). Neurophysiological measurements in human subjects performing a visual go/no-go categorization task indicate that speed of processing of visual information in human is in
Empirical evidence about temporal structure
7
the order of 150 ms (Thorpe et al., 1996). Taking into account the firing rates of neurones in the visual pathway, only few spikes at most can be generated by each neurone in performing this highly demanding task, thus providing strong arguments for the importance of precise temporal coding. Indeed, electrophysiological experiments in primates have established that correlated firing between single neurones recorded simultaneously in the frontal cortex may evolve within tens of milliseconds, in systematic relation to behavioural events, without modulation of the firing rates (Vaadia et al., 1995). Furthermore, in multiple electrode recordings performed in the primary motor cortex of monkeys trained in a delay-pointing task, spike synchronization occurred in relation to purely internal events (stimulus expectancy), during which modulations of firing rate were distinctly absent (Riehle et al., 1997). These findings provide an important support for the general view that a stronger synaptic influence is exerted by multiple converging neurones firing in coincidence, thus making synchrony of firing ideally suited for highlighting responses and for expressing relations among neurones with high temporal precision.
2. DETECTION OF SPATIOTEMPORAL FIRING PATTERNS An influential and remarkable model based on the assumptions of high temporal precision in brain processing is the “synfire” chain hypothesis. This model suggests how precise timing can be sustained in the central nervous system by means of feed-forward chains of convergent/divergent links and re-entrant loops between interacting neurones forming an assembly (Abeles, 1982). Structures like synfire chains may exhibit attractors in which a group of neurones excite themselves, maintaining elevated firing rates for long periods and allowing the same neurone to participate in many different synfire chains. A fundamental prediction of such a model is that simultaneous recording of activity of cells belonging to the same assembly and involved repeatedly in the same process should be able to reveal repeated occurrences of such spatiotemporal firing patterns. Note that the term “firing pattern” encompasses both their precision in time, and the fact that they can occur across different neurones, even when recorded from separate electrodes. The following example (Figure 2) illustrates schematically the recurrence of patterns in a network connected according to the “synfire” model. A chain of 56 model neurones is formed by 8 sets of 7 neurones connected by diverging/converging links. This means one neurone in a set (in black in Figure 2a) receives 7 inputs from the previous set and projects to all 7 neurones of the next set. In the example shown in Figure 2a the links are subdivided into 5 excitatory (open circle) and 2 inhibitory connections (gray circles). This network is embedded into a larger network (200 neurones) with random connections. Note that the last link of the chain is connected to the first link. The model neurones follow simple integrate-and-fire dynamics (Villa and Tetko, 1995; Hill and Villa, 1997). Each neurone integrates all post-synaptic potentials. If the integrated depolarization passes the threshold, the neurone fires, becoming refractory, whereas if firing does not take place, the membrane potential tends to return to the resting level. Synaptic strengths and time constants have been selected in
Time and the brain
8
order to allow “spontaneous activity” to propagate but also to prevent “epileptic” spikes to self-generate in a simulation run. Figure 2b shows 28 successive frames of the neuronal chain activity, in arbitrary time units. Let us assume that three selected neurones, i.e. c10, c36, and c46, are recorded by virtual electrodes. The corresponding spike trains are shown in Figure 2c, and a firing pattern formed by four spikes was detected in these spike trains. More generally, let the list of cell labels that appear in a pattern of c spikes be noted as S c =(i 1 ,...,i j ,...i c ). In this, any label of the recorded neurones can be assigned to i j . In particular, for a 4-tuple pattern of spikes (quadruplet) obtained from a simultaneous recording of ten spike trains, let us assume that we find S 4 =(c36, c36, c10, c46), meaning all patterns of complexity c equal to 4, formed by a spike recorded from cell c36, followed by another event of cell c36, then a spike recorded from cell c10, which in turn is followed by a spike recorded from cell c46. One specific example of such patterns with delays of 6 time units between the first two spikes from
Figure 2 . Example of spatiotemporal pattern detected in a simulated chain of diverging/converging links. (a) Basic structure of one set of the chain, formed by 7 neurones. One neurone in a set (black circle) receives 5 excitatory (white circles and solid lines) and 2 inhibitory (gray circles and dashed lines) inputs from the previous chain. The output projections of one neurone on the cells of the next set follow the same pattern. (b) The results of the simulation of the activity in a network formed by eight successive sets from time 61 to 88 in arbitrary time steps. The large dots indicate that a neurone is firing.
Empirical evidence about temporal structure
9
The events belonging to the pattern detected in panel (c) are enclosed by squares, (c) A significant pattern formed by four cells is detected. The pattern was: cell 36; 6 time units (t.u.) later cell 36 again; 8 t.u. later cell 10 and 11 t.u. later cell 46.
c36, 2 time units between c36 and c10 and 5 time units between c10 and c46 is notated as
(see Figure 2). In addition, this notation assumes that the same time jitter was found for all events forming the pattern. An apparent weak point of the “synfire” theory is the requirement for neural mechanisms able to support a precise timing of spike patterns even after large time delays, and in the presence of various neuromodulators. For instance, a prominent effect of cholinergic modulation is to reduce adaptation of spike frequency. This results in an increase of neuronal excitability and a shortening of inter-spike intervals, so that the overall effect of such modulation can be a modification of spike timing. Differential actions of acetylcholine on the excitability of two subtypes of inhibitory cortical interneurones exist, so that cortical cholinergic activation may change the direction of information flow within cortical circuits (Xiang et al., 1998). Despite this, patch-clamp experiments in rat neocortical slices have shown that timing of spikes produced in response to fluctuating current injections may be preserved during cholinergic modulation (Tang et al., 1997). Transmission across neocortical synapses depends on the frequency of presynaptic activity and studies of synaptic depression have demonstrated that different aspects of the firing patterns of their afferents are transmitted depending on the average presynaptic frequency (Abbott et al., 1997; Tsodyks and Markram, 1997). When a single sensory stimulus drives many neurones to fire at elevated rates, the spikes of these neurones may become tightly synchronized, which could be involved in “binding” together individual firing-rate feature representations into a unified object percept. However, the response to elevated rates of firing may be weak, if any, and population coding based on relative spike timing can systematically signal stimulus features following the stimulus time course even where mean firing rate does not change (deCharms and Merzenich, 1996). These properties determine the fact that little information about steady-state frequency is transferred across synapses. This result is in contradiction with the rate code hypothesis, assuming that a higher input rate corresponds to a higher output rate. In this respect, information theory applied to the coding problem in the frog and house fly (Bialek and Rieke 1992; Rieke et al., 1995) also suggests that spike timing is important for reliable information transfer from receptors to brain, and may explain the subsequent behavioural events. In summary, several lines of evidence are in favour of precise temporal activity in the brain, but the development of analytical methods to detect reliably the transient temporal relations in sequences of spike intervals therefore represents a critical step to establish their function as a temporal code. One proposed technique aimed at detecting “favoured patterns”—i.e. patterns that occur more often than can reasonably be expected at random—consists of identifying template patterns even if the detailed timing of the patterns varies slightly (Dayhoff and Gerstein, 1983a). The advantages of this method, referred to as Favoured Pattern Detection (FPD), is that it can be used to detect other favoured patterns whose occurrences may have extra or missing spikes, and the firing patterns can be tested for
Time and the brain
10
significance by Monte Carlo algorithms. Unfortunately, this technique can be applied only for analysis of temporal patterns of activity generated by a single neurone, and for fixed jitter—i.e. the fluctuation in temporal precision allowed for each spike. Moreover, the choice for settings, e.g. the threshold level parameter, may have discouraged its use among electrophysiologists, and limited its application to experimental data (Dayhoff and Gerstein, 1983b). The use of multiple microelectrode recordings, and of large artificial neural networks of spiking neurones has challenged investigators to search for methods allowing the analysis of higher complexity firing patterns, formed by several spike trains and involving several events within a pattern. An approach to the exact calculation of the probability of randomly obtaining each individual recurring pattern was proposed for three and more distinct spike trains (Frostig et al., 1984, 1990). In this method the statistical evaluation is based on the use of a 2×2 contingency tables and the application of Fisher’s exact test. At the first step, the detection stage, all possible intervals between three spike trains are tested for significance, and at a second step, the expansion stage, all significant patterns found previously are checked for higher-order associations among spike trains. The main limitations of this method consist of its inability to detect recurring patterns involving less than three different spike trains and patterns of more than three spikes, if not all subpatterns formed by a triplet of spikes are statistically significant. Several investigators have pointed out that in most experimental preparations presently available the chance of observing patterns of discharges of higher-order complexity is very low, and specific analytical methods aimed at analyzing patterns formed by three spikes (triplets) were developed. The principles of three-fold correlation among spikes are illustrated in Figure 3. Suppose that the firing times of three neurones A, B, and C are recorded simultaneously (Figure 3a). One could ask whether the firing of cells B and C depends on the time elapsed from a spike of cell A. Let us consider the time delays tAB of the discharges of cell B after neurone A fired and similarly the time delays tAC. The joint distribution histogram may be constructed as shown in Figure 3b. At each discharge of cell A the spike train of neurone B is plotted in the X-axis and the spike train of neurone C is plotted in the Y-axis of the graph. The crossings of columns and rows, falling within a defined bin size, that correspond to the spike trains of neurones B and C recorded simultaneously are counted and a Z-axis, perpendicular to the plan of drawing, is plotted. The value of the compound rate plotted on this Z-axis may be coded in shades of gray (or on a colour scale). A graphic possibility for plotting all compound joint distributions between three neurones is based on a triangular coordinate system because the three ). The X-Y graph can be variables tAB, tAC, tCB, are not independent (since skewed by 60° (Figure 3b, right panel) so that all three joint distributions can be plotted together (Figure 3c). A practical application of this method in practice is shown in Figure 3d. In this example the data were recorded from the thalamus of an anaesthetized cat during spontaneous activity (Villa, 1988, 1990). Cell 1 (discharging at a rate of 2.8 spikes/s) was recorded in the posterior group of the thalamus, cell 3 (1.7 spikes/s) in the dorsal nucleus of the medial geniculate body and cell 7 (5.7 spikes/s) in the brachium of the inferior colliculus. A black triangular bin indicates a significant event corresponding to a spike of cell 3, then 8 ms later a spike from cell 7 and a spike of cell 1 occurring 116 ms after the onset of the pattern. The time accuracy of this estimation was limited to 15
Empirical evidence about temporal structure
11
ms so that spikes of cells 7 and 1 might be considered synchronous. Detection of triplets was first achieved using the “snowflake” triple-spike renewal histogram (Perkel et al., 1975; Abeles, 1983). However, triple-spike renewal histograms do not provide a reliable estimation of the significance of the detected patterns, and to solve this problem the method was developed further into the Joint PeriStimulus Time Histogram (JPSTH) (Aertsen et al., 1989). A number of applications of this technique to
Figure 3 . Outline of the general procedure followed to study three-fold correlation among spikes, (a) Three spike trains, labelled A, B and C, are recorded simultaneously. The time of occurrence of spikes A are
Time and the brain
12
successively taken as onset time for calculation of delays of simultaneously recorded spikes of cell B (tAB) and C (tAC). Three markers indicate three successive spikes of cell A. (b) A plot of all possible patterns formed by B and C spikes as a function of the A spikes marked on the top panel is presented in form of a X-Y plot where the abscissa represents the delays of B spikes after A firing and the ordinate the delays of C spikes after A firing. The left panel shows the same plot with the Y-axis skewed by 60°. (c) A three-cell correlogram is constructed by joining three graphs in order to represent all possible time relations among the firing of three neurones. The three intervals (tAB, tAC, tCB) associated with any point on the graph can be read by projecting the point onto the three external axes, (d) The activity of three neurones, labelled 1, 3 and 7, were recorded from three different electrodes in the cat thalamus. A significant pattern is detected as shown by the black triangular bin. The pattern was: cell 3; 8 ms later cell 7 and 108 ms later cell 1. The shades of gray correspond to the number of counts per bin on a relative scale.
the study of simultaneously recorded spike trains in behaving monkeys have been reported (Abeles et al., 1993b,c). Since then, the algorithm has been improved for statistical evaluation, in order to take into account the existence of significant crosscorrelation between cells. Analyses of multiple spike trains recorded in behaving monkeys using this most recent algorithm, referred to as Joint Triplet Histogram (JTH), have been published recently (Prut et al., 1998). A careful analysis of the compound activity of three-cells may suggest interesting hypotheses about functional connections between distinct nuclei. Let us consider the result illustrated in Figure 3d. One could argue that cells in the brachium of the inferior colliculus and cells in the dorsal nucleus of the medial geniculate body receive common afferent activation from the ascending auditory pathway, and synchronous activation of these nuclei is easy to accept. However, the cross-correlation between the neurones recorded in these nuclei was flat, as indicated by the absence of dark diagonal bands centered on time zero in the snowflake plot, and the quasi-synchronous activation of cells 3 and 7 was only detected when a spike in cell 1 occurred more than 100 ms later. This delay cannot be due to direct connections through the thalamus (according to the present knowledge of thalamic organization). The observation of a complex pattern of firing in different thalamic nuclei suggests that these anatomical regions may form distributed cell assemblies that are functionally connected by some kind of reverberating activity, probably involving re-entrant activity from the cerebral cortex. In chains of diverging/converging links there is no need for each of the synaptic connections to be particularly strong, since they become effective through coactivation with others. Then, several synapses can depolarize the postsynaptic cell to reach its firing threshold, by spatial and temporal summation. In the case of pure excitatory feed-forward chains of neurones (Abeles, 1982, 1991) it has been demonstrated that a strong stimulation (e.g. input fibres firing at a high asynchronous rate) applied to all the cells of a set in the chain would elicit a spike in all of them synchronously, so that the next set of neurones down the chain would also be activated synchronously. This synchronous
Empirical evidence about temporal structure
13
activity would then propagate along the whole chain, with a limited time jitter. Thus, interconversions from firing rate code to synchronous activity, and back, can be performed by a chain of diverging/converging links. This observation is related to the origin of the “synfire” chain model (Abeles, 1982). If these chains include excitatory and inhibitory links, the analysis of the propagating activity is not so simple. In addition when one considers that distinct sets of the chain can be activated independently , then the picture of a propagating wave becomes less clear. However, these chains will tend to produce unitary events, i.e. precise spike synchronization between neurones, which occur significantly more often than expected by chance. The temporal spacing between the spikes should not be regarded as corresponding to the relative location of the parent cells along the chain. Such characteristics of the model are illustrated by Figure 4. In this example the very same simulation results as in Figure 2 are considered, but different cells were sampled by the virtual electrodes. It is important to note that cells involved in unitary events would not necessarily show signs of synchronization in crosscorrelograms, because they do not receive either common excitatory inputs or common inhibitory inputs. In Figure 4a it can be seen that, due to the presence of other inputs in some occurrences of the temporal pattern, a spike belonging to the pattern may be missing because a cell is in a refractory state. However, due to the peculiar properties of diverging/converging links the pattern is sustained by the other neurones and may persist in the network. An increasing number of studies are now investigating unitary events and preliminary experimental evidence has been reported (Riehle et al., 1997; Prut et al., 1998). It is therefore possible that unitary events in the primary motor cortex are related to pattern generation, for instance as starting points for delayed synchronous patterned activity related to the precise activation of sequences of muscles involved in motor output.
3. EMPIRICAL EXAMPLES The Pattern Detection Algorithm (PDA) developed by Abeles and Gerstein (1988) allows one to perform a comprehensive search for firing patterns and to test if there is a statistical excess of patterns. According to the complexity of the patterns to be analyzed,
Time and the brain
14
Figure 4 . Example of a unitary event detected in a simulated chain of diverging/converging links, (a) Two events corresponding to the synchronous activation of cells 1, 17, 23 and 33 have been detected, plus an event corresponding to the synchronous activation of cells 1, 17 and 33. (b) Same simulation results of Figure 2b. The large dots indicate that a neurone is firing. The events belonging to the unitary events are enclosed by squares.
two methods (“ad-hoc” and “bootstrap” algorithms tested against simulated multineurone data) are used for estimation of the expected number of repeated patterns of spikes, regardless of their complexity. Improved and computationally effective versions for estimation of the significance of patterns of spikes in this algorithm, based on probabilistic and combinatorial algorithms, were recently elaborated (Tetko and Villa, 1997a). The main restriction of PDA when compared to the other methods is that it does not allow one to establish the statistical significance of an individual pattern which repeats a number of times, but only the existence of an overall excess of precise patterns beyond expectations. This limits its main applicability to cases when pattern occurrence is very rare. Assuming a Poisson distribution of point processes, the significance of the excess of detected patterns Z, of complexity c, and repeating exactly r times in the record,
Empirical evidence about temporal structure
15
over the expected number X is estimated by pr{Z,X}=P(Z,X) where P(Z,X) represents the incomplete gamma function (Press et al., 1994). This estimation is simplified to if only one firing pattern is detected. Let us fix the level of significance to be p 0 =0.001. For example, if the number of patterns detected by PDA is Z=6, and only one pattern (i.e., X=1) is expected by chance, then a significant excess of detected patterns is observed because . However, this estimation cannot indicate which ones out of the six detected patterns are significant. In some cases PDA allows one to detect an excess of patterns when all sequences of spike intervals are significant. Consider an example where the expected number of patterns is as low as X=0.001. If one pattern is detected, its significance is equal to thus indicating that the pattern is highly significant. If in the above example we detect six patterns instead of one, then any of these six can be considered as highly significant. (In such a case each of the six pattern can be considered significant by itself.) Routine studies using the classical PDA have demonstrated that this optimal case is exceptional. Most often, patterns that could be significant by themselves under appropriate values of jitter are detected by PDA as distinct patterns if the search is performed with maximum accuracy, i.e. with time jitter equal to 1 ms. The application of PDA to several experimental models, and to recordings from various brain areas, has revealed that complex Spatiotemporal firing patterns do indeed occur (Vaadia et al., 1989; Villa and Abeles, 1990; Abeles et al., 1993a; Villa and Fuster, 1992). In these studies the algorithm was usually set to find any repeating pattern of three or more spikes, provided that the entire pattern lasted not more than 900 ms at most, and was repeated with accuracy of 1 ms. Since high frequency bursts (200 spikes/s or more) may produce patterns which relate to intracellular processes, burst filtering should be performed. A filtering procedure was originally proposed by Abeles and Gerstein (1988). A good filtering algorithm should provide a minimal loss of information, for an acceptable rate of errors in the estimation method. This would require fine tuning of this procedure that includes several combinations of three independent parameters: the time window for burst detection, the time window and the number of counts in high-frequency filtering. An alternative filtering method based only on one parameter, the filtering frequency, has been recently proposed (Tetko and Villa, 1997a). This method, with filtering frequency ranging between 200 and 400 spikes/s, provides better estimates of the number of patterns, for a comparable loss of information. A record was considered as having an excess of repeating firing patterns only when the probability of finding as much (or more) repetitions by chance was less than the significance level p 0 . Significant patterns that repeated exactly, more than four times, may be used as templates for searching approximate patterns with jitter of ±5 ms. Figure 5 shows an example of the activity of six neurones recorded in the cat thalamus during spontaneous activity (Villa, 1990; Villa and Abeles, 1990). It illustrates a pattern of 3 spikes, repeating 25 times, starting with spike 2, then after 89 ms spike 4, and then after 305 ms spike 5. On the left panel (Figure 5a) the rasters are aligned on pattern start and on the right panel (Figure 5b) the corresponding histograms (we may call peri-pattern histograms) are computed. In addition to the pattern itself, lasting hundreds of ms and formed by spikes recorded from different electrodes, this example is important because it shows an additional feature. The histogram of Figure 5b shows that two bursty neurones, cells 1 and 6, do not
Time and the brain
16
discharge in correspondence to the last spike of the triplet (cell 5 at 394 ms), but both cells show a significant increased tendency to discharge, in bursts, near 410 ms after the pattern
Figure 5 . (a) Raster display of the activities of six neurones during spontaneous activity recorded in the auditory thalamus of the cat. The rasters are aligned by displaying the first spike in the pattern at time 0. The pattern was: cell 2; 89±5 ms later cell 4, and then 305±5 ms later cell 5. The units were recorded by four microelectrodes. The first, in the thalamic reticular nucleus, picked-up units 1–2; the second, in the medial geniculate body, unit 3; the third, also in the medial geniculate body, picked-up units 4–5 and the last, in thalamic reticular nucleus, unit 6. (b) Histograms of the activities of the units triggered by the pattern onset. The abscissa full scale is 800 ms and the ordinate full scale is 85 spikes/s. Note that units 1 and 6 had a significant higher firing rate 410 ms after the pattern onset. The dashed lines correspond to the 99% confidence levels and a solid line is plotted at the average firing level. The curves are smoothed by a moving Gaussian bin shape of 10 ms.
onset. Therefore we could say that Figure 5 illustrates in fact a very complex pattern that should be noted . This is true, but no algorithms are yet available to detect such a pattern because the firing of the bursty cells is only loosely,
Empirical evidence about temporal structure
17
yet significantly, synchronized with the pattern onset. Furthermore, the selective inhibition of cells 1 and 6 at the same time as the third event of the triplet represents per se a valuable piece of information, related to the highly precise temporal processing occurring in the thalamus. Unfortunately, complex temporal firing patterns mixing spike occurrences and “missing spikes” are not detected by the usual algorithms. This is partly due to the enormous increase in computations required by taking recursively the “negative” of one spike train with all other spike trains recorded simultaneously, and partly by the fact that it is not correct to simply take the “negative”. However, there is no doubt that selective absence of spikes at specific timings would contribute to a better understanding the temporal processing in the brain, and it is reasonable to forecast that in the next few years methods will appear to deal with this problem.
4. PATTERN RECURRENCE IN RELATION TO STIMULI AND RECORDING CONDITION The next two examples illustrate some relations of firing patterns elicited by sensory stimuli, in the sense that they would not be observed during spontaneous activity (Villa 1988; Villa et al., 1991). Recordings were made in the auditory thalamus of the cat. Figure 6 shows a triplet, occurring selectively during the presentation of a stimulus which was a white noise burst lasting 200 ms, delivered to both ears. The firing pattern always started after the stimulus onset, but it always ended before the stimulus offset. However, the pattern was not time locked with the stimulus onset, as clearly shown by the irregular alignment of the stimulus onset and the start of the pattern. Similarly, a pattern could be elicited by a stimulus, but occurring after the stimulus offset and without a tight locking to it. Such an example is illustrated by the triplet in Figure 7, repeating 12 times, starting by spike 3, then after 285 ms spike 1 and then after 189 ms spike 2. Note that spikes 1 and 2 were recorded from the same electrode, thus indicating that these cells lay within a distance of few tens of microns of each other. Nevertheless the delay between these correlated discharges was near to 200 ms, and suggested long reentrant loops from the cortex to the thalamic reticular nucleus or, alternatively, from the substantia nigra and/or from modulatory mesencephalic nuclei, that are known to project to this thalamic region. In order to establish the significance of firing patterns as signposts of distributed network activity related to specific brain processes, it is crucial to demonstrate that whenever the network is working in the same state, the same temporal pattern of firing is observed. Several above-mentioned examples have suggested that cortical activity may affect the recurrent activity in the thalamus, as shown by several corticofugal studies, either experimental (Villa et al., 1991; Payne et al., 1996; Villa et al., 1999b) or theoretical (Tetko and Villa, 1997b). The next examples report data recorded in the auditory thalamus of the rat during reversible cortical deactivation by cooling (Villa et al., 1999b). This type of study illustrates an additional problem that may arise in studies of temporal coding. The number of spikes necessary for a fair statistical evaluation of the
Time and the brain
18
Figure 6 . Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.0 mm. The lowermost raster shows the onset of the stimulus, as a thick tick, and the gray bars represent the time of stimulus duration. The rasters are aligned by displaying the first spike in the pattern <2,3,1; 49,83> at time 0. The pattern is repeated 19 times with a jitter of ±3 ms. The abscissa full scale is 600 ms.
significance of the patterns may not be met in all data-sets because the experimental protocol, in this case the cortical cooling, may produce a massive decrease in firing rates of selected single units, or alternatively a massive increase of discharges, and a major change in the pattern of discharges (i.e. from regular to bursty). In case of a decrease in firing rate a partial methodological compensation might be obtained by increasing the recording time. Figure 8 shows an example of a pattern formed by three spikes belonging to two single units recorded simultaneously from two different electrodes in the rat auditory thalamus. The pattern is formed by cell 7 firing, then after 350 ms by a discharge of cell 11 and then after 21 ms cell 11 firing again. This exact pattern was used as template for searching approximate matching patterns with a jitter of ±2ms. Note that several spikes may occur in between the spikes belonging to the pattern without affecting the precise intervals separating the occurrences of the cell discharges. In this example, the significant pattern was observed 9 times in 400 s of spontaneous activity recorded during the control condition. No patterns were observed in 300 s of recording during cooling, but the very
Empirical evidence about temporal structure
19
same precise pattern was observed 5 times in 300 s during the recovery period. Note that between the first and the last occurrences of the pattern more than 90 minutes had passed. Another example (Figure 9) illustrates that significant patterns observed during cortical cooling might be completely absent prior to and after cortical deactivation. This case
Figure 7 . Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.4 mm. The rasters are aligned by displaying the first spike in the pattern <3,1,2; 285,474> at time 0. The pattern is repeated 12 times with a jitter of ±2 ms. The abscissa full scale is 1000 ms.
shows a pattern of 3 spikes repeating 9 times (with a jitter of ±2ms) starting by a spike of cell 8, followed by a spike of cell 5 180 ms later, then after 280 ms by a spike of cell 6. Note that only cells 5 and 6 were recorded from the same electrode. This second example also illustrates the tendency observed in firing patterns during cortical cooling to exhibit a higher complexity than patterns observed during control and recovery conditions (Villa and Abeles, 1990).
5. PATTERNS IN RELATION TO BEHAVIOURAL STATES Although the previous examples have provided solid evidence for the existence of precise temporal structure in neural activity, this does not yet establish its function as a temporal code. What is needed is some demonstration that Spatiotemporal firing patterns occur
Time and the brain
20
reliably under particular behavioural conditions, and are related to cognitive activity. The following example shows experimental results recorded in the primate inferotemporal cortex, which is considered one of the last in an ascending hierarchy of cortical processing stages that begins in the striate cortex. Studies with behaving monkeys have led to the conclusion that the inferotemporal cortex is not only a higher order stimulusanalyzer, but is also involved in retention of visual information, and is modulated by attention (Fuster, 1990). In the data presented here, the monkey performed a visual delayed matching-to-sample task (Villa and Fuster, 1992). A trial in one such task consisted of the following:
Figure 8 . Spatiotemporal pattern of spikes detected in the rat auditory thalamus 9 times prior to and 5 times after recovery from cooling of the auditory cortex, but absent during cortical deactivation. The firing pattern was formed by three spikes, starting by spike 7, then after 359 ms spike 11 and then after 21 ms spike 11 again. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated by thick ticks in the raster displays (right panels) aligned by displaying the first spike in the pattern at time 0. Note that spikes 7 and 11 were recorded from different electrodes. The analyzed records corresponded to 400 s of spontaneous activity
Empirical evidence about temporal structure
21
during control condition, 300 s during cortical cooling and 300 s in the recovery condition. Each block of 100 s is represented by a thick mark along the time axis of the experimental protocol, and the dots indicate the start event of the pattern.
(1) sample stimulus (a colour or a geometric figure on a luminescent button with a diameter of 25 mm); (2) 10–20 seconds of delay (retention period); and (3) choice of one stimulus (among 2 or 4) that matched the sample by a given feature (colour or symbol). Correct responses were reinforced with a squirt of juice. About 40% of all single units recorded in the inferotemporal cortex showed a sustained elevation of firing during the delay after the sample stimulus (which was retained in short-term memory for performance of the task). This sample was further subdivided into selective units (14% of all units) showing an activation during the delay which was selectively and significantly higher after one particular tested stimulus, and non-selective units (26%) which were unspecifically activated by all tested stimuli. The distribution of firing rates for both
Figure 9 . Triplet of spikes detected 9 times in the rat auditory thalamus, exclusively during cortical deactivation, starting by spike 8, then after 180 ms spike 5 and then after 280 ms spike 6. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated as in Figure 8. Note that only spikes 5 and 6
Time and the brain
22
were recorded from the same electrode. The analyzed records corresponded to 300 s of spontaneous activity during control condition, 500 s during cortical cooling and 300 s in the recovery condition.
groups in spontaneous discharge was skewed and broad, but the median firing rate for non-selective units (4.4 spikes/s) was higher than that for selective units (2.5 spikes/s). In this study, only one spike train at a time was recorded and thus the temporal firing patterns were formed by multiple spikes of one neurone. Figure 10 illustrates two patterns formed by three spikes observed in two different neurones. The triplet of cell 14 repeated 6 times during the intertrial periods, which did not reach statistical significance, whereas its repetitions were significant (19 times out of 68 trials) during the delay period. The accuracy of the pattern was ±3 ms. The triplet of cell 47 repeated 7 and 19 times during the intertrial and delay periods, respectively, out of 75 trials. One pattern involved long intervals, of the order of hundreds of ms, whereas the other pattern involved relatively short intervals. However, the striking similarity between these two patterns is that both patterns appeared to a significant extent during the retention period of the sample stimulus, and that only one occurrence of the pattern (seldom two) was observed per trial (although in only 20–25% of all trials).
Figure 10 . Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of two non-selective units recorded in the
Empirical evidence about temporal structure
23
primate inferotemporal cortex during intertrial intervals (left panels) and during the delay (right panels). The measuring time was equal in both conditions. Cell 14 shows a pattern of 3 spikes repeating 6 and 19 times during the intertrial and delay periods, respectively: The second spike of the triplet occurred 165 ms after the pattern start and the third 195 ms after the second spike. Cell 47 shows a triplet repeating 7 and 19 times during the intertrial and delay periods, respectively. The first spike of cell 47 occurred at time 0 (aligning the rasters), the second 42 ms later, and the third 48 ms later. In these rasters the pattern is allowed an accuracy of ±3 ms.
In spike trains recorded in the inferotemporal cortex it was difficult to observe firing patterns composed of 4–6 spikes recurring more than twice within a record. Figure 11 shows an example of such a complex pattern formed by spikes of a selective unit during intertrial periods, and during the delay after the preferred stimulus (the only periods characterized by an increase in firing rate). The pattern was composed of 4 spikes: a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. This quadruplet repeated 6 times during the intertrial periods. The four subpatterns corresponding to the triplets included in the original quadruplet are also displayed in Figure 11. The incidence of the highly organized pattern decreased when the cell was activated by its preferred stimulus, irrespective of an increase in the rate of discharges during the delay. However, it is important to note that one specific subpattern, i.e. was detected at a statistically-significant level during the intertrial and delay periods. This triplet occurred 5 and 9 times during the intertrial and delay periods, respectively. The example of Figure 11 is important, because it illustrates several additional characteristics of precisely time-structured activity in the cortex. Firstly, it illustrates the contribution of subpatterns to a higher order pattern, thus suggesting the robustness of a complex pattern even if at some occurrences a spike is missing. Secondly, it illustrates that subpatterns may express information processing different from that involving the superpattern (as shown by the disappearance of the event at a delay of 95 ms in Figure 11, and at the same time a relative increase in occurrences of the triplet ). On a more general level, the examples of Figures 10–11 raise an interesting problem. Notably, the incidence of firing patterns tended to be inversely related to the selectivity of the units for the stimulus that the animal held in short-term memory (Villa and Fuster, 1992). During the delay periods, and in comparison with intertrial periods, selective units showed a decrease of firing patterns, whereas non-selective units showed an increase. On the one hand, the more patterned activity of non-selective units may reflect their involvement in wide networks representing multiple and general attributes of that particular stimulus held in memory. On the other hand the term “selective unit” shows the long-held belief and
Time and the brain
24
Figure 11 . Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of a selective unit (cell 61) recorded in the primate inferotemporal cortex during intertrial intervals (25 lines) and during the delay (17 lines). The pattern was composed of 4 spikes, a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. The occurrences of the exact subpatterns formed by 3 out of 4 spikes of the original pattern are also displayed on this raster. The quadruplet is allowed an accuracy of ±3 ms, whereas the subpatterns ±1 ms. Note that during the delay the average firing rate of the cell increases, while the quadruplet pattern disappears, although one particular subpattern remains visible.
prejudicial influence of the rate code on our assumption of what is “selective”. In fact, selective units were defined on the basis of an increased firing rate during the delay period that followed a particular stimulus of the tested set. It is not possible to rule out the possibility that in another paradigm or experimental condition the “selective” units would become “non-selective” and vice versa. Thus, one should bear in mind that the term “selective” should be restricted to a particular experimental situation. However, it is interesting to note that both populations of cells increased their rate of discharges during the retention period, but only the non-selective units increased the patterned activity. An attractive interpretation of these data could be the following: to retain all stimulus characteristics, some transfer of information occurs between the selective units, which encode the specific stimulus feature being retained, and the non-selective units, which are part of a wider network encoding contextual attributes of the stimulus that may be used for binding with other features. This model supports the hypothesis of conversion from asynchronous activity to synchronous activity (and back) through diverging/converging
Empirical evidence about temporal structure
25
connections, as proposed by Abeles (1982, 1991). One important limitation of the PDA method is its application to patterns significant by themselves, as explained above. An additional limitation is the use of a fixed jitter. A new method, called Pattern Grouping Algorithm (PGA), has recently been developed, aiming to identify and evaluate the statistical significance of temporal patterns of spikes formed by three or more different events (Villa and Tetko, 1999; Tetko and Villa, 1999). In particular, this new algorithm is able to recognize patterns of spikes with slight differences in spike timing, and to cluster them as one single group of patterns. Firstly, PGA considers those patterns detected by the classic pattern detection algorithm as templates for search of similar patterns in the spike train data. There are three adjustable parameters in PGA: (i) the maximal duration of the pattern, measured as a delay between the first and the last spike in the sequence of spikes (i.e. the window duration); (ii) the level of significance to be used for detection of significant groups; and (iii) the maximum allowed jitter for timing accuracy of time delays in a pattern group. Based on the idea developed in the template search method (Dayhoff and Gerstein, 1983a) and on selection of optimal jitter, PGA is able to minimize the jitters allowed for each spike in the template independently, according to the actual distribution of spikes. Thus, the probability of detection of significant patterns of spikes is optimized. PGA is not restricted to the analysis of patterns formed by only one cell, and may be applied to the identification of precise temporal patterns regardless of their complexity and number of different cells in a pattern. Therefore, PGA can estimate the significance of each spatiotemporal pattern of firing. The method (described in detail in Tetko and Villa, 1999) consists of an independent estimation by the previously developed PDA, FPD and JTH methods, that have been updated in order to consider the variable jitters. Only those patterns that are significant by all methods are considered for the final step of PGA. Eventually, the final output of PGA consists of a central template pattern with variable jitters corresponding to previously detected patterns grouped into one class. Several tests and validation analyses have demonstrated that PGA can detect true patterns and avoid the detection of spurious patterns, even in the presence of several non-stationarities in the spike trains. These non-stationarities are likely to occur in recordings from behaving and freely-moving animals. Even so, it must be kept in mind that for most of the functional systems the available data demonstrate the presence of precisely timed neural activity whose relationship with behaviour is not always evident. To prove that these temporal patterns do indeed serve for response selection, data are required that establish causal relations between the occurrence of precise temporal relationships and cognitive or motor processes. The next examples will provide solid evidence about these processes. Spike trains from up to 15 single neurones were recorded simultaneously in the temporal cortex of freely moving rats, while animals waited for acoustic cues in a Go/NoGo task (Villa et al., 1999c). The auditory stimulus contained two types of information: pitch (“high” or “low”), and position (“left” or “right”). During the first phase of training, the location was kept constant and the rats had to discriminate between tones of high and low pitch, with one signalling “Go” and the other “NoGo”. Reinforcement was given only after correct “Go” trials. In the second phase, pairs of tones were delivered (one tone from each location), with four possible tone-position combinations. We have shown that, despite the lack of reward for correct NoGo
Time and the brain
26
performance, and the lack of painful punishment for incorrect trials, all rats learn the task to a satisfactory level (performance in the range 70–90%). The paradigm allows analysis of activity during two functional epochs: the “wait” period prior to the stimulus, during which the animal must remain at the back of the cage, and restrain responding despite an increasing probability of imminent stimulus delivery; and the “processing” time, from stimulus delivery until beginning of movement, during which the information content of the signal is processed and motor output organized. Only the “wait” period is considered here for analysis. Significant patterns were detected using PGA from a data-set of 13 hours of recording, involving over one million spikes (Villa et al., 1998a, 1999a). “Go” responses resulted both from correct movements to the feeder in response to low pitch sound at the right speaker and from incorrect movements when a high pitch was delivered to this speaker. Conversely, “NoGo” responses could be correct (failure to move in response to the high pitch from the right speaker) or incorrect (failure to move in response to low pitch from right speaker). Of particular interest were the patterns that were significantly associated with the type of response the animal made later, independent of whether the response was that prompted by the cue. Figure 12 illustrates one such pattern, a triplet formed by the spikes of two cells recorded from distinct electrodes placed in the temporal cortex within the same hemisphere. Note that there were more than twice as many trials (n=27) with patterns that were followed by “Go” responses as there were trials (n=12) with patterns followed by “NoGo” responses, despite the fact that the total number of “Go” and “NoGo” responses was nearly the same (n=283 and 269, respectively). An alternative display of the spikes belonging to the triplets is shown in Figure 12b. In this illustration, the rasters are aligned to an external event (the stimulus onset) instead of being aligned to the start of the pattern. Of these “behaviour-predicting” patterns, half were associated with an enhanced tendency to “Go” in response to the stimulus, and for about 20% of these patterns, trials including the pattern were associated with a faster or slower reaction time than those lacking the pattern (Villa et al., 1998a, 1999a). Figure 13 illustrates one such pattern, and the corresponding reaction times. The triplet shown here is formed by three different neurones, two of which were recorded from the same electrode. It is important to notice that the patterns were associated in almost half (22/37) of the “Go” responses and that pattern occurrence during the wait period led to an accelerated reaction time, by 355 ms on average. Although the patterns could start at any time during the waiting period, we
Empirical evidence about temporal structure
27
Figure 12 . (a) Raster display of the activities of two neurones recorded from two different electrodes in the temporal cortex of a freely-moving behaving rat. The rasters are aligned by displaying the first spike in the pattern at time 0. The pattern was: cell 5; 320±3 ms later cell 11, then 342±3 ms later cell 5 again. The data shown include trials recorded on two consecutive days and in which both Go (n=27) and NoGo (n=12) responses occurred, (b) Raster display of the data corresponding only to the Go responses shown in (a) but on a compressed time scale, and aligned to the time of stimulus onset instead of to time of the first spike of the pattern. Spikes involved in generating instances of the pattern during the waiting period (time to the left of the stimulus onset) are displayed as bars instead of ticks. Three spikes constituting one instance of the pattern are picked out by empty circles. Note the stability of this pattern recorded over two consecutive days.
Time and the brain
28
observed that in several cases the time course of pattern appearance was related to the reaction time. Figure 14 illustrates this finding on a cumulative display of reaction time vs. timing of pattern occurrence. The reaction times of the subjects were normalized to 800 ms on average, in order to cumulate eleven examples on the same curves. Notice that the individual timing of each pattern was by itself significantly correlated with the reaction time on a quadratic regression analysis . The left
Figure 13 . Raster display of the activities of three rat cortical neurones participating in a spatiotemporal firing pattern, aligned by the occurrence of the first spike in the pattern. The time of stimulus presentation is marked by horizontal ticks in the lower window. The pattern was formed by spike #12, 867±2 ms later cell #11, and then after 29±1 ms cell #5. The timing of the three spikes forming the pattern is indicated by the arrows below the stimulus raster. Note that this pattern repeated 22 times (p<0.001) before the onset (i.e. during the waiting period) of a stimulus which triggered a Go response (either correct or incorrect). On average, the subject responded significantly faster after stimulus onset if a pattern was detected in the waiting period.
panel of Figure 14 shows that the earlier the pattern occurred in the waiting period, the slower the reaction time. A pattern occurring just before the stimulus presentation elicited a fast reaction time, thus suggesting a kind of matching operation. The right panel of Figure 14 illustrates the opposite case, where the occurrence of the pattern just before the stimulus provoked a slower reaction.
Empirical evidence about temporal structure
29
Figure 14 . Reaction time, measured as the delay between stimulus onset and the first infra-red beam crossing, vs. the timing of patterns before the stimulus onset. The timing occurrence of five different patterns (labelled 1–5) are cumulated on the left panel, showing a faster reaction time if the pattern occurred just before the stimulus. The right panel shows six other patterns (A-F) cumulated on the same curve and showing a slower reaction time when the pattern occurred just before the stimulus. For all patterns the reaction time of the subjects was normalized to 800 ms on average in order to cumulate the data.
The remaining behaviour-predicting patterns were associated with an enhanced “NoGo” tendency. Overall mean rates of activity did not vary significantly across trials, and the patterns could not be related to the prompting auditory cues, since these occurred later and were chosen randomly. Hence, these data demonstrate for the first time that particular spatiotemporal patterns predict future behavioural responses. The information coded by means of precise spike timings has predictive value, and can therefore allow forecasts of the animal’s future response, regardless of whether or not these responses are ones demanded by subsequent unpredictable cues. The impact of the presence of specific spatiotemporal patterns on the responses of the animal suggests that such patterns participate in set-related activities that prime a particular response, and thus bias the subject’s response choice, regardless of appropriateness. Alternative, and not mutually exclusive hypotheses, can be proposed, whereas the underlying dynamics can be only a matter of conjecture. Firstly, patterns may be considered as parts of “templates” generated by preparatory processes activated in order to extract the useful information from forthcoming sensory cues (auditory “objects” in cognitive contexts requiring difficult discriminations). Templates would be activated selectively prior to the stimulus—the “Go” or the “NoGo” sound—and allow faster match-mismatch recognition; consequently, they are comparable to the “preconceived ideas” that influence human perception. “Go patterns”, for instance, would reflect states wherein signals are more likely to be interpreted as “Go” prompts. Such fine structures of brain activity, coined in more precise terms than overall mean rates, could be related to thalamo-cortical information processing. Several lines of evidence indicate that the auditory cortex exerts
Time and the brain
30
a dynamic control over the functional segregation of signals transmitted via the thalamocortical loops, and activity in such loops may generate precisely repeating interspike intervals (Villa et al., 1991; McCormick and Bal, 1994; Miller, 1996; Tetko and Villa, 1997b).
6. PATTERNS IN RELATION TO MOTOR PROGRAMMING The next examples investigate a second hypothesis which cannot be ruled out, that patterns may be parts of motor programs, each pre-prepared for active performance of either a “Go” or a “NoGo” response. The activity of multiple neurones, recorded simultaneously from the monkey primary motor cortex during the performance of a delayed-pointing task, was analyzed by PDA and PGA (Villa et al., 1995, 1998b). Each trial was initiated by the monkey pressing a lever. Following a fixed interval (predelay period), a preparatory signal and then, after a second delay, an imperative (response) signal were presented (Figure 15a). This protocol was designed to manipulate the temporal aspects of preparation for visually guided pointing movements by varying the interval between these two signals. On any one trial, this interval (the preparatory period) could take one of 4 values (600, 900, 1200, or 1550 msec). Thus, during trials with the longest delay, there were three earlier times at which a response signal could have occurred, and which might have been expected by the animal. These moments are referred to as expected signal times. Spatiotemporal firing patterns were indeed detected preferentially in relation to the expectancy of behaviourally relevant events. The timing of several of these spatiotemporal patterns was found to be related to the preparatory signal. Figure 15b shows such an example, in which a pattern of three spikes occurred 20 times, started by neurone 4, then after 113±1.0 ms neurone 6 and then after
Empirical evidence about temporal structure
31
Figure 15 . (a) Diagram of the time course of a trial in the delayed-pointing task. Each trial was initiated by the monkey pressing a lever (“start”). Following a fixed interval (pre-delay period), a preparatory signal (PS) and then, after a second delay, an imperative (response) signal (RS) were presented. The preparatory periods were randomly selected out of four possible fixed durations (600,900, 1200, and 1500 ms). For the longest preparatory period, three expected signal delays (ESI at 600 ms, ES2 at 900 ms, and ES3 at 1200 ms) are identified, (b) Raster displays of spiking activities of a group of four neurones recorded in the primate motor cortex. In the last row of the panel, the markers indicate the time of occurrence of the corresponding events. The rasters are aligned by the first spike in the pattern. The scale bar corresponds to 500 ms. The pattern was: cell 4; 113±1.0 ms later cell 6, and then 294±2.0 ms later cell 6 again. In
Time and the brain
32
three trials this pattern occurred twice. These trials can be recognized by the labels 1 and 2 near the corresponding PS. Note that the majority of patterns precede the onset of the preparatory signal. PS: preparatory signal; RS: response signal; ES: expected occurrence of RS.
294±2.0 ms neurone 6 again. In 15 out of 20 occurrences this firing pattern ended a few hundreds of ms before the onset of the preparatory signal. This result suggests that neurones in the primary motor cortex can undergo temporally patterned activity in relation to the expectancy of signal occurrence.
Empirical evidence about temporal structure
33
Figure 16 . Raster displays of spiking activities a group of four neurones (different from Figure 15) recorded in the primate motor cortex. The rasters are aligned by the last spike in the pattern. The pattern was: cell 4 and cell 5 firing almost synchronously (within ± 1.0 ms); 55±3.5 ms later cell 5, and 223±4.5 ms later cell 4. All types of trials were pooled and the patterns were equally distributed among expected signal times (ES1: n=4; ES2: n=3) and response signals (RS1: n=3; RS2: n=2; RS3: n=1; RS4: n=1). Note that when the pattern was preceding an expected signal (labeled by empty solids) the time of occurrence of the corresponding response signal is also marked. The response signals are labeled by black dots. The question mark indicates the brain process associated with the expected signal.
Of particular interest are several patterns that occurred preferentially prior to the preparatory signal and/or expected signal times. A complex pattern formed by four spikes was observed fourteen times (each time in a different trial) and always ended over a
Time and the brain
34
narrow interval (20-230 ms) before the actual or expected signal time (Figure 16). For example, in the case of the longest preparatory period (i.e. 1500 ms) a response signal could have been presented at any of four moments. Since trials were presented at random, the animal could not know in advance the duration of PP. The timing of the pattern shown in Figure 16, which predicts with reasonable accuracy the times of possible responses signals, suggests that the occurrences of the pattern correspond to some brain process associated with expectancy of upcoming signals. Furthermore, patterned activity might be initiated by synchronous volleys of spike discharges, as shown by the two quasi-synchronous events shown in Figure 16, and ended before the occurrence of the response signal. This pattern is interesting also because it exemplifies a typical result of PGA, since the optimal time jitters were all unequal. An alternative display to show the relationship between the preparatory period and the pattern occurrence is based on a raster plot of the spike trains triggered by onset of the PS. In addition, this raster should be restricted to those trials characterized by an occurrence of a specific pattern during the preparatory period. Figure 17a shows an example of a brief pattern (lasting less than 50 ms) formed by three spikes from two different neurones: the first two spikes are synchronous, and the third occurs 48 ms later. The histogram of the 28 occurrences of the end-times of the pattern (Figure 17b) demonstrates that this distribution is not uniform. Half of the patterns (14/28) ended within 100 ms before the actual or expected end of the preparatory period. Both neurones involved in the pattern showed only a slight modulation in firing rate during the preparatory period compared to the predelay period, as shown in the peri-stimulus time histograms for the longest preparatory periods (Figure 17c). However, the most interesting feature of this pattern is its predictive value for the subsequent behaviour. The total response time, i.e. the interval between signal onset and touching the target, was 512±12 ms (mean and SEM) in the 22 trials containing a spiking pattern, whereas in the 101 trials without pattern, response time was 569±13 ms. In this particular example, reaction time, i.e. the interval between signal onset and the movement onset, was significantly faster as well (248±9 vs. 297±9 ms).
7. SUMMARY OF EMPIRICAL EXAMPLES In summary, experimental results of the coordinated timing of spikes, and the synchronous firing or network synchronization have provided several conceptual frameworks in which temporal coding in spiking activity is important. Temporal codes do not preclude a rate code being superimposed on it simultaneously. The presence of sensory stimuli or motor output often correlates with single-unit firing rates averaged over many trials and/or with asynchronous compound activity of a neuronal population (Riehle and Requin, 1995; Georgopoulos, 1995). In addition, it has been well established that hippocampal place cells signal by their rates whether the animal is within the place field, independently of whether the animal is entering or leaving that field (Wilson and McNaughton, 1993). However, as such studies increase in complexity, a simple rate code may be rendered inadequate as a predictor of behaviour. In this section a number of
Empirical evidence about temporal structure
35
bodies of experimental evidence for the existence of precise neuronal timing has been provided. Spatiotemporal patterns do in fact occur spontaneously, but occur recurrently, and in a more stable way, in particular temporal relationships to stimuli, or movement, and during the retention period of a delayed response task; and under particular experimental conditions “behaviour-predicting” patterns have been detected. It has been proposed (Ferster and Spruston, 1995) that clear proof for a
Figure 17 . (a) Raster display of the spiking activities of two neurones recorded simultaneously in the primate motor cortex. The spike trains are aligned on the onset of the preparatory signal (PS). Only trials characterized by the presence of a pattern are displayed. The pattern was: cell 1 and cell 6 firing almost synchronously (within ±1.0 ms) and 48±5.0 ms later cell 1 again. In one trial the three spikes participating in the pattern are indicated by circles. In the second row (cell 1) thick ticks indicate the last spiking event of each pattern. In
Time and the brain
36
the third row the ticks indicate the response stimuli (RS) associated with each pattern (i.e., RS1, RS2, RS3, and RS4 after 600, 900, 1200, and 1500 ms, respectively). Note that pattern occurrences were distributed in 22 trials, i.e. in few cases one pattern occurred in relation to the expected stimulus (ES) and one in relation to the response stimulus in the same trial, (b) Histogram of the end-times of pattern occurrences. The bin size is 100 ms. Note that the last spike of this pattern preceded either an expected stimulus (12 times, empty bins) or RS (16 times, filled bins), (c) Peri-stimulus time histograms of the activity of cells 1 and 6 for 31 trials characterized by preparatory periods lasting 1500 ms (PP4). The abscissa is time (ms) and the ordinate is scaled in rate units (spikes/s). The vertical full scale for cell 1 and cell 6 is 27 and 45 spikes/s, respectively. The curve is convoluted by a Gaussian shape bin of 22 ms.
temporal code would require that distinct stimuli could reliably produce different temporal spike patterns. However, this restrictive view, to assimilate the temporal code to a kind of “Morse alphabet”, cannot account for the complex dynamics existing in the brain described in the next section.
8. CHAOS AND ATTRACTORS An alternative way to investigate neural dynamics in spike trains assumes that the whole time series of spike occurrences is an expression of some fundamental process governing the activity of the neurones being recorded. When a specific input pattern activates a cell assembly, it is assumed that the neurones will be activated either in a timely organized mode or in an asynchronous mode. Then, a mode of activity defines how information is processed within a neural network and how it is associated with the output pattern of activity that is generated. This assumption does not exclude the possibility that the same cell would respond in an opposite mode to other stimuli. Furthermore, the same neural network may subserve several modes of activity through modulation of its connectivity (for instance during learning or pathological processes) or by modulation of its excitability, e.g. by modulation of the resting potential or of the synaptic time constants. In this framework the state of the neural network is defined by a set of parameters characterizing the neural network at a certain time. Then, the state of the network at any given time is represented by the values of these parameters, and a network state is fully determined if all parameters are known for each neurone. A state, in general, changes over time. Neurones are embedded in an inextricably large neural networks whose coordinated activities determine the timing of action potentials. Ab absurdo if we were able to set the same initial conditions for all elements of the neural networks we would obtain the same spike trains. Such a dynamical system as a whole is said to be deterministic, as it is possible to predict precisely the evolution of the system in time if one knows exactly the initial conditions. However, a slight change or incorrect measurement in the initial conditions results in a seemingly unpredictable evolution of
Empirical evidence about temporal structure
37
the system. Such sensitive dependence on initial conditions is a hallmark of chaotic systems. A process is defined by the passage of a state over time. The simplest definition of a chaotic process, or simply chaos, is that it is completely deterministic at each step of its temporal evolution, yet over the long term, its response is unpredictable. An equivalent definition of a process is a “path” over time, or trajectory, in the state space. The attractor is defined by the set of points approached by the trajectory as time increases to infinity. If the evolution in time of the system is described by a trajectory forming a closed loop (also referred to as a periodic orbit), then the system is said to have a limit cycle, also referred to as a periodic attractor. In general, brief initial perturbations applied to any combination of the governing set of parameters move such a system away from the closed loop, but, with passage of time, the trajectory collapses asymptotically to the same attractor. If, on the other hand, the system is deterministic, yet sensitive to small initial perturbations, the trajectory defining its dynamics is an aperiodic orbit, then the system is said to have a chaotic attractor, often referred to as a strange attractor.
9. DYNAMICAL SYSTEM ANALYSIS Let us consider a simple example of a point process, whose dynamics is characterized by discrete steps in time. Let be a time series with K points, where x represents the state of the system. In a dynamical system the subsequent state of the system is determined by its present state (Addison, 1997). The simplest expression would be to consider a map defined by where α is a control parameter. The biological systems, and the brain in particular, are often characterized by feedback mechanisms. The expression known as the logistic map, illustrates a simple dynamical system with a negative non-linear feedback, defined for x [0,l]. It is clear from this expression that the time arrow is non-reversible, because it is always possible for each x i to obtain a value x i+1 but there are two possible x i for each x i+1 . The behaviour of this equation is very interesting. For all values 0≤α≤1 the iterated series decay towards zero. Conversely, for some other values of α , e.g. 1.7 or 2.1, the series converges to a fixed point, equal to 0.52381 and 0.41176 respectively. Conversely, for values α equal to 3.2 and 3.52, the series converge to two alternating fixed points, i.e. and four alternating fixed points, i.e. respectively. The trajectories of the three last examples are periodic orbits, with periods equal to 1,2, and 4, respectively. However, it is possible to find other intersesting cases. With a control parameter equal to 4.0 and an initial condition x 0=0.6 the system tends to decay to zero, but with an initial condition x 0=0.4 the dynamics never produces a repeating sequence of states. This aperiodic behaviour is different from randomness, or white noise, because an iterated value x i can occur only once in the series, otherwise, due to the deterministic dynamics of the system, the next value must be also a repetition and so on for all subsequent values. In formal terms, let us define a function ƒ which describes how a state x is mapped into the space of state. Such a function is the probability function f of the mode of activity of
Time and the brain
38
the network. If the function is set by a control parameter µ we can write . A dynamical system x’ is a subset of the space of state and can be obtained by taking the gradient of the probability function with respect to the state variable, that is . Mathematically speaking, the space of state is a finite dimensional smooth manifold, assuming that ƒ is continuously differentiable and the system has a finite number of degrees of freedom (Smale, 1967). For periodic activity the set of all possible perturbations define the inset of the attractor or its basin of attraction. In case of the logistic map, for a=3.2 all initial conditions in the interval end up approaching the period 2 attractor. This interval is known as the basin of attraction for the period 2 attractor, whereas the value x 0=0.6875 is a fixed point. If the activity is generated by chaotic attractors, whose trajectories are not represented by a limit set either before or after the perturbations, the attracting set may be viewed through the geometry of the topological manifold in which the trajectories mix. The function ƒ corresponding to the logistic map, is the parabolic curve containing all the possible solutions for x . This function belongs to the single humped map functions which are smooth curves with single maxima (in here the single maximum is at x =0.5). It is easy to conceive that in large neural networks the complexity of the system is such that several attractors may appear, moving in space and time across different areas of the network. Such complex Spatiotemporal activity may be viewed more generally as an attracting state, instead of simply an attractor. In particular, simulation studies demonstrated that a neural circuit activated by the same initial pattern tends to stabilize into a temporally-organized mode or into an asynchronous mode if the excitability of the circuit elements is adjusted to the first order kinetics of the postsynaptic potentials (Villa and Tetko, 1995; Hill and Villa, 1997).
10. DETECTION OF CHAOS IN EXPERIMENTAL DATA SETS It is obvious that it is not possible to know all variables determining brain dynamics, yet the progress made by computational and statistical physics during the two past decades has brought a number of methods allowing one to differentiate between random (i.e. unpredictable) and chaotic (i.e. seemingly unpredictable) time series. It is difficult to prove that biological systems generate chaotic attractors, owing primarily to their shortlived and apparently nonstationary behaviour (Mpitsos, 1989). Several lines of experimental observation suggest that chaotic processes may be generated by brain activity (Babloyantz et al., 1985; Rapp et al., 1985). In particular, chaotic discrete dynamics observed in spike trains allow simple networks of neurones to perform complicated tasks that would require considerably more complex networks to perform if the signals were generated by non-chaotic discrete or continuous processes (Mpitsos and Burton, 1992). Among several methods available for classification of discrete time series the Grassberger-Procaccia algorithm (Grassberger and Procaccia, 1983) has been widely applied to theoretical and experimental cases. This algorithm is based on the following
Empirical evidence about temporal structure
39
idea. Let { x 1 ......x k } be a time series corresponding to interspike interval sequences generated by K +1 number of spikes (Cutler and Kaplan, 1997). Delay coordinates { y 1 ....y n } in a d -dimensional embedding space are defined by setting y 1 = ( x 1 .....x d ), y and Y≡ { y 1 ....y N } is the 2 = { x 2 ....x d+1),....y N ={ x N ....x K ), where set of delay vectors. One fixes an embedsding space with dimension d and computes the values of the so-called correlation integrals, corresponding to the average of the number of points falling inside an hypersphere (with arbitrary small radius) around each point of the original time series. These quantities are plotted against the radius for several values d of the dimension of the embedding space (Figure 18). When the curves are definitely parallel, the system is said to be deterministic and their slope defines the correlation dimension, often referred to as D 2 , of the attractor underlying the chaotic structure of the system (Figure 18b). The value of d at which the convergence starts is the embedding dimension. In practical applications, the correlation dimension D 2 must be calculated in a meaningful range of values of the radius. This interval, denoted as the scaling region, must have an amplitude of at least a decade (Eckmann and Ruelle, 1992). Moreover, the number K of points defining the time series should be significantly large in order to get reliable estimates of the correlation dimension. In particular, for typical experimental time series with 800≤ K≤10,000 points the embedding spaces should have a realistic dimension d≤6. Higher dimensional systems cannot be reliably detected over relatively
Figure 18 . Correlation integrals of two spike trains, (a) No attractor is detected in the embedding space of dimensions d up to d=8 for this spike train (n=1094 spikes), (b) This spike train (n=876 spikes) is characterized by a correlation dimension D 2=2.1 embedded in a 4-dimensional space. The thick black bars indicate the scaling region. Note that for time series with less than 900 points the computation of the correlation integrals is stopped at an embedding space of dimension d=7.
short time series and their dynamics cannot be distinguished from random noise Rapp et al., 1993). Furthermore, the embedding dimension of the interspike interval attractor
Time and the brain
40
reconstruction must be greater than or equal to the correlation dimension of the attractor in order to obtain a meaningful result (Castro and Sauer, 1997). Misapplications of the Grassberger-Procaccia algorithm led, in the 1980s, to unreliable estimations of deterministic structures for time series (see Rapp et al., 1993; Theiler and Rapp, 1996, for details). Nonetheless, its careful use in recent years has indicated that numerous neurones (up to 25% in some samples) recorded in the substantia nigra and thalamus of anaesthetized rats can be characterized by deterministic dynamics (Celletti and Villa, 1996; Celletti et al., 1997,; Celletti et al., 1999). The analysis of neuronal activity recorded in the primate motor cortex during the delayed-pointing task (see Section §6) revealed the existence of chaotic dynamics in and across paired spike trains (Villa et al., 1998b). The attractors were generally embedded in a space of low dimension, ranging from 2 to 5, and were characterized by a correlation dimension between 0.15 and 3.34 (Figure 19). In the intertrial periods, the dynamical system tended to be characterized by a correlation dimension near 1, and by an attractor embedded in a 4-dimensional space. Conversely, during the preparatory period the geometric scaling properties of the neural dynamics were broadly distributed in the sense that no clear clustering around a specific value of the correlation dimension could be observed (Figure 19). This might suggest the disruption of a common state of activity by the ongoing motor preparation. Attractors observed during the preparatory period were not related to the activity associated with anticipation of the response signal, but rather to preparatory motor processes. Thus, the question is raised whether such dynamics bear some message,
Figure 19 . Neuronal activity recorded in the primate motor cortex (see Figure 15a). Scattergram and marginal distributions of the embedding and
Empirical evidence about temporal structure
41
correlation dimensions of the attractors determined by the GrassbergerProcaccia algorithm in 34 spike trains. The filled symbols and the black bars refer to observations during the preparatory period whereas empty circles and light gray bars refer to the control period.
or may represent a “side effect” of some global self-organizing dynamics within the cerebral cortex. Indeed, deterministic dynamics in the cortex may be interpreted as a collective phenomenon arising from the expression of local attractor dynamics, sustained by a synaptic matrix formed in the process of learning (Herrmann et al., 1993; Amit et al., 1994; Amit, 1995; Amit et al., 1997). However, with respect to information processing in the brain, the body of observations derived from the application of these methods strongly depends upon the dynamics of the underlying neural circuitry. The detection of deterministic dynamics necessarily requires the stability of the generating processes over a relatively long period of time. Thus, significant deterministic dynamics across spike trains may reveal neuronal interactions involved in long-term processes (e.g. memory traces, learned motor programs), whereas the previously cited methods could reveal short-term operational processes at attentional or feature-binding level. Therefore, pattern detection algorithms and dynamical systems analysis may both be characterized by repetitive dynamics of neuronal activity. The question can be raised if there are some relations between these two approaches.
11. SPATIOTEMPORAL FIRING PATTERNS AND CHAOTIC ATTRACTORS PGA was applied to two records from a large scale simulated network described elsewhere (Amit and Brunel, 1997) with maximum allowed jitter equal to 7 ms. The network was composed of 15,000 integrate-and-fire cells, from which 12,000 were excitatory and 3,000 were inhibitory. Each cell in the network had a probability 0.15 of a direct contact with other cells and received, on average, 1,800 excitatory and 450 inhibitory synapses from neurones belonging to the network and 1,800 excitatory synapses from outside the network. In the first record, referred to as “random”, the strength of neuronal connections was initially set as random (Amit and Brunel, 1997) and was not modified. Thirty units were selected by chance, and their activity was recorded during 180 seconds with an accuracy of 1 ms. The activity in the “random” record corresponded to sustained spontaneous rates, in the range 0.1–11.7 spikes/s (mean=4.0 spikes/s and median=3.1 spikes/s). In the second record, referred to as “Hebbian”, the weights of specific populations of synapses were modified according to some type of learning described in detail elsewhere (Amit and Brunel, 1997). The activity of the same set of units was recorded following a stimulation correlated with the learned stimulus. Discharge rates of cells were increased after the Hebbian learning (mean and median equal to 6.5 and 4.2 spikes/s, respectively). No firing pattern was detected in the “random” record by PGA, whereas most cells (20/30 of this sample) formed patterns in the “Hebbian” record. Note that all detected patterns were formed by one cell. The firing rates (mean and median equal to 7.1 and 5.3
Time and the brain
42
spikes/s, respectively) of cells involved in significant group patterns were higher than the rates (mean and median equal to 5.3 and 3.9 spikes/s, respectively) of cells that never formed a temporal pattern. However, the firing rate or the shape of the autocorrelogram were not predictive of the presence of patterns. Most of the autocorrelograms were almost flat and indicated that the spike trains deviated only slightly from a Poisson renewal point process (Figure 20a). In this example, cell 16 was characterized by a firing rate of 17.6 spikes/s but no significant temporal pattern was detected. Cell 17 fired at a similar rate, 17.2 spikes/s, and its autocorrelogram was similar to that of cell 16 (Figure 20a), but 238 patterns of complexity 3–5 were detected. Figure 20b illustrates a significant pattern formed by 5 events repeating 10 times, i.e. <17, 17, 17, 17, 17; 31±2.0, 168±1.5, 186±3.5, 231±2.0.>. In this example the event at delay 186 ms from the pattern onset was detected only at the maximum allowed variability, whereas the other four events formed a significant subpattern at a much greater precision. The relation between spatiotemporal patterns of firing and attractors can be illustrated further by the following example. Let us consider a classical dynamical system, the 2dimensional Hénon mapping (Hénon, 1976). This map is characterized by chaotic behaviour within a certain range of input parameters. With a particular set of such initial parameters it is possible to generate a discrete time series, scaled to a base frequency of 3 spikes/s, thus simulating a possible cortical spike train recording (Tetko and Villa, 1997c). One thousand spikes were generated, corresponding approximately to 330 seconds of recording time. Figure 21a illustrates an example of a spike train generated in this way. In addition to the raster display the return map provides a useful tool to determine visually the presence of a structure. The X-Y plot of the return map is constructed by taking the value of an interval between two successive spikes as
Empirical evidence about temporal structure
43
Figure 20 . High complexity pattern elicited by an attractor mode of activity in a randomly connected network after “Hebbian” learning, (a) Autocorrelograms of three cells recorded simultaneously. The abscissa is lag (in ms) and the ordinate is scaled in rate units (spikes/s). The curves are smoothed by a 10 ms Gaussian shaped bin; the dashed lines indicate the 99% confidence level assuming a Poisson distribution. Note that the shapes of the curves are similar for the three cells, but cell #16 produced no significant patterns, (b) Ten repetitions of a pattern of complexity five, formed by repeated events of cell #17, are displayed as rasters aligned on pattern start. Note that the fourth event, at a delay of 186 ms, was detected on the limit of the maximum allowed accuracy (±3.5 ms).
X-coordinate and the immediately next interspike interval as Y-coordinate. Even if some
Time and the brain
44
regularity may be detected visually on the inspection of the raster display (Figure 21a, left panel), the geometric structure of the return map (Figure 21a, right panel) revealed deterministic dynamics more clearly. In particular, notice the values of the intervals in the range 190–260 ms. The dynamical systems can enter a chaotic regime corresponding to small perturbations of some parameters. The counterpart is that a chaotic regime can be perturbed to
Figure 21 . Raster display and return map of a spike train either generated directly following the Hénon mapping (a) or after deletion of 30% of the spikes and addition of 30% dynamical noise (b). The values of successive intervals In-1 and In are expressed in ms.
some extent, but remains chaotic because of its attractor structure. Let us consider the Hénon spike train perturbed by random deletion of some spikes, and with insertion of new spikes generated according to a uniform distribution on the actual interval. An example of such noisy Hénon spike train is illustrated on Figure 21b. The usual algorithmic methods used to determine if such noisy time series were deterministic may fail to demonstrate any significant structure in the dynamics (Tetko and Villa, 1997c) although in the present example the visual inspection of the return map suggests some regularities (Figure 21b, right panel). It is important to note that chaotic dynamics found in neurophysiological spike trains may be low-dimensional (Celletti and Villa, 1996; Celletti et al., 1998; Villa et al., 1998b), but still with attractors embedded in spaces of dimensions 3–5 whose observation is difficult on a 2-dimensional projection. The
Empirical evidence about temporal structure
45
application of pattern detection algorithms to noisy spike trains generated by attractor dynamics may indeed reveal the presence of recurrent temporal patterns. A typical example is illustrated by Figure 22a, showing a triplet characterized by spike intervals equal to 210 and 191 ms, that fall in the range of the original attractor (Figure 21a). An additional example illustrates an effect due to the noise in the pattern detection results of this type of data-set. The occurrence of 12 triplets <1, 1, 1; 60±4.0, 204±1.5>, characterized by the second event of the pattern having a much higher variability than the third, are shown in Figure 22b. In fact the delay of the third spike is equal to 204 ms, a value included in the range of the original attractor dynamics. Conversely the second event is contributed either by chance or by dynamical noise. In fact, an increase in firing rate loosely time-locked to the onset of the pattern can be observed at delays of approximately 448 and 1440 ms, thus suggesting a non-random origin. However, this example is interesting because it shows that events belonging to the same pattern may have been generated by different mechanisms, so that the temporal pattern might represent a signpost of a sort of binding between those mechanisms. Similar conclusions were discussed from experimental observations described above (Figure 11). The time series generated by chaotic attractors can produce patterns of spikes that recur significantly more often than expected by chance. This finding raises the question whether significant patterns of spikes detected in experimental data are the product of attractor behaviour of neural networks, or if they are produced solely by synfire chains. As mentioned above, the comparison between dynamics of primate motor cortical cells recorded during the period preceding the preparatory signal and during the preparatory period showed important differences of the attractors characteristics (Villa et al., 1998b). The intertrial period corresponds to an interval during which the monkey is quiet and waits for the occurrence of the preparatory signal. In such behavioural conditions, it is
Figure 22 . Raster display, aligned by displaying the first spike in the pattern at
Time and the brain
46
time 0, of the activity of a simulated neurone whose dynamics was generated according to the noisy Hénon mapping (Figure 21b). (a) The pattern repeated 5 times, and was composed of 3 spikes: a spike at time 0, the second 210 ms later, and the third spike 191 ms later. Six additional occurrences of the exact subpatterns formed by the two pair of spikes <1,1; 210> and <1,1; 191> are also displayed on this raster. The triplet was detected with a fixed accuracy of ±2 ms. (b) The pattern repeated 12 times and was composed of 3 spikes: a spike at time 0, the second 60±4.0 ms later, and the third spike 144±1.5 ms later. Notice the variable jitter. An increase in firing rate which is loosely time-locked with the pattern start is indicated by the asterisks. The abscissa full scale is 2000 ms.
possible that the primary motor cortex is not yet involved in motor preparation, because the actual salient cognitive activity is the expectancy of the preparatory signal. Then, the firing activities of motor cortical cells may be attracted in the same basin, a kind of “common attractor” as postulated theoretically for spontaneous activity (Villa, 1992; Amit, 1995; Villa and Tetko, 1995; Amit and Brunel, 1997). In the next section a model of neural dynamics is presented, based on formal definitions and on the above considerations of chaotic activity in spike trains.
Empirical evidence about temporal structure
47
12. A MODEL OF NEURAL DYNAMICS System identification rests upon a vast quantitative methodology (Nieman et al., 1971), but most of it refers to linear operations. In previous sections significant sensory, cognitive, behavioural, and simulation correlates of non-linear temporal patterns of firing have been provided. Spike trains are statistically expressed by point-like processes (Segundo et al., 1966). Thus, a crucial requirement for a theoretical framework is to identify these point process systems without any assumption as to whether or not they are linear. Specific methods for this purpose have been developed (Brillinger 1974, 1975; Rapp et al., 1994). Point process systems—systems whose input and output are point processes—are said to be identified when an acceptable model is found. The first step of the identification is to estimate certain conditional rate functions, called kernels, of the spike trains. The one of zero-order, i.e. a constant, simply measures the mean rate. The one of first-order, a function of a single time argument, relates to the average effect of a single trigger spike (presynaptic) on the spike train; the one of second-order, a function of two time arguments, relates to the interactions between pairs of spikes; and so forth for higher-order functions. Then successive models are constructed recursively and based on the kernel of zero-order, on the kernels of zero- and first-order, on the kernels of zero-, first- and second-order, and so on. The acceptance of a kernel estimate as plausible does not necessarily mean that the series up to the corresponding term is a good predictor. The predictability may be evaluated by means of coherence (Brillinger, 1975). On the basis of the above assumptions it is possible to identify the neural network system (see Appendix I) that fulfils the necessary requirement of the probability function f describing its mode of activity. With respect to the state variable µ, the system may be qualified as a dynamical system x’ where , as mentioned above (Section §9). The next step consists of assuming that the dynamical system is structurally stable. In terms of topology, that means that, for a dynamical system x’ there exists a neighbourhood N(x’) in the state space with the property that every is topologically equivalent to x’. This concept is extremely important, because a structurally stable dynamical system cannot degenerate. There is no need to know the exact equations of the dynamical system because qualitative, approximate equations—i.e. “in the neighbourhood”—show the same qualitative behaviour (Andronov and Pontryagin 1937) (Figure 23). In the case of two control parameters, the probability function ƒ is 2 defined as the points µ of IR with a structurally stable dynamics of (Peixoto, 1962). That means the qualitative dynamics x’ is defined in a neighbourhood of a pair ( x 0, µ 0) at which ƒ is in equilibrium (e.g. minima, maxima, saddle point). With these assumptions, the equilibrium surface is geometrically equivalent to the RiemannHugoniot or cusp catastrophe described by Thorn (1975). Figure 24 illustrates a possible topological interpretation of neural dynamics according to this model. The equilibrium surface represents stable modes of activity with postsynaptic potential kinetics and the membrane excitability as control parameters.
Time and the brain
48
Let us start at a point (a) on the equilibrium surface, corresponding to a high level of excitability and a relatively long decay time of the postsynaptic potentials, e.g. 12 ms. This may correspond to the tonic mode of firing described in the thalamo-cortical circuit (Steriade et al., 1990). With these “settings” an input pattern will evoke precisely timestructured activity. Different firing patterns may be evoked by the same input if the synaptic dynamics are modified with minor changes of the cellular excitability, as suggested for neuromodulatory mediators (Levitan, 1988). Furthermore, different input patterns of activity may subserve the same mode of activity, somewhat like attractors. The transitions between these states are represented graphically in Figure 24 by paths (ab-a), (a-e-a) and (a-g-a). When the excitability is decreased, several types of neurones in the cortex and in the thalamus tend to switch towards a rhythmic or bursty type of firing. This effect may be provoked by a hyperpolarization of the cell membrane, or by modifying the spike threshold level (Foote and Morrison, 1987). In the former case a smooth passage between time-structured activity and asynchronous firing is likely to occur, as suggested by path (b-c-b), especially if the synaptic decay is long. Conversely, if the synaptic decay is fast, and a modulatory input modifies the threshold potential, a sudden switch from temporal patterns of firing to desynchronized activity will occur, as indicated by paths
Figure 23 . Two-dimensional representation of dynamical systems that are topologically equivalent (see text for definition of equivalency).
Empirical evidence about temporal structure
49
Figure 24 . Topological interpretation of neural dynamics as a function of two control parameters, the cell excitability and the kinetics of the postsynaptic potentials. The equilibrium surface is represented by a cusp catastrophe, where transitions can occur either suddenly or continuously between temporally organized firing patterns and asynchronous activity. This equilibrium surface refers to the activity of a specific neural network, and it is possible that the same cell belongs to more than one cell assembly. If the cell assemblies are controlled only by one parameter in common, then temporal and rate code are not mutually exclusive.
(a-d) and (e-f). Complex spatiotemporal firing patterns may also occur with low levels of excitability, i.e. point (e) in the graph of Figure 24, as suggested by cholinergic switching within neocortical networks (Villa et al., 1996; Xiang et al., 1998). Such a point (e) on the equilibrium surface can be particularly unstable, because a further decrease in excitability [path (e-f)], but also an increase in synaptic decay [path (e-d)], may provoke a sudden change in the mode of activity, as observed in simulation studies (Villa, 1992; Hill and Villa, 1997). It is important to notice that if the excitability is low, e.g. during long lasting hyperpolarization, the kinetics of the postsynaptic potential is often irrelevant with regard to the input pattern, so that the output activity will always tend to be organized in rhythmic bursts. Conversely, if the excitability is increased from a starting point (f) and the time constant of the synaptic decay is fast (say 4–5 ms), the input patterns could switch on either stable [path (f-g)] or unstable temporally-organized modes
Time and the brain
50
of activity [path f-e] only through sudden transitions.
13. SUMMARY OF CHAOTIC DYNAMICS Point processes defined by spike trains can be analyzed from alternative conceptual frameworks. Although few reasons are generally considered for adopting these frameworks as complementary approaches to the same biological system, several observations, either from experimental or simulated data, and theoretical assumptions have lead us to raise serious questions about the relationships between Spatiotemporal firing patterns and chaotic processes in neural networks. The dual description of brain activity could be regarded as similar to the dual representation of the light, with corpuscular and wave equations representing two irreducible expressions of the same phenomenon. A careful investigation of the dual nature of highly converging/diverging neural networks could open unexpected new ways to answer the problems arising from observations at multiple scales (ranging from sets of single neurones to brain imaging) of brain activity. In mathematics, few years ago, the demonstration of the Taniyama-Shimura conjecture filled the gap between two worlds and opened the way to the demonstration of Fermat’s last theorem by Andrew Wiles. This conjecture says that every rational elliptic curve (e.g. characterized by doughnut shapes) is a modular form, and modular forms have special properties with their many symmetries. Likewise, it is appealing to suggest that the relationships between chaotic dynamics and complex patterns of firing may represent a dictionary where questions, intuitions and insights in the one world get translated to questions, intuitions and insights in the other world.
14. FUTURE PROGRAM The detection of precise Spatiotemporal firing patterns or attractors necessarily requires the stability of the generating processes over a relatively long period of time. Thus, precise spike patterns in single neurones, or across multiple neurones, may be mostly involved in long-term processes (e.g. memory traces, learned motor programs), whereas the ensemble coding based on systematic modulations of firing rate may be related to short-term operational processes (e.g. motor action, attentional or feature-binding). The research presented here is not asking the questions that most neurophysiologists usually ask: which is the most adequate stimulus for a given neurone?; how is the external world mapped in the cortex?; what precisely are the receptive fields of single units? What is being asked here is how to detect the participation of nerve cells in distributed brain information processing, e.g. sensory discrimination, preparatory activity or short-term memory. The quantitative measure of neuronal activity available to neurophysiologists is the time series of spike epochs—i.e. the spike train. Therefore, it is of utmost interest to study temporal patterns of firing and assess whether they might be carriers of information. It is true that in the nervous system the problem of learning is crucial and
Empirical evidence about temporal structure
51
can hardly be approached without taking into account synaptic modification. However, not all neural information needs to be processed through modifiable networks. Changes in strength of a particular link between two cells become meaningless, because, in a distributed system, equivalent changes at one synapse might correspond to different patterns of activity. Therefore, the information is not necessarily resident in the links among the units, but might be provided by activity organized in a highly precise temporal mode. Indeed a connectionist network with the same set of connection weights may transmit different signals (Mpitsos et al., 1988). Accurate application of complex pattern detection methods enables correlation between discrete brain states and measures of behavioural performance on a trial-by-trial basis. The fact that the timing of pattern occurrence could relate to reaction time indicates that the network phenomena underlying them reflect some state of the animal that is able to influence behavioural output. Changes in cortical network activity during the waiting period may therefore be related to the concepts of “attention” and “set”, with emphasis on processes related to motor output in the former (Wise and Kurata, 1989) and to sensory processing in the latter (Shinba et al., 1995). It is interesting to notice that in the behavioural experiments reported here we found relatively few significant patterns, in the order of few hundreds, for tens of hours of analyzed recording time. The recording of cell activity is done over short periods (10 s) per trial, but one behavioural experiment could last several hours. It is possible to postulate either that the single unit detection might be unstable over such a prolonged time, or that only those cells involved repeatedly in activity directly related to the animal behaviour can participate in the detected patterns, thus decreasing the chance of detection with a limited sample of 15 spike trains recorded simultaneously. The problem of finding several neurones from a synfire chain is analyzed by Abeles (example 7.6.2, p. 255 in Abeles, 1991). In this example, finding two or more neurones from the same chain, when recording 10 neurones simultaneously, where p 0 =0.15 is the estimated probability of finding one cell from a chain, can be performed with an approximate probability of 0.44. The probability of detecting the same number of neurones in simultaneous recordings of 15, 20 or 30 cells, increases significantly. If the assumptions of this example are correct, it becomes possible to detect 6 or more neurones belonging to the same synfire chain, when recording from 30 single units simultaneously, with a probability near to 0.30. These estimations take into account only one synfire chain, but a single neurone may be involved simultaneously in multiple synfire chains and a synfire chain can include the same neurone twice, or even a higher number of times (Abeles et al., 1994). Altogether the experimental data and theoretical assumptions presented here make it possible to outline a general framework of coding principles, in which time and rate coding do not represent mutually exclusive coding schemes. Temporal and/or noisy rate codes might operate simultaneously in a given network, and several tools of analysis should be used to test the general hypothesis that the activation of widespread cell assemblies is distributed through multiple patterns of neuronal activity. Synchronous or oscillatory activity might activate and start complex temporally-organized processes, e.g. encompassing distributed sensorimotor information over widespread brain areas or thalamo-cortical recurrent loops (Villa et al., 1996). State-of-the-art techniques have allowed several laboratories to record tens of cells simultaneously in behaving rats
Time and the brain
52
(Skaggs and McNaughton, 1996; Nicolelis et al., 1997). Simulation of large-scale neuronal networks formed by integrate-and-fire model neurones are also becoming popular to test hypotheses on the significance of temporal information processing in the brain (Amit and Brunel, 1997; Lumer et al., 1997). The possibility of inserting virtual electrodes in such networks and of recording from hundreds or thousands of virtual neurones will certainly become a routine analysis in the forthcoming years. Then, the possibility of detecting and estimating carefully the significance of high complexity spatiotemporal patterns of spikes appears to be a crucial step for the evaluation of coding schemes. The spatiotemporal firing pattern should not be considered as representing a Morse alphabet, but rather the co-representation of specific information and a certain state of “central arousal”. Future developments of temporal dynamics and topological interpretation of neuronal network activity could be an important issue in understanding the functional properties and information processing in the central nervous system. Whether this prediction is upheld or not, there is no doubt that these studies will open up new possibilities for research (new areas, new methods of analysis, new recording technique, computer-modeling assistance, etc.) and no possible outcome should be a dead end. After all, we are only 200 years from the intellectual legacy of Galvani and although we may say that the pre-history of Neuroscience is over we should not rule out the possibility that we are still in its proto-historical phase.
ACKNOWLEDGEMENTS This work was made possible particularly by the contribution of Igor V.Tetko and by the members of the Laboratory of Neuro-heuristics (www-lnh.unil.ch) and the staff personnel of the Institute of Physiology of the University of Lausanne. Partial support by the Swiss National Science Foundation (Bern) and the Istituto della Enciclopedia Italiana (Roma) is acknowledged.
APPENDIX I The following paragraphs illustrate a possible framework for system identification. Suppose that A is a stationary spike train. The overall mean rate of the A train is defined by Prob { A spike in ( t,t + h )}/ h. Suppose that B is another stationary spike train occurring simultaneously to spike train A and m B its mean rate. A first-order conditional rate function m AB (u) is defined as follows. For one cell B and close to any particular time t the AB cross-intensity function, m AB (u), measures the instantaneous rate of the B train—i.e. the probability whether there will be a B spike- u time units after an A spike, say
(1) The average change of the instantaneous rate at time u in a spike train B, when a single
Empirical evidence about temporal structure
53
spike occurs at time 0 in a spike train A is predicted at best by the kernel of first order a (u). If one defines m AA (u) as the auto-intensity function of A —i.e. the cross-intensity function of the A train with itself-then, where satisfies the following condition (Brillinger 1974),
(2)
Because of stationarity this function does not depend on t. If the spike trains are independent, then m AB (u)=m B for any u . If spike train A were Poisson noise, m AA (v) would be equal to m A for all v , and the solution of Equation (2) would be a(u)=m AB (u) −m B . In this case the kernel function would correspond to the cross-intensity function up to an additive constant and it could be estimated by the corresponding cross-correlation histogram. In order to construct a general solution a(u) of Equation (2) let us define two parameters, φ AA (λ) and φ AB (λ). The parameter
(3)
denotes the power spectrum of the A spike train (Brillinger 1974). This is proportional to the variance of the component of frequency λ of the A spike train, hence it may be interpreted as reflecting the power in each frequency component of that train. For pure Poisson noise, it is identically constant and equal to (2 p)−1 m A . The parameter (4)
denotes the cross-spectrum of the A and B spike trains. This is proportional to the covariance of the component of frequency λ of the A train with the corresponding component of the B train. Hence it may be interpreted as reflecting how a certain frequency component in the A train is associated with one in the B train. The Fourier transform of the solution a(u) is defined by
(5)
Taking the Fourier transform of Equation (2) and the definitions (3) and (4) leads to φ AB , the solution of Equation (λ)=A(λ) φAA (λ). It follows that, if (2) may be written as
Time and the brain
54
(6)
Finally, the measure of predictability of the kernel is provided by the coherence at frequency λ defined by It is never less than zero or greater than one. It is a measure of the association of the spike trains. A coherence of one would imply perfect linear prediction of the B train by the A train at frequency λ. A coherence of zero results if the two trains are statistically independent or if at frequency λ. Several studies have applied this approach to study the functional interactions between spike trains (Rosenberg et al., 1989; Brillinger and Villa, 1994, 1997).
REFERENCES Abbott, L.F., Varela, J.A., Sen, K., and Nelson, S.B. (1997) Synaptic depression and cortical gain control. Science, New York 275, 220–224. Abeles, M. (1982) Local Cortical Circuits: An Electrophysiological Study , Springer Verlag, Berlin. Abeles, M. (1983) The quantification and graphic display of correlations among three spike trains. IEEE Transactions on Biomedical Engineering 30, 235–239. Abeles, M. (1991) Corticonics: Neural Circuits of the Cerebral Cortex . Cambridge University Press, New York. Abeles, M., and Gerstein, G.L. (1988) Detecting Spatiotemporal Firing Patterns Among Simultaneously Recorded Single Neurones. Journal of Neurophysiology , 60, 909–924. Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993a) Spatiotemporal Firing Patterns in the Frontal Cortex of Behaving Monkeys. Journal of Neurophysiology , 70, 1629–1638. Abeles, M., Prut, Y., Bergman, H., Vaadia, E., and Aertsen, A. (1993b) Integration, synchronicity and periodicity. In: A.Aertsen, (ed.) Brain theory: spatio-temporal aspects of brain function , pp. 149–181. Elsevier, Amsterdam. Abeles, M., Vaadia, E., Bergman, H., Prut, Y., Haalman, I., and Slovin, H. (1993c) Dynamics of neuronal interactions in the frontal cortex of behaving monkeys. Concepts in Neuroscience , 4, 131–158. Abeles. M., Prut, Y., Bergman, H., and Vaadia, E., (1994) Synchronization in neuronal transmission and its importance for information processing, Progress in Brain Research , 102, 395–404. Addison, P.S. (1997) Fractals and Chaos: An illustrated Course . Institute of Physics Publishing, Bristol. Adrian, E.D. (1912) On the conduction of subnormal disturbances in normal nerve. Journal of Physiology (London) , 45, 389–412. Adrian, E.D. (1934) The basis of sensation: the action of the sense organs . Christophers, London. Aertsen, A.M.H.J., Gerstein, G.L., Habib, M.K., and Palm, G. (1989) Dynamics of neuronal firing correlation: modulation of effective connectivity. Journal of Neurophysiology , 61 , 900–917.
Empirical evidence about temporal structure
55
Amit, D.J. (1995) The Hebbian paradigm reintegrated: local reverberations as internal representations. Behavioral Brain Sciences , 18, 617–657. Amit, D.J., Brunel, N., and Tsodyks, M. (1994) Correlations of cortical Hebbian reverberations: theory versus experiment. Journal of Neuroscience , 14 , 6435–6445. Amit, D.J. and Brunel, N., (1997) Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex , 7, 237–252. Amit, D.J., Fusi, S., and Yakovlev, V. (1997) Paradigmatic working memory (attractor) cell in IT cortex. Neural Computation , 9, 1071–1092. Andronov, A., and Pontryagin, L. (1937) Systèmes grossiers. Doklady Akademii Nauk. SSSR , 14 , 247–251. Babloyantz, A., Salazar, J.M., and Nicolis, C. (1985) Evidence of chaotic dynamics of brain activity during the sleep cycle. Physics letters , 111A , 152–155. Bair, W., and Koch, C. (1996) Temporal precision of spike trains in extrastriate cortex of the behaving Macaque monkey. Neural Computation , 8, 1184–1202. Bialek, W., Rieke, F., de Ruyter van Steveninck, R.R., and Warland, D.S. (1991) Reading a neural code. Science, New York , 252, 1854–1857. Bialek, W., and Rieke, F. (1992) Reliability and information transmission in spiking neurons. Trends in Neuroscience , 15 , 428–434. Braitenberg, V., and Schüz, A. (1991) Anatomy of the cortex: statistics and geometry . Springer Verlag, Berlin. Brillinger, D.R. (1974) Cross-spectral analysis of processes with stationary increments including the stationary G/G/8 queue. Annals of Probability , 2, 815–827. Brillinger, D.R. (1975) The identification of point process systems. Annals of Probability , 3, 909–924. Brillinger, D.R., and Villa, A.E.P. (1994) Examples of the investigation of neural information processing by point process analysis. In: V.Z.Marmarelis (ed.) Advanced Methods of Physiological System Modeling, volume 3 , pp. 111–127, Plenum Press, New York. Brillinger, D.R., and Villa, A.E.P. (1997) Assessing connections in networks of biological neurons. In: D.R.Brillinger, L.T.Fernholz, S.Morgenthaler (eds) The Practice of Data Analysis , pp. 77–92, Princeton University Press, Princeton, NJ. Brillinger, D.R., Bryant, H.L. Jr., and Segundo, J.P. (1976) Identification of synaptic interactions. Biological Cybernetics , 22, 213–228. Bullock, T.H. (1961) The problem of recognition in an analyzer made of neurons. In: W.A. Rosemblith (ed.) Sensory communication , pp. 717–724, Wiley, New York. Castro, R., and Sauer, T. (1997) Correlation dimension of attractors through interspike intervals. Physics Reviews , A55, 287–290. Celebrini, S., and Newsome, W.T. (1994) Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. Journal of Neuroscience , 14 , 4109–4124. Celletti, A., and Villa, A.E.P. (1996) Low dimensional chaotic attractors in the rat brain. Biological Cybernetics , 74, 387–394. Celletti, A., Bajo Lorenzana, V.M., and Villa, A.E.P. (1997) Correlation dimension for paired discrete time series. Journal of Statistical Physics , 89, 877–884. Celletti, A., Tetko, I.V., and Villa, A.E.P. (1998) Analysis of the deterministic behaviour of experimental series. Journal de Physique IV 8, 209–214. Celletti, A., Froeschlé, C., Tetko, I.V., and Villa, A.E.P (1999) Deterministic behaviour of short time series. Meccanica . 34 . 145–152.
Time and the brain
56
Cutler, C.D., and Kaplan, D.T. (1997) Nonlinear dynamics and time series: building a bridge between the natural and statistical sciences. Fields Institute communications, vol. 11 , pp. 252, American Mathematical Society, Providence, R.I. Dayhoff, J., and Gerstein, G.L. (1983a) Favored patterns in spike trains. I. Detection. Journal of Neurophysiology , 49, 1334–1348. Dayhoff, J., and Gerstein, G.L. (1983b) Favored patterns in spike trains. II. Application. Journal of Neurophysiology , 49, 1349–1363. de Ruyter van Steveninck, R.R., Lewen, G.D., Strong, S.P., Koberle, R., and Bialek, W. (1997) Reproducibility and variability in neural spike trains. Science, New York , 275, 1805–1808. deCharms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds by the coordination of action-potential timing. Nature, London , 381, 610–613. Douglas, R., and Martin, K. (1991) A functional micro-circuit for cat visual cortex. Journal of Physiology (London) , 440, 735–769. Eckmann, J.P., and Ruelle, D. (1992) Fundamental limitations for estimating dimensions and Lyapunov exponents in dynamical systems. Physica D , 56, 185–187. Edmonds, B., Gibb, A., and Colquhon, D. (1995) Mechanisms of activation of glutamate receptors and the time course of excitatory synaptic currents. Annual Review of Physiology , 57, 495–519. Eckhorn, R., Grusser, O.J., Kroller, J., Pellnitz, K., and Popel, B. (1976) Efficiency of different neuronal codes: information transfer calculations for three different neuronal systems. Biological Cybernetics , 22, 49–60. Ferster, D., and Spruston, N. (1995) Cracking the neural code. Science, New York , 270, 756–757. Fetz, E.E. (1997) Neuroscience—temporal coding in neural populations. Science, New York , 278, 1901–1902. Foote, S.L., and Morrison, J.H. (1987) Extrathalamic modulation of cortical function. Annual Review of Neuroscience , 10, 67–96. Friston, K.J. (1995) Neuronal transients. Proceedings of the Royal Society of London— Series B , 261, 401–405. Frostig, R.D., Frostig, Z., and Harper, R.M. (1984) Information trains. The technique and its uses in spike train and network analysis with examples taken from the nucleus parabrachialis medialis during sleep-waking states. Brain Research , 322, 67–74. Frostig, R.D., Frostig, Z., and Harper, R.M. (1990) Recurring discharge patterns in multiple spike trains. I. Detection. Biological Cybernetics 62, 487–493. Fuster, J.M. (1990) Inferotemporal units in selective visual attention and short-term memory. Journal of Neurophysiology , 64, 681–697. Georgopoulos, A., Taira, M.L., and Lukashin, A. (1993) Cognitive neurophysiology of the motor cortex. Science, New York , 260, 47–52. Georgopoulos, A.P. (1995) Current issues in directional motor control. Trends in Neuroscience , 18, 506–510. Gerstein, G.L., and Mandelbrot, B. (1964) Random walk models for the spike activity of a single neuron. Biophysical Journal , 4, 41–68. Grassberger, P., and Procaccia, I. (1983) Characterization of strange attractors. Physics Review Letters , 50, 346–349. Hénon, M. (1976) A two-dimensional mapping with a strange attractor, Communications in Mathematical Physics , 50, 69–77. Herrmann, M., Ruppin, E., and Usher, M. (1993) A neural model of the dynamic activation of memory. Biological Cybernetics , 68, 455–463.
Empirical evidence about temporal structure
57
Hill, S.L., and Villa, A.E.P. (1997) Dynamic transitions in global network activity influenced by the balance of excitation and inhibition, Network , 8, 165–184. Klemm, W.R., and Sherry, C.J. (1981) Serial ordering in spike trains: what’s it trying to tell us? International Journal of Neuroscience , 14, 15–23. König, P., Engel, A.K., and Singer, W. (1996) Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neuroscience , 19, 130–137. Legéndy, C.R. (1975) Three principles of brain function and structure. International Journal of Neuroscience , 6, 237–254. Levitan, I.B. (1988) Modulation of ion channels in neurons and other cells. Annual Reviews of Neuroscience , 11, 119–136. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997) Neural dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cerebral Cortex , 7, 207–727. MacGregor, R.J. (1987) Neural and brain modeling . Academic Press, San Diego. Mainen, Z.F., and Sejnowski, T.J. (1995) Reliability of spike timig in neocortical neurons. Science, New York , 268, 1503–1506. Markram, H., Lubke, J., Frotscher, M., and Sakmann, B. (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APS and EPSPS. Science, New York , 275, 213–215. Martignon, L., Von Hasseln, H., Grim, S., Aertsen, A., and Palm, G. (1995) Detecting higher-order interactions among the spiking events in a group of neurons. Biological Cybernetics , 73, 69–81. McCormick, D.A., and Bal, T. (1994) Sensory gating mechanisms of the thalamus. Current Opinion in Neurobiology , 4, 550–556. McCulloch, W.S., and Pitts, W. (1943) A logical calculus of the ideas imminent in nervous activity. Bulletin of Mathematical Biophysics , 5, 115–133. Miller, R. (1996) Cortico-thalamic interplay and the security of operation of neural assemblies and temporal chains in the cerebral cortex. Biological Cybernetics , 75, 263–275. Mountcastle, V.B., Poggio, G.F., and Werner, G. (1963) The relation of thalamic cell response to peripheral stimuli varied over an intensive continuum. Journal of Neurophysiology , 26, 807–834. Mpitsos, G.J., Burton, R.M., and Creech, H.C. (1988) Connectionist networks learn to transmit chaos. Brain Research Bulletin , 21, 539–546. Mpitsos, G.J. (1989) Chaos in brain function and the problem of nonstationarity: a commentary. In: E. Başar, T.H.Bullock (eds) Dynamics of sensory and cognitive processing by the brain , pp. 521–535, Springer-Verlag, New York, N.Y. Mpitsos, G.J., and Burton, R.M.Jr. (1992) Convergence and divergence in neural networks: processing of chaos and biological analogy. Neural Networks , 5, 605–625, Mukhametov, L.M., Rizzolatti, G., and Tradardi, V. (1970) Spontaneous activity of neurones of nucleus reticularis thalami in freely moving cats. Journal of Physiology (London) , 210, 651–667. Nafe, J.P. (1968) Neural correlates of sensation. In: D.R.Kenshalo (ed.) The Skin Senses pp. 5–14, Clarence Thomas, Springfield. Nicolelis, M.A.L., Ghazanfar, A.A., Faggin, B.M., Votaw, S., and Oliveira, L.M.O. (1997) Reconstructing the engram—simultaneous, multisite, many single neuron recordings. Neuron , 18, 529–537. Nieman, R.E., Fisher, D.G., and Seborg, D.E. (1971) A review of process identification and parameter estimation techniques. International Journal of Control , 13 , 209–264.
Time and the brain
58
Payne, B.R., Lomber, S.G., Villa, A.E.P., and Bullier, J. (1996) Reversible deactivation of cerebral network components. Trends in Neuroscience , 19 , 535–542. Peixoto, M. (1962) Structural stability on 2-dimensional manifolds. Topology, 1, 101– 120. Perkel, D.H., and Bullock, T.H. (1965) Neural coding. Neurosciences Research Program Bulletin , 6 , 221–348. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967a) Neuronal spike trains and stochastic point processes. I. The single spike train. Biophysical Journal , 7, 391–418. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967b) Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophysical Journal , 7, 419– 440. Perkel, D.H., Gerstein, G.L., Smith, M.S., and Tatton, W.G. (1975) Nerve-impulse patterns: a quantitative display technique for three neurons. Brain Research , 100 , 271–296. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. (1994) Numerical Recipes in C. (2nd ed.) Cambridge University Press, New York. Prut. Y., Vaadia, E., Bergman, H., Haalman, I., Slovin, H., and Abeles, M. (1998) Spatiotemporal Structure Of Cortical Activity—Properties and Behavioral Relevance. Journal of Neurophysiology , 79, 2857–2874. Rapp, P.E., Zimmerman, I.D., Albano, A.M., Deguzman, G.C., and Greenbaum, N.N. (1985) Dynamics of spontaneous neural activity in the simian motor cortex: the dimension of chaotic neurons. Physics Letters , 110A , 335–338. Rapp, P.E., Zimmerman, I.D., Vining, E.P., Cohen, N., Albano, A.M., and JimenezMontano, M.A. (1994) The algorithmic complexity of neural spike trains increases during focal seizures. Journal of Neuroscience , 14 , 4731–4739. Rapp, P.E., Albano, A.M., Schmah, T.I., and Farwell, L.A. (1993) Filtered noise can mimic low-dimensional chaotic attractors. Physical Review , E47, 2289–2297. Riehle, A., and Requin, J. (1995) Neuronal correlates of the specification of movement direction and force in four cortical areas of the monkey. Behavioral Brain Research , 70, 1–13. Riehle, A., Grün, S., Diesmann, M., and Aertsen, A. (1997) Spike synchronization and rate modulation differentially involved in motor cortical function. Science, New York , 278, 1950–1953. Rieke, F., Bodnar, D.A., and Bialek, W. (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proceedings of the Royal Society of London—Series B , 262, 259–265. Rosenberg, J.R., Amjad, A.M., Breeze, P., Brillinger, D.R., and Halliday, D.M. (1989) The Fourier approach to the identification of functional coupling between neuronal spike trains. Progress in Biophysics and Molecular Biology , 53, 1–31. Segundo, J.P. (1986) What can neurons do to serve as integrating devices? Journal of Theoretical Neurobiology , 5, 1–59. Segundo, J.P., Moore, G.P., Stensaas, L., and Bullock, T.H. (1963) Sensitivity of neurons in Aplysia to temporal patterns of arriving impulses. Journal of Experimental Biology , 40, 643–667. Segundo, J.P., Perkel, D.H., and Moore, G.P. (1966) Spike probability in neurones: influence of temporal structure in the train of synaptic events. Kybernetik , 3, 67–82. Segundo, J.P., and Perkel, D.H. (1969) The nerve cell as an analyzer of spike trains. UCLA Forum in Medical Sciences , 11, 349–390. Shadlen, M.N., and Newsome, W.T. (1994) Noise, Neural Codes and Cortical
Empirical evidence about temporal structure
59
Organisation. Current Opinion in Neurobiology , 4, 569–579. Shadlen, M.N., and Newsome, W.T. (1998) The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. Journal of Neuro science , 18, 3870–3896. Shinba, T., Sumi, M., Iwanami, A., Ozawa, N., and Yamamoto, K. (1995) Increased neuronal firing in the rat auditory cortex associated with preparatory set. Brain Research Bulletin , 37, 199–204. Skaggs, W.E., and McNaughton, B.L. (1996) Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science, New York , 271, 1870–1873. Smale, S. (1967) Differential dynamical systems. Bulletin of the American Mathematical Society , 73, 747–817. Softky, W., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random epsps. Journal of Neuroscience , 13, 334–350. Steriade, M., Jones, E.G., and Llinás, R.R. (1990) Thalamic oscillations and signalling , Wiley & Sons, New York, NY. Stuart, G.J., and Sakmann, B. (1994) Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature, London , 367, 69–72. Tang, A.C., Bartels, A.M., and Sejnowski, T.J. (1997) Effects of cholinergic modulation on responses of neocortical neurons to fluctuating input. Cerebral Cortex , 7, 502–509. Tetko, I.V., and Villa, A.E.P. (1997a) Fast combinatorial methods to estimate the probability of complex temporal patterns of spikes. Biological Cybernetics , 76, 397– 407. Tetko, I.V., and Villa, A.E.P. (1997b) Efficient partition of learning datasets for neural network training. Neural Networks , 10, 1361–1374. Tetko, I.V., and Villa, A.E.P. (1997c) A comparative study of pattern detection algorithm and dynamical system approach using simulated spike trains. Lecture Notes in Computer Science , 1327, 37–42. Tetko, I.V., and Villa, A.E.P. (1999) Temporal interactions in multiple spike trains detected by pattern grouping algorithm. Journal of Neurophysiology (submitted). Theiler, J., and Rapp, P.E. (1996) Re-examination of the evidence for low-dimensional, nonlinear structure in the human electroencephalogram. Electroencephalography and Clinical Neurophysiology , 98, 213–222. Thorn, R. (1975) Structural stability and morphogenesis (English translation). W.A. Benjamin, Reading, MA. Thomson, A., and Deuchars, J. (1994) Temporal and spatial properties of local circuits in neocortex. Trends in Neuroscience , 17, 119–126. Thorpe, S., Fize, D., and Marlot, C. (1996) Speed of processing in the human visual system. Nature, London , 381, 520–522. Tsodyks, M.V., and Sejnowski, T. (1995) Rapid state switching in balanced cortical network models. Network , 6, 111–124. Tsodyks, M.V., and Markram, H. (1997) The neural code between neocortcal pyramid neurones depends on neurotransmitter release probability. Proceedings of the National Academy of Sciences, U.S.A. , 94, 719–723. Tsukada, M., Terasawa, M., and Hauske, G. (1982) Temporal pattern sensitive and nonsensitive responses in the cat’s retinal ganglion cells. Biological Cybernetics , 44, 197–203. Usher, M., and Stemmler, M. (1995) Dynamic pattern formation leads to 1/f noise in
Time and the brain
60
neural populations. Physical Review Letters 74, 326–329. Vaadia, E., Bergman, H., and Abeles, M. (1989) Neuronal activities related to higher brain functions—theoretical and experimental implications. IEEE Transaction in Biomedical Engineering , 36, 25–35. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London , 373, 515–518. Villa, A.E.P. (1988) Influence de I'écorce cérébrale sur I’activité spontanée et evoquée du thalamus auditifdu chat . Imprivite, Lausanne. Villa, A.E.P. (1990) Functional differentiation within the auditory part of the reticular nucleus of the cat. Brain Research Reviews . 15. 25–40. Villa, A.E.P. (1992) Les catastrophes cachées du cerveau. Le Nouveau Golem , 1, 33–63. Villa, A.E.P., and Abeles, M. (1990) Evidence for spatio-temporal firing patterns within the auditory thalamus of the cat. Brain Research , 509, 325–327. Villa, A.E.P., Rouiller, E.M., Simm, G.M., Zurita, P., de Ribaupierre, Y., and de Ribaupierre, F. (1991) Corticofugal modulation of information processing in the auditory thalamus of the cat. Experimental Brain Research , 86 , 506–517. Villa, A.E.P., and Fuster, J.M. (1992) Temporal correlates of information processing during visual short-term memory. NeuroReport , 3, 113–116. Villa, A.E.P., and Tetko, I.V. (1995) Spatio-temporal patterns of activity controlled by system parameters in a simulated thalamo-cortical neural network. In: H.Hermann, (ed.) Supercomputing in Brain Research: from tomography to neural networks , pp. 379–388, World Scientific Publishing, Singapore. Villa, A.E.P., Tetko, I.V., Riehle, A., and Requin, J. (1995) Selective spatio-temporal firing patterns in the motor cortex during a reaction-time task. Society for Neuroscience Abstracts , 21 , 516. Villa, A.E.P., Bajo, V.M., and Vantini, G. (1996) Nerve Growth Factor (NGF) modulates information processing in the auditory thalamus. Brain Research Bulletin , 39, 139– 147. Villa, A.E.P., Hyland, B., Tetko, I.V., and Najem, A. (1998a) Dynamical cell assemblies in the rat auditory cortex in a reaction-time task. Biosystems , 48, 269–278. Villa, A.E.P., Tetko, I.V., Celletti, A., and Riehle, A. (1998b) Chaotic dynamics in the primate motor cortex depend on motor preparation in a reaction-time task. Current Psychology of Cognition . 17, 763–780. Villa, A.E.P., Tetko, I.V., Hyland, B., and Najem, A. (1999a) Significance of spatiotemporal activity patterns among rat cortex neurons in performance of a conditioned task. Proceedings of the National Academy of Sciences, U.S.A. , 96, 1006– 1011. Villa, A.E.P., Tetko, I.V., Dutoit, P., de Ribaupierre, Y., and de Ribaupierre, F. (1999b) Corticofugal modulation of functional connectivity within the auditory thalamus of rat, guinea pig and cat revealed by cooling deactivation. Journal of Neuroscience Methods , 86, 161–178. Villa, A.E.P., Eriksson, J., Eriksson, C., Haeberli, C, Hyland, B., and Najem, A. (199c) Stimulus congruence affects perceptual processes in a novel Go/Nogo conflict paradigm in rats. Behavioural Processes , 48, 69–88. Villa, A.E.P., and Tetko, I.V. (1999) Spatiotemporal Activity Patterns Detected from Single Cell Measurements from Behaving Animals, Proceedings SPIE (International Society for Optical Engineering) , 3728, 20–34. Wilson, M.A., and McNaughton, B.L. (1993) Dynamics of the Hippocampal Ensemble
Empirical evidence about temporal structure
61
Code for Space. Science, New York , 261 , 1055–1058. Wilson, H.R., and Cowan, J.D. (1974) Excitatory and inhibitory interactions of localized populations of model neurons. Biophysical Journal , 12 , 1–24. Wise, S.P., and Kurata, K. (1989) Set-related activity in the premotor cortex of rhesus monkeys: effect of triggering cues and relatively long delay intervals. Somatosensory and Motor Research , 6, 455–476. Xiang, Z., Huguenard, J.R., and Prince, D.A. (1998) Cholinergic switching within neocortical inhibitory networks. Science, New York 281, 985–988. Xing, J., and Gerstein, G.L. (1996) Networks with lateral connectivity: I. Dynamic properties mediated by the balance of intrinsic excitation and inhibition. Journal of Neurophysiology , 75, 184–199. Zurita, P., Villa, A.E.P., de Ribaupierre, Y., de Ribaupierre, F., and Rouiller, E.M. (1994) Changes of single unit activities in the cat’s auditory thalamus and cortex associated to different anesthetic conditions. Neuroscience Research . 19 , 303–316.
2 Cross-Correlograms for Neuronal Spike Trains. Different Types of Temporal Correlation in Neocortex, their Origin and Significance Lionel G. Nowak 1 and Jean Bullier 2 1. Section of Neurobiology, Yale University School of Medicine, C303 Sterling Hall of Medicine, 333 Cedar Street, New Haven, CT06510, USA. e-mail: [email protected] 2. Jean Bullier: Centre de Recherche Cerveau et Cognition, CNRS-UPS UMR 5549, 133, route de Narbonne, 31062 Toulouse Cedex email: [email protected]
After a presentation of the method of computation of the cross-correlation histogram, the major interpretations of cortical synchronization are reviewed. These include indicators of different brain states (sleep, waking state), indication of cortical connectivity, participation in cell assemblies, and a different form of neural coding. It is then argued that there are several types of synchronization, reflected in the existence of CCH peaks of different widths. The function of these different forms of synchronization in cortical processing and their possible underlying generation mechanisms are discussed. KEYWORDS: Synchronization; Cell assembly; Binding; Cortex; Vision; Crosscorrelogram Synchronization of neuronal activity is a topic that has recently received a lot of interest. The major reason for this is certainly related to the publication of experimental results that supported the idea that the relative timing of action potentials between different neurones could play an essential role in perception. The enthusiasm raised by these experiments has not been shared by the whole community, and a number of criticisms have been made. Scientific controversies are often related to a misunderstanding of certain definitions. In our case, the essential question is: what does synchronization really mean and imply? This chapter is an attempt to examine synchronization in depth. First, we recall the basic principles of the cross-correlation technique, as it is the method used most frequently for tracking synchrony at the level of action potentials. Next, we present a historical perspective on synchronization, which leads to the realization that the meaning of synchronization has varied a lot at different periods. In a third section we show that one should not talk about synchronization as a
Time and the brain
64
single factor, but about several different types of synchronization. Having acknowledged this basic fact, we attempt to review the possible mechanisms that lead to the generation of these different forms of synchronization. Finally, we review recent results and the current status on the functional roles of synchronization.
1. THE CROSS-CORRELATION HISTOGRAM AS A TOOL The technique that has been used most commonly to reveal the temporal coherence in the firing of cortical neurones is to compute a cross-correlation histogram (abbreviated as CCH, also known as cross-correlogram) from their spike trains. Although the crosscorrelation technique had been used before, Perkel et al. (1967b) were the first to present formally its computation and interpretation: consider two cells that are recorded extracellularly (A and B in Figure 1). The CCH computed from the spikes emitted by neurones A and B is an estimate of the probability of firing of neurone B as a function of the time before or since a spike was fired by neurone A. This estimate is calculated by placing each spike of neurone B in a given time bin depending on its difference in time of emission from that of a spike in neurone A. It is important to note that in a CCH, a spike from neurone B usually appears in several bins, since the intervals between this spike and all the spikes of neurone A are taken into account. However, each interval between a spike in neurone A and a spike in neurone B is counted only once. Thus, a CCH is best described as a histogram of intervals between spikes-times in two neurones. Different patterns of connectivity can give rise to different forms of temporal coherence, and hence to different shapes in a CCH. We schematically illustrate these in Figure 1, considering the case of 2 neurones recorded during spontaneous activity.
Cross-correlograms for neuronal spike trains
65
Figure 1 . Typical shapes of cross-correlation histograms (CCH). Neurones A and B produce the spike trains 1 and 2. The CCH gives an estimate of the probability of a spike in neurone B at time t before or after a spike in neurone A. Uncorrelated neurones give a flat CCH. A single sharp displaced peak is the signature of a monosynaptic drive. A peak centered on the origin corresponds to synchronous firing that is usually interpreted as resulting from a common drive.
If they share no direct or indirect link, the CCH computed from the activities of the two cells (A and B in Figure 1) appears flat. This is interpreted as evidence that the cells fire independently (Figure 1a). A monosynaptic excitatory connection between 2 neurones appears in the CCH as a peak displaced from the origin of time by a latency corresponding to the sum of the conduction time in the axon and the synaptic delay (Figure 1b). In the vast majority of brain structures, not all spikes in cell A trigger a spike in cell B; conversely, not all spikes in B are triggered by a spike in cell A. As a consequence, the CCH peak emerges from a baseline probability level corresponding to those spikes in B that were not related to the spikes in A. In other words, those spikes that occurred independently in the two cells. The strength of the connection can then be estimated by different means, for example as the percentage of spikes in the presynaptic neurone that are related to spikes in the postsynaptic neurone (the “effectiveness”; Levick et al., 1972); as the percentage of spikes in the postsynaptic neurone that are related to spikes in the presynaptic neurone (the “contribution”; Levick et al., 1972); as the peak area normalized by the area under the peak (the “mean percent increase”, or “MPI”; Cope et al., 1987); as the peak height normalized by the CCH baseline (the “RMA”; Engel et al., 1990); as the number of
Time and the brain
66
spikes contained in the peak (the “asynchronous gain”; Abeles, 1991); etc. A monosynaptic inhibitory connection produces a dip in the CCH, provided the inhibitory postsynaptic potential (IPSP) is strong enough to reduce the firing probability in neurone B and that there is enough activity to be reduced by the IPSP (Aersten and Gerstein, 1985; Melssen and Epping, 1987). If the two cells are driven at the same time by a common presynaptic neurone (assuming homogeneous axonal conduction times and synaptic delays), they would tend to fire at the same time, which, in the CCH, results in a peak centered on the time origin (Figure 1b). The two cells in this case are said to synchronize their activities. Finally, a broad peak should appear in the CCH if the two neurones are simultaneously inhibited by a third neurone. This might appear counter-intuitive at first sight, but can be explained by the following reasoning: On average, the common inhibition tends to decrease their firing at the same time. When inhibition is released, the 2 cells increase their firing rate at the same time, hence the broad peak centered on the time origin. Sensory stimuli are often used to activate neurones, especially in regions of the cortex where spontaneous activity is too low to accumulate enough spikes to compute CCHs in a reasonable time. However, the use of a stimulus introduces a complication. In that case, the presence of a peak in the CCH can have two different origins: one corresponds to the stimulus-locked temporal coherence. It reflects the fact that the two neurones under study are activated with a constant temporal relationship by the sensory stimulation. This is exemplified in Figure 2 for the case of two neurones recorded, one in area V1 and the other in area V2 of the macaque visual cortex. The spike trains are shown schematically in Figure 2a. Figure 2b shows the peri-stimulus time histograms of the responses of the two cells to the onset and the offset of a flashing stimulus. The stimulus induced an increase in firing rate in both cells, at roughly the same time, and for a comparable duration. This coincident increase in firing rate contributes to the peak in the CCH (Figure 2c1, raw CCH). The delay and the size of the peak corresponding to the stimuluslocked temporal coherence depend on the
Cross-correlograms for neuronal spike trains
67
Figure 2 . Examples ofCCH and methods of the shift predictor. Recordings from units in areas V1 and V2 of the macaque monkey. A: examples of spike trains, as recorded (above) and after shifting the V2 spikes by one stimulus period (below). B: Post-stimulus histograms (PSTHS) of the responses of the two neurones to a small stimulus flashed in the receptive fields. The light is ON at time 0 and OFF at 3 s. C1: CCH computed from the normal spike trains (raw CCH) and after shifting the V2 spike train (Shift predictor), for the parts of the spikes trains corresponding to visual responses in the CCH. The deviation of the raw CCH from the Shift predictor indicates the presence of neural coupling. C2: Subtracted CCH and side peak. The subtracted CCH is the difference between the raw CCH and the shift predictor. The side peak is obtained by subtracting two shift predictors, one shifted by one and one shifted by two stimulus periods. It gives an indication of the amount of variability of the CCH trace. D1: same calculation as for C1 except that the spikes come from the period with no stimulus. D2: same as C2 for spontaneous activity (no stimulus). Note that a broad peak (H type) is observed in spontaneous activity whereas an intermediate peak (C type) is observed during visual stimulation.
latency difference in the activation of the neurones by the stimulus, as well as the duration of their responses. The other form of temporal coherence observed with sensory stimuli is the temporal
Time and the brain
68
coherence of neuronal origin. This form of coherence results from any of the neuronal interactions described above (Figure 1). To distinguish between the 2 forms of temporal coherence it is common to use a shift (or shuffle) predictor method. This method was first presented by Perkel et al. (1967b): a sensory stimulus, if repeated with a fixed periodicity, induces a periodic increase in firing rate (Figure 2a). The effect of the sensory stimulus, therefore, can be predicted, in isolation, by computing another CCH (called the shift predictor) after shifting one spike train with respect to the other by an amount of time corresponding to one (or several) periods of the stimulus (Figure 2a). Usually, the periodicity of the stimulus is several seconds. The temporal coherence of neuronal origin, on the other hand, exists only for a limited amount of time, most commonly below 10 msec in the case of fast synaptic transmission, and generally below 1 second in the case of slower (metabotropic) interactions. Therefore, by comparing the original CCH (the raw CCH) with its shift predictor it is possible to determine what is produced by the stimulation and what is due to the neuronal interactions (Figure 2c1. The subtracted CCH, corresponding to the subtraction of the shift predictor from the raw CCH, is used to characterize the temporal coherence of neuronal origin in isolation (Figure 2c2). Experimental studies were then designed to test the prediction from the theoretical study by Perkel et al. (1967b). Moore et al (1970); and Bryant et al. (1973) applied the cross-correlation technique to the study of the functional connectivity in Aplysia. The peaks observed in the CCH could be ascribed to different forms of synaptic interactions, which were directly inferred by virtue of dual intracellular recording for the same cell pairs. They showed that, indeed, a monosynaptic excitatory connection produces a displaced peak, and that a common input produces a peak centered on the origin of time. Modeling studies (Knox, 1974), and studies in the spinal cord (Kirkwood and Sears, 1978, 1982b; Fetz and Gustafsson, 1983; Gustafsson and McCrea, 1984; Cope et al., 1987) and recently in the cortex (Matsumura et al., 1996), attempted to establish a relationship between the shape of an excitatory postsynaptic potential (EPSP) and the shape of the CCH. The shape of the CCH does not reproduce the shape of the underlying EPSP. Instead, it is usually found that the shape of the peak in the CCH is similar to the temporal derivative of the EPSP; in other words, the width of the CCH peak is proportional to the rise time of the EPSP. Departure from this relationship is observed in cases of EPSPs of small size compared to the fluctuations of the membrane potential. In such cases the shape of the CCH peak can be reproduced by a linear combination of the EPSP shape and its derivative. Moore et al. (1970) and Bryant et al. (1973) showed further that the shape of the CCH peak is determined not only by the connectivity pattern, but also by the firing pattern of the neurones under study. This pattern is revealed by computing the autocorrelogram (ACH) of the neurone activity (cf. Perkel et al., 1967a). It is done in the same way as the CCH except that the ACH is a histogram of all the intervals (with or without intervening spikes) between the spikes recorded in one neurone. Moore et al. (1970) and Bryant et al. (1973) distinguished between the primary effects, which result from the synaptic interactions, and the secondary effects, which result from the firing pattern of the neurones under study. The ACH can show that some neurones fire action potentials that are not independent of each other. For example a neurone might be bursty, in which case
Cross-correlograms for neuronal spike trains
69
one action potential will usually be followed by the other action potentials of the burst. If a postsynaptic neurone shows a bursty firing, then the CCH will present a broader peak, because a spike in neurone A is followed on average by a burst in neurone B. Another case corresponds to an oscillatory common input neurone: the oscillatory firing will be replicated symmetrically in the CCH, leading to an oscillatory structure. Although the impact of the firing pattern of the cells was recognized in these early papers, many crosscorrelation studies have neglected to take it into account when interpreting the various shapes and widths shown by the peaks in the CCHs. We return to this question in Section 4.
2. SYNCHRONIZATION AND ITS MEANINGS: AN HISTORICAL PERSPECTIVE The cross-correlation technique has been used to characterize the neuronal interactions that take place within the central nervous system. In the cerebral cortex, it has shown that neurones tend to synchronize their activity in the majority of cases. However, this basic result has been open to multiple interpretations. To show how the interpretation of synchronization has evolved in parallel to the conceptual field within which it has been studied, we now trace the history of synchronization in the cortex. 2.1. Awakening and Desynchronization Synchronization of neuronal firing was first associated with sleep and anaesthesia, and was supposed to disappear when subjects or animals were awake. This notion can be traced back to the first studies on electroencephalographic activity, initiated by Berger, in which rest, sleep and anaesthesia were found to be associated with EEG traces showing large-amplitude slow waves, that appeared coherent over large distances. In contrast, awakening was associated with lower amplitude events occurring at a higher frequency, and usually independently at the different recording sites (e.g. Adrian and Matthews, 1934). This led to the hypothesis that the underlying neuronal activity should be synchronized in anaesthesia and sleep, while sensory stimulation and awakening would disrupt and suppress this synchronization. Adrian (1935), for example, stated in 1935 that “[…] if a large mass of nerve-cells is to pulsate with a co-ordinated beat it is clear that all of them must tend to pulsate at the same frequency. If afferent messages excite some to a higher frequency the co-ordinated beat must break down; and so these cortical rhythms are broken up by external stimulation, to return again when the animal is undisturbed”. Or, said in another way by the same author, “[…] groups of nerve-cells very often tend to act in unison when there is nothing to prevent them”. Hence, the first studies in which dual extracellular recording of action potentials was performed in cortex were designed to test the hypothesis that neurones synchronize their
Time and the brain
70
activity during anaesthesia and sleep, while sensory stimulation and awakening suppress this synchrony. Li (1959), recording from pairs of neurones in the somatosensory cortex of cats anaesthetized with a barbiturate, found that spontaneous activity is made up of simultaneous bursts of 5 to 10 action potentials when the animal is deeply anaesthetized. The tendency of neurones to discharge simultaneously tended to decrease as the anaesthesia level was reduced. Holmes and Houchin (1966) conducted a study in urethane anaesthetized rats, and constructed the equivalent of a CCH to detect synchronization of neuronal activity. They found it to be more prevalent in deeply anaesthetized animals, and observed it even for cells that were separated by several millimeters on the cortical surface. Synchronization was reduced, or suppressed, when the anaesthesia was lighter. They also used “strong peripheral stimuli” (which presumably were more like awakening stimuli than subtle sensory stimuli), and found that these also decreased the synchronization. Noda and Adey (1970) recorded the ongoing activity in the suprasylvian gyrus of unanaesthetized, behaving cats. They found that pairs of neurones strongly synchronize their activity during sleep, while synchronization, for the same pairs of neurones, was strongly reduced, if not completely absent, during arousal and rapid eye movement sleep. Noda and Adey (1973) completed their study by testing the action of the anaesthetic pentobarbital on neuronal activity and synchronization. The anaesthetic increased the synchrony and the burstiness of the neurones to a level comparable to that observed during sleep. With this anaesthetic, the bursts of action potentials occurred mostly in coincidence with the spindle waves appearing in the EEG. Burns and Webb (1979) added another argument in favour of a desynchronizing role for afferent sensory stimuli: They managed to deafferent pieces of cortex and to record from them. The neurones in these deafferented cortices synchronized their activities in all the cases, independently of the behavioural state of the cat, while the non-deafferented cortex only rarely showed signs of synchronization, especially when the animal was awake. As a conclusion for this section we quote Burns and Webb (1979): “[…] it is not unreasonable to state that when the brain is processing information, neighbouring cortical neurones are very unlikely to be found doing the same thing at the same time.” Hence experimental results supported the hypothesis of Adrian that neuronal activity is strongly synchronized during sleep and anaesthesia, while synchronization would be absent or at best strongly reduced in the awake brain. However, these studies can be criticized on one point, in that, in most cases, the neurones were studied during spontaneous activity; the effects of the sensory stimulation, if studied, were only poorly controlled. 2.2. Neuronal Connectivity and Common Inputs Within a different conceptual framework, inherited from the study by Perkel et al. (1967b), cross-correlation has been used as a method to provide data on the connectivity between neurones. In this context, what was studied was not the relationship between synchronization of neuronal firing and the behavioural state of the animal; instead, crosscorrelation was used as a tool to test for the presence of synaptic relationships between extracellularly recorded neurones.
Cross-correlograms for neuronal spike trains
71
Compared to other methods, calculating an index of cross-correlation between the firing of two neurones would appear as a better alternative to study their connectivity: the tracing of anatomical pathways says nothing about the physiological properties of the labelled neurones or their target cells. Electrical stimulation, as a method of identifying neurones that are targeted by a given structure, has a number of drawbacks, including the possible activation of axonal tracts issuing from other structures and passing near the stimulating electrode, or the problem of the activation of axon collaterals (e.g. Nowak and Bullier, 1998a,b). One powerful method consists of performing dual intracellular recordings, allowing the physiological and morphological identification of both the preand postsynaptic neurones involved in a synaptic connection. The use of dual intracellular recording in vitro has provided a number of important data (for recent reviews see Thomson and Deuchars, 1997; Markram, 1997), but its low yield and its requirement for long lasting, stable recordings, makes it impossible to use for in vivo mammalian studies. In the mammalian visual system, CCHs displaying narrow peaks offset from the time origin, indicating monosynaptic connections, have been observed in the subcortical visual pathways. Cleland et al. (1971a,b) and Levick et al. (1972) traced the functional connectivity between the retina and the lateral geniculate nucleus (LGN). Among other things, their results showed that a retinal ganglion cell projects on a thalamic neurone with a similar receptive field subtype. They also showed that a small number of retinal ganglion cells, sometimes only one, provides most of the excitatory drive to a thalamic neurone. After these investigations, the study of the thalamocortical pathway followed, with the studies by Tanaka (1983), who showed that ON-centre thalamic neurones provide the main drive to the ON subfield of cortical simple cells. The corresponding CCH peaks, a few msec wide, were displaced from the time origin by a latency 0.9 to 2.7 msec. Tanaka (1983) also showed that, on average, one spike in a thalamic neurone increases the discharge probability of a cortical simple cell by 10%. These results were recently confirmed by Reid and Alonso (1995), although the strength of the thalamic input appeared lower in their study, presumably due to the use of different visual stimuli (see also Swadlow, 1995, for the case of the somatosensory thalamus and cortex). The cross-correlation technique has also been used to study synaptic connectivity between cortical neurones recorded within the same area. After a pioneering work by Griffith and Horn (1963), the first cortical studies were those by Dickson and Gerstein (1974) in the auditory cortex, by Renaud and Kelly (1974) in the motor cortex, by Toyama et al. (1981a, b) and Michalski et al. (1983) in the visual cortex. The CCHs showed significant peaks, indicating temporal coherence of neuronal origin, in roughly half the cases when the neurones were close to each other (e.g. recorded from the same electrode). The incidence of significant CCHs decreased with increasing separation between the electrodes. In contrast to what was observed for the retinothalamic and thalamocortical pathways, the significant CCHs rarely displayed features indicating the presence of a monosynaptic connection: Displaced peaks and displaced troughs were observed in only 5–10% of the cases, and only when the neurones were located close to each other. Instead, most of the CCH peaks were centred on the time origin when neurones were close to each other, and only centred peaks were
Time and the brain
72
observed when they were distant. In other words, the activity of two simultaneously recorded cortical neurones appeared synchronized in the vast majority of the cases even when sensory stimuli were used to activate them. It was then inferred that such synchronized activity reflects the presence of common inputs (Figure 1), from other cortical areas or from subcortical structures. Gerstein and Dickson (1974) concluded that the connectivity between cortical neurones is made by an “arrangement that consists mostly of shared inputs.” 2.3. Functional Connectivity, Synchronization and Cell Assemblies Although cross-correlation was once thought to be a method for tracing connections in vivo, this hope had to be abandoned. The modelling study by Boven and Aertsen (1990) illustrates how a simple monosynaptic connection between two neurones can give rise to different CCHs, depending on the activity of the other neurones in the network. In their model, a neurone 1 projects to a neurone 2, with a relatively small synaptic strength α (Figure 3). For its firing to be influenced by that of neurone 1, it is necessary for neurone 2 to have its membrane potential close enough to the spike threshold, such that the relatively small synaptic events produced by neurone 1 would lead to an action potential. The membrane potential of neurone 2 is determined by the activity in the population of neurones (N) projecting onto it. If the activity in this population of neurones is low, then the membrane potential of neurone 2 would not be depolarized enough to allow the EPSPs produced by neurone 1 to trigger an action potential. As the activity in the population N increases, the probability (called effective connectivity—α’ in Figure 3) that an EPSP produced by neurone 1 leads to an action potential in neurone 2 increases, until this probability saturates. From this we can also extrapolate that if the activity in the population of neurones (N) is high, but out of phase with respect to that of neurone 1, then the signature of the monosynaptic connection will not be seen in the CCH. Furthermore, the size of the peak cannot be interpreted as an estimate of the synaptic strength α’, since it reflects the combined effects of the connection between neurones 1 and 2 and the activity of the network in which they are embedded. At this point, “one should distinguish between structural (or anatomical) connectivity on one hand and functional (or effective) connectivity on the other” (Aertsen and Preiβ1, 1990). The lack of a peak is not proof of a lack of connectivity. Conversely, the presence of a peak in a CCH computed between two neurones indicates not only the presence of a synaptic connection. It also means that the whole cortical network was in a state that allowed the expression of this connection… The appearance and disappearance of centred peaks can be envisaged simply in a situation where one common input neurone, C, projects to two neurones A and B (cf. Figure 1c). Neurones A and B may also receive a number of inputs they do not share, such that their activity with different stimuli might be of comparable strength. Neurone C could be activated by only one stimulus configuration, and only in this case
Cross-correlograms for neuronal spike trains
73
Figure 3 . Relationship between strength of effective connectivity α’ as a function of network activity Nl for different values of the synaptic weight β between the pool of neurones N and neurone 2. Neurones 1 and 2 are related by a monosynaptic connection of synaptic weight a. Nλ is the mean firing rate of the neurones in the pool N driven by the stimulus S. α’ is the ratio of the peak area in the CCH to the number of presynaptic spikes (equivalent to the “contribution” index of Levick et al., 1972). In all cases there is a strong dependence of the effective connectivity upon the activity of the network and a saturation for high activity levels. Note that the variation of the effective connectivity is much larger for small values of the network activity, which is likely to be similar to the case in the cortex, from Boveu and Aertoe (1990)
would a centred peak be visible in the CCH. The anatomical wiring is always present, but may not be expressed in all cases. Obviously the situation in the cortex is far more complex, where one neurone receives several thousands of synapses from neurones located in different cortical and subcortical structures, all with different stimulus selectivities. In support of these theoretical considerations, several experimental studies have shown that the temporal coherence of neuronal origin observed in the spike trains from the same neurone pair depends on the stimulus used to activate them, as well as on the behavioural
Time and the brain
74
context. In the auditory cortex, Frostig et al. (1983) found that 88% of the CCHs were modified by the stimulus presented: some CCHs, showing displaced peaks during spontaneous activity, became flat in the presence of a stimulus while other CCHs behaved in the opposite way. Later studies in auditory cortex confirmed this strong stimulus dependency (Vaadia and Abeles, 1987; Ahissar et al., 1992a; Eggermont, 1994; de Charms and Merzenich, 1996). Changes in correlated firing under different stimulus conditions have also been observed in subcortical structures (e.g. Epping and Eggermont, 1987; Sillito
Figure 4 . Oscillatory coupling between area 17 neurones of opposite cortical hemispheres. Decomposition of oscillatory correlation over peristimulus time. Right and left hemisphere multiunit signals are
Cross-correlograms for neuronal spike trains
75
correlated in response to a bar moving for 3 s, stationary for 5 s and moving back for 3 s. The receptive fields were partially overlapping and had optimal orientations differing by 45 degrees. A: The net peri-stimulus CCH (PSCCH) shows oscillations relative to time difference, with the position of the peak shifting slightly during the movement of the bar, and shifting in the other direction when the bar returns. Lowest panel shows PSTHs for right hemisphere (shaded) and left hemisphere (outlined). Note the high level of spontaneous activity between stimuli. Middle panel shows sections along the PST dimension for time difference windows of −3 to 3 msec (shaded) and 10 to 16 msec (outlined). Right hand panel shows CCHs obtained by summing over PST dimension. B: Array of equivalent Z-values indicating the statistical significance of the coupling. The middle panel shows profiles of the significance along the PST dimension, for time differences at the central peak (shaded) and at a negative lobe (outlined), along with the 2.5% one-sided significance level (dotted) for positive coupling (Z=1.96) and negative coupling (Z=−1.96). Coupling is highly significant during movement of the bar (Z=+/−5), while dropping below significance between movements. The panel to the right shows sections along the time-difference axis of mean Z value for PST in windows 10.3 to 11.3 seconds (shaded), and in the interstimulus period, for PST from 5 to 9 seconds (outlined) along with the one-sided 2.5% significance leves (z=+/−1.96; dotted line). The windows for the horizontal and vertical sections are indicated by brackets on their respective axes. Several lobes of the oscillatory profile during stimulation are significant, while the interstimulus profile is well below significance, in spite of high levels of spontaneous activity, from Nowak et al (1995)
et al, 1994). In frontal and prefrontal cortex, the correlated firing between pairs of neurones can also change in relation to behaviour, with a time course of a fraction of a second (Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995; Riehl et al., 1997). In the visual cortex, the first description of a change in synchronization related to a change of the stimulus-configuration can be found in the paper by Ts’o et al. (1986). They showed an example, where the CCH peak height decreased when the neurones under study were activated with 2 bars with optimal orientation, rather than with one single bar with an orientation in between the two optima. In the visual cortex, stimulus-dependent changes of synchronized activity have been investigated mostly for one form of synchronization, the synchronization of gamma range (between 20 and 80 Hz) oscillation initially described by Gray and Singer (1989) and Eckhorn et al. (1988). One example is shown in Figure 4, for the case of interhemispheric synchronization in cat. These synchronized oscillations are observed under stimulation by a moving stimulus, but not during spontaneous activity (Figure 4). Most importantly, they depend on the configuration of the stimuli in such a way that they are more prominent for one continuous stimulus activating groups of cells together and coherently, than for different stimuli activating the different neuronal populations separately in space and/or
Time and the brain
76
time (Gray et al., 1989; Engel etal., 1991c). In summary, these results show that the temporal coherence in the firing of two neurones is not invariant; instead it depends on the presence of one type of stimulus, and/or on one phase of the behaviour in which the animal is engaged. Therefore crosscorrelation is not simply an anatomical method. However, cross-correlation of spike trains can be used to show that, at a particular moment and in presence of a given stimulus, one cell can be linked to a larger group of cells, and that this link might disappear in other conditions. The acknowledgement of the specificity of this link brought the cross-correlation technique into another conceptual field, that of the cell assembly. The initial concept of the cell assembly was proposed by Hebb (1949) as a link between neuronal activity and behaviour (for other types of cell assemblies see also Gerstein et al., 1989). Hebb, as well as most contemporary neuroscientists, did not envisage a one-to-one relationship between the activity of one neurone and one perception or one behaviour. Instead perception and motor activity had to be based on the activity of pools of neurones (or “population coding”). The novelty of Hebb’s theory resided in the involvement of the connectivity between neurones, and most importantly in the functional expression of this connectivity, as the neuronal basis of perception and behaviour. He defined the cell assembly as: “a diffuse structure comprising cells in the cortex and diencephalon (and also, perhaps, in the basal ganglia of the cerebrum), capable of acting briefly as a closed system, delivering facilitation to other such systems and usually having a specific motor facilitation” (Hebb, 1949, p. XIX of introduction). Hebb (1949) also conceived that memory (in a broad sense) and perception are two aspects of the same phenomenon, supported by the same cell assemblies. Establishment of a memory trace would proceed in two steps: First, reverberation of activity within the cell assembly would provide a physiological basis for a transient memory of the stimulus. Second, this transient memory can become a long lasting one by means of structural changes. The way by which such structural changes could occur is Hebb’s famous rule, according to which the coincidence of pre- and postsynaptic activities leads to a strengthening of the synapse. Strengthening of synapses can indeed be obtained by pairing pre- and postsynaptic activities, as has been established in several studies, both in vivo and in vitro (Baranyi and Szente, 1987; Gustafsson et al., 1987; Frégnac et al., 1988; Bonhoeffer et al., 1989; Ahissar et al., 1992b; Frégnac et al., 1994b; Jester et al., 1995; Markram et al., 1997). Hebb conceived that some basic perceptual capacities are already present at birth. Some inherited neuronal mechanisms therefore would provide the basis for a primary form of segmentation and figure-ground segregation: “The primitive unity of the figure is defined here as referring to the unity and segregation from the background which seems to be a direct product of the pattern of sensory excitation and the inherited characteristics of the nervous system on which it acts” (Hebb, 1949, p. 19).
Cross-correlograms for neuronal spike trains
77
If enough repetitions of the same pattern occur, activations in these low levels would be combined and would lead to the generation of a cell assembly, supposedly taking place in association areas. Once established, the cell assemblies would provide support for the identity of a perceived figure and for the permanence of learning. One important feature of the cell assembly is the presence of convergence and divergence in the projection scheme. This provides one means by which a cell can be activated by different presynaptic elements, which in turn enables pattern completion: presentation of one part of a figure would activate the whole assembly, hence creating the internal representation of the whole. The associational properties of the cell assemblies would also enable the property of figure invariance (the fact that the recognition of an item is independent of the viewing distance, illuminating condition, viewing angle, etc.) and the property of generalization (the ability to recognize a whole class of items after familiarization with a subpopulation of it). The role of temporal coherence in the assembly was not unambiguous in Hebb’s words. On the one hand, temporal correlation of spike activities appears necessary for the strengthening of synapses. It is also clear that once the strengthening of synapses between two neurones has taken place, these will not act independently anymore: “A functional relationship of activity” should result. “The fundamental meaning of the assumption of growth at the synapse is in the effect this would have on the timing of action potentials by the efferent cells” (Hebb, 1949, p. 72). Using present day terminology and technology, this should appear as a displaced peak in a CCH. On the other hand, Hebb explicitly rejected the importance of a tight temporal coherence for the functioning of the assembly: “At each synapse there must be a considerable dispersion in the time of arrival of impulses, and in each individual fibre a constant variation of responsiveness; and one could never predicate a determinate pattern of action in any small segment of the system. In the larger system, however, a statistical constancy might be quite predictable” (Hebb, 1949, p. 76). There is one important limitation in Hebb’s theory. The paradox is that this limitation results exactly from what makes the cell assembly such an attractive theory, and can be traced back to the anatomical basis of the assembly: neighbouring neurones are strongly interconnected, and this presumably results from the application of Hebb’s rule, since neurones located close to each other are activated together by sensory stimuli. The presence of this net-like structure, with its convergence and divergence, creates a multipotentiality by virtue of which one neurone can belong to different assemblies and generalization and completion can be realized. However, when two, instead of one, figures are presented simultaneously and close to each other, it is likely that some neurones will be activated by both of them, such that the resulting increase in activity in these neurones cannot be traced back to one or the other figure: the cell assembly system is not able to encode more than one object without confounding them. This ambiguity as to what figure induced an increased activity in which assembly, was pointed out by von
Time and the brain
78
der Malsburg (1981) and is known as the “superposition catastrophe”. Hence there must be some mechanism that maintains different assemblies, segregated from one another. In terms of anatomo-functional connectivity, there must be a selection process, by which neurones will belong to one assembly or another, but not to two assemblies at the same time. Synchronization of neuronal firing appeared as one possible mechanism by which this selection procedure might be performed. A role for synchronization in this process of selection was first proposed by Milner (1974), who was concerned with the way that even elements as simple as lines can be perceived as different entities: “[…] we must attempt to explain how lines are immediately seen as units or ‘wholes’. One possibility worth considering is that primitive unity depends upon the contiguity of active cells in topographically organized parts of the projection pathway. […] If adjacent, or nearly adjacent, cells interact when excited, in such a way as to synchronize and perhaps intensify each other’s activity, this could provide the unifying characteristic that ties the elements of a figure together. At subsequent levels of the pathway, impulses from cells fired by one whole would arrive as synchronous volleys, whereas impulses from different figures would have a random temporal relationship to each other. The resulting difference in the effectiveness of temporal summation could serve to funnel the signals from different objects in the field into different paths, through the feature detection network, and thus minimize their mutual interference”. 1 A role for synchronization (or using a broader term, “correlation”) in figure-ground segregation was also proposed by von der Malsburg (1981) for whom “a figure is decomposed in parts, the strongest correlations signalling affiliation to one part of the figure, 1. Recent modeling studies by Carlos Brody (J. Neurophysiol. 80:3345–3351; Neural Comput. 11:1527–1536; Neural comput. 11:1537–1551) have shown how artefactual peaks can be obtained in cross correlation histograms, that are the consequences of 1) slow covariation of firing rates and 2) correlation of visual response latencies. Brody’s papers also provide indication as to how such correlations can be distinguished from those related to neuronal connectivity.
weaker ones affiliation to adjacent parts, and so on.” There is a “decomposition of the visual scene into a hierarchy of correlates”. Therefore, the use of a temporal coding, based on short term correlation, would solve the superposition problem—and by extension would provide a mechanism for segmentation and for figure-ground segregation. Neurones activated by one object would synchronize between each other, but not with the neurones activated by another object. Obviously the temporal segregation of neuronal activities can be more efficient if neurones have the possibility of engaging in oscillatory synchronization: then one figure would have its representation in the synchronous oscillation of one population of neurones, while the other population would have its own synchronized oscillation out of phase with those of the first. This provided the basis for a modelling study in which a fully connected set of neurones was nevertheless capable of segmentation by virtue
Cross-correlograms for neuronal spike trains
79
(among other things) of synchronized oscillatory activities (von der Malsburg and Schneider, 1986). Another important question also came into focus, which is how neuronal activity is propagated from one place to another. The basic fact is that one cortical neurone produces only small depolarizations in another cortical neurone. According to in vitro studies the average size of an EPSP elicited by one pyramidal neurone upon another is less than 1 mV in the majority of the cases (for reviews: Thomson and Deuchars, 1997; Markram, 1997). The amplitude of the depolarization required for triggering an action potential in a cortical neurone is of the order of 10–20 mV. Therefore it is required that a summation of EPSPs takes place. The number of EPSPs that have to be summated is much higher when these EPSPs occur independently from each other in time, than when they occur all together, that is, in synchrony. This led to the notion that neurones can behave as coincidence detectors, rather than as temporal summators. The role of synchronization for the propagation of activity is the starting point for the synfire chain concept proposed by Abeles (1982): “According to this hypothesis the activity of the neurons that transmit information is organized along a chain of sets of neurons. Each link in the chain is made of a set of neurons that fire in exact synchrony whenever the chain becomes active. Neurons of each set converge on neurons of the next, and therefore synchronized activity of one set elicits synchronized activity in the next set and so on. […] the relevant code is the combination of neurons that fire in synchrony.” Here synchronization serves a selection procedure, for only those neurones that are synchronized have a chance to transmit their information to the next stage. “This concept of dynamic formation and dissolution of functional chains is similar to the ‘cell assembly’ postulated by Hebb thirty years ago, except for the requirement that the transmission along the chain is secured by the synchronous firing of sets of cells.” One advantage of the synfire chain, is that it allows for the existence of “cross over” such that one neurone can belong to two different chains. Even so, the independence of the two chains is preserved, as required for the segregation of cell assemblies with common neurones, as long as they are not active in exact synchrony. Experimental supports were later given for the existence of synfire chains in the awake behaving monkey (Abeles et al., 1993). The stimulus-dependent changes in synchronization observed in visual cortex (Gray et al., 1989; Engel el al., 1991c; see also Engel et al., 1991b; Ghose et al., 1994; Sillito et al., 1995; Kreiter and Singer, 1996; Livingstone, 1996; Brosch et al., 1997) are consistent with a role for synchronization in the process of segmentation, and appeared as the experimental demonstration of the theoretical ideas developed by Milner, von der Malsburg and Abeles. Hence, this gave support to the notion that “synchrony might be the ‘glue’ that binds distributed neuronal activity into unique representations” (Engel et al., 1992).
Time and the brain
80
2.4. Synchronization and Attention We have just reviewed evidence indicating a role for synchronization in scene segmentation. There are, however, other experiments showing that, in awake behaving animals, synchronization does not always have a clear relationship with sensory stimulation or motor activity. Early studies demonstrated that high frequency oscillatory events are visible, if not prominent, in the EEG traces under behavioural conditions requiring attention and vigilance: Lopes da Silva et al. (1970) observed the presence of a “beta” rhythm (around 20 Hz) above the visual cortex of dogs that were trained to press a lever when the visual stimulation (whole field, sinusoidally modulated light) was ON. This rhythmic signal was absent if the animals were anaesthetized. Rougel et al. (1979) demonstrated the presence of a fast somatoparietal rhythm during combined focal attention and immobility in baboon and squirrel monkey. The frequencies were between 13 and 25 Hz, and appeared independently in two loci (S1 and area 5). Bouyer et al. (1981) reported the presence of 35–5 Hz oscillations, with their focus located in the frontal (area 4 or 6) and parietal cortex (area 5) in immobile cats during focused attention. In these last two studies, the oscillations appeared in cortical areas where no information had to be processed—sensory and motor activities were presumably shut down during this attentive immobile behaviour. Some of the changes in synchronization observed in relation to different phases of behaviour in the frontal cortex of awake monkeys (Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995; but see Riehle et al., 1997) and in cat visual and motor areas (Roelfsema et al., 1997) might be related to changes in attentiveness. In the somatosensory and motor cortex, Murthy and Fetz (1992) observed 25–35 Hz synchronized oscillatory activity, that did not depend on any particular movement or stimulus, but instead appeared to be more prominent during a difficult task presumably requiring attention. Sanes and Donoghue (1993), in a visuo-motor task, attributed the origin of these oscillations to preparation of movements, rather than to their execution, while Baker et al. (1997) suggested these oscillations could be associated with the motor cortex “resting” between demanding motor tasks. Finally, in monkey parietal cortex, Cardoso de Oliveira et al. (1997) showed that synchronization between pairs of neurones is stronger before the presentation of the visual stimulus that the monkey has to attend to, than during its presentation. Studies in humans also indicate several possible roles for gamma oscillations. The power of such oscillations is higher when real and illusory (Kanizsa) triangles are presented than for non-coherent stimuli (Kanizsa triangles with rotated inducers), thus suggesting a role in binding for object perception (Tallon-Baudry et al, 1996). However, using similar recording techniques, the same authors demonstrated that increased power of gamma oscillations can be recorded during a task of active visual search (TallonBaudry et al., 1997) and during the expectation period in a delayed match to sample task (Tallon-Baudry et al., 1998). These results suggest that gamma oscillations may be involved in the recollection of stored representations and in visual attention (see also Llinás et al., 1994).
Cross-correlograms for neuronal spike trains
81
2.5. Summary Four different interpretations have been advanced for the presence of synchronization in the cortex. First, it has been proposed that the synchronization of action potentials between pairs of neurones reflects the EEG waves that characterize the brain in an unconscious state—during sleep or when anaesthetized. Second, synchrony has been considered to be the manifestation of a connectivity pattern typical of the cortex, in which a pair of neurones receives common inputs from another group of cells. Third, it has been proposed that the synchronized firing of two cells indicates their participation in a cell assembly. From this it was proposed that synchronization constitutes another neuronal code, a new dimension for the neuronal representation of sensory inputs and motor outputs, to be added to the rate coding. Finally, it has been shown that synchronization of neuronal activity can occur with no relationship to sensory input or motor output, and reflects the state of the cortical network when the subject is in a state of expectancy or attention.
3. DIFFERENT TYPES OF SYNCHRONIZATION We have reviewed the accumulated evidence that synchronization is the form of temporal coherence that dominates when recording from pairs of neurones in cortex. So far we have shown that synchronization has received different and sometimes contradictory interpretations. However, different interpretations for the presence of synchronized firing may not be contradictory if there is not one, but several forms of synchronization. We will now show that, indeed, different forms of synchronization exist. 3.1. Non-oscillatory Synchronization within One Cortical Area As mentioned earlier, the large majority of CCH peaks obtained from neurones recorded in the same cortical area reflects synchronization of neuronal activity. Different types of synchronization can be distinguished on the basis of the width of the CCH peaks. One common finding among the earliest studies on cortical synchronization, when it was studied in relationship to sleep and anaesthesia, is the presence of rather large CCH peaks. The peaks presented by Holmes and Houchin (1966) were several tens or hundreds of msec wide; the peaks presented by Noda and Adey (1970, 1973) were similarly quite wide on average, although a large variability could be observed in their data, such that narrow peaks a few msec wide as well as very large peaks, several hundred msec wide, were reported. In contrast, in those reports in which neuronal interactions were studied within the context of neuronal connectivity, with sensory stimuli or motor responses, the widths of the peaks often appeared to be narrow. In cat visual cortex for example, Toyama et al. (1981a) reported the presence of very narrow centered peaks 1 msec wide, while
Time and the brain
82
Michalski et al. (1983) reported centered peaks with widths ranging from 10 to 120 msec. Several studies showed that two or three different types of peaks can be distinguished, when attempts have been made to classify them according to their widths (Murphy et al., 1985; Krüger and Aiple, 1988; Krüger and Mayer, 1990; Aertsen et al., 1991; Gochin et al., 1991; Hata et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992). Compilation of these results suggests a relatively clear segregation between three types of CCH: Some narrow peaks, with a width less than about 10 msec when measured at half width; a second class of peaks, (intermediate peaks), with widths between approximately 10 msec and a few hundred of msec, and broad peaks with widths broader than one hundred msec. These different types of peaks have been reported both in anaesthetized animals and in awake behaving monkeys. As a rule, the three different peak types are observed for neurones that are close to each other (for example, recorded from the same electrode), while mostly the intermediate and broad peaks are detected when neurones are separated by more than a few hundred microns and up to several mm (Murphy et al., 1985; Krüger and Aiple, 1988; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992; Cardoso de Oliveira et al., 1997). In primary visual cortex, orientation selectivity is another property that appears to be related to the incidence of different peak types: Narrow peaks are seen more often when neurones share a similar orientation preference, especially if the receptive fields do not overlap (Krüger and Aiple, 1988; Schwartz and Bolz, 1991; Hata et al., 1991; Livingstone, 1996; Tamura et al., 1996). Notice, however, that the probability of observing a peak in CCHs between cross-oriented or iso-oriented pairs of neurones might also depend on the type of stimulus configuration used to activate the cells (Sillito et al., 1995). In primate area V2, Tamura et al. (1996) did not observe any relationship between orientation selectivity and the probability of observing narrow peaks. The occurrence of intermediate peaks does not depend strongly on orientation selectivity (Krüger and Aiple, 1988). 3.2. Non-oscillatory Synchronization between Different Cortical Areas CCHs have been computed for neurones recorded in different visual areas. The incidence of significant peaks, indicating interactions of neuronal origin, seems to be lower than for within-area recording. The only study in which these incidences have been compared under the same conditions (Cardoso de Oliveira et al., 1997) shows a drop from 46 to 15%. In our study on inter-areal interactions in monkey visual cortex, significant peaks were observed in only 10% of the dual single unit recordings. In the other 90% of cases the subtracted CCH appeared flat, indicating the absence of neuronal interactions in the great majority of the cases (Nowak et al., 1999). The coupling strength also seems to be much weaker for between- compared to within-area CCHs (Ana Wang Roe, personal communication). Similar to reports for within-area recordings, the CCHs showed peaks centred on the time origin in the very large majority of the cases (Nelson et al., 1992; Nowak et al., 1995; Wang Roe and Ts’o, 1997; Cardoso de Oliveira et al,, 1997; Nowak et al., 1999).
Cross-correlograms for neuronal spike trains
83
Therefore, neurones belonging to different areas tend to synchronize their activity through mechanisms of neuronal origin. On the basis of the width of the peaks, synchronization between different areas can also be subdivided in three different groups. The tripartite distribution of peak width is illustrated in Figure 5 for three different studies in two animal species: Interactions between areas 17 and 18 in the cat (Nelson et al., 1992), interactions between areas 17 of opposite hemispheres in the cat (Nowak et al., 1995), and interactions between areas V1 and V2 of the same hemisphere in the monkey (Nowak et al., 1999). The tripartite distribution of peak widths becomes quite obvious when one makes histograms of the distribution of the logarithm of the peak width, as shown in Figure 5. From one study to another, there is some variability in the exact positions of the modes and of the gaps between the different groups of peaks. Nevertheless, it is quite clear that inter-areal neuronal interactions produce some narrow peaks, which we called T (“tower”) peaks, with width of a few msec up to approximately 15 msec; some intermediate peaks, which we called C (for “castle”) peaks, with widths between 10–20 msec and 100–200 msec (one example of a C peak is shown in Figure 2C2); and finally, some broad peaks (H peaks, “hills”), with width of several hundred msec up to 2 sec (one example is shown in Figure 2D2). Similar to what has been reported for intra-areal synchronization, the incidence of the different peak types observed in the inter-area studies depends on the separation of the neurones (in this case, not in terms of physical separation but in terms of receptive field separation) and on their orientation selectivities. T peaks are preferentially obtained when neurones have overlapping receptive fields and similar orientation preferences (Nelson et al., 1992; Nowak et al., 1995). C peaks can be observed even if the receptive fields do not overlap; however, the receptive field separation for which C peaks are observed appears bounded by an upper limit of 10 degrees in cat (Nelson et al., 1992; Nowak et al., 1995) and 5 degrees in macaque (Nowak et al., 1999), for the corresponding eccentricities of the recordings. In the cat, the incidence of C peaks is not related to the difference in orientation selectivity when the receptive fields are in overlap. However, C peaks show a weak dependence on orientation preference when the neurones have nonoverlapping receptive fields (Nelson et al., 1992). The broadest peaks (H peaks) occurred without any relationship with receptive field properties, except for a decreased incidence with increased receptive field separation in the monkey V1-V2 study (Nowak et al., 1999). Thus, the different types of peaks observed in the inter-area studies are not homogeneously distributed with respect to the properties of the neurones recorded. 3.3. Synchronized Oscillations Although it has received much of recent attention, gamma-range oscillatory synchronization (GROS, one example of which is shown in Figure 4) was the last one to be described in the cortex, although it was occasionally mentioned in early studies. In terms of incidence, this form of synchronization seems to be totally absent in some reports, but overwhelmingly present in others. One possible reason for this discrepancy might be
Time and the brain
84
Figure 5 . Distribution histograms of the logarithm of the width of CCH peaks. The width is measured at half height. CCHs are calculated between neurones in areas 17 and 18 of the cat (top), between areas 17/18 of both cortical hemispheres in the cat (middle) and between areas V1 and V2 in the macaque monkey (bottom). Note the tripartite distribution of peak width with narrow (T type), middle (C type) and
Cross-correlograms for neuronal spike trains
85
broad (H type) peaks for each case of cross-correlation between neurones located in different cortical areas.
the method used to reveal it: cross-correlation can reveal synchronized oscillations in the spike trains of neurones only if they are regular enough. Other methods are less strict in this regard. For instance, some methods based on signal frequency analysis, or synaptic or field potentials (like the power spectrum analysis) might reveal more or less broad “peaks”, centred close to 40 Hz, that correspond to signals which fluctuate within this frequency range (field potentials: Eckhorn et al., 1988, 1993; Munk et al., 1996; TallonBaudry et al., 1996, 1997; synaptic potentials: Jagadeesh et al., 1992; Gray and McCormick, 1996; Steriade et al., 1996a). Whether these fluctuations are large and regular enough to be termed “oscillations” or “aperiodic oscillations” (Freeman and Barrie, 1994) is not always clear. However, a membrane potential that contains a large amount of power in the gamma-range frequency does not necessarily lead to oscillatory auto- (or cross) correlation histograms (Nowak et al., 1997). The problem of the presence or absence of oscillations then becomes a methodological and terminological issue (see also Young et al., 1992; Frégnac et al., 1994a). When the presence of oscillations is tested in spike trains, another methodological problem appears, that is related to the use of averaging over long periods of recording: there is some variability in the frequency from one oscillatory event to another, if not within one period, such that averaging across multiple oscillatory episodes might result in a smoothing of the side peaks that characterise the oscillations in auto- and crosscorrelograms (Gray et al., 1990; Kreiter and Singer, 1992; Livingstone, 1996; Murthy and Fetz, 1996; Gray and Viana Di Prisco, 1997). Thus, the presence of one isolated peak in a CCH, like a T peak for instance, does not exclude the possibility that the neurones were, at one time or another, engaged in an oscillatory synchronous activity. We will concentrate on the cases where oscillations are clearly visible. Unfortunately, gamma-range oscillations do not constitute a homogeneous ensemble. Instead gammarange oscillations have been shown to occur in 3 different conditions: 1. Gamma-range oscillations have been reported in field potentials and intracellular recordings—but, to our knowledge, not in spike trains—during spontaneous activity in anaesthetized preparations. 40 Hz “brief waves” were observed by Adrian and Matthews (1934) and have been described anew with different anaesthetics, in various cortical areas and in different animal species (Steriade et al., 1996a,b; Brett and Barth, 1997). 2. GROS have also been described, that appear only during sensory stimulation, for neurones recorded within one visual cortical area in the cat (Gray and Singer, 1989; Gray et al., 1989; Eckhorn et al., 1988; Engel et al., 1990, 1991c; Roelfsema et al., 1994; König et al., 1995; Munk et al., 1996; Kruse and Eckhorn, 1996; Brosch et al., 1997; Gray and Viana Di Prisco, 1997) and the primate (area V1: Eckhorn et al., 1993; Livingstone, 1996; area V2: Frien et al., 1994; area MT: Kreiter and Singer, 1992). 3. Synchronized oscillation can also appear without any clear relationship to the presence of stimulation, but presumably in relation to the state of attentiveness (Lopes da Silva et al., 1970; Rougeul et al., 1979; Bouyer et al., 1981; Murthy and Fetz, 1992; Sanes and Donoghue, 1993; Baker et al., 1997).
Time and the brain
86
Gamma-range oscillations can be synchronized between two different cortical areas: Between areas 17 and 18 in the cat (Eckhorn et al., 1988); between areas 17 of the two hemispheres in the cat (Engel et al., 1991a, Nowak et al., 1995); between areas 17 and PMLS of the cat (Engel et al., 1991b). Oscillatory synchronization also occurs between the motor areas of the 2 hemispheres and between somatosensory and motor areas (Murthy and Fetz, 1996). The width of the central peak observed in the gamma oscillatory CCHs (3–5 msec in Engel et al., 1990) is similar to the width of the narrow peaks described above. Furthermore, there are instances in which a non-oscillatory CCH containing one narrow peak can be made to become oscillatory (Schwartz and Bolz, 1991; Nowak et al., 1995). This would suggest that non-oscillatory narrow peaks and gamma-range synchronized oscillation are somehow related. There is however one important difference: the gamma oscillations can be synchronized between neurones that are separated by large distances on the cortical surface (up to 7 mm in cat area 17; Eckhorn et al., 1988; Gray et al., 1989; Engel et al., 1990; König et al., 1995), while the non-oscillatory narrow peaks are restricted to neurones located close to each other, or with overlapping receptive fields (Murphy et al., 1985; Krüger and Aiple, 1988; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992; Nelson et al., 1992; König et al., 1995; Nowak et al., 1995). Similarly, the high frequency oscillations reported in the motor cortex of awake behaving monkeys appear coherent despite significant separation between the recording sites (Murthy and Fetz, 1996) while narrow peaks are observed only for small interneuronal separations (Murphy et al., 1985). In area 17, the stimulus-related GROS have been reported to occur between neurones displaying different orientation selectivities for overlapping receptive fields. Beyond overlap, the probability of observing this type of synchronization is higher when neurones display similar orientation preferences (Gray et al., 1989; Engel et al., 1990; Engel et al., 1991a; Livingstone, 1996). The motion of the stimuli also seems to be an important stimulus parameter to obtain GROS, as they do not appear with stationary stimuli (Engel et al., 1990; but see Ghose and Freeman, 1992). In fact, oscillations turned on by a moving stimulus can be disrupted by stationary stimuli (Kruse and Eckhorn, 1996). Summary Four different types of synchronization can be distinguished on the basis of the shape of the CCH peaks. One form corresponds to the gamma-range oscillatory synchronization. The three other forms correspond to non-oscillatory synchronization, that can be distinguished on the basis of different peak widths. These non-oscillatory forms of synchronization show different incidences depending on the properties of the recorded cells.
Cross-correlograms for neuronal spike trains
87
4. WHERE DOES SYNCHRONIZATION COME FROM? Before trying to determine the underlying mechanisms, we must first review the relationship between the different forms of synchronized activity, the waking state of the brain and the presence of a sensory stimulus. 4.1. Behavioural and Stimulus Dependency of the Different Types of Synchronization 4.1.1. Awakening and anaesthesia We have mentioned above that sleep and waking states, as well as the use of anaesthesia, affect the probability of observing synchronization between pairs of cortical neurones. Here we re-examine this issue with respect to the four different forms of synchronization defined above, in an attempt to determine their origins and functional roles. We have investigated the effects of different anaesthetic agents on the three types of non-oscillatory activities observed between areas V1 and V2 (Nowak et al., unpublished). Three anaesthetic agents have been used, which were isoflurane, halothane (both volatile anaesthetics), and the opiate fentanyl. The level of spontaneous activity was lower by a factor of 1.5 to 2 with isoflurane and halothane compared to fentanyl, but the strength of the visual responses was not significantly different between the three anaesthetics. In terms of incidence and strength of synchronized activity however, there was a major difference. The three non-oscillatory forms of synchronization were observed under all anaesthesia conditions. With isoflurane or halothane, the probability of observing a significant peak in a CCH computed from single unit activities was five time larger than with fentanyl. In parallel with this large difference in incidence, a strong difference was also observed in the proportion of action potentials contributing to the CCH peaks, which was more than twice as high with the volatile anaesthetics compared with fentanyl. This indicates a considerable reduction in the number of action potentials that were correlated by mechanisms of neuronal origin under fentanyl anesthesia. The origin of these differences is not clear, but it may be hypothesized that isoflurane and halothane have a reinforcing effect on the neuronal mechanisms generating sleepassociated rhythms, like the spindle wave and the “slow (<1 Hz) oscillation” (Adrian and Matthews, 1934; Steriade et al., 1993a). For example, the strength of GABAergic inhibition is increased by these anaesthetics (e.g., Franks and Lieb, 1994), which might lead to a strengthening of the spindling mechanisms in the thalamus (Keifer et al., 1994; for the cellular mechanisms of spindle wave generation see McCormick and Bal, 1997; Destexhe et al., 1998). Fentanyl, on the other hand, is likely to act on more specific targets related to nociception, and is less likely to influence directly the thalamocortical
Time and the brain
88
network (e.g., Pasternak, 1993). Therefore most of the C peaks observed under halothane or isoflurane might reflect the presence of spindling activity. In a similar line of evidence, barbiturate anaesthesia is known to induce spindles. At the cortical level this results in cross-correlation peaks with width very similar to that of C peaks (e.g., Contreras et al., 1997). Concerning H peaks, it is interesting to notice their similarity with the peaks observed in CCHs computed during the slow (<1 Hz) cortical oscillations associated with sleep (Amizca and Steriade, 1995). Concerning gamma-range oscillations, Livingstone (1996) found their occurrence not to be influenced by the halothane, nitrous oxide or sufentanyl. Gray and Viana di Prisco (1997) found that the incidence of gamma-range oscillations associated with visual stimulation is not different in awake behaving cats and in cats anaesthetized with halothane. This would suggest that gamma-range oscillations are neither disrupted nor reinforced by anaesthesia. On the other hand, the gamma-range oscillatory activities associated with attentive behaviour are unlikely to be preserved under anaesthesia. In keeping with this observation, Munk et al. (1996) found that stimulation of the reticular formation in anaesthetized cats, as a paradigm that would mimic awakening, increases the incidence and strength of oscillatory synchronization. Similarly, the gamma-range fluctuations that are observed transiently during the slow sleep oscillations become more sustained after stimulation of the mesopontine cholinergic nuclei (Steriade et al., 1996b). 4.1.2. Stimulus dependency The four different types of synchronized activity show different behaviour with respect to the presence of a sensory stimulus. As mentioned above, the GROS observed in the visual cortex occur only during the presentation of a visual stimulation, and are not found during periods of spontaneous activity (see Figure 4). Under fentanyl anaesthesia, we studied the stimulus dependency for two of the three classes of non-oscillatory peaks we observed between areas V1 and V2 of the monkey (Nowak et al., 1999). An example of this stimulus dependency is presented in Figure 2. It shows that the presentation of a stimulus led to the occurrence of a C peak, while during spontaneous activity an H peak was present instead (Figure 2C2 and 2D2). This behaviour summarizes the general tendency we observed. We could not establish any statistically significant trend for T peaks, due to the paucity of their occurrence. We found that C peaks were mostly associated with the presentation of a stimulus (Nowak et al., 1999). Hence, based on the effects of the presentation of a visual stimulus and on the effect of anaesthesia (see above), it might be hypothesized that there are not one, but two types of C peaks, one associated with the processing of sensory signals (see also Aertsen et al., 1991; Cochin et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995 for the occurrence of intermediate peaks in awake behaving animals), the other reflecting sleepand anaesthesia-induced rhythms. The H peaks on the other hand, showed a different behaviour with respect to the presentation of a visual stimulus, suggesting that most of them are associated with spontaneous activity rather than with visual responses. Notice that, contrary to what
Cross-correlograms for neuronal spike trains
89
Figure 2 may suggest, the C and H peaks do not necessarily replace each other depending on whether the neurones were visually driven, or were spontaneously active. For example, C peaks could be found for the visual response period while the CCH corresponding to the spontaneous activity period could be flat. This questions the relevance of H peaks as a neuronal correlate of sensory processing. However, a minority of H peaks do appear during visual stimulation only (Nowak et al., 1995, 1999), and peaks similar to H peaks have been shown to occur in awake, behaving monkeys (Aertsen et al., 1991; Gochin et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995). Here again it might be that two different neuronal mechanisms generate CCH peaks with similar widths. Besides the difference between visually driven and spontaneous activity, the configuration of the visual stimuli can strongly influence the probability of observing synchronization. This has been well documented for the GROS, which tend to occur more often when the neurones are activated by one common stimulus rather than when two stimuli activate them separately or in a conflicting way (Gray et al., 1989; Engel et al., 1991b, c; Kreiter and Singer, 1996; Livingstone, 1996; Brosch et al., 1997). Whether non-oscillatory synchronization is modified when different types of stimulus configuration are used to activate the cells has not been studied as extensively as in the case of oscillatory synchronization. In cat area 17, Engel et al. (1991c) showed that nonoscillatory CCHs display a stimulus dependency similar to that shown for the oscillatory synchronization. In cat and monkey primary visual cortex, Sillito et al. (1995) showed examples of narrow CCHs that depended on the orientation context of the stimuli used. Some H peaks observed between the two hemispheres in cat appeared to be disrupted by the use of conflicting stimuli (Nowak et al., 1995). The stimulusdependent synchronization observed in frontal and auditory cortex of awake animals seems to apply for all the different types of peaks (Frostig et al., 1983; Vaadia and Abeles, 1987; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Ahissar et al., 1992a; Vaadia et al., 1995). Altogether these data indicate that stimulus dependent changes are not restricted to gamma-range synchronous oscillations: they can also be observed for all the different types of non-oscillatory synchronization. 4.2. Neuronal Mechanisms Generating Different Types of Synchronization 4.2.1. Common inputs The peaks centred on the time origin in a CCH may result from the activation of the two recorded cells by a third cell, or by a pool of common presynaptic neurones. In the retina, two neighbouring ganglion cells can show synchronized firing within a short time scale, and Mastronade (1983a) concluded that this was due to a common input from amacrine cells. In the lateral geniculate nucleus of the cat, Sillito et al. (1994) described a form of synchronization, characterized by relatively broad peaks (whose width was in the range of tens of msec), that appears between neurones with non overlapping receptive fields,
Time and the brain
90
when they are simultaneously activated by light bars or gratings long enough to cover both receptive fields. This form of synchronization also depends on the orientation of the stimulus with respect to the alignment of the receptive fields. It is not observed, or is much smaller, when small spots of light covering only each receptive field are flashed simultaneously. This form of synchronization disappears when the visual cortex is lesioned. This suggests that this form of thalamic synchronization is due to common input fed back from cortical layer VI. As already stressed, synchronization of neuronal origin is the most often encountered type of neuronal interaction found in the cortex. Dickson and Gerstein (1974) calculated that the source of common inputs responsible for the synchronized activity they observed within the auditory cortex should be located more than 1.5 mm away from the recorded area. This obviously suggests the thalamus as one possible structure containing the neurones providing the common inputs. However, Engel et al. (1991c) and Nowak et al. (1995) showed that gamma-range oscillatory synchronization, as well as the three types of non-oscillatory synchronization, are present in the interactions between areas 17 of the two hemispheres. Since the projections of the thalamus onto the visual cortex are strictly ipsilateral, this shows that synchronized activity can be obtained without common thalamic inputs. Obviously this does not mean that the thalamic inputs do not contribute to the synchronization observed within the same hemisphere, although in this case, it is unlikely to be the source of shared inputs to neurones that are separated by a distance larger than the width of the terminal arbor of thalamic axons (Nelson et al., 1992). The corpus callosum contains only axons of cortical origin. Engel et al. (1991c) showed that lesion of the corpus callosum suppressed all instance of GROS between the two hemispheres. Munk et al. (1995) extended this result to the case of T and C peaks. This indicates that the corticocortical connections transiting through the corpus callosum are required for the intermediate and narrowest type of coupling, whether or not it is oscillatory. H peaks, on the other hand, remained quite numerous if only the posterior half of the corpus callosum was cut. They were reduced in incidence, although still not completely suppressed, only after 80% of the corpus callosum was severed (Munk et al., 1995). What types of connections sustain the different types of interhemispheric synchronization? Two types of corticocortical connections are present in the corpus callosum. The first corresponds to the fibres that link together the two areas 17. This group of connections is homologous to the horizontal connections within area 17. The other set of axons corresponds to feedback corticocortical connections: the areas 17 of both hemispheres receive common feedback inputs from several “higher” cortical areas (areas 19, 21a, PMLS and PLLS), by means of axons that bifurcate such that one branch innervates the hemisphere ipsilateral to its parent neurone; while the other innervates the hemisphere contralateral to it. Notice also that some form of “functional” bifurcation can occur if two adjacent neurones projecting one to one side, the other to the other hemisphere, are made to synchronize their activities through mechanisms of neuronal origin. Interestingly, C and H peaks are observed for separations that correspond to the known values of divergence for the feedback connections. The next step in the study of Munk et al. (1995) was therefore to establish the role of one or the other set of connections in the generation of synchronized activity. To this end, extensive lesions of
Cross-correlograms for neuronal spike trains
91
the extrastriate cortex were made, aimed at sparing only area 17 and the neighbouring strip of area 18 that represents the vertical meridian. In this way, the feedback connections were removed and the route for the polysynaptic propagation of activity through multiple cortical steps was interrupted as well. The corpus callosum was left intact in these instances (hence the “horizontal connections”). Despite these extensive lesions, the strength of the visual responses was not different from that in intact animals. The results showed that narrow peaks were not strongly affected. Similarly, gammarange oscillations were still observed, although their strength appeared to be reduced (Munk et al., unpublished). However, the incidence of C peaks was reduced, approximately by one half, in single unit paired recordings, and that of H peaks was reduced by two thirds. Together with this reduced incidence, a reduced coupling strength was observed for those H and C peaks that remained. H peaks remained unless all but the most anterior portion of the callosum was removed, but disappeared when the direct feedback inputs were removed. From this we can conclude that a majority of H peaks seem to reflect the transmission of a slow rhythm reflecting the anaesthetized or sleeping state of the animal, by way of multiple, polysynaptic pathways through the cortical mantle. Half of the C type synchronization seems to be associated with monosynaptic common inputs provided by feedback connections. It is not clear however, if higher cortical areas contain common input neurones that generate C peaks, or whether they are involved in modulating activity in a way that would make neurones more bursty, hence generating larger peaks (see below). T peaks and GROS do not seem to require a common input source from higher areas; they seem to be supported by the direct reciprocal connections between the two hemispheres. 4.2.2. The emergent properties of the cortical network At this stage we are faced with a paradox for the generation of the narrowest CCH peaks (T peaks and gamma-range oscillatory synchronization): They seem to be generated solely by the reciprocal connections linking the two synchronized neurones, but they are nonetheless centred on the time origin despite the large separation and large conduction delay between the recording sites. How is this possible? This issue has been addressed by modelling studies. A number of such studies have shown that networks composed of realistic neurones, interconnected with realistic synaptic strengths and conduction delays, show synchronized oscillatory activities when activated by an “input” that does not contain any temporal structure (some of the studies relevant to this question are: Sporns et al., 1989; Bush and Douglas, 1991; König and Schillen, 1991; Lytton and Sejnowski; 1991; Wilson and Bower, 1991; Ritz et al., 1994; Whittington et al., 1995; Bush and Sejnowski, 1996, Hansel and Sompolonski, 1996; Wang and Buzsáki, 1996; Lumer et al., 1997a,b). These studies obviously differ in their details but all suggest that synchronization can be obtained as an emergent property within the cortical network. These models provide a counter-intuitive result, namely, that the presence of a conduction delay between neurones does not prevent them from being
Time and the brain
92
engaged in synchronized activity. This is determined by the connectivity pattern, characterized by its divergence (one cell projects to hundreds of other cells), by the fact that excitatory neurones project to both excitatory and inhibitory neurones (and vice versa), by the small strength of the individual synapses, to some extent by the membrane properties of individual neurones, and, most important, to GABAergic inhibition that plays a key role in oscillatory synchronization. Model studies specifically designed to recreate the gamma-range oscillatory activity showed that neurones can be synchronized as long as the conduction delay is less than, approximately, one fourth of the oscillation period (see also Eckhorn, this volume). The presence of oscillatory forms of synchronization between the two cortical hemispheres for example, is not incompatible with the large distance between neurones and conduction delays up to about 5 msec—the conduction delay between the two hemispheres in cat cortex is below 5 msec in the majority of the cases (Harvey, 1980; Innocenti, 1980). Another issue related to the generation of gamma-range synchronous oscillation is the requirement for pacemaker neurones and their location. Some experimental results suggest that fast rhythms might be generated through synaptic interactions taking place within the cortical network, without pacemaker neurones, since oscillatory activity can be independent of the membrane potential of the recorded neurones (Bringuier et al., 1997). In some of the modelling studies mentioned above however, synchronized oscillations were obtained only after including excitatory neurones discharging in bursts of action potentials. Neurones displaying this property might be the “chattering cells”, recently described in cat cortex (Gray and McCormick, 1996; Steriade et aL, 1998), that have the property to discharge repetitive bursts of action potentials at frequencies up to 70 Hz. (However the majority of these neurones depart from “pure” pacemaker neurones in the sense that the burst frequency depends on the strength of their activation [Gray and McCormick, 1996].) In addition, some cells display subthreshold oscillations in their membrane potential when depolarized by DC injection (Llinás et al., 1991; Nuñez et al., 1992). Oscillatory activities in the gamma range do occur in subcortical structures such as the retina and the visual thalamus (Rodieck, 1967; Robson and Troy, 1987; Funke and Worgotter, 1995; Ghose and Freeman, 1992; Neuenschwander and Singer, 1996). It should be pointed out however, that the apparent “rhythmicity” observed at the level of some retinal ganglion cells during sustained activity basically reflects the regularity (low variability) of the discharges and that the frequency shown by these “rhythms” changes widely depending on the luminance conditions (Rodieck, 1967; Robson and Troy, 1987; Troy and Robson, 1992). According to Bouyer et al. (1981), the posterior thalamic group could be the pacemaker for the 40 Hz rhythms observed in the parietal cortex of attentive cats. Cells with rhythmic discharges within the gamma range have also been reported in the reticular and the intralaminar nuclei (Pinault and Deschênes, 1992a; Steriade et al., 1993c). Other modelling studies therefore, have suggested that neurones in the thalamus act as pacemakers for the production of cortical 40 Hz oscillations (Llinás et al., 1994; Funke and Worgotter, 1995; Ghose and Freeman, 1997). Nevertheless, these models cannot account for the presence of oscillations synchronized between the two hemispheres.
Cross-correlograms for neuronal spike trains
93
4.2.3. Generation of different peak widths Compared to the case of synchronized oscillations, the other forms of synchronization, despite being more commonly observed, have received less interest from modelling studies (but see: Erb and Aertsen, 1992; Koch and Schuster, 1992; Abeles et al., 1994; Arnoldi and Brauer, 1996; Hansel and Sompolinski, 1996). One aspect that has not received much attention is the variability of peak widths observed in the CCHs. Experimental data, however, can give a hint about how different peak widths (and therefore different peak classes) can be obtained. As far back as 1970, it was shown that the width of a CCH is strongly influenced by the presence of burstiness in one or both of the neurones (Moore et al., 1970). More recent studies have shown that variable amounts of burstiness generate different CCH peak widths. First, there is a similarity between the width of a CCH and the width of the ACHs of one or both of the neurones of the pair (Gochin et al., 1991; Nowak et al., 1995). Second, replacing the bursts by the first action potential of the burst, or using deconvolution methods, usually leads to a narrowing of the peak width (Mastronarde, 1983b; Eggermont et al., 1993; Eggermont and Smith, 1996). Recognizing that burstiness accounts for the width of a CCH peak just moves the problem a step farther: what neuronal mechanisms are responsible for the presence of bursty firing? Burstiness can be an intrinsic property of the neurones. Cortical neurones have been classified in different types, based on their spike response to current pulse injection in vivo and in vitro. Among these different types of neurones, two are characterized by their propensity to generate bursts of action potentials. One type corresponds to the “intrinsic bursting neurones” identified both in vitro (Connors et al., 1982; McCormick et al., 1985; Jones and Heinemann, 1988; Chagnac Amitai et al., 1990; Mason and Larkman, 1990) and in vivo (Pockberger, 1990; Baranyi et al., 1993; Gray and McCormick, 1996). The other category has been observed in vivo and corresponds to the “chattering cells” (Gray and McCormick, 1996; Steriade et al., 1998) that differ from the “intrinsic bursting” neurones by their thinner action potentials and their ability to discharge bursts in a sustained manner. Nevertheless, in both types of cells, each burst lasts for only a few msec. This would produce peaks in ACHs of a few msec width only. This would account only for the larger width shown by the narrowest class of CCHs. Intrinsic membrane properties of cortical neurones cannot therefore explain the width of the intermediate and broad CCH peaks. Neurones in the thalamus, provided they are sufficiently hyperpolarized, can discharge bursts of action potentials. The duration of a typical burst in the thalamus is of the order of ten to several tens of msec (Jahnsen and Llinás, 1984; Kim et al., 1995; Guido and Weyand, 1995). During spindle waves, these bursts are strongly synchronized such that the thalamus can be brought into a globally synchronized state (Kim et al., 1995; Contreras et al., 1997). In these conditions, the topographic organization of the thalamus, and the lack of direct interactions between relay cells, do not prevent neurones located in separate territories from behaving in unison. The corresponding ACH and CCH peaks display widths of several tens of msec (Kim et al., 1995). This synchronized activity,
Time and the brain
94
transmitted to cortex, would produce bursts of action potentials there, that would not be of intrinsic but of synaptic origin. This would generate CCH peaks with a width compatible with that of intermediate (C) peaks between spatially remote loci in cortex. Therefore, some intermediate peaks (those that are associated with sleep and anaesthesia) could be generated through the thalamocortical network, with a prominent role played by the membrane properties of thalamic neurons. For those C peaks that appear without a relationship to spindle activity (for example during sensory stimulation only, or in awake animals), it can be expected that the synaptic burstiness originates from within the cortex itself. If it is the same C peaks that appear to depend on feedback connections, then it could be that feedback connections are able to control the duration of cortical neuronal bursts. Since bursts are more likely to produce the temporal summation that is required to trigger action potentials in a post synaptic neurone, it could be hypothesized that feedback inputs from higher cortical areas would ultimately control the transmission of activity through the cortex. We have proposed that the majority of H peaks are correlates in the spike synchronization domain of the slow (<1 Hz) oscillations associated with sleep (Adrian and Matthews, 1934; Steriade et al., 1993a). Steriade et al. (1993b) showed that the slow oscillations survive extensive lesion of the thalamus, which indicates that it is generated within the cortex. The neuronal mechanisms that lead to the slow build up of the associated depolarizing synaptic potential, and to its resetting, remain to be elucidated. 4.3. A Global Picture As we entered into a more detailed examination of cortical synchronization, we found that there are not one, but four different forms of synchronization in cortex. Then, examining the stimulus and sleep/waking dependency of these different forms of synchronization, as well as the possible mechanisms that could account for their generation, we are forced to recognize that there might also be some subclasses within the different forms of synchronization. 1. The gamma-range synchronized oscillations might not be a single form of synchronization. One form appears in the membrane potential and in the field potential of neurones when animals are asleep or anaesthetized. Another form of gamma-range synchronized activity, that displays a strong stimulus dependency, depends apparently on the direct reciprocal connections between the recorded cells, although subcortical structures might contribute as well. There is a third form of gamma-range synchronized oscillation, that cannot be associated with the presentation of a stimulus, but which seems to be associated with attentiveness and expectancy. Whether these three forms of gamma-range synchronization share similar mechanisms for their generation, or whether their similarity is just coincidental, remains to be determined. 2. Non-oscillatory, narrow peaks. There are two types of narrow peaks. Those that are neatly separated from the time origin can be considered as representing a serial synaptic connection between the two recorded cells; this form of temporal coherence nonetheless appears extremely rare in the cortex. The majority of narrow peaks are centred on the time origin. They could be produced by common inputs from other
Cross-correlograms for neuronal spike trains
95
structures, especially when they are observed between neurones located close to each other. When the participation of common inputs can be dismissed, they appear to be generated by the direct reciprocal connections between the two recorded cells. 3. Intermediate and C peaks. Because they are observed between neurones that are spatially separated, it is necessary to postulate that they are generated through mechanisms that do not respect a strict topographic organization. In the visual cortex, feedback connections do not respect the retinotopic organization of the lower cortical areas, and it has been shown that these connections contribute to their generation. The feedback connections could further sustain the stimulus dependency exhibited by this type of synchrony. Another form of C peak could be related to spindle wave generation in the thalamus during sleep and anaesthesia. In that case their width could be related to the interaction taking place within the thalamocortical network. 4. Broad and H peaks. A majority of H peaks seem to reflect the transmission of a slow rhythm reflecting the anaesthetized or sleeping state of the animal, through multiple, eventually polysynaptic pathways. Since this form of coupling does not really show any relation to sensory processing, it is not surprising to observe it between neurones that have nothing in common in terms of receptive field properties. Nevertheless, there is another form of broad peak, that is observed in awake behaving animals. In these conditions too, they are observed between neurones that are spatially separated from each other. Since this type of synchrony can change with respect to the behavioural state of the animal, it is tempting to propose at it reflects slow, global changes, associated perhaps with the generation of sustained attention.
5. FUNCTIONAL ROLES FOR SYNCHRONIZATION We have shown that synchronization, in its different forms, can be explained as the consequence of the anatomical organization of corticocortical and thalamocortical connections and of their functional expression. The question to be solved now is whether synchrony is only the consequence of the network at work, or whether synchronization is also used for some purposes. In this attempt to examine the functional significance of synchronization, we will not examine the significance of the synchronization that occurs in the states of sleep and anaesthesia. Likewise, we will not discuss the importance of synchronization for plastic changes of neuronal connectivity. 5.1. Multiplexing of Information Synchronization of neuronal origin can be explained by the presence of common input to the two recorded cells. In two studies, one in retina (Meister et al., 1995) and the other in cortex (Ghose et al., 1994b), attempts have been made to map the receptive field of the synchronized action potentials. One result from the two studies is that the receptive field corresponding to the synchronized action potentials can differ, in size and properties,
Time and the brain
96
from those of the two recorded units. This means that the synchronized action potentials from two cells contain information not available from the response of each cell taken individually. By this way there would be a multiplexing of information, allowing for an increased amount of information transmission in the axons. However attractive this hypothesis is, it requires that neurones beyond the synchronized ones be able to demultiplex the inputs they receive. Whether such a subtle information processing can be performed in the cortex remains to be determined. 5.2. Binding by Synchronization 5.2.1. Recent results, challenges, and constraints Synchronization of neuronal activity has been proposed as a solution to the binding problem. In short, neurones that respond to the same object would synchronize their activity, while remaining desynchronized from neurones responding to different objects. Synchronization would provide a means of selecting specific neuronal populations, by coincident arrival of EPSPs. Electrophysiological evidence has been provided, showing that neurones synchronize their activities when activated by a common stimulus, but remain desynchronized when activated by conflicting stimuli (see above). Additional support for a role of synchronization in perception was given in two studies. In the first one, Roelfsema et al. (1994) studied amblyopia (poor vision in one eye) that was induced by strabismus in cats: cats are not able to perceive gratings of high spatial frequency through their amblyopic eye, but the strength of the neuronal responses to these high frequency gratings is similar for the two eyes. What differed instead between the two eyes was the ability for neurones to synchronize, which was lower for neurones activated by the amblyopic eye, compared to those activated by the normal eye. A deficit in perception appears correlated with a deficit in synchrony. In a second study, Fries et al. (1997) studied the effects of interocular rivalry in awake strabismic (but not amblyopic) cats. They found that presenting two competing stimuli to both eyes did not affect the strengths of visual responses. However, in the condition of rivalry, synchronization was stronger between neurones activated by the dominant eye, and lower in neurones activated by the non-dominant eye. This suggested that “at early stages of visual processing the degree of synchronicity rather than the amplitude of the responses determines which signals are perceived and control behavioural responses.” Additional support for a role of synchrony in binding comes from large scale modelling studies (Neven and Aertsen, 1992; Tononi et al., 1992). Based on Hebbian cell assembly organization, these artificial neural networks are able to learn and to recognize stimuli. In doing so they can solve the in variance and the occlusion problems. However, as already stressed, segmentation cannot easily be performed in a pure Hebbian cell assembly system. In these models therefore, it is synchronization that has been used to tag which parts belong to the same object. Hence these modelling studies demonstrate the plausibility of synchronization as a means of solving the binding problem. Nevertheless, there are a number of recent experimental studies that have challenged
Cross-correlograms for neuronal spike trains
97
the binding-by-synchrony hypothesis. First, Gochin et al. (1991), recording in inferotemporal cortex of awake behaving monkeys, found that different visual stimuli did not produce different CCH peaks, despite the differences of neuronal responses. Likewise, Gawne and Richmond (1993) in inferotemporal cortex, and Gawne et al. (1996) in primary visual cortex of awake behaving monkeys, found that correlation in the millisecond range does not correlate with the similarity of the responses of the neurones to a battery of different stimuli. This does not prove that synchrony does not play a role, but it suggests that a rate coding, eventually combined with a slow temporal coding, would do a better job in discriminating between the different stimuli used in these studies, than would a code based solely on synchronized activity in the millisecond range. Second, a group of studies in the motor cortex has failed to find a clear relationship between oscillatory synchronization and movement execution (Murthy and Fetz, 1992; Sanes and Donoghue, 1993; Baker et al., 1997). These results do not argue against a role for synchronization in binding in sensory cortex. However, the recent study by Cardoso de Oliveira et al. (1997), in parietal visual cortex of monkey, showed the intriguing result of a suppression of synchronization by visual stimulation, rather than an enhancement. In awake behaving animals therefore, synchronization might have a stronger relationship to attention than to segmentation. It has been suggested that 40 Hz oscillatory activities are instrumental for the generation of long range synchronization (König et al., 1995). Therefore perturbing 40 Hz oscillations should prevent synchronization, and ultimately prevent accurate perception. In two psychophysical studies however (Fahle and Koch, 1995; Kiper et al., 1996), it was found that perceptual performances are neither improved not impaired by the use of stimuli flickering in time near 40 Hz. If synchronization plays a role in binding, then it could be argued that either oscillations are not required for the generation of synchronization, or that non-oscillatory forms of synchronization play a role. Thorpe et al. (1996) demonstrated that it takes only 150 msec of processing for recognition of a visual image. The images used in this study were complex colour photographs, therefore requiring binding and figure-ground segregation to allow recognition. This does not argue against a role of synchronization in segmentation, but this imposes serious constraints on the onset of synchronous activity with respect to the onset of the neuronal responses in different cortical areas. Therefore, one important issue that remains to be investigated is that of the onset of synchronized activity in cortex.
Time and the brain
98
5.2.2. Coincidence detection The cornerstone of the binding-by-synchronization hypothesis is the requirement for neurones to behave as coincidence detectors: an action potential is triggered only after enough EPSPs have been summated (see above). The time it takes for this summation to occur corresponds to the integration time. The largest time window within which the summation of two EPSPs can take place is approximately equivalent to the time constant of the neurone. Beyond this time interval, the depolarization elicited by the first EPSP has vanished, since the membrane potential has returned to its resting level. Theoretical and modelling studies show that, for the same mean presynaptic firing rate, the number of randomly occurring EPSPs required to fire an action potential is much larger than the number of EPSPs required if they occur in synchrony (Abeles, 1982; Bernander et al., 1991; Reyes and Fetz, 1993b). This makes the neurone extremely sensitive to the occurrence of synchronized EPSPs. For this reason it has been proposed that cortical neurones may behave as coincidence detectors. All neurones perform temporal summation. The distinction between the temporal integrator and the coincidence detector resides in the time window during which this summation is accomplished. König et al. (1996) proposed a frontier between the two types, such that a neurone is considered as a coincidence detector if the integration time is smaller than the mean interspike interval, while the integration time of a temporal integrator would be equal or larger than the mean interspike interval. The critical value would be around 5 msec. Experimental measurements of the integration time of cortical neurones during sensory stimulation are still lacking. However, the variability of action potential timing has provided one entry to study the way neurones integrate synaptic activity: For a constant mean firing rate, the interspike intervals (ISI) can be highly variable, such that short and long ISIs can be observed adjacent to each other. The distribution of ISIs is then characterized by its skewed nature, giving a Poisson-like shape to the ISI histogram. The coefficient of variation (the standard deviation divided by the mean ISI) is often close to unity, indicating apparent randomness of spike timing (e.g. Softky and Koch, 1993). The variability of ISIs has been noticed since the earliest electrophysiological investigations of the central nervous system. Softky and Koch (1993) attempted to determine its origin by confronting experimental data obtained from awake monkey visual cortex with different computer models of synaptic integration. One important conclusion they reached is that the variability observed in vivo is incompatible with the integration of random EPSPs that are simply summated: if EPSPs of small size have to be summated in order to yield a constant mean firing rate, then the total synaptic current is almost constant in time, and action potential timing would not vary. One way, however, to obtain the high variability in the membrane potential and therefore in spike timing, is to have the neurone depolarized not by small random EPSPs, but by large compound EPSPs. In other words, it is possible to explain the presence of a high variability of action potential timing if the action potentials are triggered by synchronous EPSPs. This provided evidence favouring the role of synchronization in determining action potential triggering. In the light of these results, it was proposed that, indeed, cortical neurones do
Cross-correlograms for neuronal spike trains
99
behave as coincidence detectors. Unfortunately, later studies have indicated that variability of action potential timing might be explained by other mechanisms. Shadlen and Newsome (1994) showed that variability of action potential timing can be obtained if neurones integrate not only random EPSPs, but also random IPSPs. For this neuronal model to work in accordance with the in vivo data, the EPSPs and the IPSPs have to be balanced (that is to say, the total inhibitory current should be equivalent or close to the total excitatory current a neurone receives). The role of inhibition in generating spike timing variability has been confirmed in other modelling studies (Hansel and Sompolinski, 1996; van Vreeswijk and Sompolinski, 1996; Schindler et al., 1997; Troyer and Miller, 1997). Large scale modelling studies might provide an indication about the usefulness of synchronization in eliciting activity in cortical neurones. In their model, Lumer et al. (1997b) could modify the amount of synchrony between neurones without changing other parameters (a procedure impossible to achieve experimentally). They found that reducing synchronization decreases the firing rate of the neurones in the network. However, in the worst case, the firing rate was not reduced to zero, but to a value approximately half the original one. This suggests that synchronization can help the transmission of signal and maintenance of activity, but also that synchronization is not absolutely necessary for spike generation. Altogether it could be concluded that neurones might behave as coincidence detector, in which case synchronization would help transmitting information. It does not mean, however, that neurones cannot work as temporal summators. The experimental demonstration that cortical neurones are activated by synchronized EPSPs remains to be done. 5.2.5. Alternatives to the binding-by-synchronization The central issue in the question of segmentation within a Hebbian cell assembly is that of the selection, among those anatomical connections that are present, of those that have any relevance to the stimulus. Synchronization has been proposed as one possible mechanism, but it is important to ask whether there are other neuronal mechanisms of segmentation. Another possible solution to the segmentation problem, which at the same time provides the correct selection of neuronal pathway, is to suppress the possibilities of cross talk: this is falling back into the labelled line and cardinal neurone hypotheses, that have been largely criticized (for example for the lack of flexibility they imply), and will not be discussed further here. Another possibility to be considered is that the selection among a set of anatomical connections does rely on the inhibition of the activity of neurones that are not relevant to the situation. Centre-surround interactions, that have been well documented in various visual cortical areas (Nelson and Frost, 1975; Allman et al., 1985; Schein and Desimone, 1990; Knierim and Van Essen, 1992; DeAngelis et al., 1994; Lamme, 1995; Sillito et al., 1995), might play this role. In short, centre-surround interactions are such that, as long as no discontinuity is present in a stimulus pattern, the activity of neurones is actively
Time and the brain
100
maintained at a relatively low level. It is only when there are discontinuities that neuronal activities are higher, and therefore are more likely to transmit information to the next processing stage. In this case the transmission of information does not have to depend on the presence of synchronization, whether or not it is oscillatory, and the postsynaptic neurones do not have to behave as coincidence detectors. In the future it will be important to determine, for the same neurone-pairs, which of the two mechanisms (selection by synchronization, selection by inhibition) have the largest impact on neuronal processing. In conclusion, we have shown that there are multiple types of synchronization in cortex. These different forms of synchronization can be related, to some extent, to different states of the cortical network related to sleep and anaesthesia, to information processing, or to attentiveness and expectancies. Synchronization associated with information processing has been proposed as a solution to the binding problem. However, not all experimental results seem to agree with this notion, and alternative mechanisms besides synchronization exist, that could also potentially play a role in segmentation and figure-ground segregation. Whatever the outcome of this question, synchronization of neuronal activity will remain an important phenomenon, for it reflects, in one way or another, the functioning of the cortex.
ACKNOWLEDGEMENTS Thanks to Joshua Brumberg for comments and help with the English. Thanks to Naura Chounlamountri for help with the Figures.
REFERENCES Abeles, M. (1982) Local cortical circuits: An electrophysiological study . Springer Verlag. Abeles, M. (1991) Corticonics: Neuronal circuits of the cerebral cortex . Cambridge University Press. Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993) Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology , 70, 1629–1638. Abeles, M., Prut, Y., Bergman, H., and Vaadia, E. (1994) Synchronization in neuronal transmission and its importance for information processing. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal Coding in the Brain , Springer Verlag, Heidelberg, pp. 39–50. Adrian, E.D. (1935) The electrical activity of the brain. Proceedings of the Royal Society of Medicine , 29 , 197–200. Adrian, E.D., and Matthews, B.H.C. (1934) The interpretation of potential waves in the cortex. Journal of Physiology (London) , 81 , 440–471. Aertsen, A., and Gerstein, G.B. (1985) Evaluation of neuronal connectivity: sensitivity of cross-correlation. Brain Research , 340 , 341–354. Aertsen, A., and Preißl, H. (1990) Dynamics of activity and connectivity in physiological neuronal networks. In: H.G.Schuster (ed.) Nonlinear Dynamics and Neuronal Network , VCH, pp. 281–301.
Cross-correlograms for neuronal spike trains
101
Aertsen, A., Vaadia, E., Abeles, M., Ahissar, E., Bergman, H., Karmon, B., Lavner, Y., Margalit, E., Nelken, I., and Rotter, S. (1991) Neural interactions in the frontal cortex of a behaving monkey: signs of dependence on stimulus context and behavioral state. Journal für Hirnforschung , 32, 735–743. Ahissar, M., Ahissar, E., Bergman, H., and Vaadia, E. (1992a) Encoding of sound-source location and movement: Activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. Journal of Physiology (London), 67, 203–215 Ahissar, E., Vaadia, E., Ahissar, M., Bergman, H., Arieli, A., and Abeles, M. (1992b) Dependence of cortical plasticity on correlated activity of single neurons and on behavioral context. Science, New York , 257, 1412–1415. Allman, J., Miezin, F., and McGuinness, E. (1985) Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception , 14 , 105–126. Amzica, F., and Steriade, M. (1995) Short- and long-range neuronal synchronization of the slow (<1 Hz) cortical oscillation. Journal of Neurophysiology , 73, 20–38. Arnoldi, H.M., and Brauer, W. (1996) Synchronization without oscillatory neurons. Biological Cybernetics , 74, 209–223. Baker, S.N., Olivier, E., and Lemon, R.N. (1997) Coherent oscillations in monkey motor cortex and hand muscle EMG show task-dependent modulation. Journal of Physiology (London) , 501 , 225–241. Baranyi, A., and Szente, M.B. (1987) Long lasting potentiation of synaptic transmission requires postsynaptic modification in the cortex. Brain Research , 423, 378–384. Baranyi, A., Szente, M.B., and Woody, C.D. (1993) Electrophysiological characterization of different types of neurons recorded in vivo in the motor cortex of the cat. I. Patterns of firing activity and synaptic responses. Journal of Neurophysiology , 69, 1850–1864. Bernander, O, Douglas, R.J., Martin, K.A.C., and Koch, C. (1991) Synaptic background activity influences spatiotemporal integration in single pyramidal cells. Proceedings of the National Academy of Sciences, U.S.A. , 88, 11569–11573. Bonhoeffer, T., Staiger, V., and Aertsen, A. (1989) Synaptic plasticity in rat hippocampal slice cultures: Local “hebbian” conjunction of pre- and postsynaptic stimulation leads to distributed synaptic enhancement. Proceedings of the National Academy of Sciences, U.S.A. , 86, 8113–8117. Bouyer, J.J., Montaron, M.F., and Rougeul, A. (1981) Fast fronto-parietal rhythms during combined focused attentive behavior and immobility in cat: cortical and thalamic localizations. Electroencephalograhy and Clinical Neurophysiology , 51 , 244–252. Boven, K.-H., and Aertsen, A. (1990) Dynamics of activity in neuronal networks give rise to fast modulations of functional connectivity. In: R.Eckmiller, G.Hartmann and G.Hauske (eds) Parallel Processing in Neural System and Computers , Elsevier Science Publishers B.V. (North Holland), pp. 53–56. Bringuier, V., Frégnac, Y., Baranyi, A., Debanne, D., and Shulz, D. (1997) Synaptic origin and stimulus dependency of neuronal oscillatory activity in the primary visual cortex of the cat. Journal of Physiology (London) , 500 , 751–774. Brett, B., and Barth, D.S. (1997) Subcortical modulation of high-frequency (gamma range) oscillating potentials in auditory cortex. Journal of Neurophysiology , 78, 573– 581. Brosch, M., Bauer, R., and Eckhorn, R. (1997) Stimulus-dependent modulations of correlated high-frequency oscillations in cat visual cortex. Cerebral Cortex , 7, 70–76. Bryant, H.L., Ruiz Marcos, A., and Segundo, J.P. (1973) Correlation of neuronal spike discharges produced by monosynaptic connections and by common inputs. Journal of
Time and the brain
102
Neurophysiology , 36, 205–225. Burns, B., and Webb, A.C. (1979) The correlation between discharge times of neighbouring neurons in isolated cerebral cortex. Proceedings of the Royal Society of London, series B. , 203, 347–360. Bush, P.C., and Douglas, R.J. (1991) Synchronisation of bursting action potentials discharge in a model network of neocortical neurons. Neural Computation , 3, 19–30. Bush, P., and Sejnowski, T. (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network model. Journal of Computational Neuroscience . 3 , 91–110. Cardoso de Oliveira, S., Thiele, A., and Hoffman, K.-P. (1997) Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. Journal of Neuro science , 17, 9248–9260. Chagnac-Amitai, Y., Luhmann, H.J., and Prince, D.A. (1990) Burst generating and regular spiking layer 5 pyramidal neurons of rat neocortex have different morphological features. Journal of Comparative Neurology , 296, 598–613. Cleland, B.C., Dubin, M.W., and Levick, W.R. (1971a) Simultaneous recording of input and output of lateral geniculate neurones. Nature (New Biology) , 231, 191–192. Cleland, B.G., Dubin, M.W., and Levick, W.R. (1971b) Sustained and transient neurones in the cat’s retina and lateral geniculate nucleus. Journal of Physiology (London) , 217, 473–496. Connors, B.W., Gutnick, M.J., and Prince, D.A. (1982) Electrophysiological properties of neocortical neurons in vitro. Journal of Neurophysiology, 48, 1302–1320. Contreras, D., Destexhe, A., Sejnowski, T.J., and Steriade, M. (1997) Spatiotemporal patterns of spindle oscillations in cortex and thalamus. Journal of Neuroscience , 17, 1179–1196. Cope, T.C., Fetz, E.E., and Matsumura, M. (1987) Cross-correlation assessment of synaptic strength of single Ia fibre connections with triceps surae motoneurones in cats. Journal of Physiology (London) , 390, 161–188. DeAngelis, G.C., Freeman, R.D., and Ohzawa, I. (1994) Length and width tuning of neurons in the cat’s primary visual cortex. Journal of Neurophysiology , 71, 347–374. de Charms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds by the coordination of action potential timing. Nature, London. , 381, 610–613. Destexhe, A., Contreras, D., and Steriade, M. (1998) Mechanisms underlying the synchronizing action of corticothalamic feedback through inhibition of thalamic relay cells. Journal of Neurophysiology , 79, 999–1016. Dickson, J.W., and Gerstein, G.L. (1974) Interactions between neurons in auditory cortex of the cat. Journal of Neurophysiology , 37, 1239–1261. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboek, H.J. (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Biological Cybernetics , 60, 121–130. Eckhorn, R., Frien, A., Bauer, R., Woelbern, T., and Kehr, H. (1993) High frequency (60–90 Hz) oscillations in primary visual cortex of awake monkey. NeuroReport , 5, 2273–2277. Eggermont, J.J. (1992) Neural interaction in cat primary auditory cortex. Dependence on recording depth, electrode separation, and age. Journal of Neurophysiology , 68, 1216– 1228. Eggermont, J.J., Smith, G.M., and Bowman, D. (1993) Spontaneous burst firing in cat primary auditory cortex: age and depth dependence and its effect on neural interaction measures. Journal of Neurophysiology , 69, 1292–1313.
Cross-correlograms for neuronal spike trains
103
Eggermont, J.J. (1994) Neural interaction in cat primary auditory cortex. II. Effects of sound stimulation. Journal of Neurophysiology , 71, 246–270. Eggermont, J.J., and Smith, G.M. (1996) Neuronal connectivity only accounts for a small part of neural correlation in auditory cortex. Experimental Brain Research , 110, 379– 391. Engel, A.K., König, P., Gray, C.M., and Singer W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: inter-columnar interaction as determined by crosscorrelation analysis. European Journal of Neuroscience , 2, 588–606. Engel, A.K., König, P., Kreiter, A.K., and Singer, W. (1991a) Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, New York , 252, 1177–1179. Engel, A.K., Kreiter, A.K., König, P., and Singer, W. (1991b) Synchronization of oscillatory neuronal reponses between striate and extrastriate visual areas of the cat. Proceedings of the National Academy of Science, U.S.A., 88, 6048–6052. Engel, A.K., König, P., and Singer, W. (1991c) Direct physiological evidence for scene segmentation by temporal coding. Proceedings of the National Academy of Science, U.S.A. , 88, 9136–9140. Engel, A.K., König, P., Kreiter, A.K., Schillen, T.B., and Singer, W. (1992) Temporal coding in the visual cortex: new vistas on integration in the nervous system. Trends in Neuroscience , 15, 218–226. Epping, W.J.M., and Eggermont, J.J. (1987) Coherent activity in the auditory midbrain of the grass-frog. Journal of Neurophysiology , 57, 1464–1483. Erb, M., and Aertsen, A. (1992) Dynamics of activity in biology-oriented neural network models: Stability at low firing rates. In: A.Aertsen and V.Braitenberg (eds.) Information processing in the cortex: Experiments and theory , Springer, Heidelberg, pp. 201–223. Fahle, M., and Koch, C. (1995) Spatial displacement, but not temporal asynchrony, destroys figural binding. Vision Research , 35, 491–494. Fetz, E.E., and Gustafsson, B. (1983) Relation between shapes of post-synaptic potentials and changes in firing probability of cat motoneurones. Journal of Physiology (London) , 341, 387–410. Franks, N.P., and Lieb, W.R. (1994) Molecular and cellular mechanisms of general anesthesia. Nature, London , 367, 607–614. Freeman, W.J., and Barrie, J.M. (1994) Chaotic oscillations and the genesis of meaning in cerebral cortex. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 13–37. Frégnac, Y., Shulz, D., Thorpe, S., and Bienenstock, E. (1988) A cellular analog of visual cortical plasticity. Nature, London , 333, 367–370. Frégnac, Y., Bringuier, V., and Baranyi, A. (1994a) Oscillatory neuronal activity in visual cortex: a critical re-evaluation. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 81–102. Frégnac, Y., Burke, J.P., Smith, D., and Friedlander, M.J. (1994b) Temporal covariance of pre- and postsynaptic activity regulates functional connectivity in the visual cortex. Journal of Neurophysiology , 71, 1403–1421. Frien, A., Eckhorn, R., Bauer, R., Woelbren, T., and Kehr, H. (1994) Stimulus-specific fast oscillations at zero phase between areas V1 and V2 of awake monkey. NeuroReport , 5, 2273–2277. Fries, P., Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997)
Time and the brain
104
Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proceedings of the National Academy of Science, U.S.A. , 94, 12699–12704. Frostig, R.D., Gottlieb, Y., Vaadia, E., and Abeles, M. (1983) The effects of stimuli on the activity and functional connectivity of local neuronal groups in the cat auditory cortex. Brain Research , 272, 211–221. Funke, K., and Wörgötter, F. (1995) Temporal structure in the light response of relay cells in the dorsal lateral geniculate nucleus of the cat. Journal of Physiology (London) , 485, 715–737. Gawne, T.J., and Richmond, B.J. (1993) How independent are the messages carried by adjacent inferior temporal cortical neurons? Journal of Neuro science , 13, 2758–2771. Gawne, T.J., Kjaer, T.W., Hertz, J.A., and Richmon, B.J. (1996) Adjacent visual cortical complex cells share about 20% of their stimulus-related information. Cerebral Cortex , 6, 482–489. Gerstein, G.L., Bedenbaugh, P., and Aertsen, A.M.H.J. (1989) Neuronal assemblies. IEEE Transactions on Biomedical Engineering , 36, 4–14. Ghose, G.M., and Freeman, R.D. (1992) Oscillatory discharge in the visual system: Does it have a functional role? Journal of Neurophysiology , 68 , 1558–1574. Ghose, G.M., Ohzawa, I., and Freeman, R.D. (1994) Receptive-field maps of correlated discharge between pairs of neurons in the cat’s visual cortex. Journal of Neurophysiology , 71, 330–346. Ghose, G., and Freeman, R.D. (1997) Intracortical connections are not required for oscillatory activity in the visual cortex. Visual Neuroscience , 14, 963–979. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of awake macaque. Experimental Brain Research , 84 , 505–516. Gray, C.M., and Singer, W. (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 86, 1698–1702. Gray, C.M., König, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-colomnar synchronisation which reflects global stimulus properties. Nature, London . 338, 334–337. Gray, C.M., Engel, A.K., König, P., and Singer, W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: receptive field properties and feature dependence. European Journal of Neuroscience , 2, 607–619. Gray, C.M., and McCormick, D.A. (1996) Chattering cells: Superficial pyramidal neurons contributing to the generation of synchronous oscillations in the visual cortex. Science, New York , 274, 109–113. Gray, C.M., and Viana Di Prisco, G. (1997) Stimulus dependent neuronal oscillations and local synchronization in striate cortex of the alert cat. Journal of Neuroscience , 17, 3239–3253. Griffith, J.S., and Horn, G. (1963) Functional coupling between cells in the visual cortex of the unrestrained cat. Nature, London , 199, p. 876 and 893–895. Guido, W., and Weyand, T. (1995) Burst responses in thalamic relay cells of the awake behaving cat. Journal of Neurophysiology , 74, 1782–1786. Gustafsson, B., and McCrea, D. (1984) Influence of stretch-evoked synaptic potentials on firing probability of cat spinal motoneurones. Journal of Physiology (London) , 347, 431–451. Gustafsson, B., Wigström, H., Abraham, W.C., and Huang, Y.-Y. (1987) Long-term
Cross-correlograms for neuronal spike trains
105
potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials. Journal of Neuroscience , 7, 774–780. Hansel, D., and Sompolinsky, H. (1996) Chaos and synchrony in a model of a hypercolumn in visual cortex. Journal of Computational Neuroscience , 3, 7–37. Harvey, A.R. (1980) A physiological analysis of subcortical and commissural projections of area 17 and 18 of the cat. Journal of Physiology (London) , 302, 507–535. Hata, Y., Tsumoto, T., Sato, H., and Tamura, H. (1991) Horizontal interactions between cortical neurones studied by cross-correlation analysis in the cat. Journal of Physiology (London) , 441, 593–614. Hebb, D.O. (1949) The organization of behavior: A neuropsychological theory . John Wiley & Sons, New York. Holmes, O., and Houchin, J. (1966) Units in the cerebral cortex of the anesthetized rat and the correlations between their discharges. Journal of Physiology (London) , 187, 651–671. Innocenti, G.M. (1980) The primary visual pathway through the corpus callosum: Morphological and functional aspects in the cat. Archives Italiennes de Biologie , 118, 124–188. Jagadeesh, B., Gray, C.M., and Ferster, D. (1992) Visually evoked oscillations of membrane potential in cells of cat visual cortex. Science, New York , 257, 552–554. Jahnsen, H., and Llinás, R. (1984) Electrophysiological properties of guinea-pig thalamic neurons, an in vitro study. Journal of Physiology (London) , 349, 205–226. Jester, J.M., Campbell, L.W., and Sejnowski, T.J. (1995) Associative EPSP-spike potentiation induced by pairing orthodromic and antidromic stimulation in rat hippocampal slices. Journal of Physiology (London) , 484, 689–705. Jones, R.S.G., and Heinemann, U. (1988) Synaptic and intrinsic responses of medial enthorinal cortical cells in normal and magnesium-free medium in vitro. Journal of Neurophysiology , 59 , 1476–1496. Keifer, J.C., Baghdoyan, H.A., Becker, L., and Lydic, R. (1994) Halothane decreases pontine acetylcholine release and increases EEG spindles. NeuroReport , 5, 577–580. Kim, U., Bal, T., and McCormick, D.A. (1995) Spindle waves are propagating synchronized oscillations in the ferret LGNd in vitro. Journal of Neurophysiology , 74, 1301–1323. Kiper, D.C., Gegenfurtner, K.R., and Movshon, J.A. (1996) Cortical oscillatory responses do not affect visual segmentation. Vision Research , 36, 539–544. Kirkwood, P.A., and Sears, T.A. (1978) The synaptic connexions to intercostal motoneurones as revealed by the average common excitation potential. Journal of Physiology (London) , 275, 103–134. Kirkwood, P.A., and Sears, T.A. (1982) The effect of single afferent impulses on the probability of firing of external intercostal motoneurones in the cat. Journal of Physiology (London) , 322, 315–336. Knierim, J.J., and Van Essen, D.C. (1992) Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. Journal of Neurophysiology , 67, 961–980. Knox, C.K. (1974) Cross-correlation functions for a neuronal model. Biophysical Journal , 14 , 567–582. Koch, C., and Schuster, H. (1992) A simple network showing burst synchronization without frequency locking. Neural Computation , 4, 211–223. König, P., and Schillen, T.B. (1991) Stimulus-dependent assembly formation of oscillatory responses: I.Synchronization. Neural Computation , 3, 155–166. König, P., Engel, A.K., and Singer, W. (1995) Relationship between oscillatory activity
Time and the brain
106
and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 92, 290–294. König, P., Engel, A.K., and Singer, W. (1996) Integrator or coincidence detector? the role of cortical neurons revisited. Trends in Neuroscience , 19 , 130–137. Kreiter, A.K., and Singer, W. (1992) Oscillatory neuronal responses in the visual cortex of the awake monkey. European Journal of Neuroscience , 4, 369–375. Kreiter, A.K., and Singer, W. (1996) Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. Journal of Neuroscience , 16 , 2381–2396. Krüger, J., and Aiple, F. (1988) Multimicroelectrode investiguation of monkey striate cortex: Spike train correlations in the infragranular layers. Journal of Neurophysiology , 60 , 798–828. Krüger, J., and Mayer, M. (1990) Two types of neuronal synchrony in monkey striate cortex. Biological Cybernetics , 64 , 135–140. Kruse, W., and Eckhorn, R. (1996) Inhibition of sustained gamma oscillations (35–80 Hz) by fast transient responses in cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 93, 6112–6117. Lamme, V.A.F. (1995) The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience , 15, 1605–1615. Levick, W.R., Cleland, E.G., and Dubin, M.W. (1972) Lateral geniculate neurons of cats: Retinal inputs and physiology. Investigative Ophthalmology , 11 , 302–311. Li, C.-L. (1959) Synchronization of unit activity in the cerebral cortex. Science, New York , 129, 783–784. Livingstone, M.S. (1996) Oscillatory firing and interneuronal correlations in squirrel monkey striate cortex. Journal of Neurophysiology , 75, 2467–2485. Llinás, R.R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10- to 50-Hz frequency range. Proceedings of the National Academy of Science, U.S.A. , 88, 897–901. Llinás, R., Ribary, U., Joliot, M., and X.-J. Wang (1994) Content and context in temporal thalamocortical binding. In: G. Buzsáki, R. Llinás, W. Singer, A. Berthoz and Y. Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 251– 272. Lopes da Silva, F.H., van Rotterdam, A., Storm van Leeuwen, W., and Tielen, A.M. (1970) Dynamic characteristics of visual evoked potentials in the dog. I. Beta frequency selectivity in evoked potentials and background activity. Electroencephalography and Clinical Neurophysiology , 29 , 260–268. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997a) Neuronal dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cerebral Cortex , 7, 207–227. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997b) Neuronal dynamics in a model of the thalamocortical system. II. The role of neuronal synchrony tested through perturbations of spike timing. Cerebral Cortex , 7, 228–236. Lytton, W.W., and Sejnowski, T.J. (1991) Simulation of cortical pyramidal neurons synchronized by inhibitory interneurons. Journal of Neurophysiology , 66 , 1059–1079. Markram, H. (1997) A network of tufted layer 5 pyramidal neurons. Cerebral Cortex , 7, 523–533. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, New York , 275, 213–215.
Cross-correlograms for neuronal spike trains
107
Mason, A., and Larkman, A. (1990) Correlations between morphology and electrophysiology of pyramidal neurons in slices of rat visual cortex. II. Electrophysiology. Journal of Neuroscience , 10, 1415–1428. Mastronarde, D.N. (1983a) Correlated firing of cat retinal ganglion cells. I. Spontaneously active inputs to X- and Y-cells. Journal of Neurophysiology , 49 , 303– 324. Mastronarde, D.N. (1983b) Correlated firing of cat retinal ganglion cells. II. Responses of X-and Y-cells to single quantal events. Journal of Neurophysiology , 49, 325–349. Matsumura, M., Chen, D.-F., Sawaguchi, T., Kubota, K., and Fetz, E.E. (1996) Synaptic interactions between primate precentral cortex neurons revealed by spike-triggered averaging of intracellular membrane potentials in vivo. Journal of Neuroscience , 16 , 7757–7767. McCormick, D.A., Connors, B.W., Lighthall, J.W., and Prince, D.A. (1985) Comparative electro-physiology of pyramidal and sparsely spiny stelate neurons of the neocortex. Journal of Neurophysiology , 54, 782–806. McCormick, D.A., and Bal, T. (1997) Sleep and arousal: thalamocortical mechanisms. Annual Review of Neuroscience , 20 , 185–215. Meister, M., Lagnado, L., and Baylor, D.A. (1995) Concerted signaling by retinal ganglion cells. Science, New York , 270, 1207–1210. Melssen, W.J., and Epping, W.J.M. (1987) Detection and estimation of neuronal connectivity based on crosscorrelation analysis. Biological Cybernetics , 57, 403–414. Michalski, A., Gerstein, G.L., Czarkowska, J., and Tarnecki, R. (1983) Interactions between cat striate cortex neurons. Experimental Brain Research , 51 , 97–107. Milner, P.M. (1974) A model for visual shape recognition. Psychological Review , 81 , 521–535. Moore, G.P., Segundo, J.P., Perkel, D.H., and Levitan, H. (1970) Statistical signs of synaptic interaction in neurons. Biophysical Journal , 10, 876–900. Munk, M.H.J., Nowak, L.G., Nelson, J.I., and Bullier, J. (1995) The structural basis of cortical synchronization. II. Localization of the common input. Journal of Neurophysiology , 74, 2401–2414. Munk, M.H.J., Roelfsema, P.R., König, P., Engel, A.K., and Singer, W. (1996) Role of reticular activation in the modulation of intracortical synchronization. Science, New York , 272, 271–274. Murphy, J.T., Kwan, H.C., and Wong, Y.C. (1985) cross-correlation studies in primate motor cortex: synaptic interaction and shared input. Canadian Journal of Neurological Science , 12 , 11–23. Murthy, V., and Fetz, E.E. (1992) Coherent 25- to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys. Proceedings of the National Academy of Science, U.S.A. , 89, 5670–5674. Murthy, V.N., and Fetz, E.E. (1996) Synchronization of neurons during local field potential oscillations in sensorimotor cortex of awake monkeys. Journal of Neurophysiology , 76, 3968–3982. Nelson, J.I., and Frost, B.J. (1978) Orientation-selective inhibition from beyond the classical receptive field. Brain Research , 139 , 359–365. Nelson, J.I., Salin, P.A., Munk, M.H.-J., Arzi, M., and Bullier, J. (1992) Spatial and temporal coherence in cortico-cortical connections: A cross-correlation study in area 17 and 18 in the cat. Visual Neuroscience , 9, 21–37. Neuenschwander, S., and Singer, W. (1996) Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature, London , 379 ,
Time and the brain
108
728–733. Neven, H., and Aertsen, A. (1992) Rate coherence and event coherence in the visual cortex: a neuronal model of object recognition. Biological Cybernetics , 67 , 309–322. Noda, H., and Adey, W.R. (1970) Firing of neuron pairs in cat association cortex during sleep and wakefulness. Journal of Neurophysiology , 33, 672–684. Noda, H., and Adey, W.R. (1973) Neuronal activity in the association cortex of the cat during slepp, wakefulness and anesthesia. Brain Research , 54, 243–259. Nowak, L.G., Munk, M.H.J., James, A.C., Nelson, J.I., and Bullier, J. (1995) The structural basis of cortical synchronization. I. 3 types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–2400. Nowak, L.G., Sanchez-Vives, M.V., and McCormick, D.A. (1997) Influence of low and high frequency fluctuations on spike timing in visual cortical neurons. Cerebral Cortex , 7, 487–501. Nowak, L.G., and Bullier, J. (1998a) Axons, but not cell bodies, are activated by electrical stimulation in the cortical gray matter. I. Evidence from chronaxie measurements. Experimental Brain Research , 118, 477–488. Nowak, L.G., and Bullier, J. (1998b) Axons, but not cell bodies, are activated by electrical stimulation in the cortical gray matter. II. Evidence from selective inactivation of cell bodies and axon initial segment.Experimental Brain Research , 118, 489–500. Nowak, L.G., Munk, M.H.J., James, A.C., Girard, P., and Bullier, J (1999) Crosscorrelation study of the temporal interactions between areas V1 and V2 of the macaque monkey. Journal of Neurophysiology . 81:1057–1074. Nuñez, A., Amzica, F., and Steriade, M. (1992) Voltage-dependent fast (20–40 Hz) oscillations in long-axoned neocortical neurons. Neuroscience , 51, 7–10. Pasternak, G. (1993) Pharmacological mechanisms of opioid analgesics. Clinical Neuropharmacology , 16 , 1–18. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967a) Neuronal spike trains and stochastic point processes. I. The single spike train. Biophysical Journal , 7, 391–418. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967b) Neuronal spike trains and stochastic point processes. II. simultaneous spike trains. Biophysical Journal , 7, 419– 440. Pinault, D., and Deschênes, M (1992) Voltage-dependent 40 Hz oscillations in rat reticular thalamic neurons in vivo. Neuroscience , 51, 245–258. Pockberger H. (1991) Electrophysiological and morphological properties of rat motor cortex neurons in vivo. Brain Research. 539, 181–190. Reid, R.C., and Alonso, J.-M. (1995) Specificity of monosynaptic connections from thalamus to visual cortex. Nature, London , 378, 281–284. Renaud, L.P., and Kelly, J.S. (1974) Identification of possible inhibitory neurons in the pericruciate cortex of the cat. Brain Research , 79, 9–28. Reyes, A.D., and Fetz, E.E. (1993) Effects of transient depolarizing potentials on the firing rate of cat neocortical neurons. Journal of Neurophysiology , 69 , 1673–1683. Riehle, A., Grün, S., Diesmann, M., and Aertsen, A. (1997) Spike synchronization and rate modulation differentially involved in motor cortical function. Science, New York , 278, 1950–1953. Ritz, R., Gerstner, W., Fuentes, U., and van Hemmen, J. (1994) A biologically motivated and analytically soluble model of collective oscillations in the cortex. II. Application to binding and pattern segmentation. Biological Cybernetics , 71, 349–358. Robson, J.G., and Troy, J.B. (1987) Nature of the maintained discharge of Q, X, and Y
Cross-correlograms for neuronal spike trains
109
retinal ganglion cells of the cat. Journal of the Optic Society of America , Series A, 4, 2301–2307. Rodieck, R.W. (1967) Maintained activity of cat retinal ganglion cells. Journal of Neurophysiology , 30, 1043–1071. Roelfsema, P.R., König, P., Engel, A.K., Sireteanu, R., and Singer, W. (1994) Reduced synchronization in the visual cortex of cats with strabismic amblyopia. European Journal of Neuroscience , 6, 1645–1655. Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997) Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature, London , 385, 157–161. Rougeul, A., Bouyer, J.J., Dedet, L., and Debray, O. (1979) Fast somato-parietal rhythms during combined focal attention and immobility in baboon and squirrel monkey. Electroencephalography and Clinical Neurophysiology , 46, 310–319. Sanes, J.N., and Donoghue, J.P. (1993) Oscillations in local field potentials of the primate motor cortex during voluntary movement. Proceedings of the National Academy of Science, U.S.A. , 90 , 4470–4474. Schein, S.J., and Desimone, R. (1990) Spectral properties of V4 neurons in the macaque. Journal of Neuroscience , 10, 3369–3389. Schindler, K.A., Bernasconi, C.A., Stoop, R., Goodman, P.H., and Douglas, R.J. (1997) Chaotic spike patterns evoked by periodic inhibition in rat cortical neurons. Zeitschrift für Naturforschung . 52a, 509–512. Schwarz, C, and Bolz, J. (1991) Functional specificity of a long-range horizontal connection in cat visual cortex: A cross-correlation study. Journal of Neuroscience , 11, 2995–3007. Shadlen, M.N., and Newsome, W.T. (1994) Noise, Neural codes and cortical organization. Current Opinion in Neurobiology , 4, 569–579. Sillito, A.M., Jones, H.E., Gerstein, G.L., and West, D.C. (1994) Feature linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London , 369, 479–82. Sillito, A.M., Grieve, K.L., Jones, H.E., Cudeiro, J., and Davies, J. (1995) Visual cortical mechanisms detecting focal orientation discontinuities. Nature, London , 378, 492– 496. Softky, W., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. Journal of Neuroscience , 13, 334–350. Sporns, O., Gally, J.A., Reeke, G.N., and Edelman G.M. (1989) Reentrant signaling among simulated neuronal groups leads to coherency in their oscillatory activity. Proceedings of the National Academy of Science, U.S.A. , 86, 7265–7269. Steriade, M., Nuñez, A., and Amzica, F. (1993a) A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. Journal of Neuroscience , 13, 3252–3265. Steriade, M., Contreras, D., Curro Dossi, R., and Nuñez, A. (1993b) The slow (<1 Hz) oscillation in reticular thalamic and thalamocortical neurons: scenario of sleep rhythm generation in interacting thalamic and neocortical networks. Journal of Neuroscience , 13, 3284–3299. Steriade, M., Curró Dossi, R., and Contreras, D. (1993c) Electrophysiological properties of intralaminar thalamocortical cells discharging rhythmic (~ 40 Hz) spike-bursts at ~ 1000 Hz during waking and rapid eye movement sleep. Neuroscience , 56, 1–9. Steriade, M., Nuñez, A., and Amzica, F. (1996a) Synchroniszation of fast (30–40 Hz)
Time and the brain
110
spontaneous cortical rhythms during brain activation. Journal of Neuroscience , 16, 392– 417. Steriade, A., Contreras, D., Amzica, F., and Timofeev, I. (1996b) Synchronization of fast (30–40 Hz) spontaneous oscillations in intrathalamic and thalamocortical networks. Journal of Neuroscience , 16, 2788–2808. Steriade, M., Timofeev, I., Dürmüller, N., and Grenier, F. (1998) Dynamic properties of corticothalamic neurons and local cortical interneurons generating fast rhythms (30–40 Hz) spike bursts. Journal of Neurophysiology , 79, 483–90. Swadlow, H.A. (1995) Influence of VPM afferents on putative interneurons in S1 of the awake rabbit: evidence from cross-correlation, microstimulation, and latencies to peripheral sensory stimulation. Journal of Neurophysiology , 73, 1584–1599. Tallon-Baudry, C., Bertrand O., Delpuech C., and Pernier, J. (1996) Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. Journal of Neuroscience , 16, 4240–4249. Tallon-Baudry, C., Bertrand, O., Delpuech, C., and Pernier, J. (1997) Oscillatory gammaband (30–70 Hz) activity induced by a visual search task in human. Journal of Neuroscience , 17, 722–734. Tallon-Baudry, C., Bertrand, O., Peronnet, F., and Pernier, J. (1998) Induced gammaband activity during the delay of a visual short-term memory task in humans. Journal of Neuroscience , 18, 4244–4254. Tamura, H., Sato, H., Katsuyama, N., Hata, Y., and Tsumoto, T. (1996) Less segregated processing of visual information in V2 than in V1 of the monkey visual cortex. European Journal of Neuroscience , 8, 300–309. Tanaka, K. (1983) Cross-correlation analysis of geniculostriate neuronal relationship in cats. Journal of Neurophysiology , 49 , 1303–1318. Thomson, A.M., and Deuchars, J. (1997) Synaptic interactions in neocortical local circuits: Dual intracellular recordings in vitro. Cerebral Cortex , 7, 510–522. Thorpe, S., Fize, D., and Marlot, C. (1996) Speed of processing in the human visual system. Nature, London , 381, 520–522. Tononi, G., Sporns, O., and Edelman, G.M. (1992) Reentry and the problem of integrating multiple cortical areas: simulation of dynamic integration in the visual system. Cerebral Cortex , 2, 310–335. Toyama, K., Kimura, M., and Tanaka, K. (1981a) Cross-correlation analysis of interneuronal connectivity in cat visual cortex. Journal ofNeurophysiology , 46, 191– 201. Toyama, K., Kimura, M., and Tanaka, K. (1981b) Organization of cat visual cortex as investigated by cross-correlation technique. Journal of Neurophysiology , 46, 202–214. Troy, J.B., and Robson, J.G. (1992) Steady discharges of X and Y retinal ganglion cells of cat under photopic illuminance. Visual Neuro science , 9, 535–553. Troyer, T.W., and Miller, K.D. (1997) Physiological gain leads to high ISI variability in a simple model of a cortical regular spiking cell. Neural Computation , 9, 971–983. Ts’o, D.Y., Gilbert, C.D., and Wiesel, T.N. (1986) Relationship between horizontal interactions and functinal architecture in cat striate cortex as revealed by crosscorrelation analysis. Journal of Neuroscience , 6, 1160–1170. Vaadia, E., and Abeles, M. (1987) Temporal firing patterns of single units, pairs and triplets of units in the auditory cortex. Israel Journal of Medical Science , 23, 75–83. Vaadia, E., Ahissar, E., Bergman, H., and Lavner, Y. (1991) Correlated activity of neurons: A neural code for higher brain functions? In: Krüger, J. (ed) Neuronal Cooperativity , Springer-Verlag Berlin Heidelberg, 1991, 205–279.
Cross-correlograms for neuronal spike trains
111
Vaadia, E., Haalman, I, Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London , 373, 515–518. van Vreeswijk, C., and Sompolinski, H. (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, New York , 274, 1724–1726. von der Malsburg, C. (1981) The correlation theory of brain function. Internal Report 81–2, Dept. of Neurobiology, Max-Plank-Institute for Biophysical Chemistry, 3400 Göttingen, W-Germany. von der Malsburg, C., and Schneider, W. (1986) A neural cocktail-party processor. Biological Cybernetics , 54, 29–40. Wang, X.-J., and Buzsáki, G. (1996) Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. Journal of Neuroscience , 16, 6402–6413. Wang Roe, A., and Ts’o, D.Y. (1997) The functional architecture of area V2 in the macaque monkey. In: K.E.Rockland, J.H.Kaas and A.Peters (eds), Cerebral Cortex, Volume 12, Extrastriate Cortex in primate , Plenum Press, New York and London, pp. 295–257. Wilson, M.A., and Bower, J.M. (1991) A computer simulation of oscillatory behavior in primary visual cortex. Neural Computation , 3, 498–509. Whittington, M.A., Traub, R.D., and Jeffreys, J.G.R. (1995) Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, London , 373, 612–615. Young, M.P., Tanaka, K., and Yamane, S. (1992) On oscillating neuronal responses in the visual cortex of the monkey. Journal of Neurophysiology , 67, 1464–1474.
3 The Space-Time Continuum in Mammalian Sensory Pathways Asif A.Ghazanfar and Miguel A.L. Nicolelis Department of Neurobiology, Box 3209 Duke University Medical Center, Durham, NC 27710 correspondence to: Miguel A.L.Nicolelis, M.D., Ph.D, Department Neurobiology, Box 3209, Duke University Medical Center, 101 Research Drive, Bryan Research Building, Durham, NC, 27710 USA Phone: 919–684–4580; FAX: 919–684–5435; e-mail: [email protected]
Animals in their natural environments actively process spatiotemporally complex sensory signals in order to guide adaptive behaviour. It therefore seems likely that the properties of both single neurones and neural ensembles should reflect the dynamic nature of such interactions. We review several recent studies that demonstrate the existence of timedependent receptive fields and distributed coding of sensory signals by neurones in the auditory, visual and somatosensory thalamocortical pathways—features that seem to be constrained by the need to interpret actively time-varying stimuli. Based on these results, the central nervous system does not appear to operate as a static decoder, composed of highly-specialized feature-detector neurones, which sits idly until inputs from the environment are conveyed through it. On the contrary, the evidence indicates that perception emerges as the outcome of highly distributed interactions between populations of broadly-tuned neurones throughout the multiple and reciprocally-connected subcortical and cortical areas that define a sensory system. KEYWORDS: Neural ensembles; Coding; Spatiotemporal; Thalamocortical; Feedback “The trace of any activity is not an isolated connexion between sensory and motor elements. It is tied in with the whole complex of spatial and temporal axes of nervous activity which forms a constant substratum of behaviour”—Lashley, 1950.
1. INTRODUCTION It is no longer tenable to ignore the fact that the temporal structure of both natural signals
The space-time continuum in mammalian sensory pathways
113
and exploratory behaviours used by animals in their ecological niche constrain the mechanisms employed by the central nervous system to acquire, process, and decode sensory information. Until recently, however, very few neurophysiological theories of perception incorporated these dynamic aspects of sensory information processing. Instead, the emphasis of mainstream neural coding theories was placed on the existence of static and topographic mapping functions, which had as their central elements highly specialized “feature extracting” single neurones (Barlow, 1972; Hubel and Wiesel, 1962). The fact that sensory cues in the natural environment typically have intricate timevarying properties, and that animals can perceive such sensory stimuli very rapidly, suggests that the mammalian central nervous system is capable of processing complex time-dependent signals very efficiently. For example, speech and other vocalization signals have unique spectro-temporal signatures in which the timing and order of basic components (e.g. syllables, phonemes, motifs, etc.) play a critical role in distinguishing the meaning of different sounds. Thus, neural circuits have to decode the temporal structure of these signals in order to allow the subject to perceive and respond adaptively to them. The necessity of having time-dependent neural processes underlying perception is also indicated by the fact that, in natural settings, animals gather information about the surrounding environment by actively moving their sensory receptor arrays as they engage in exploratory behaviours. For example, during active exploration of their haptic environments, primates employ stereotypical movements of their hands to perceive minute tactile differences in object size, texture, and shape. When primates voluntarily move their fingers across an object, their ability to discriminate a variety of tactile features is improved considerably when compared to the passive reception of the same features (Darian-Smith, 1984). Analogous dependencies between active exploration and optimal sensory discrimination have been observed in other sensory modalities, such as saccadic eye movements during visual search and object recognition or head orientation movements for sound localization and recognition. Thus, most of the sensory information gleaned from the environment is sampled through actions, and leads to the genesis of percepts that seem to unfold continuously over time rather than in a series of discrete events. In this review, we address one of the central questions in modern neuroscience: how does the central nervous system rely on time-dependent computations to process behaviourally relevant sensory signals? In the last decade or so, many laboratories have dealt with a variety of questions that revolve around this central theme. Here, an attempt is made to summarize some of these new results. In the first half of this chapter, we describe how the discovery of spatiotemporal receptive fields across the thalamocortical loop has revealed a new dimension of sensory information-coding, which emphasizes the ability of the CNS to process time-varying signals during exploratory behaviours. In the second half of this chapter, we discuss how time-dependent neural coding schemes can be exploited by populations of neurones in the visual, auditory, and somatosensory thalamocortical pathways. The data reviewed here suggest that the presence of stimulus-dependent changes in the temporal structure of neuronal spike trains can be used to encode different types of sensory information. The temporal precision required in most of these schemes is of the
Time and the brain
114
order of 5–50 milliseconds, a time-scale very similar to those used to describe the spatiotemporal organization of thalamic and cortical neuronal receptive fields.
2. DYNAMIC RECEPTIVE FIELDS 2.1. Beyond the Classical Receptive Field Since the early work in the visual system (Hartline, 1950), the analysis of the receptive field (RF) properties of single neurones has occupied centre stage in sensory neurophysiology and, as a consequence, has guided the definition of a variety of cardinal principles in the investigation of sensory systems. Classically, the RF of a neurone is defined in terms of spatial co-ordinates (for the visual and somatosensory system), spectral co-ordinates (in the case of the auditory system), or by an arbitrary parameter space that has behavioural relevance (such as a set of odours or tastes for the olfactory and gustatory systems). Simply put, the RF of a neurone has been defined as the area of the particular “parameter space” within which a stimulus can elicit a neuronal response (measured in terms of the number of action potential) which is significantly above the spontaneous firing rate. Although the response properties of neurones are inherently both spatial and temporal, most studies have focused only on the spatial aspects of the RF of single neurones. This is evident from the voluminous amounts of work that have focused on “mapping” the organization of topographic representations of sensory space found in cortical and subcortical areas of the brain. As a result of the limited scope of these studies, neurones with particular spatial or spectral RF properties have been characterized simply as static filters of sensory inputs—filters that describe a neurone’s selectivity for features such as orientation or frequency tuning. The development of new RF mapping techniques has facilitated the characterization of RFs in both the spatial and temporal domains. Application of these methods in different sensory systems has demonstrated the existence of time-dependent RFs in multiple processing levels of the neuraxis. The central results of all these studies was the demonstration that it is inadequate to describe stimulus selectivity or representation in terms of the maximal or average spike counts averaged over a large time epoch, because the structure of RFs varies as a function of time. 2.2. Dynamic Frequency Tuning of Auditory Cortical Neurones: Adaptations to Complex Sound Processing In no other system has the role of time-dependent variables been emphasized more than in the auditory system. Over the years, information embedded in the form of time duration, interval and temporal order of stimulus features has been demonstrated to convey important information-bearing parameters for auditory signal processing in different species. Elegant studies (Hauser et al., 1998; Knudsen and Konishi, 1979; Margoliash and Fortune, 1992; Suga et al., 1983) have established that different nervous
The space-time continuum in mammalian sensory pathways
115
systems use a variety of temporal cues (i.e. interaural time difference, echolocation, sequences of sounds, etc.) to interpret auditory signals which are used to guide adaptive behaviours. In this context, even a small change in the temporal structure of an auditory signal can have a dramatic effect on its behavioural significance. Despite this wealth of information, very few neurophysiological studies have addressed how the spectrotemporal information embedded in complex sounds is reflected in the receptive fields of single neurones in the mammalian auditory system. In general, studies of the auditory cortex have focused on neuronal responses to simple stimuli, such as pure tunes and broad-band noise at different intensities. Only a handful of studies have been carried out to examine the time-dependence of frequency tuning in the mammalian auditory cortex. These studies have revealed that auditory cortical neurones can exhibit “spectrotemporal” RF, i.e. their frequency tuning may vary as a function of post-stimulus time. In other words, these neurones have “multi-peaked” tuning curves, such that they respond best to a particular frequency interval at one post-stimulus time, and to another frequency range at a later post-stimulus epoch. Thus, whereas most neurones in the peripheral auditory pathways have RFs that respond to a clearly definable frequency range, there is a small proportion of cortical neurones whose RFs contain two or more frequencies that can drive the neurones at the same intensity level (Abeles and Goldstein, 1972; Sutter and Schreiner, 1991). In a systematic study of these neurones, Sutter and Schreiner (1991) presented 675 different frequency-level combinations to each primary auditory cortical neurone in a pseudo-random order. The approach was very similar to the “response plane” technique used in some visual RF mapping studies (see below). These authors found that the two frequencies that elicit robust responses in multi-peaked neurones are often separated by frequencies to which the neurones do not respond at the same intensity levels. Moreover, the bandwidth of each peak in a multi-peaked RF was not significantly different from that of single-peaked neurones. The authors found that there were latency differences between peaks, and that these differences were highly variable; most often, however, the lowest frequency stimulus induced the shortest latency neuronal response, while the highest frequency stimulus led to the longest latency responses. Thus, these auditory cortical neurones seem to exhibit what one would consider a spectrotemporal RF. This means that the frequency tuning of the neurone changes as a function of post-stimulus time, or that the frequency response and time are coupled. Interestingly, the spectral and temporal ranges of these multi-peaked RFs corresponded to the ranges of behaviourally-relevant auditory signals for the species used for these studies. This raised the question of how these spectrotemporal RFs relate to how auditory neurones respond to natural auditory signals. Studies of the auditory responses to vocalizations in squirrel monkeys have demonstrated that neurones in the auditory cortex are sensitive to species-specific calls and have unique firing patterns for different call types (Winter and Funkenstein, 1973; Wollberg and Newman, 1972). Auditory cortical neurones responded with temporally-complex patterns of discharges, and for a given call type, a variety of temporal firing patterns were observed among different cortical neurones. Likewise, a given cortical neurone could respond to different call types. Editing out parts of a particular vocalization pattern indicated that single cortical neurones could dynamically integrate information from different call components into
Time and the brain
116
their firing patterns (Wollberg and Newman, 1972). Of the cortical neurones that responded to vocalizations, the majority of them responded to steady pure tones as well. Winter and Funkenstein (1973) reported that the responses to vocalizations could be predicted qualitatively by the neurone’s response to simpler stimuli, such as clicks and pure tones. More recent experiments on call-selective neurones in the auditory cortex of anaesthetized rhesus (Rauschecker et al., 1995) and marmoset monkeys (Callithrix jacchus) (Wang et al., 1995) have largely supported the latter results from squirrel monkeys. In the marmoset, primary auditory neurones phaselocked to the envelope portion of the vocalization, which was centred around the neurone’s frequency tuning (Wang et al., 1995). In the rhesus monkey, when different segments of a particular vocalization were used as stimuli for a call-selective neurone, the response could also be predicted by the frequency tuning curve of the neurone (Rauschecker et al., 1995). Thus, the spectral and temporal nature of a particular auditory neurone’s RF can be used, at least to a first approximation, to predict the firing patterns of a neurone to behaviourally-relevant vocal signals. Further testing of this assumption was performed by mapping the RF of primate auditory cortical neurones with reverse-correlation techniques, and by generating corresponding synthetic sounds (deCharms et al., 1998; Tramo et al., 1996). In these cases, the auditory stimuli used for the reverse-correlation consisted of rapid sequences of tones or chords. As mentioned above, for cats, in the primate primary auditory cortex, neurones have RFs whose best-frequency varies as a function of time. In addition, these spectro-temporal RFs can be used to a certain extent to predict the responses of these neurones to synthetic versions of species-specific vocalizations (Tramo et al., 1996). In the primary auditory cortex of the owl monkey, deCharms et al. (1998) showed that neurones have complex patterns of spectrotemporal RFs that indicate preferences for stimulus edges, stimulus transitions in frequency or intensity, and conjunctions of different stimulus features. To test the integrative properties of these neurones, different classes of auditory stimuli were generated based on the structure of their spectro-temporal RFs. Once these synthetic stimuli were presented they were able to drive the corresponding auditory cortical neurone at much higher rates than simple stimuli, the when the optimal stimulus was expanded or contracted in either the spectral or temporal domain (deCharms et al., 1998). These results support the notion that spectrotemporal RF properties reflect the solution found by the central nervous system for processing and interpreting natural, time-varying, and behaviourally-relevant auditory signals. In this sense there is a reciprocal relationship between the dynamic structure of single neurone RFs and the time-varying signature of natural sounds. Indeed, the responses to natural vocal signals could be used to predict, to some extent, the spectrotemporal structure of a neurone’s RF, and, conversely, the spectrotemporal RF could be used to predict a given neurone’s response to complex synthetic sounds. 2.3. Receptive Field Dynamics in the Lateral Geniculate Nucleus Traditionally, the RFs of lateral geniculate nucleus (LGN) neurones have been described
The space-time continuum in mammalian sensory pathways
117
in purely spatial terms as circular with an “on-centre”, which responds to the onset of a bright stimulus, and an “off-surround”, which responds to the onset of a dark stimulus (or the off-set of a bright stimulus). LGN neurones of the opposite polarity exist as well. Perhaps, the earliest study suggesting that time was an important dimension in the structure of LGN RFs was conducted by Stevens and Gerstein (1976). To characterize the RF of LGN neurones, these authors used the “response plane” technique, a method in which the visually-responsive area of a neurone is divided into a matrix and the spiking of the neurone elicited by a brief flash of light (a spot, or any other stimulus feature) is measured at each position of the matrix. A number of post-stimulus time histograms (equal to the size of the stimulus matrix) are generated and then used to compile the temporal sequences of responses. Using this approach, Stevens and Gerstein (1976) found that the simple, spatial description of cat LGN RFs, such as “on-off” and “centresurround”, was insufficient to describe completely even the simplest of LGN RFs. Overall, these authors reported two different types of LGN RFs. One type displayed spatially homogeneous distributions of excitatory and inhibitory domains, and another type exhibited spatially heterogeneous excitatory and inhibitory domains, which shifted as a function of post-stimulus time. These different dynamic behaviours were hypothesized to originate from two different functional streams, X and Y, within the cat visual system. Thus, these authors state that, “time is at least as important an analytical variable as space to visual neurones” (Stevens and Gerstein, 1976) emphasizing that the coupling between space and time is an integral component of LGN RFs. More recent experiments have supported the original results of Stevens and Gerstein by demonstrating the existence of spatiotemporally coupled visual RFs in both the cat (Dan et al., 1996) and primate LGN (Golomb et al., 1994). These studies employed “reverse-correlation” techniques to map the spatiotemporal response function of a given neurone. In this approach, the structure of the receptive field of a single neurone can be described fully by the measured response to any complete set of stimuli presented in rapid, pseudo-random sequences. One complete set of stimuli is defined by localized flashes of light presented in several random sequences. The average spike count of the neurone is then correlated with the past location and time of each flash, i.e. “reversecorrelated”. In the primate LGN, Golomb et al. (1994) found that both parvocellular and magnocellular neurones had spatiotemporal RFs. Thus, although these neurones still exhibited centre-surround organization, the response to the surround was delayed relative to the response of the centre. Thus, the RFs of these neurones could only be described as space-time inseparable, since at a given moment in time the overall spatial distribution of the RF was different from those observed in a previous or a future time epoch. These RFs could also be used to predict the neurone’s response to complex pattern stimuli (Golomb et al., 1994; see below). Do the spatiotemporal response properties of LGN neurones elicited by simple stimuli relate to their function in coding and representing natural visual signals? This is an important issue, because the demonstration that the spatiotemporal structure of a single neurone RF could be used to predict the neurone’s temporal coding of different visual stimuli would support the notion that the coupling of spatial and temporal dimensions are relevant for sensory information processing. This question was addressed by experiments done in the cat LGN, where the RF structure of single neurones was used to predict the
Time and the brain
118
responses to natural, time-vary ing visual stimuli (Dan et al., 1996). Natural scenes are highly redundant, having both spatial and temporal correlations. Because coding efficiency could be advantageous, both evolutionarily and computationally, it was hypothesized that LGN neurones would decorrelate natural scene stimuli in order to transform this information into a less redundant signal (Atick and Redlich, 1990). This was tested by recording the activity of LGN neurones in response to natural, time-varying scenes and measuring the power spectra of the neural signals (Dan et al., 1996). Indeed, the responses of LGN neurones to movies of natural scenes revealed that their individual spike trains were temporally decorrelated, that is their power spectra were essentially flat from 3Hz to 15Hz in response to natural scenes. Thus, although the natural scenes have intrinsic correlations in their structure, the LGN neuronal responses do not reflect these redundancies and, hence, code the information more efficiently. Furthermore, these authors reported that this decorrelation of LGN responses to natural scenes could only be predicted by the filtering properties of their spatiotemporal RFs (Dan et al., 1996). Thus, in both cat and primate the time-dependent RF structures of LGN neurones have been used to predict the responses and temporal coding of stimuli, for both artificial (Golomb et al., 1994) and natural stimuli (Dan et al., 1996). In both of these studies, feedforward circuitry was hypothesized to account for the dynamic behaviour LGN RFs. However, no experimental test of this “feedforward” hypothesis has been carried out thus far. 2.4. Dynamic Organization of the Primary Visual Cortex Traditionally, visual cortical neurones are classified according to their response selectivity and the shapes of their RFs. “Simple” cells of the primary visual cortex receive most of the LGN inputs, and have RFs that are classically described as being oriented spatially. These RFs exhibit subregions that respond primarily to bright or dark elongated stimuli. “Complex” cells respond to both bright and dark stimuli anywhere within their RFs. Recently, approaches similar to those used for mapping the spatial and temporal properties of LGN neurones have been used to characterize the RFs of visual cortical neurones, in both the cat and the monkey. Both response-plane and reverse-correlation approaches have demonstrated that the RF structure of visual cortical neurones is not static, but varies as function of post-stimulus time. These studies revealed that primary visual cortical neurones also have spatiotemporal RFs whose time-dependent structure can be measured by using a variety of stimulus parameters (overall RF size, location of on-off regions, orientation tuning, directional tuning, and length tuning). Receptive field mapping, in both space and time, has shown that V1 neurones respond to flashes of light in different subregion of their RFs at different times. Using the response plane technique, Dinse et al. (1991) reported that each subfield within the RF of visual cortical neurones, for both on- and off-responses, exhibits its own temporal characteristics. When each response-plane is plotted in sequence (from 20 to 200 milliseconds), a wave of excitation seems to travel and spread across the entire RF. Using the reverse-correlation analysis, where the stimuli consisted of rapidly-flashed patterns of
The space-time continuum in mammalian sensory pathways
119
spots or bars, DeAngelis et al. (1993) found that simple cells of the cat visual cortex have bright and dark excitatory elongated subregions that move as a function of time. The elongated on-off subregions of these spatiotemporal RFs could be used to predict the responses of these simple cells’ responses to bars of light (DeAngelis et al., 1995). In addition to the time-dependent organization of on-off responses, other properties of visual cortical neurones, such as orientation tuning, end-stopping, and directional selectivity, also show a robust time-dependence on a millisecond scale. The first evidence for this time-dependence was provided by Dinse et al. (1991), who employed flickering bars of light of different lengths to investigate the time course of orientation tuning and end-stopping in cat visual cortical neurones. Classically, orientation tuning is defined simply as the selectivity of neuronal responses to bars of a particular orientation, while end-stopping is the selectivity to bars of a particular length. Compiling the sequences of tuning curves for each of these stimulus features in post-stimulus time-steps of 20 milliseconds, these authors demonstrated that the selectivity of single neurones to these features evolved over post-stimulus time. For instance, in the case of orientation tuning, some neurones first exhibited a state of excitation that did not show selectivity for any particular orientation, followed by the gradual sharpening of the selectivity over time. Other neurones showed selectivity for a particular orientation that remained stable over time, while for yet another group of neurones, the selectivity changed over time (i.e. responding best to one orientation at one time epoch, and another orientation at a later time epoch). Nearly identical results were found for length-tuning (Dinse et al., 1991). A laminar comparison of dynamic orientation tuning in the macaque primary visual cortex was performed by applying the reverse-correlation technique in the orientation domain. Ringach et al. (1997) found that orientation selectivity developed after approximately 30–45 milliseconds, and persisted for up to 85 milliseconds over poststimulus time. Comparing the nature of orientation selectivity across different layers of macaque V1, these authors observed that the degree of dynamic orientation tuning depended on the laminar position of the neurones. The cortical layers which received direct input from the LGN (layers 4Ca and 4Cb) contained neurones which showed a single, broadly-tuned orientation preference, that did not change over time, while neurones in the output layers of V1 (layers 2, 3, 4B, 5 and 6) showed a range of dynamic behaviours. In these “output” layers, orientation tuning was narrower than neurones of the input layers, and their preferred orientations changed over post-stimulus time. Furthermore, as reported by Dinse et al. (1991) for cat visual cortex, a subset of neurones in V1 showed orientation tuning that increased in sharpness over time (Ringach et al., 1997). Taken together, the results from these experiments indicate that the neurones in the primary visual cortex are not simply a collection of static spatial filters or feature detectors; instead, their RF size and feature selectivity vary as function of post-stimulus time. 2.4.1. Potential circuit mechanisms underlying visual cortical dynamics An important and unresolved question concerns the neural circuitry underlying the
Time and the brain
120
genesis of visual spatiotemporal receptive fields. Both feedforward and feedback mechanisms have been proposed to account for the type of cortical and subcortical dynamics described above. Feedforward mechanisms could include the convergence of multiple cortical neurones, which do not exhibit spatiotemporal RFs, upon neurones which do. Alternatively, feed-forward inputs from two different classes of LGN neurones, each with different temporal dynamics (lagged versus non-lagged LGN neurones [Saul and Humphrey, 1992]) could account for some of the time-dependency of RFs in V1. As proposed by DeAngelis et al. (1995), simultaneous, multi-site recordings of LGN and V1 neurones (Reid and Alonso, 1995) are needed to test these hypotheses. Feedback cortical circuitry, on the other hand, could account for the dynamic feature tuning and the increases in selectivity of V1 neurones observed over poststimulus time (Dinse et al., 1991; Ringach et al., 1997). For orientation tuning and end-stopping, many neurones displayed orientation tuning which became narrower over time. It is conceivable that this increase in selectivity results from a “sculpturing” process, carried out by the asynchronous convergence of multiple feedback projections to the same set of V1 neurones (Dinse et al., 1991). According to this hypothesis, the initial state of widespread excitation of the cortex caused by the onset of the stimulus would be followed by reverberating activity from nearby local circuitry that could increase the selectivity of cortical neurones. In addition to local circuitry, other cortical areas could contribute, via feedback projections, to the definition of the RF structure of V1 neurones at longer latencies (Mignard and Malpeli, 1991). Overall, these results from both the LGN and visual cortex indicate that spatiotemporal RFs properties are likely to exist at most, if not all, processing levels of the visual system, and that this time-dependency in single neurone firing likely results from the asynchronous convergence of many inputs from different feedforward and feedback projections to individual neurones. To test such a hypothesis, one would need to carry out simultaneous multi-site recordings of neuronal responses (i.e. from thalamus and cortex) that are obtained before, during, and after selective inactivation of feedforward and/or feedback pathways. Experiments of this type are more feasible using the rat somatosensory pathway as a model system. 2.5. Spatiotemporal Receptive Fields Across the Somatosensory Thalamocortical Loop 2.5.1. Spatiotemporal receptive fields in the somatosensory thalamus Ensembles of well-isolated single neurones from the ventral posterior medial (VPM) nucleus of the thalamus, and the posteromedial region of the primary somatosensory (SI) cortex, of awake and anaesthetized rats have been simultaneously recorded using a technique based upon chronically-implanted arrays of microwires (Nicolelis and Chapin, 1994; Nicolelis et al., 1997a). These brain regions are known to process tactile information coming from the large facial whiskers on the rat snout. As in traditional studies, the RFs of thalamic and cortical neurones were reconstructed in these studies,
The space-time continuum in mammalian sensory pathways
121
following repetitive mechanical deflections of individual whiskers. For this analysis, a single neurone’s response to the stimulation of ~5x5 matrix of whiskers (deflected one at a time) was divided into 5 millisecond post-stimulus epochs. For each epoch, the whisker that elicited the greatest response (in terms of spike counts) was defined as the “RF centre” for that time epoch. As the “centre” whisker often varied as a function of poststimulus time, we were able to map spatiotemporal RFs for most neurones in both the VPM and SI cortex (Nicolelis and Chapin, 1994; Ghazanfar et al., 1995). This approach is identical to the response plane techniques used to map spatiotemporal RFs in the visual system (Dinse et al., 1991; Stevens and Gerstein, 1976). The chronic and simultaneous neural ensemble recording approach is uniquely suited to carrying out a quantitative analysis of the response properties of neurones across the thalamocortical pathway. First, the stable recordings obtained using the chronic placement of microwires allows one to sample from populations of well-isolated single neurones for several hours at a time, permitting the use of a large stimulus set (5 whiskers) and a large number of trials (360) per site. Second, since many neurones are recorded in parallel, several non-stationary effects (e.g. anaesthesia, placement of stimulus, etc.) can be reduced or eliminated. Finally, this approach permits one to record from the exact same set of neurones before, during and after experimental manipulations, such as deafferentation and/or pharmacological inactivation of different pathways. Traditionally, the RF of a somatosensory neurone is defined as the area of the body surface that, when stimulated, triggers a significant increase in this neurone’s firing rate. As in other species, most classical electrophysiological studies of the “whisker region” of rodent somatosensory system have focused on the characterization of the spatial attributes of the RF of neurones located in the subcortical and cortical processing centres of the trigeminal pathway. Recently, Nicolelis, Chapin and their colleagues (Nicolelis et al., 1993a; Nicolelis and Chapin, 1994) employed simultaneous neural ensemble recordings to sample the sensory responses of populations of neurones distributed across the whisker representation area of the main thalamic nucleus of the trigeminal somatosensory system, the VPM. These authors carried out a detailed analysis of the RFs of individual VPM neurones in both awake and anaesthetized animals, and revealed that single VPM neurones have large, multiwhisker RFs whose centre is defined by one whisker whose stimulation elicits the strongest sensory response at the shortest latency. This whisker is classically known as the principal whisker (PW) of the RF, and is commonly used to identify the location of a given neurone in the topographic maps of the whisker pad observed across the trigeminal system. Further analysis, however, revealed that the location of the PW (or RF centre) of a given VPM neurone could vary as a function of poststimulus time. Indeed, VPM neurones could be divided into two functional classes, according to the time-dependency of their RF centres. In about 41% of the recorded VPM neurones, the spatial position of the RF centre shifted over poststimulus time (Figure 1, bottom panel). The remaining neurones exhibited RF centres that remained in the same whisker over time (Nicolelis and Chapin, 1994).
Time and the brain
122
Figure 1 . Top panel. Spatiotemporal receptive field of a cortical neurone located in layer V of the rat primary somatosensory cortex. Bottom panel Spatiotemporal receptive field of neurone located in the ventral posterior medial nucleus of the rat thalamus. Notice that the spatial domain of these receptive fields varies as a function of post-stimulus time.
The first class of neurones exhibited a spatiotemporal coupling of their RFs, because the position of the RF centre, and its surround, varied as a function of post-stimulus time. In other words, the PW in the short latency component (PWSL) was different from the PW in the long latency component (PWLL) of the response. Interestingly, all spatiotemporal VPM RFs exhibited time-dependent RF shifts of centres in a caudal-to-rostral direction, and, consequently, were termed C->R cells. For example, one particular C->R neurone responded best to its PWSL, whisker C1, at 5–10 milliseconds post-stimulus time, while still responding significantly to many other caudal and rostral whiskers which defined its RF surround. By 15–20 milliseconds, the strongest response for this neurone was no longer elicited by whisker C1, but by whisker B3, which became the PW of the RF at long-latency (or PWLL). Thus, the RF shift observed in the VPM was clearly defined by a smooth transition in the spatial distribution of sensory responses observed in the first 30 ms of post-stimulus time. In general, C->R neurones had the largest RFs (average of 17.2 whiskers). The time-dependent shifting of the centres of these cell’s RFs seemed to evolve gradually, and appears to involve two factors: 1) an early component composed of short-latency responses from the caudal whiskers; and 2) a time-dependent
The space-time continuum in mammalian sensory pathways
123
enhancement of responses to more rostral whiskers. Conversely, a second class of VPM neurone was observed, which did not exhibit any evident time-dependent shift in their RFs. Invariably, this group of neurones had their PWs located in the rostral part of the whisker pad, which contains small and immobile vibrissae. These neurones were labelled “rostral position (RP) cells”. The RFs of these neurones was characterized by the existence of a single whisker (the PW) which elicited the largest response at all post-stimulus times, and by the presence of surrounding whiskers which constituted the RF surround. 2.5.2. Dynamic receptive fields in the primary somatosensory cortex Classically, neurones located within a given cortical barrel column are believed to respond preferentially to the deflection of one “principal” whisker (PW), and to fire less vigorously to stimulation of several adjacent whiskers that define the surround region of the neurone receptive field (SRF) (Armstrong-James and Fox, 1987). As in the VPM, responses to the PW occur at shorter latencies than responses to SRFs; the distribution of response latencies to centre and surround whiskers is clearly bimodal. In a series of experiments, Ghazanfar and Nicolelis, (1999) investigated whether the differences in latency observed between a stimulus applied to a PW or the SRFs could define spatiotemporal RF in the rat SI cortex. This was an interesting question to ask, because until recently most studies of rat SI cortex had focused primarily on the spatial aspects of cortical RF organization. Although multi-whisker RFs had been described in the rat SI before (Chapin, 1986; Simons, 1978), and the existence of a time-dependent spread of cortical recruitment had been proposed (Armstrong-James et al., 1992), the spatiotemporal organization of SI cortical neurone RFs was never characterized. Ghazanfar and Nicolelis, (1999) analyzed in detail the time-dependent RF dynamics of 197 cortical neurones located in layer V of the rat SI cortex. Overall, these authors observed that, for the vast majority of these neurones, the spatial structure of their RFs changed as a function of post-stimulus time (Figure 1, top panel). However, the patterns of these SI cortical spatiotemporal RFs varied widely, unlike those reported for the VPM, which shifted primarily in a caudal to rostral direction, or not at all (Nicolelis and Chapin, 1994). We identified four directions of SI cortical RF shifts: 1) rostral-to-caudal (RC); 2) caudal-to-rostral (CR); 3) dorsal-to-ventral (DV); and 4) ventral-to-dorsal (VD). Neurones whose RFs traversed an equal distance in two or more directions were defined as unclassifiable (UC). Out of the 197 neurones in our sample, 175 (88.8%) exhibited time-dependent shifts in their RF centres, while 22 (11.2%) did not. Of those with spatiotemporal RFs, 17.7% were RC shifters, 16.0% CR shifters, 23.4% DV shifters, and 6.3% VD shifters. The remaining 36.6% (64 out of 175 neurones) could not be classified as having a directional bias. In this study, Ghazanfar and Nicolelis, (1999) also demonstrated that the direction of the spatiotemporal RF shift was dependent upon the location of the PWSL. If a neurone’s PWSL was located in the dorsal part of the whisker pad, then the neurone was more likely to exhibit a DV spatiotemporal RF. If a neurone’s PWSL was in the more rostral part of the whisker pad, then it was more likely to exhibit a RC spatiotemporal RF. In summary,
Time and the brain
124
the authors observed that the large, asymmetrical RFs of layer V SI cortical neurones, and the variations in minimal latencies for surround whiskers, define complex spatiotemporal RFs in which the centre (or the PW) varies as function of post-stimulus time. Four classes of spatiotemporal RF types could be characterized based on directional biases of the RF movement, and these biases were, in part, dependent upon the location of the PWSL. Therefore, the existence of such time-dependent changes in RF structure suggests that populations of SI neurones located in layer V provide rather dynamic representations of the whisker pad, which incorporates both spatial and temporal domains of tactile sensory responses. 2.5.3. Hypothesis concerning the origins of spatiotemporal RFs in the somatosensory thalamocortical pathway The organization of VPM and SI cortical layer V RFs indicated that both thalamic and cortical neurones exhibit a variety of directional biases and complex spatiotemporal coupling. In their original studies (Nicolelis et al., 1993a; Nicolelis and Chapin, 1994), Nicolelis and Chapin postulated that the dynamic nature of these RFs resulted from the asynchronous convergence of feedforward and feedback pathways on to VPM neurones. The same hypothesis was put forth later by Ghazanfar and Nicolelis, (1999) to account for the genesis of spatiotemporal RFs in the SI cortex. For both VPM and SI cortical neurones, asynchronous inputs could arise from parallel, feedforward inputs from the ascending lemniscal and paralemniscal pathways. These pathways arise from the trigeminal brain-stem nuclei, and are known to have different temporal lags. Such a mechanism would be similar to that proposed for the genesis of spatiotemporal RFs in the visual system (DeAngelis et al., 1995). Moreover, feedback connections from a variety of cortical somatosensory areas also converge upon VPM and SI neurones (Fabri and Burton, 1991; Beck et al., 1998). These cortical feedback projections could also contribute to the time-dependent changes in RF centre, observed in both the VPM and the SI cortex. For instance, examination of the VPM C->R RF shifts suggested that these RFs could be broken down into early and late temporal components. These different temporal components of the RFs could emerge as the result of the temporally-asynchronous convergence of multiple excitatory and inhibitory afferents to VPM neurones. In this model, the early component of VPM RFs would result from inputs from the fast ascending somatosensory lemniscal pathways that include afferents from the principal and spinal trigeminal complex of the brainstem (which terminate on the same neurones in the VPM). The late component is a little more difficult to account for, but potential sources of input would include slower-conducting trigeminal pathways, and the very dense excitatory corticothalamic inputs that converge on the distal dendrites of VPM neurones. Although the spatioternporal properties of cortical neurones are likely to vary according to their laminar location, the dynamic RFs of SI cortical neurones described above are also likely to arise from the asynchronous convergence of multiple afferents on the same population of cortical neurones. Such inputs could arise from parallel, feedforward inputs from the lemniscal and paralemniscal pathways, which converge onto
The space-time continuum in mammalian sensory pathways
125
layer V neurones at different timelags, from intracortical feedback connections within SI (Gottlieb and Keller, 1997; Hoeflinger et al., 1995), or from other sensory and motor cortical areas (Fabri and Burton, 1991; Koralek et al., 1990). 2.5.3.1. Testing the asynchronous hypothesis for generation of thalamic spatiotemporal RFs Two types of experiments were employed to test whether an “asynchronous” convergence of inputs could account for the generation of spatiotemporal RFs in the rat VPM nucleus. First, the contribution of feedforward somatosensory pathways to the genesis of their RFs was tested by inducing reversible deafferentation of the whisker pad, through subcutaneous infusions of a local anaesthetic, lidocaine. This local anaesthetic allowed measurements of changes in RF organization which emerge as a consequence of altering the flow of tactile information through ascending feedforward trigeminal somatosensory pathways to the VPM and SI cortex (Nicolelis et al., 1993b; Faggin et al., 1997; Nicolelis, 1997). As predicted, this reversible peripheral deafferentation induced immediate reorganization in both the spatial and temporal properties of VPM and SI RFs. The two major effects were 1) a temporal shift resulting in the elimination or reduction of early, short-latency component of RFs, located within the “anaesthetized zone”, with the enhancement of the later components, and 2) a spatial shift where RFs that were originally centred in the “anaesthetized region” of the whisker pad shifted, immediately following the induction of the deafferentation, to neighbouring territories of the whisker pad. These experiments suggest that peripheral deafferentation largely effects the early, “feedforward” components of the VPM and SI cortical RFs. In a second series of experiments, the potential role of corticofugal projections in the genesis of spatiotemporal structure of VPM RFs was investigated (Ghazanfar et al., 1997; Krupa et al., 1999). The results obtained in these studies were complimentary to those reported for peripheral deafferentation (Nicolelis et al., 1993b; Faggin et al., 1997). Using a combination of multi-site chronic microwire arrays and a stereotaxically-placed cannula, Ghazanfar et al. (1997) and Krupa et al. (1999) were able to inactivate the SI cortex pharmacologically, using the GABA agonist, muscimol. Since the cannula was placed adjacent to a chronically implanted microwire array in the SI cortex, these authors were able to monitor the onset, duration, and tangential spread of muscimol. These experiments revealed that corticofugal projections contribute very importantly to the genesis of longer latency neuronal responses in the VPM nucleus, since, in most cases, blockade of cortical activity attenuates these later response components. As mentioned before, these long-latency components are critical to the definition of spatiotemporal RFs in VPM (Nicolelis and Chapin, 1994), and for the C->R RF shifts observed in this nucleus, since the PWLL (principal whisker or RF centre at long-latency) differs from the one defined at short-latency (the PWSL). Furthermore, it was also observed that cortical feedback projections can account for up to 50% of the immediate plastic effects observed in the VPM following peripheral deafferentation (Krupa et al., 1999). As described above, the results obtained with both peripheral deafferentation and inactivation of corticofugal projections support the asynchronicity hypothesis, and
Time and the brain
126
demonstrate that spatiotemporal RFs, at least in the rat VPM thalamus, are defined by large-scale interactions between multiple brain structures. 2.5.4. Active tactile exploration is required for the genesis of spatiotemporal RFs How do the spatiotemporal structures of VPM and SI RFs emerge in the first place? Upon the discovery of directional RF shifting in the rat VPM, a series of studies tested the possibility that experience-dependent mechanisms, via active whisker movements occurring from the time of early postnatal life, could account for the “sculpturing” of directional bias into the spatiotemporal structure of VPM and cortical RFs. Rats use rhythmic protractions of their long caudal whiskers to explore objects that are placed in front of them (Welker, 1964), while the small, immobile rostral whiskers are used primarily to establish and maintain contact with objects. Because active tactile exploration plays a critical role in the normal development of haptic perception in mammals (Bushnell and Boudreau, 1991), Nicolelis et al. (1996) and Oliveira et al. (1997) postulated that the spatiotemporal RFs of layer V SI cortical and VPM thalamic neurones in the rat probably reflect the pattern of spatiotemporal whisker inputs that occurs during the active “whisking” of object surfaces during natural behaviours. Patterned tactile stimulation generated by active exploration during postnatal development may be required for the normal functional organization of the somatosensory system. To test this hypothesis, the effects of neonatal disruption of active whisker movements on the development of thalamic and cortical spatiotemporal RFs was investigated (Nicolelis et al., 1996; Oliveira et al., 1997). In rats, the facial muscles used during whisker movements are innervated solely by branches of the facial nerve, which is clearly separated from the infraorbital nerve, a branch of the trigeminal nerve, which carries low-threshold tactile information from the facial whiskers. Because of this anatomical peculiarity, it was possible to eliminate the production of active whisker movements in neonatal rats by sectioning the facial nerve before these animals experienced any sort of active whisker movements. Complete peripheral nerve section prevented rats from making active whisker movements on one side of the face throughout their development and adulthood, without affecting the sensory innervation—via the infraorbital nerve—of their facial vibrissae. After the rats reached adulthood, the spatiotemporal structure of somatosensory RFs was examined and it was observed that preventing the expression of active tactile exploration during development dramatically alters the spatiotemporal RF organization of VPM and SI neurones. First, the size of neuronal RFs in both structures was significantly reduced. In the VPM, for instance, the reduction was on average from 13 whiskers (normal animals) to 6 whiskers (in animals with facial nerve section). The spatiotemporal organization of RFs was also disrupted in the VPM and SI cortex. For example, in normal adult rats 100% of VPM neurones whose PWSL were located in the caudal part of the whisker pad exhibited a caudal-to-rostral shift in their RF centres over post-stimulus time. In rats with neonatal facial nerve sections, on the other hand, only 19% of VPM neurones with PWSL in the caudal whisker pad exhibited time-dependent shifts in their RF centres (Nicolelis et al., 1996). In other words, the dynamic properties of VPM neurones were dramatically reduced by
The space-time continuum in mammalian sensory pathways
127
preventing these animals from actively exploring their environment during development. Therefore, these results underscore the importance of active movements for the functional maturation of normal spatiotemporal RFs, in both cortical and subcortical relays of the rat somatosensory system, and implicate experience-dependent mechanisms for the establishment of spatiotemporal RFs in the somatosensory system.
3. SPATIOTEMPORAL CODING BY SINGLE NEURONES AND NEURAL ENSEMBLES 3.1. Rate Versus Temporal Coding? Although the analysis of single neurone RFs continues to play an important role in classical neurophysiology, more contemporary views recognize that the RF of a neurone offers only a probabilistic description of the “potential” responses that can be exhibited by that neurone within the parameter space of a particular type of sensory stimulus (e.g. all the facial whiskers, or all the orientations of a bar of light). Thus, the investigation of how neurones encode information derived from a subset of the parameter space of a sensory stimulus does not rely exclusively on the analysis of single neurone RFs. The investigations of neural codes revolves around identifying a measurement or metric of neuronal activity to describe a particular stimulus parameter, behaviour, or cognitive skill. Neural codes are also studied in the context of how different areas of the brain exchange information and relate to each other. Many neural metrics have been proposed to operate as candidate neural codes in different neural structures and in different species. In general, most of these algorithms have been dichotomized as variations of two main themes: “rate” or “temporal” codes (Shadlen and Newsome, 1994). The prevailing view that firing rate is the universal code was first advocated by Adrian (1928), who suggested that a neural code is simply the mean firing rate of the action potentials evoked from a neurone, and rate variability is “noise”. Because this noise can be filtered out, or reduced, by averaging across time or by recording the activity of large populations of neurones, a rate code performs very well in the presence of noise, but has limited information-carrying capacity. In contrast to the classical rate code hypothesis, multiple schemes have been proposed in which temporal aspects of neuronal firing play a significant role in conveying information. Temporal coding schemes include measurements based on: 1) the precise timing of individual spikes (in a 1–3 millisecond range); 2) the precise temporal sequence of firing across a population of neurones (in a range of <5 millisecond); 3) the occurrence of widely distributed synchronous neuronal firing activity at frequency ranges varying from 1–100 Hz (periods between >10 ms and <1000 ms); 4) the time-dependent modulation of firing rate exhibited by single neurones; and, 5) the occurrence of stimulus or behaviour-related spatiotemporal patterns of firing across populations of neurones (ranging from 10 to 100 ms). Although the jury is still out and not likely to return an early verdict, it seems fair to say that the discussion between rate versus temporal coding is an artificial one. First,
Time and the brain
128
most of the schemes of temporal coding actually represent a particular case of a rate code, which is limited to a particular time epoch. On the other hand, as discussed above, the high level of stimulus selectivity of neurones seems to vary over time, and thus, the identity of a stimulus cannot be ascertained simply by time-averaging each neurone’s response function. This suggests that a strict rate code does not seem to do justice to the richness of neuronal dynamics observed throughout the central nervous system. Instead, the coding of stimuli seems to be best represented when the temporal modulation of the sensory response is taken into account. 3.2. Temporal Coding of Space by Single Neurones 3.2.1. The presence of stimulus-related information in the response waveform ofLGN neurones For many decades, the dominant assumption in visual physiology has been that the strength of a visual neurone’s response is the only variable which contains relevant information about the stimulus (Barlow, 1972). Consequently, the description of temporal structures in the sensory responses of neurones was used simply to classify different LGN cell types (Marrocco, 1976) or to investigate how these neurones integrate their inputs (Dawis et al., 1984). More recently, however, studies using complex patterns of visual stimulation have demonstrated that neurones in the visual pathway respond to a variety of different stimulus attributes. These studies have also revealed that, by including the temporal structure of their sensory responses as an additional dimension of coding, one can increase the information carrying capacity of these neurones. Using behaving primates performing a fixation task, McClurkin et al. (1991a,b) studied the visual responses of single neurones in the parvocellular layers of LGN to a set of two-dimensional black-and-white patterns, called Walsh patterns, that could be used for the linear reconstruction of any picture of the same resolution (in this case, 8 x 8 pixels). The neuronal responses to these different patterns were characterized by examining the strength and temporal pattern of the responses using principle component analysis (PCA). These authors found that parvocellular LGN neurones displayed a variety of temporal firing patterns, and that these firing patterns varied in a reproducible way as a function of the patterns presented. The differences in firing patterns included changes in the magnitude, width, and number of peaks seen in the initial response evoked by the stimulus, as well as changes in the magnitude of the sustained, longer-latency component of the responses. These results were interpreted as an indication that the time-dependent modulation of neuronal firing, i.e. the temporal distribution of LGN spikes, might also encode information in addition to the total number of spikes fired in a given epoch of time (McClurkin et al., 1991a). McClurkin et al. (1991b) then employed PC A to quantify the differences in temporal response patterns elicited by the set of Walsh patterns. By definition, PCs are linearly weighted polynomia which are orthogonal to each other. By correlating the derived PCs with different attributes of neuronal responses, these authors showed that the first
The space-time continuum in mammalian sensory pathways
129
principal component was highly correlated with the overall strength of the response. Higher principal components either were related to the shape of the response (i.e. the temporal distribution of spikes) or identified the particular stimulus eliciting the response. To assess how much stimulus-related information was carried simply in the firing rate versus both the magnitude and shape of the response, information theory analysis was applied to the response strength alone versus the first three principal components, which carry both response strength and temporal patterning information. This analysis revealed that the inclusion of both the response strength and temporal distribution of spikes elicited by a given stimulus increased the information carrying capacity of LGN neurones 1.5 times when compared to the response strength alone. Based on these results, the authors hypothesized that this form of temporal coding cannot be explained by traditional models of LGN neurone properties, and proposed that these neurones and other neurones along the visual pathway should be considered as spatiotemporal filters. To test this hypothesis directly, Golomb et al. (1994) analyzed the spatiotemporal RF properties of primate LGN neurones and used this information to predict their responses to the Walsh patterns used by McClurkin et al. (1991a,b). They found that the spatiotemporal RF properties could indeed be used to predict the temporal response patterns seen by McClurkin et al. (1991a,b). Similarly, information theory analysis revealed that these spatiotemporal firing patterns could be used to distinguish between different Walsh stimuli on the basis of a single response. Therefore, these two studies (McClurkin et al., 1991a,b; Golomb et al., 1994) provided a direct link between the spatiotemporal RF properties and temporal coding capacity of LGN neurones. 3.2.2. Stimulus-dependent nature of temporal coding in the visual cortex Using an experimental approach identical to that used to study LGN neurones McClurkin et al. (1991a,b), Richmond et al. (1990) and Richmond and Optican (1990) studied the potential coding dimensions of primate primary visual cortical neurones. Overall, the results were very similar to those found for LGN neurones. Responses of both simple and complex V1 neurones showed robust differences in their response strength, and temporal modulation to different Walsh patterns (Richmond et al., 1990). Once again, principal component and information theory analyses revealed that the nature of the stimulus could best be predicted by taking into account both the response magnitude and the temporal pattern of spiking (Richmond and Optican, 1990). These authors also reported that V1 complex neurones exhibited substantially different temporal response profiles when Walsh patterns (or oriented bars) and their contrast-reversed mates were employed as stimuli. Along similar lines, McClurkin and Optican (1996) reported that the number of spikes elicited by V1, V2 and V4 primate visual cortical neurones varied only for one or the other of the stimulus types (colour vs. pattern). However, the temporal distribution of spikes was affected by both stimulus types, suggesting that neurones in multiple visual cortical areas can encode multiple stimulus features simultaneously, both through changes in number of spikes and by time-dependent modulation of firing rates (McClurkin and Optican, 1996; Richmond et al., 1990). Put in another way, these results indicate that different features of a stimulus do not need to be encoded by distinct
Time and the brain
130
populations of neurones, each devoted to a particular stimulus attribute. Instead, the same population of neurones could encode multiple stimulus attributes simultaneously. Using an entirely different analytical approach, Victor and Purpura (1996) have proposed that temporal coding of stimuli in V1 and V2 visual cortical areas of primates performing a fixation task is dependent upon the nature of the stimulus. These authors developed a metric to compare the similarities between two spike trains, based on three basic parameters: the total number of spikes, the intervals between spikes, and the absolute spike time. Each of these parameters was then used to calculate the distance between all pairs of spikes within each data-set. By varying the width of their analysis window, these authors were able to characterize parametrically the precision of spike timing across different stimulus conditions. Their analyses revealed that five different stimulus attributes (contrast, check size, orientation, spatial frequency, and texture) could elicit responses from V1 and V2 neurones that varied systematically in both the number of spikes and their temporal patterns (Victor and Purpura, 1996). Interestingly, there was evidence for temporal coding for all the stimulus attributes tested, but the precision of the temporal code varied. For instance, contrast stimuli elicited spike trains with a ~ 10–30 millisecond precision, while texture stimuli elicited responses with a ~ 100 millisecond precision. Like the results of Richmond et al. (1990), these findings suggest that multiple stimulus attributes could be represented simultaneously in the spike trains of visual cortical neurones. For example, at a high temporal resolution, stimulus contrast could be encoded, while at a lower resolution, the same spike trains could encode texture (Victor and Purpura, 1996). 3.2.3. Coding of sound location by the temporal discharge patterns of single auditory cortical neurones In addition to identifying different types of sounds such as vocalization, the auditory system must also determine the source of sounds. In the visual and somatosensory systems, it is presumed that stimulus location, at least at early stages of these pathways (i.e. up to the level of the primary sensory cortices) is somehow represented by the orderly retinotopic or somatotopic maps found in these brain regions. Although neuropsychological studies have demonstrated that lesions of the auditory cortex result in sound localization deficits (Wortis and Pfeiffer, 1948), no analogous topographic maps, formed by neurones sharply-tuned to sound-source location, have been found in the auditory cortex (Middlebrooks et al., 1998). For instance, auditory cortical neurones in the anterior ectosylvian sulcus of the cat respond to noise bursts, and are broadly-tuned for sound source location (Middlebrooks et al., 1994, 1998). Thus, by using only their average firing rates, one cannot get a very precise estimation of the location of a sound. Middlebrooks et al. (1994, 1998) tested whether the patterns of spiking of single neurones could more reliably encode the location of a sound. An artificial neural network (ANN) was trained to classify the temporal spike patterns of single auditory cortical neurones to sounds coming from different locations in azimuth. Surprisingly, it was found that the spiking pattern of single neurones could encode the location of a sound across 360 degrees of azimuth at more than twice the level of chance. In other words, the
The space-time continuum in mammalian sensory pathways
131
neurones were not tuned to a single “best area” of sound localization. When the resolution of the spike trains was decreased beyond 1 millisecond, the ability of the ANN to discriminate different patterns degraded. Based on these results, Middlebrooks et al. (1998) argued that the organization of the auditory system is uniquely suited to preserving the temporal fidelity of sounds. Nevertheless, as they pointed out, since the performance of single neurones was still below the psychophysical ability of the animal to locate sounds (Jenkins and Masterton, 1982), it is likely that a distributed population of neurones is necessary to identify the precise location of a sound in a real world scenario. 3.3. Dynamic and Distributed Representations of Sensory Inputs by Neural Ensembles The presence of neurones with large receptive fields, whose tuning changes as a function of time, across different sensory pathways, makes it apparent that even the simplest of stimuli is likely to elicit temporally-complex responses from large populations of cortical and sub-cortical neurones. The distinction between temporal and rate coding, in this context, can be seen only as a heuristic dichotomy, since both measures supply one with important information about different stimuli. It, therefore, seemed reasonable to focus our experimental efforts to measure the dynamics of populations of neurones distributed across a given brain region, or multiple brain regions simultaneously, in order to address the question of how time is used by the central nervous system to encode information. With the recent advent of electrophysiological and analytical techniques for multielectrode recordings (Nicolelis, 1998 in press), multi-site neural ensemble recordings have sparked renewed interest in the properties of neural assemblies and their potential roles in brain function. Much of this has focused on pair-wise analysis of neurones, but in some instances larger-scale approaches have been successful in recording and analyzing small ensembles across different brain structures (Ghazanfar and Nicolelis, 1997; Nicolelis et al., 1997a). Therefore, it is not surprising that many laboratories have begun applying neural ensemble recordings to investigate how neuronal populations encode sensory and motor information. In the next sections, some of this recent work carried out in the mammalian visual, auditory, and somatosensory systems is reviewed. 3.3.1. Ensemble coding of visual features Due to the extensive interconnections between neurones both within and across multiple brain structures, even the simplest stimuli will elicit a response from a large population of neurones. This gives rise to the problem of how multiple stimuli can be represented by overlapping populations of neurones. To address this problem, a relational neural code based on synchronous neuronal firing has been suggested as a solution. According to this hypothesis, the joint activity of neurones could allow the many different features of a particular stimulus to be “bound” together; i.e. “feature extractor” neurones should fire synchronously in order to represent the multitude of stimulus features that define a visual scene (Singer and Gray, 1995). In support of this contention, multi-electrode experiments
Time and the brain
132
have been carried out in a variety of visual structures, and these studies have shown that, under certain conditions, neurones can synchronize their firing in response to global stimulus features. In the visual cortex of the anaesthetized cat, Gray and Singer (1989) showed that neurones in a cortical column fire synchronously during the presentation of a moving bar of light. From this result, they hypothesized that synchronous firing may occur between neurones in two spatially distinct cortical columns. To test this idea, they recorded multiunit responses to oriented bars simultaneously from several spatially separate sites in cat primary visual cortex (Gray et al., 1989). Using cross-correlations, they demonstrated that neurones with non-overlapping RFs from spatially distinct locations (up to 7 mm apart), but with similar orientation preferences, could robustly synchronize their firings if the stimulus extended across both neurones’ RFs. When two separate, smaller stimuli were moved together, at the same time and in the same direction, to drive each neurone, weak correlations occurred. However, moving the two stimuli in opposite directions completely abolished synchronization. These results suggest that this neuronal synchronization depended on global features of the stimuli, such as coherent motion (movement in the same direction vs. opposing directions), and continuity (one long bar vs. two short bars) (Gray et al., 1989). Similar results have been reported for neurones located in different cortical areas and between hemispheres (Nelson et al., 1992; Nowak et al., 1995). Nonetheless, it is important to emphasize that, implicitly, this theory still supports the notion that single neurones behave as feature extractors and that explicit mechanisms for “binding information” are required. The presence and potential relevance of neuronal synchronization in the visual system has also been investigated by studying the correlated activity of LGN neurones in the cat. Sillito et al. (1994) have shown that neurones in the LGN of the cat may show significant correlated activity under the influence of stimulus continuity and coherent motion. These authors reported that LGN neurones with non-overlapping receptive fields did not show any significant temporal correlations when activated by a pair of stationary spots. However, when a single bar or moving gratings that covered both neurones’ receptive fields were presented, the same cells would synchronize their firing. By inactivating the overlying visual cortex, these authors also demonstrated that cortical feedback to the LGN was required for the occurrence of this synchronization. Similarly, Alonso et al. (1996) recorded from pairs of cat LGN neurones and found that neurones that had overlapping RFs and monosynaptic connections with primary visual cortical neurones would correlate their firing during visual stimulation. These authors reported that this correlated firing of LGN neurones enhances the efficacy of their inputs upon visual cortical neurones. Whether synchronization under these circumstances was due to feedforward or feedback influences remains to be investigated. In a departure from pairwise cross-correlation studies, Radons et al. (1994) investigated visual neuronal population responses to moving bars, both light and dark. The data-set, containing both multiple- and single-units, was obtained from a 30microelectrode array implanted in the primary visual cortex of an anaesthetized monkey (Kruger and Aiple, 1988). These authors used hidden Markov models, a patternrecognition technique, to determine how well the spatiotemporal firing patterns of neural ensembles could be used to recognize the different visual stimuli. Using this approach,
The space-time continuum in mammalian sensory pathways
133
they found that the identity of the stimulus types could be classified with 90% accuracy, several times above chance levels, when the time window of the analyses was between 25 and 100 milliseconds; increasing or decreasing the time window resulted in a drop in performance. Because of this relatively coarse time scale, the recognition of stimulus types was based simply on the spatiotemporal patterns of mean firing rates and not on the covariance of firing among neurones within the ensemble. Thus, these results argue that the representation of a stimulus can be achieved using the graded responses of a population of neurones without the need for high levels of neuronal synchrony. Another solution to “feature binding” is the convergence of many neurones coding for particular features upon a small numbers of neurones which code for the conjunction of those features (Hubel and Wiesel, 1962). Higher order visual areas are the places where one would expect to find such “grandmother” neurones. Gochin et al. (1994) tested whether neural ensemble coding was necessary for the complex feature detection in a higher order visual area. This was accomplished by focusing on the inferior temporal (IT) cortex, a region located along the ventral pathway of the macaque visual cortex (the “what” pathway, encoding object identities). Because of its position in the visual pathway, this cortical region is ideally-suited for investigating the existence of local versus distributed coding schemes. Although some of these IT neurones are highly selective, and can respond selectively to complex visual patterns such as faces and hands (Desimone et al., 1984), most of them do not respond only to one particular visual stimulus. It follows, therefore, that the stimulus discrimination capability of the IT area would best be assessed by studying groups of neurones. Thus, Gochin et al. (1994) tested the ability of ensembles of IT neurones, in a paradigm in which each neurone was recorded sequentially from a monkey performing a paired-association task, to encode different complex visual patterns. Data were analyzed by constructing firing rate vectors (time-scale =100 milliseconds) from serially recorded neurones, and applying linear discriminant analysis techniques to assess the ability of these reconstructed ensembles to distinguish between five different visual images. The main finding was that ensembles of IT neurones were able to discriminate between different stimuli much better than single IT neurones. Furthermore, the discrimination capacity varied as a function of ensemble size: the bigger the ensemble, the better the performance. Systematically dropping neurones from the ensemble resulted in “graceful” degradation of discrimination capacity. Consistent with this finding, the application of information theory analysis revealed that the distribution of information across the IT neural ensemble was unimodal, and thus, not segregated to particular neurones within the ensemble. Interestingly, the amount of information carried by the ensemble was less than the arithmetic sum of the information carried by individual neurones, suggesting that there is considerable redundancy across the population. In an important control, data from simultaneously recorded IT neurones were analyzed in the same way, and there was no significant increase in the amount of information contained in the ensembles. Based on this result, the authors suggest that, at least in IT cortex, little information is contained in the covariance of neural activity. Nevertheless, in a previous study Gochin et al. (1991) demonstrated, by recording small ensembles of IT neurones simultaneously from a 3-microwire bundle, that nearby neurones can synchronize their firing patterns in response to a visual stimulus, while neurones that
Time and the brain
134
were far apart are weakly correlated. The role of such synchronous firing, if any, in information processing of complex visual images remains to be investigated. 3.3.2. Encoding of sounds by neural ensembles in the auditory cortex In the auditory cortex of the rat, Maldonado and Gerstein (1996) were interested in determining the changes in neuronal ensemble dynamics that follow sensory reorganization induced by intracortical microstimulation (ICMS). ICMS is known to produce a broadening of the receptive fields of cells located at the stimulation site. In addition, neurones recorded from adjacent electrodes have been shown to increase their responsiveness to the best frequency of the cells recorded from the stimulating electrode. The functional relationships between neurones distributed within the primary auditory cortex were assessed by using the gravity analysis (Gerstein et al., 1985), a method in which the temporal relationships between neuronal spike trains are represented as a series of clusters in a multidimensional space. In this multidimensional space, neurones attract or repel each other depending on the coincidence of their neuronal firing. In this study, the responses of up to eight neurones were recorded simultaneously following auditory stimuli and ICMS. The experiments demonstrated that the functional clustering of a subset of the simultaneously recorded neurones, obtained during the delivery of the auditory stimuli, could be strengthened following ICMS. Interestingly, the formation of these functional clusters of neurones did not necessarily relate to the anatomical distance between the cells. In other words, neurones that were anatomically close did not necessarily have a stronger interaction, and neurones that were far apart did not necessarily have weaker interactions. Maldonado and Gerstein’s results indicate that neural ensembles can be established transiently in time, and that they are not necessarily composed of neurones within a highly circumscribed location (e.g. a cortical column). Moreover, their data suggest that the membership of a neural ensemble is mutable as a function of reorganization induced by ICMS. In a direct test of Middlebrooks et al.’s (1994, 1998) distributed coding of sound location hypthesis, Ahissar et al. (1992) demonstrated that broadly-tuned auditory neurones could encode sound location and movement by their pair-wise interactions. By recording the simultaneous extracellular activity of up to nine auditory cortex neurones in awake monkeys trained to sit still, Ahissar et al., studied the sensitivity of primary auditory cortical activity to azimuth and angular movements of acoustic stimuli. They found that two neurones, each broadly-tuned for spatial location, could synchronize their firing to predict spatial location more accurately. Analysis of cross-correlation histogram shapes revealed that different stimulus conditions could modulate the “effective connectivity” of the two neurones; that is, the magnitude and time course of synchronization varied according to stimulus parameters. These authors also noted that correlation of firing between neurones could occur in the absence of changes in the firing rates of each individual neurone. This suggests that temporal interactions between pairs of broadly-tuned auditory neurones, whose firing rates alone were unable to predict spatial location, could be used to discriminate between different spatial locations and angular movements of sound.
The space-time continuum in mammalian sensory pathways
135
The results of Ahissar et al. (1992), suggesting a role for relational coding in auditory processing, are supported by another data set obtained in the primary auditory cortex of anaesthetized marmosets. In this study, cross-correlation analyses revealed that pairs of neurones, or multiunit clusters, may also synchronize their firing according to another stimulus parameter, sound duration (deCharms and Merzenich, 1996). Using pure tone stimuli appropriate to the tuning of the recorded neurones, deCharms and Merzenich showed that short duration stimuli could change the firing rates of auditory cortical neurones. However, the same stimuli delivered for a longer duration could coordinate the firing patterns of neurones recorded simultaneously from two nearby locations. Importantly, this synchronization occurred without changes in the overall firing rates of each of the neurones. The coordination of activity between neurones could last for the duration of the stimulus, while the firing rate remained unchanged. Thus, only pair-wise corticocortical interactions were able to represent duration of a sound. Similar to the results reported by Ahissar et al., (1992), the degree of synchronization varied according to stimulus parameters. The correlation of firing between neurones was strongest when the stimuli were adjusted to fit their frequency tuning profiles, and tended to weaken when the distance between neurones increased. As mentioned in the introduction to this chapter, both the behavioural relevance and the past experience of an animal are believed to play an important role in the function of sensory systems. The recognition of species-specific vocalizations is particularly relevant for many animals since an important part of their everyday life depends on these sensory experiences. In the anaesthetized marmoset, Wang et al. (1995) have shown that subpopulations of neurones in the primary auditory cortex respond selectivity to a speciesspecific vocalization, the “twitter” call. Although most of the neurones were recorded serially, the time-varying and spatially distributed population responses were reconstructed according to their frequency tuning and post-stimulus time. This study revealed that the twitter call could elicit activity from a population of neurones that were spatially distributed across the primary auditory cortex, and that the pattern of firing was correlated with the spectro-temporal characteristics of the vocalization, but on a coarser time scale. In other words, the firing of these neurones phase-locked to large portions of the stimulus, and not to the finer-grained changes in frequency modulation. Two different vocalizations elicited two different cortical population response profiles, with considerable overlap in terms of the members of the population. Temporally-manipulated synthetic versions of the call, which retained the appropriate spectral content, resulted in a degradation of neuronal responses and decreased the amount of synchronization across the population of neurones. This suggests that the temporal characteristics of the individual neurones’ RFs play a role as important as the spectral features. 3.3.3. Encoding of tactile stimuli by thalamocortical somatosensory ensembles Temporal interactions between neuronal populations located in different brain regions played a fundamental role in the concept of cell assemblies first put forward by Donald Hebb (1949). Testing Hebb’s assumptions requires one to record from populations of neurones distributed across multiple cortical areas and subcortical nuclei. Using a chronic
Time and the brain
136
multiple-electrode recording preparation, Nicolelis et al. (1995, 1997a) were able to record from populations of neurones distributed throughout the brainstem, thalamic and cortical processing levels of the trigeminal somatosensory pathway in both anaesthetized and awake rats. The results of these studies have emphasized that the co-ordinated activity of large ensembles of neurones, distributed across multiple cortical and subcortical structures, may provide the basis for the encoding of tactile information in the mammalian somatosensory system. Ensemble recordings of neurones in both VPM and SI cortex have revealed that, even in structures classically thought to employ a labelled line coding scheme, punctate tactile stimuli can elicit distributed responses across each of these brain regions (Figure 2). During normal exploratory behaviours, rats make many simultaneous whisker contacts with objects in the environment. It is important, therefore, to investigate how the somatosensory system integrates information that are more complex than punctate stimuli such as
The space-time continuum in mammalian sensory pathways
137
Figure 2 . Population histograms reveal the distributed nature of responses to punctate tactile stimuli in both the SI cortex (top panel) and VPM thalamus (bottom panel). Notice that individual neurones, all recorded simultaneously, within each population vary in their response magnitudes and latencies.
single whiskers. Studies in other sensory systems have uncovered non-linear neuronal properties when naturalistic stimuli are used to elicit sensory responses (e.g., Rauschecker et al., 1995; Wang et al., 1995). Towards this goal, Ghazanfar and Nicolelis
Time and the brain
138
(1997) recorded concurrently the responses of several tens of well-isolated single neurones from the VPM and primary somatosensory (SI) cortex following single- and multi-whisker deflections. These multi-whisker deflections consisted of three coincidentally-displaced whiskers oriented in rows or columns within the matrix of facial whiskers. Examination of ensemble responses revealed that both single and multiwhisker stimuli could elicit widespread and concurrent activation of neurones across the thalamocortical loop for several milliseconds. Multi-whisker stimuli oriented along columns of whiskers elicited non-linear cortical ensemble responses by activating significantly SI cortex more vigorously, and faster than the arithmetic sum of singlewhisker ensemble responses. This was not the case for multi-whisker stimuli oriented along rows of whiskers. Interestingly, this bias towards whisker columns was not apparent in the single neurone data for SI cortex. Therefore, these results suggest that ensembles of cortical neurones can generate complex spatial transformations that are not seen at the single neurone level. 3.3.4. Large-scale neural ensemble interactions across the somatosensory system In another set of studies, simultaneous multi-site neural ensemble recordings were used to investigate the large-scale interactions between populations of neurones distributed across multiple processing levels of the rat trigeminal somatosensory system (Nicolelis et al., 1995). These recordings included neurones located in the trigeminal ganglion, the principal and spinal subnuclei of the trigeminal brainstem complex, the VPM thalamus, and the infragranular layers of the SI cortex in freely behaving animals. The first important observation derived from this study was the discovery that cortical and subcortical somatosensory neurones can exhibit widespread 7–12 Hz synchronous oscillations (peak frequency at 10Hz), which tend to precede the initiation of tactile exploratory behaviours (whisker twitching movements). By correlating the electrophysiological data with behavioural measurements, obtained through quantitative analysis of facial electromyograms and frame-by-frame video records, Nicolelis et al. (1995) observed that these oscillations tended to appear during periods in which rats assume an immobile and alert posture (Figure 3). Interestingly, these oscillations preceded, by an average of 581 ms, the occurrence of small amplitude rhythmic whisker movements (whisker twitches) in the same frequency range (7–12 Hz). In about 80% of these episodes, the oscillations were first observed in the SI cortex where, on average, 92% of the neurones exhibit oscillatory activity as measured by autocorrelation functions. About 20 ms after the oscillations appeared in cortex, 98% of the VPM neurones, located in both hemispheres, were entrained into these oscillations (Figure 3). Just prior to the initiation of whisker twitches, about 49% of the neurones located in the spinal nucleus of the trigeminal brainstem complex began to exhibit the same oscillations. Neurones located in the principal nucleus of the trigeminal complex or in the trigeminal ganglion did not exhibit these oscillations. By stimulating individual whiskers during the occurrence of these oscillations, we observed that only stimuli delivered during the peak of the oscillations were capable of eliciting sensory-evoked responses. Stimuli delivered outside this small time window
The space-time continuum in mammalian sensory pathways
139
would either induce a phase shift in the underlying oscillatory activity or have no effect. Interestingly, stimuli delivered in a particular phase of these oscillations could block the oscillatory activity for several seconds. In some instances, tactile stimuli could also trigger the occurrence of such oscillations. Altogether, these results raised the hypothesis that top-down influences, generated primarily in the neocortex, are capable of entraining most of the somatosensory system into a pattern of firing (oscillatory mode), which may be ideal for processing sensory information obtained during rhythmic active exploration (through whisker movements) of the surrounding environment.
Figure 3 . Local field potentials recorded simultaneously from VPM and SI cortex before and during whisker twitching behaviour. Oscillations begin first in SI cortex, then in VPM, just prior to whisker twitching movements. Arrows denote the beginning of SI cortical oscillations. First six channels were recorded from electrodes in SI cortex, and last two were recorded in VPM (figure courtesy of Erika E.Fanselow).
3.3.5. Effects of chronic peripheral deafferentation on somatosensory ensembles Many studies have investigated the effects of peripheral sensory deafferentation on the properties of topographic maps in different brain regions (Kaas, 1991; Nicolelis, 1997). In the somatosensory system, peripheral deafferentation is known to result in plastic changes across multiple cortical and subcortical relays of the pathway (Florence and Kaas, 1995). For example, neonatal rats that underwent a complete unilateral removal of their facial whiskers had an extensive reorganization of their ventral posterior medial (VPM) nucleus (Nicolelis et al., 1991). Specifically, this reorganization resulted in the
Time and the brain
140
expansion of RFs to include abnormal regions of the body, such as the shoulder and forepaw, an increase in the response latencies, and an increase in the reverberatory bursting activity of VPM neurones. The effects of this reorganization of the dynamic properties of VPM neurones on the ability of VPM neural ensembles to represent the location of a tactile stimulus were investigated by Nicolelis et al. (1997b). Until recently, no study had examined how such peripheral manipulations affect the ability of the nervous system to encode various stimulus attributes. Ensembles of 13–20 single neurones were recorded simultaneously, and their spatiotemporal firing patterns were analyzed using discriminant analysis (DA). DA was used to quantify the effects of neonatal sensory deprivation on the ability of VPM neural ensembles to “discriminate”, on a trial-by-trial basis, the location of a tactile stimulus. Using this approach, it was determined that when the spatiotemporal patterns of VPM neural ensemble responses in normal rats were used, the neural ensembles were able to classify correctly the location of a tactile stimulus, among three different sites, on 88.2% of the trials. In contrast, ensembles of VPM neurones from whisker-deprived animals performed significantly worse, classifying correctly the location of the stimulus on 73.5% of the trials, even though the three different stimulus locations chosen were much farther apart (making it easier to discriminate) than in the normal animals. Furthermore, removing, one-by-one, the neurones that contributed the most weight in the population response, revealed a graceful degradation of ensemble performance, demonstrating the distributed nature of normal VPM function. Overall, these results suggest that peripheral deafferentation, by altering both the spatial and temporal aspects of single VPM neuronal properties, can decrease the ability of somatosensory neural ensembles to discriminate the location of a tactile stimulus and provides an unique experimental/neurophysiological model for studying chronic peripheral sensory deprivation that is similar to clinical disorders such as phantom limbs. 3.3.6. Simultaneous coding of tactile stimulus location across multiple primate somatosensory cortical areas One of the most basic functions of the nervous system is to identify the location of sensory stimuli. Information about stimulus location is transduced first by activation of the peripheral receptor array. From there, this information may travel along several parallel pathways in the central nervous system until reaching the neocortex. In the complex somatosensory system of primates, which includes many reciprocally-connected cortical areas, a fundamental question is whether information about tactile stimulus localization is preserved throughout these many cortical areas, or whether such information is conserved within distinct cortical areas that then serve as a general reference for other cortical areas. It has been suggested that the degradation of topographic maps of the body surface, due to increases in receptive field sizes as one proceeds from early to later stages of cortical processing, should result in a corresponding degradation in the ability of higher order areas to localize a tactile stimulus. For example, the secondary somatosensory cortex and the different fields of the posterior parietal cortex have long been considered to be uninvolved in stimulus location coding, because
The space-time continuum in mammalian sensory pathways
141
the large receptive fields of neurones in these regions would seem to preclude the representation of this information (Kaas and Pons, 1988). It is seemingly easier to locate a stimulus using a population of neurones with very small RFs, and such a population would be, functionally, a labelled line system. Thus, a fundamental question is whether information about tactile stimulus location is lost in cortical areas where neurones have large receptive fields. It is possible that information about stimulus location could be transformed from a spatial code at the periphery, easily recognized at a single neurone level, to a population code, best recognized in the simultaneous activities of large populations of cortical neurones. Nicolelis et al. (1998) tested these two potential coding schemes by chronically and simultaneously recording populations of neurones from the hand, arm and face regions of three different somatosensory cortical areas in owl monkeys (Aotus trivirgatus). Quantitative analyses of single neurone RF sizes in the infragranular layers of primary somatosensory (area 3b), secondary somatosensory (SII), and parietal area 2, revealed that most neurones had broadly-tuned RFs. As such, single neurones were not reliable indicators of stimulus location on a single-trial basis. As reported previously for the rat somatosensory system (Ghazanfar et al., 1995; Ghazanfar and Nicolelis, 1997; Nicolelis and Chapin, 1994), punctate stimuli elicited unique spatiotemporal patterns of population responses from each of these primate somatosensory cortical areas. In order to test whether these distributed population responses could identify the location of a stimulus on a single-trial basis, neural ensemble analyses were carried out using multivariate statistical pattern recognition approaches (Nicolelis et al., 1998 in press). These analyses, which included an artificial neural network based on learning vector quantization (LVQ), revealed that small ensembles of neurones (~20–40 neurones) from area 3b, SII, or area 2 could encode the location of a stimulus reliably, on a single trial basis (Figure 4a). For example, an ensemble of 24 neurones from area 3b could correctly identify the stimulus location among three different skin sites on 72.5% of the trials (chance is 33.3%). Similarly sized neural ensembles in SII and area 2 also demonstrated above-chance discrimination on a single trial basis. Decreasing ensemble sizes, by removing sequentially the “best predictor” neurone, revealed a graceful degradation of ensemble performance in all three cortical areas, a hallmark of distributed coding (Figure 4a). In order to investigate which parameters of the neural ensemble response were important for localizing the tactile stimulus, two data manipulations were carried out: 1) decreasing the temporal resolution of the response by increasing bin sizes; and 2) shuffling the trials, to eliminate the covariance structure between elements of the population. Increasing the bin size, from 3 milliseconds to 45 milliseconds, significantly reduced the performance of SII ensembles to discriminate the location of a stimulus, suggesting that the temporal modulation of ensemble firing plays an important role for encoding stimulus location in this cortical area (Figure 4b). Surprisingly, ensembles in areas 3b and 2 did not show a deficit in performance, demonstrating that simple firing rate changes were sufficient (Figure 4b). This finding, which is reminiscent of the results in the visual system (Richmond and Optican, 1990; Victor and Purpura, 1996), suggests that different somatonsensory cortical areas may employ different encoding strategies to represent the same type of tactile information (i.e. stimulus location).
Time and the brain
142
Finally, independent shuffling of trial order for each single neurone had little effect on the ability of neural ensembles in any of the three areas to correctly classify stimulus location. If anything, ensemble performance sometimes improved following trial shuffling. These results indicate that, for the set of analytical tools used in this study, neither intra-trial precise timing of ensemble spike patterns, neural synchronization, nor linear covariance across the population of neurones were necessary for effective single trial discrimination of tactile stimulus location. Instead, the location of a stimulus could be predicted by the unique spatiotemporal patterns of sensory responses across the neural population. In consideration of these results, it is important to realize that the use of multi-site neural ensemble recordings from areas 3b, SII, and 2 revealed that neural responses occur in these regions concurrently. As such, it does not appear likely that information is passed only sequentially from one cortical area to another; rather, the different encoding strategies used by each area may occur almost simultaneously. Indeed, analysis of the
The space-time continuum in mammalian sensory pathways
143
Figure 4 . a. For two cortical areas analyzed (SII and 3b) the single trial discrimination capability varied as a function of the ensemble size. This figure illustrates the effects of sequential removal of the best predictor neurons from the population on single trail discrimination of the stimulus location. These curves also revealed a graceful decay in discrimination performance as a function of ensemble size. b. Effect of bin size on the discrimination capability of two different cortical areas. While a significant reduction in discrimination capability was observed when bin clumping (i.e. increasing the bin describing neuronal firing from 3 to 45 ms) was applied to SII neural ensembles, no effect was observed in area 3b neural ensembles. These results suggest that the temporal structure of population responses could play a more fundamental role in area SII than in the other two somatosensory regions for encoding stimulus location.
time epoch in which the maximum amount of information is contained revealed that there was a 15 millisecond overlap between area 2 and areas 3b and SII sensory responses.
Time and the brain
144
4. CONCLUSION The findings reviewed in this chapter underscore the fundamental relevance of time in sensory information processing by the auditory, visual, and somatosensory systems of mammals. Altogether, these data reveal that the existence of spatiotemporal coupling in both single neurone responses and neural ensemble firing is pervasive throughout these sensory pathways. Furthermore, dynamic RFs and the spatiotemporal coding of stimulus attributes, which have been now identified in all three sensory systems, seem to emerge and to be constrained by both the need to interpret time-varying stimuli and the fact that active exploratory behaviours are used by animals to scan their surrounding environment. Regarding the mechanisms used to generate the spatiotemporal properties of single neural RFs and neural ensemble coding, the evidence suggests that the asynchronous convergence of multiple afferent pathways gives rise to the dynamic neural properties observed in all three sensory systems. In this context, descending corticofugal projections play as crucial a role in sensory information processing as the feedforward pathways, since they provide the anatomical substrate for propagating top-down signals from the neocortex to subcortical relays and, consequently, act upon the ascending flow of sensory information. It is also likely that the recurrent connectivity of sensory systems contributes to the finding that slight alterations at either peripheral or central levels can trigger system-wide alterations in the functional organization of a sensory pathway. Therefore, embedded in this model is the notion that adaptive neural phenomena (i.e. immediate and long-term sensory plasticity, learning, etc.), occurring at different time scales (i.e. seconds, minutes, hours, days, etc.) both influence and are constrained by the dynamics of sensory information processing at a millisecond scale. Thus, in addition to contributing to an extra dimension for information coding, the dynamic properties of neural ensembles provide the substrates for triggering immediate system-wide reorganization that follow changes in sensory experience over the course of the organism’s life (Nicolelis, 1997). Fundamentally, mammalian sensory systems can no longer be considered as a collection of passive filters or feature extractors that are specialized in detecting the presence of particular stimulus attributes (i.e. orientation, colour, tones, etc.). The experimental evidence reviewed in this chapter indicates that perception emerges as the outcome of highly distributed interactions between populations of broadly tuned neurones dispersed throughout the multiple and reciprocally-connected subcortical and cortical areas that define a sensory system. Put in another way, the central nervous system does not operate as a static decoder, which sits idle until inputs from the environment are conveyed through it, by feedforward neural pathways that connect the sensory periphery to the cerebral cortex. Instead, a more parsimonious hypothesis is that during adulthood, the spatiotemporal organization of sensory systems reflect their active attempt, during exploratory behaviours, to compare the meaning encoded into a time-varying stimulus with the accumulated representation of the sensory experiences acquired during a lifetime.
The space-time continuum in mammalian sensory pathways
145
REFERENCES Abeles, M., and Goldstein, M.H. (1972) Responses of single units in the primary auditory cortex of the cat to tones and to tone pairs. Brain Research , 42, 337–352. Adrian, E.D. (1928) The basis of sensation . Norton, New York. Ahissar, M., Ahissar, E., Bergman, H., and Vaadia, E. (1992) Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. Journal of Neurophysiology , 67, 203–215. Alonso, J.-M., Usrey, W.M., and Reid, R.C. (1996) Precisely correlated firing in cells of the lateral geniculate nucleus. Nature, London , 383, 815–819. Armstrong-James, M., and Fox, K. (1987) Spatiotemporal convergence and divergence in the rat SI “barrel” cortex. Journal of Comparative Neurology , 263, 265–281. Armstrong-James, M., Fox, K., and Das-Gupta, A. (1992) Flow of excitation within rat barrel cortex on striking a single vibrissa. Journal of Neurophysiology , 68, 1345– 1358. Atick, J.J., and Redlich, A.N. (1990) Towards a theory of early visual processing. Neural Computation , 2, 308–320. Barlow, H.B. (1972) Single units and sensation: a neuron doctrine for perceptual psychology? Perception , 1, 371–394. Beck, P.D., Fitzpatrick, L., and Nicolelis, M.A.L. (1998) Corticothalamic projections of somatosensory and motor cortical areas in the rat. Society for Neuroscience Abstracts , p. 131. Bushnell, E.W., and Boudreau, J.P. (1991) The development of haptic perception during infancy. In: M.A.Heller and W.Schiff (eds.) The psychology of touch . Erlbaum, Hillsdale, NJ, pp. 139–161. Carvell, G.E., and Simons, D.J. (1990) Biometric analyses of vibrissal tactile discrimination in the rat. Journal of Neuroscience , 10, 2638–2648. Chapin, J.K. (1986) Laminar differences in sizes, shapes, and response profiles of cutaneous receptive fields in the rat SI cortex. Experimental Brain Research , 62, 549– 559. Dan, Y., Atick, J.J., and Reid, R.C. (1996) Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. Journal of Neuroscience , 16, 3351–3362. Darian-Smith, I. (1984) The sense of touch: performance and peripheral neural processes. In: I. Darian-Smith (ed.) Handbook of physiology. I. the nervous system. Sensory processes. Vol. 3 , American Physiological Society, pp. 739–787. Dawis, S., Shapley, R., Kaplan, E., and Tranchina, D. (1984) The receptive field organization of X-cells in the cat: spatiotemporal coupling and asymmetry. Vision Research , 24, 549–564. DeAngelis, G.C., Ohzawa, I., and Freeman, R.D. (1993) Spatiotemporal organization of simple-cell receptive fields in the cat’s striate cortex. I. General characteristics and postnatal development. Journal of Neurophysiology , 69, 1118–1135. DeAngelis, G.C., Ohzawa, I., and Freeman, R.D. (1995) Receptive field dynamics in the central visual pathways. Trends in Neurosciences , 18, 451–458. deCharms, R.C., Blake, D.T., and Merzenich, M.M. (1998) Optimizing sound features for cortical neurons. Science, New York , 280, 1439–1443. deCharms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds
Time and the brain
146
by the coordination of action-potential timing. Nature, London , 381, 610–613. Desimone, R., Albright, T.D., Gross, C.G., and Bruce, C.J. (1984) Stimulus selective properties of inferior temporal neurons in the macaque. Journal of Neuroscience , 4, 2051–2062. Dinse, H.R., Kruger, K., Mallot, H.A., and Best, J. (1991) Temporal structure of cortical information processing: cortical architecture, oscillations, and non-separability of spatio-temporal receptive field organization. In: J.Kruger (ed.) Neuronal Cooperativity Springer-Verlag, Berlin, pp. 68–104. Fabri, M., and Burton, H. (1991) Ipsilateral cortical connections of primary somatic sensory cortex in rats. Journal of Comparative Neurology , 311, 405–424. Faggin, B.M., Nguyen, K.T., and Nicolelis, M.A. (1997) Immediate and simultaneous sensory reorganization at cortical and subcortical levels of the somatosensory system. Proceedings of the National Academy of Sciences, U.S.A. , 94, 9428–33. Florence, S.L., and Kaas, J.H. (1995) Large-scale reorganization at multiple levels of the somato-sensory pathway follows therapeutic amputation of the hand in monkeys. Journal of Neuroscience , 15, 8083–8095. Gerstein, G.L., Perkel, D.H., and Dayhoff, J.E. (1985) Cooperative firing activity in simultaneously recorded populations of neurons: detection and measurement. Journal of Neuroscience , 5, 881–9. Ghazanfar, A.A., and Nicolelis, M.A.L. (1999) Spatiotemporal properties of layer V neurons of the rat primary somatosensory cortex. Cerebral Cortex , 9, 348–361. Ghazanfar, A.A., Krupa, D.J., and Nicolelis, M.A.L. (1997) Tactile processing by thalamic neural ensembles: the role of cortical feedback. Society for Neuroscience Abstracts . Ghazanfar, A.A., and Nicolelis, M.A.L. (1997) Nonlinear processing of tactile information in the thalamocortical loop. Journal of Neurophysiology , 78, 506–510. Gochin, P.M., Colombo, M., Dorfman, G.A., Gerstein, G.L., and Gross, C.G. (1994) Neural ensemble coding in inferior temporal cortex. Journal of Neurophysiology , 71 , 2325–2337. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of the awake monkey. Experimental Brain Research , 84, 505–516. Golomb, D., Kleinfeld, D., Reid, R.C., Shapley, R.M., and Shraiman, B.I. (1994) On temporal codes and the spatiotemporal response of neurons in the lateral geniculate nucleus. Journal of Neurophysiology , 72 , 2990–3003. Gottlieb, J.P., and Keller, A. (1997) Intrinsic circuitry and physiological properties of pyramidal neurons in rat barrel cortex. Experimental Brain Research , 115 , 47–60. Gray, C.M., Konig, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, London , 338 , 334–337. Gray, C.M., and Singer, W. (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A. , 86 , 1698–1702. Hartline, H.K. (1950) The receptive fields of optic nerve fibres. American Journal of Physiology , 130 , 690–699. Mauser, M.D., Agnetta, B., and Perez, C. (1998) Orienting asymmetries in rhesus monkeys: effect of time-domain changes on acoustic perception. Animal Behaviour , 56, 41–47. Hebb, D.O. (1949) The organization of behavior. A neuropsychological theory . John Wiley and Sons.
The space-time continuum in mammalian sensory pathways
147
Hoeflinger, B.F., Bennett-Clarke, C.A., Chiaia, N.L., Killackey, H.P., and Rhoades, R.W. (1995) Patterning of local intracortical projections within the vibrissae representation of rat primary somatosensory cortex. Journal of Comparative Neurology , 354, 551– 563. Hubel, D.H., and Wiesel, T.N. (1962) Receptive fields, binocular interactions, functional architecture in the cat’s visual cortex. Journal of Physiology (London) , 195 , 215–243. Jenkins, W.M., and Masterton, R.B. (1982) Sound localization: effects of unilateral lesions in central auditory system. Journal of Neurophysiology , 47 , 987–1016. Kaas, J.H. (1991) Plasticity of sensory and motor maps in adult mammals. Annual Review of Neuroscience , 14 , 137–168. Kaas, J.H., and Pons, T.P. (1988) The somatosensory system of primates. In: H.D. Steklis and J. Erwin (eds) Comparative Primate Biology, Vol. 4 , Alan R.Liss, New York, pp. 421–468. Knudsen, E.I., and Konishi, M. (1979) Mechanisms of sound location in the barn owl (Tyto alba). Journal of Comparative Physiology , 133 , 13–21. Koralek, K.-A., Olavarria, J., and Killackey, H.P. (1990) Areal and laminar organization of corticocortical projections in the rat somatosensory cortex. Journal of Comparative Neurology , 299 , 133–150. Kruger, J., and Aiple, F. (1988) Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. Journal of Neurophysiology , 60, 798–828. Krupa, D.J., Ghazanfar, A.A., and Nicolelis, M.A.L. (1999) Immediate thalamic sensory plasticity depends on corticothalamic feedback. Proc. Natl. Acad. Sci . USA, 96, 8200– 8205. Lashley, K.S. (1950) In search of the engram. Symposium of the Society of Experimental Biology , 4, 454–482. Maldonado, P.E., and Gerstein, G.L. (1996) Neuronal assembly dynamics in the rat auditory cortex during reorganization induced by intracortical microstimulation. Experimental Brain Research , 112 , 431–41. Margoliash, D., and Fortune, E.S. (1992) Temporal and harmonic combination-sensitive neurons in the zebra finch’s HVc. Journal of Neuroscience , 12 , 4309–4326. Marrocco, R.T. (1976) Sustained and transient cells in monkey lateral geniculate nucleus: conduction velocities and response properties. Journal of Neurophysiology , 39 , 340– 353. McClurkin, J.W., Gawne, T.J., Optican, L.M., and Richmond, B.J. (1991 a) Lateral geniculate neurons in behaving primates. II. Encoding of visual information in the temporal shape of the response. Journal of Neurophysiology , 66 , 794–808. McClurkin, J.W., Gawne, T.J., Richmond, B.J., Optican, L.M., and Robinson, D.L. (1991b) Lateral geniculate neurons in behaving primates. I. Responses to twodimensional stimuli. Journal of Neurophysiology , 66 , 777–93. McClurkin, J.W., and Optican, L.M. (1996) Primate striate and prestriate cortical neurons during discrimination. I. simultaneous temporal encoding of information about color and pattern. Journal of Neurophysiology , 75, 481–495. Middlebrooks, J.C., Clock, A.E., Xu, L., and Green, D.M. (1994) A panoramic code for sound location by cortical neurons. Science, New York , 264 , 842–844. Middlebrooks, J.C., Xu, L., Eddins, A.C., and Green, D.M. (1998) Codes for soundsource location in nontonotopic auditory cortex. Journal of Neurophysiology , 80 , 863–881. Mignard, M., and Malpeli, J.G. (1991) Paths of information flow through visual cortex.
Time and the brain
148
Science, New York , 251, 1249–1251. Nelson, J.I., Salin, P.A., Munk, M.H., Arzi, M., and Bullier, J. (1992) Spatial and temporal coherence in cortico-cortical connections: a cross-correlation study in areas 17 and 18 in the cat. Visual Neuroscience , 9, 21–37. Nicolelis, M.A.L. (1997) Dynamic and distributed somatosensory representations as the substrate for cortical and subcortical plasticity. Seminars in Neuroscience , 9, 24–33. Nicolelis, M.A.L. (1998, in press) Methods in Neural Ensemble Recordings . CRC Press, New York. Nicolelis, M.A., Chapin, J.K., and Lin, R.C. (1991) Thalamic plasticity induced by early whisker removal in rats. Brain Research , 561, 344–9. Nicolelis, M.A., Lin, R.C., Woodward, D.J., and Chapin, J.K. (1993a) Dynamic and distributed properties of many-neuron ensembles in the ventral posterior medial thalamus of awake rats. Proceedings of the National Academy of Sciences, U.S.A. , 90, 2212–6. Nicolelis, M.A.L., Lin, R.C.S., Woodward, D.J., and Chapin, J.K. (1993b) Induction of immediate spatio-temporal changes in thalamic networks by peripheral block of ascending cutaneous information. Nature, London , 361, 533–536. Nicolelis, M.A.L., and Chapin, J.K. (1994) Spatiotemporal structure of somatosensory responses of many-neuron ensembles in the rat ventral posterior medial nucleus of the thalamus. Journal of Neuroscience , 14, 3511–3532. Nicolelis, M.A., Baccala, L.A., Lin, R.C., and Chapin, J.K. (1995) Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system. Science, New York , 268, 1353–8. Nicolelis, M.A.L., Oliveira, L.M.O., Lin, R.C.S., and Chapin, J.K. (1996) Active tactile exploration influences the functional maturation of the somatosensory system. Journal of Neurophysiology , 75, 2192–2196. Nicolelis, M.A.L., Ghazanfar, A.A., Faggin, B.M., Votaw, S., and Oliveira, L.M.O. (1997a) Reconstructing the engram: simultaneous, multisite, many single neuron recordings. Neuron , 18, 529–537. Nicolelis, M.A.L., Lin, R.C.S., and Chapin, J.K. (1997b) Neonatal whisker removal reduces the discrimination of tactile stimuli by thalamic ensembles in adult rats. Journal of Neurophysiology , 78, 1691–1706. Nicolelis, M.A.L., Ghazanfar, A.A., Stambaugh, C.R., Oliveira, L.M.O., Laubach, M., Chapin, J.K., Nelson, R.J., and Kaas, J.H. (1998) Simultaneous encoding of tactile information by three primate cortical areas. Nature, Neuroscience , 1 (in press). Nicolelis, M.A.L., Stambaugh, C.R., Brisben, A., and Laubach, M. (1998 in press) Methods for simultaneous multisite neural ensemble recordings in behaving primates. In: M.A.L.Nicolelis (ed.) Methods in Neural Ensemble Recordings . CRC Press, New York. Nowak, L.G., Munk, M.H., Nelson, J.I., James, A.C., and Bullier, J. (1995) Structural basis of cortical synchronization. I. Three types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–400. Oliveira, L.M.O., Shuler, M.G., Casal, S.G., and Nicolelis, M.A.L. (1997) Effects of neonatal unilateral facial nerve cuts on tactile responses in adult rat SI cortex. Society for Neuroscienc Abstracts. , p. 1798. Radons, G., Becker, J.D., Dulfer, B., and Kruger, J. (1994) Analysis, classification, and coding of multi-electrode spike trains with hidden Markov models. Biological Cybernetics , 71, 359–373. Rauschecker, J.P., Tian, B., and Hauser, M. (1995) Processing of complex sounds in the
The space-time continuum in mammalian sensory pathways
149
macaque nonprimary auditory cortex. Science, New York , 268, 111–114. Reid, R.C., and Alonso, J.-M. (1995) Specificity of monosynaptic connections from thalamus to visual cortex. Nature, London , 378, 281–284. Richmond, B.J., and Optican, L.M. (1990) Temporal encoding of two-dimensional patterns by single units in primate primary visual cortex. II. Information transmission. Journal of Neurophysiology , 64, 370–80. Richmond, B.J., Optican, L.M., and Spitzer, H. (1990) Temporal encoding of twodimensional patterns by single units in primate primary visual cortex. I. Stimulusresponse relations. Journal of Neurophysiology , 64, 351–69. Ringach, D.L., Hawken, M.J., and Shapley, R.M. (1997) Dynamics of orientation tuning in macaque primary visual cortex. Nature, London , 387, 281–284. Saul, A.B., and Humphrey, A.L. (1992) Evidence of input from lagged cells in the lateral geniculate nucleus to simple cells in cortical area 17 of the cat. Journal of Neurophysiology , 68, 1190–1207. Shadlen, M.N., and Newsome, W.T. (1994) Noise, neural codes and cortical organization. Current Opinion in Neurobiology , 4, 569–579. Sillito, A.M., Jones, H.E., Gerstein, G.L., and West, D.C. (1994) Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London , 369, 479–482. Simons, D.J. (1978) Response properties of vibrissa units in rat SI somatosensory neocortex. Journal of Neurophysiology , 41, 798–820. Singer, W., and Gray, C.M. (1995) Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience , 18, 555–586. Stevens, J.K., and Gerstein, G.L. (1976) Spatiotemporal organization of cat lateral geniculate receptive fields. Journal of Neurophysiology , 39 , 213–238. Suga, N., O’Neill, W.E., Kujirai, K., and Manabe, T. (1983) Specificity of combinationsensitive neurons for processing biosonar signals in auditory cortex of the mustached bat. Journal of Neurophysiology , 49 , 1573–1626. Sutter, M.L., and Schreiner, C.E. (1991) Physiology and topography of neurons with multipeaked tuning curves in cat primary auditory cortex. Journal of Neurophysiology , 65 , 1207–1226. Tramo, M.J., Bellew, B.F., and Hauser, M.D. (1996) Discharge patterns of auditory cortical neurons evoked by species-specific vocalizations and synthetic complex signals in alert Macaca mulatta. Society for Neuroscience Abstracts , p. 1623. Victor, J.D., and Purpura, K.P. (1996) Nature and precision of temporal coding in visual cortex: a metric-space analysis. Journal of Neurophysiology , 76 , 1310–1326. Wang, X., Merzenich, M.M., Beitel, R., and Schreiner, C.E. (1995) Representation of a species-specific vocalization in the primary auditory cortex of the common marmoset: temporal and spectral characteristics. Journal of Neurophysiology , 74, 2685–2706. Welker, W.I. (1964) Analysis of sniffing of the white albino rat. Behaviour , 12, 223– 244. Winter, P., and Funkenstein, H.H. (1973) The effect of species-specific vocalizations on the discharge of auditory cortical cells in the awake squirrel monkey (Saimiri sciureus) Experimental Brain Research , 18, 489–504. Wollberg, Z., and Newman, J.D. (1972) Auditory cortex of squirrel monkeys: response properties of single cells to species-specific vocalizations. Science, New York , 175, 212–214. Wortis, S.B., and Pfeiffer, A.Z. (1948) Unilateral auditory-spatial agnosia. Journal of Nervous and Mental Disease , 108, 181–186.
4 Information Flow along Neocortical Axons Harvey A.Swadlow Department of Psychology (U-20), The University of Connecticut, Storrs, Connecticut 06269 Tel: (860) 486–2252; FAX: (860) 486–2760; e-mail: [email protected]
Neocortical axonal systems convey the results of cortical processing to cortical target structures. This chapter is concerned with the timing of this information flow and with delays introduced by different types of cortical axon. Neocortical white matter is composed of axons that range from <0.1 µm to >10 µm in diameter. Such axons generate a similarly wide range of conduction velocities, and conduction times between soma and axon terminals. Work from the author’s laboratory is reviewed in which conduction properties of corticocortical and descending corticofugal efferent systems of rabbit neocortex were examined in each of four sensory and motor cortical regions. These data are compared with corresponding axonal systems of primates, cats and rodents. Whereas some cortical efferent systems are composed of exclusively fast-conducting axons, others have a large number of very slowly conducting elements, and impulse conduction times along these latter elements may take several tens of milliseconds. Properties of specific thalamocortical axonal systems are also reviewed, and the role of axonal conduction time in the generation of sharply and broadly synchronous cortical activity is explored. KEYWORDS: Axonal conduction; Cerebral cortex; Cortico-cortical axons; Conduction velocity; Conduction delay; Synchrony
1. INTRODUCTION Neocortical axonal systems convey the results of cortical processing to a host of destinations. This chapter is concerned with the timing of this information flow, and with delays introduced by different types of cortical axons. Considerable information is available concerning the laminar and regional origins of neocortical efferent pathways and the locus of terminal synapses. Less attention has been paid to the axon lying between soma and synaptic terminals. General surveys of cortical white matter (Bishop and Smith, 1964; Waxman and Swadlow, 1977) have shown that neocortical axons exhibit a wide spectrum of diameters. Unmyelinated axons range from <0.1–0.6 µm, and often have a median diameter of ~0.2 µm (e.g. Waxman and Swadlow, 1976b; Swadlow et al., 1980). Most such axons cannot be visualized without the aid of electron
Information flow along neocortical axons
151
microscopy. Myelinated axons may be as fine as 0.2 µm in diameter (e.g. Harding and Towe, 1985; Waxman and Bennett, 1972). The diameter of the largest myelinated axons found in a given cortical system is species dependent (see below), and may be >10 µm (e.g. Biedenbach et al., 1986). Thus, neocortical axons span a range of >1:10,000 in cross-sectional area, and a range in expected conduction velocities of >1:100 (Waxman and Bennett, 1972, see below). This chapter reviews work from the author’s laboratory in which conduction properties of corticocortical and descending corticofugal efferent systems of rabbit neocortex were examined in each of four sensory and motor cortical regions. These data are compared with results from corresponding axonal systems of primates, cats, and rodents. This is not intended to be an all-inclusive review of neocortical axonal systems, and many axonal systems are not included. The cortical regions examined in rabbit were primary visual cortex (V1), primary and secondary somatosensory cortices (S1, S2) and motor cortex (Swadlow, 1988, 1990, 1991, 1994). Efferent systems studied in each of these regions include: (1) the contralateral callosal projection (CC axons); (2) ipsilateral corticocortical projections (C-IC axons) linking primary and secondary sensory cortical areas, or motor and sensory cortical areas; (3) descending corticofugal projections of layer 5 (CF-5 axons); and (4) descending corticofugal projections of layer 6 (CF-6 axons). In addition, sharply synchronous (±1 ms), non-oscillatory activity between putative GABAergic neocortical interneurones is described (Swadlow et al., 1998), and the relationship of such activity to properties of thalamocortical axons is explored.
2. METHODOLOGICAL CONSIDERATIONS Information about axonal conduction delays can be derived from several sources, and each has characteristic limitations. 2.1 Conduction Delays Inferred from Fibre-diameter Spectra The theoretical basis for the relationship between axonal diameter and conduction velocity has been well-explored, and conduction time can be inferred from a knowledge of axon diameter and conduction distance. The conduction velocity of myelinated axons is expected to be linearly related to axon diameter (Rushton, 1951; Goldman and Albus, 1968) and that of non-myelinated axons should be proportional to the square root of diameter (Bennett and Waxman, 1972; Goldman and Albus, 1968; Matsumoto and Tasaki, 1977; Rushton, 1951). These relations assume that axons are “dimensionally similar”, and are dependent upon on factors such as specific membrane properties, internodal distances and myelin thickness. The scaling factor for myelinated axons has been derived largely from peripheral nerve, and considerable variability among axonal systems has been observed (cf., Boyd and Kalu, 1979). Assuming a scaling factor of 5.5 (5.5 m/s for a one micron fibre, including the myelin) and a minimal diameter of 0.2 µm (Waxman and Bennett, 1972),
Time and the brain
152
the smallest myelinated central axons should conduct at velocities of 1.1 m/s, and a 10 µm axon should conduct at about 55 m/s. Central unmyelinated axons are generally from 0.08–0.6 µm in diameter (e.g. Swadlow et al., 1980; Waxman and Swadlow, 1976b). In the periphery they are considerably larger, and axons of 1.1 µm in diameter have conduction velocities of ~2.1 m/s (e.g. Rushton, 1951). However, no systematic studies have examined the scaling factor of central unmyelinated axons, which often conduct at velocities as low as 0.3 m/s (e.g. Bester et al., 1995; Kaba et al, 1986; Manis, 1989; Swadlow, 1994). Assuming that this velocity value represents the finest unmyeli-nated axons, the largest unmyelinated central axons should conduct at a velocity of ~0.8 m/s (given the square root relationship described above). Assuming that axon diameter-conduction velocity relationships are known, two practical considerations seriously limit inferences of conduction time based on axonal diameter and conduction distance: (1) Only a few cortical efferent systems (e.g. the corpus callosum, pyramidal tract) are clearly packaged as fibre bundles accessible to the electron microscope. The axons of most cortical efferent systems mingle in both grey and white matter, and specific systems cannot be readily disentangled (however, see Woodward and Coull, 1984). In classical studies of axonal degeneration following cortical lesions, efferent axons seen coursing towards their targets were often characterized as “course” or “fine” (e.g. Guillery, 1966, 1967). However, such methods, using light microscopy, are generally biased towards the detection of myelinated axons because most unmyelinated axons fall below the level of resolution of the light microscope. The same limitation applies to most studies of cortical axons using retrograde or orthograde tracers (e.g. Boyapati and Henry, 1984; Rockland, 1992). (2) Electron microscopic procedures generally result in each axon being measured at a single point along it’s trajectory. Since axons may vary considerably in diameter and myelination along their length, inferences regarding conduction delays are thereby limited. 2.2. Conduction Delays Inferred from Antidromic Latencies Electrical stimulation of the axonal pathway will elicit antidromic responses in the soma, which can be differentiated from synaptic responses using collision and other procedures (e.g. Bishop et al, 1962; Fuller and Schlag, 1976). A major advantage of this procedure, when compared with inferring conduction time from single measurements of axon diameter, is that the antidromic latency reflects the conduction time along the entire length of axon between stimulation site and soma. However, this method is subject to numerous sampling biases, most of which converge to select against the detection of neurones with slowly conducting axons (see Swadlow, 1998, for a recent review). Factors that contribute to this bias include: (1) Neurones with fine-diameter axons often have small somata which are more difficult to sample than are neurones with larger somata (Humphrey and Weller, 1988; Stone, 1973; Towe and Harding, 1970).
Information flow along neocortical axons
153
(2) Cortical neurones with very fine diameter axons often show no spontaneous impulse activity, and lack sensory receptive fields (Swadlow, 1988, 1990, 1991, 1994). Such neurones are difficult to detect, and are often passed or killed by the microelectrode before they can be studied. (3) Fine diameter axons have higher thresholds to electrical stimulation than do larger diameter axons (Nowak and Bullier, 1996; Ranck, 1975). Thus, whereas a stimulating electrode in close proximity (e.g. <0.5 mm) to a small group of fibres may activate both large and small diameter axons, a somewhat more distant stimulating electrode will selectively activate the larger diameter axons. Increasing stimulus intensity will activate larger axons yet further from the stimulation site, but few unmyelinated axons will be activated at distances of >0.5 mm, even when stimulus intensity is very high (e.g. Swadlow, 1998 [Figure 3]). This results in a strong bias against fine-diameter axons unless stimulating electrodes are in very close proximity to the axons under study. This bias will be heightened when it is deemed necessary to keep stimulus intensity to very low levels in order to limit stimulus spread to a small cortical region. (4) The conduction time along fine preterminal branches may easily be excluded from the measured antidromic conduction time. Although great care may be taken in placing stimulating electrodes near the expected site of axon termination, the impulse may be first initiated some distance away, in a myelinated segment of the axon (where threshold is much lower). Because unmyelinated axons may conduct at velocities of <0.3 m/s (below), one or two millimeters of unmyelinated terminals can add several milliseconds to axonal conduction. (5) Fine-diameter axons often exhibit significant variations in axonal conduction velocity, that result from prior impulse activity (e.g. Ferreyra Moyano and Molina, 1980; Segal et al., 1983; review by Swadlow et al., 1980; see below). Antidromic latency may, therefore, vary considerably from trial-to-trial (in some cases by >15%), depending on the timing of stimulus pulses to antecedent “spontaneous” impulse activity (Swadlow et al., 1978a, b). Although the antidromic nature of the response in such cells can be verified by collision (e.g. Bishop et al., 1962; Fuller and Schlag, 1976) and other procedures (Swadlow et al., 1978b), many experimenters employ a “constant latency” (often defined as <0.1 or 0.2 ms variability) as a defining requirement for antidromic identification; cells showing greater latency variability are thereby rejected from analysis. This creates a strong bias towards the fast-conducting axons, which generally do have a relatively constant antidromic latency. 2.3. Conduction Delays Inferred from Synaptic Latencies Stimulation of one region of cortex will elicit synaptic effects in other regions, and the latencies of these effects depend on conduction times along intervening axons. There are a number of limitations to such procedures: (1) The number of intervening synapses mediating a specific effect is often unknown; (2) In reciprocal pathways, orthodromic synaptic effects cannot be readily differentiated from synaptic effects that are mediated by antidromically-activated recurrent collaterals; and
Time and the brain
154
(3) Synaptic effects mediated by large diameter axons with short conduction times are usually easily observed (because impulses from many such axons arrive nearly simultaneously, and result in considerable temporal summation of EPSPs). However, long latency effects are often temporally dispersed and below threshold for generating spikes or evoked potentials. These problems place limits on the relevance of measured synaptic latencies to understanding the normal conduction delays occurring between cortical areas (cf., Swadlow et al., 1979). However, such procedures may be very useful in determining the fastest possible conduction times between cortical regions.
3. RESULTS FROM THE RABBIT Extracellular recordings were obtained from antidromically identified efferent neurones in rabbit V1, S1, S2 and motor cortex (Swadlow, 1988, 1990, 1991 and 1994). These studies, conducted under virtually identical experimental conditions, were aimed at understanding the common features of parallel processing within diverse neocortical regions, and axonal conduction times were one of the variables under study. Estimated axonal conduction velocities and antidromic latencies of the efferent populations studied are presented in Figures 1 and 2. Also shown are conduction velocities and antidromic latencies of specific thalamocortical axons projecting to V1 and S1 (Swadlow, 1985, 1995). Conduction velocities presented in Figure 1 reflect mean axonal conduction velocities between stimulation sites and recording sites near the soma. 3.1. Callosal Axons Figure 1a shows the estimated axonal conduction velocities for rabbit callosal axons originating in V1, S1, S2 and motor cortex. Note that the axonal stimulation sites in these studies were just contralateral of the midline, so slowing of impulses as they invade preterminal branches in not reflected here. In each of the cortical systems of the rabbit, the mean conduction velocities ranged from 1.47–3.5 m/s, and the slowest callosal axons conducted at velocities of 0.29–0.6 m/s. In each system, a significant population was seen that conducted at velocities of <1 m/s, and a small number of axons conducted at velocities of >5 m/sec. Thus, these axon conduction velocities are indicative of a mixed population of unmyelinated and finediameter myelinated axons. In fact, ultrastructural studies (Waxman and Swadlow, 1976b) found 45% of rabbit splenial axons to be unmyelinated (diameters of 0.08–0.6 µm, mean=0.2 µm) and the remaining to be myelinated with diameters of 0.3–1.85 µm (mean=0.74 µm). Figure 2a shows the antidromic conduction times for the same callosal axons shown in Figure 1a. Note that these conduction times represent somewhat less than 1/2 of the interhemispheric conduction time (this is because the terminal 1/2 of the axonal length is not included, and the excluded portion contains the highly branched and more slowly
Information flow along neocortical axons
155
conducting terminal region). In each of the cortical regions studied in rabbit, a few fast conducting callosal axons result in a small number of very short conduction times (1–2 ms). However, in each system, the conduction times along most fibres are quite long. In S2, for example, conduction times of up to 29.9 ms were seen, implying an interhemispheric conduction time of >60 ms for this neurone. In the above studies of V1, S1, S2 and motor cortex, some rabbit callosal axons projected a collateral to some ipsilateral cortical areas (to V2, S2, S1 and S1 for the studies of V1, S1, S2 and motor cortex, respectively). The proportion of identified callosal axons that projected such branches was smaller in V1 (8%) and S1 (12%) than in S2 or motor cortex (24% in each: Swadlow, 1988, 1990, 1991, 1994). In nearly all cases the site of branching was shown to be temporally near the soma (<1 ms), and conduction velocities were considerably higher in the callosal branch than in the ipsilateral
Time and the brain
156
Figure 1 . Histograms of axonal conduction velocities of various axonal populations. From above downwards: CC, C-IC, CF-5 and CF-6 axons of V1, S1, S2 and motor cortex. In lowest histogram, conduction velocities of thalamocortical axons projecting from LGNd and VB thalamus to V1 and SI, respectively, are shown. These data, and those presented in Figure 2 are derived from Swadlow, 1988–
Information flow along neocortical axons
157
1995. Asterisks shown above uppermost histogram show comparable results from callosal axons of monkey prelunate gyrus (Swadlow et al., 1978a)
Figure 2 . Antidromic latencies of the same populations shown in Figure 1
corticocortical branch (Swadlow, 1991, 1994). A similar ipsilateral projection of callosal
Time and the brain
158
axons has been found in primate (Schwartz and Goldman-Rakic, 1982). 3.2. Ipsilateral Corticocortical Axons Figure 1b presents results from ipsilateral cortical systems studied in each of the cortical regions of the rabbit. In V1 and SI, the projections to V2 and S2, respectively, were studied. In S2 the “feedback” projection to SI was studied, and in motor cortex the projection to SI was studied. In each of these systems, stimulating electrodes were placed, under physiological guidance, in the topographically aligned region of the target structure under study. The diverse nature of these corticocortical projections might lead one to expect considerable differences in their axonal conduction properties. However, this was not found to be the case. In each of the cortical systems, the mean conduction velocities ranged from 0.86–1.4 m/s. In each system, a very significant population was seen to conduct at velocities of <1m/s (lowest velocities of 0.21–0.4 m/s in the four cortical regions), and a small number of axons conducted at velocities of >4 m/sec. Thus, these axon conduction velocities are appropriate for a mixed population of unmyelinated and fine-diameter myelinated axons. Figure 2b shows the antidromic conduction time for the C-IC axons shown in Figure 1b. In each of these systems studied in rabbit, the few fastest conducting axons resulted in a small number of very short conduction times (1–2 ms). However, in each system, the conduction times along most fibres were quite long. In the motor cortical projection to S1, for example, conduction times of up to 32.5 ms are seen. 3.3. Descending Corticofugal Axons of Layer-5 Figure 1c presents results for CF-5 axons studied in the four cortical regions of the rabbit. In V1, results are limited to corticotectal neurones that were antidromically activated via stimulating electrodes placed in topographically aligned regions of the superior colliculus. In SI and in S2, stimulating electrodes were placed within and above the topographically aligned region of ventrobasal thalamus, and in the motor cortex stimulating electrodes were placed within and above ventrolateral thalamus. In the studies of S1, S2 and motor cortex, the thalamic stimulating electrodes were in close proximity to the internal capsule, and high-intensity stimulation was used to activate axons terminating within the thalamus as well as those descending to regions beyond the thalamus. Thus, the CF-5 axons studied in each of these cortical regions undoubtedly represent a heterogeneous mixture of descending corticofugal neurones originating in layer 5. It is, therefore, somewhat surprising that their conduction properties were so homogeneous, and distinct from those of the callosal and ipsilateral corticocortical axons described above, and also distinct from the CF-6 axons described below. Figure 1c shows that mean axonal conduction velocities of each of the CF-5 axonal systems studied ranged from 10.9–12.9 m/s. In none of the systems were axons observed which conducted at velocities of <2 m/s (lowest velocities for the four regions were 3.7–5.7 m/s). Thus, these axon conduction velocities are suggestive of a population of myelinated axons, and no
Information flow along neocortical axons
159
evidence was seen for the presence of any unmyelinated axons. Figure 2C shows the antidromic conduction time for the CF-5 axons. In each of the systems studied, many axons had very short conduction times (1–2 ms), and very few axons with conduction times of >3 ms were seen. 3.4. Descending Corticofugal Axons of Layer-6 Figures 1d and 2d present results for CF-6 axons. In V1, stimulating electrodes were placed within and above the topographically aligned region of the dorsal lateral geniculate nucleus. In SI and in S2, stimulating electrodes were placed within and above the topographically aligned region of ventrobasal thalamus, and in motor cortex stimulating electrodes were placed within and above ventrolateral thalamus. Highintensity stimulation was used to maximize the probability of activating the axons of all of the layer-6 neurones projecting to the thalamus. The mean conduction velocities of CF-6 axons ranged from 1.73–2.4 m/s in the four cortical regions studied. In each region, a large population of CF-6 axons conducted at velocities of <1m/s (lowest velocities of 0.26–0.52 in the four cortical regions), and a small number of axons conducted at velocities of >6 m/sec. Thus, as was the case for the CC and C-IC populations, these axon conduction velocities are consistent with a mixed population of unmyelinated and finediameter myelinated axons. Figure 2d shows the antidromic conduction time for the above CF-6 axons. In each of the regions studied in rabbit, the few fastest conducting axons result in a small number of very short antidromic latencies (1–2 ms). However, in each region, the conduction times along most CF-6 axons were quite long, and antidromic latencies of >25 ms were not uncommon. 3.5. Specific Thalamocortical Axons Figures le and 2e show the high axonal conduction velocities and short conduction times, respectively, of “specific” thalamocortical axons projecting to V1 (from LGNd; Swadlow and Weyand, 1985) and S1 (from VB; Swadlow, 1995). 3.6. Axonal Conduction Velocities and Neuronal Response Properties In each of the cortical regions studied in rabbit, CF-5 axons were more rapidly conducting than CC, C-IC or CF-6 axons. CF-5 neurones also had much higher rates of spontaneous impulse activity, and were much more responsive to peripheral sensory stimulation than were CC, C-IC or CF-6 neurones. Many CC, C-IC and CF-6 neurones lack both spontaneous impulse activity and a demonstrable suprathreshold peripheral receptive field, and in each of the cortical regions studied these characteristics were observed most frequently in neurones with axonal conduction velocities of <1 m/s (Swadlow, 1988, 1990, 1991,1994). It has been shown that many such “silent”, slowly conducting cortical elements do have subthreshold receptive fields (Swadlow and Hicks,
Time and the brain
160
1996, 1997), and the conditions under which such fields reach threshold is currently under study. These observations indicate that cortical neurones with very slowly conducting axons form functionally distinct subclasses within the CC, C-IC and CF-6 populations.
4. RESULTS FROM CATS, MONKEYS AND RODENTS 4.1. Callosal Axons 4.1.1. Anatomy The corpus callosum is one of the most thoroughly studied axonal systems of the neocortex. A large body of information is available concerning the regional and laminar distribution of both the cells of origin and the sites of synaptic contact in the contralateral hemisphere (e.g. Innocenti, 1986 for review). Because callosal axons remain segregated from other white matter systems for considerable distances on either side of the midline they are readily studied using anatomical procedures. Callosal axons have been examined in mouse (Tomasch, 1957), rat (Seggie and Berry, 1972), rabbit (Waxman and Swadlow, 1976), cat (Fleischhauer and Wartenburg, 1967), monkey (LaMantia and Rakic, 1990; Swadlow et al., 1978) and man (Aboitiz et al., 1989, 1992; Tomasch, 1954). In each of these species, a considerable proportion of unmyelinated axons (19–45%) is reported, and the mean diameter of myelinated axons is ~1 µm. Although the mean axonal diameters of all species studies are very similar, a very few large-diameter axons (5–9 µm) axons are present in species with larger brains (i.e., cat, monkey and man). In monkey, regional differences in axonal composition have been observed in the anterior-posterior axis of the corpus callosum (LaMantia and Rakic, 1990). The most rostral sector, for example, contains ~28% unmyelinated axons, and the mean diameter is 0.7 µm (not including the myelin sheath). The most posterior section (splenium), in contrast, was found to contain only 3.5% unmyelinated axons and the mean axonal diameter was 0.9 µm. (However, Swadlow et al., 1980, reported 31% of splenium axons of a single macaque monkey to be unmyelinated). 4.1.2. Conduction velocities and conduction delays Conduction velocities similar to those of rabbit have been reported in sensorimotor cortex of rat (mean axonal velocity=2.9 m/s, range=0.4–8.3 m/s, Catsman-Berrevoets et al., 1980) and visual cortex of the mouse (mean conduction velocities of <1 m/s Simmons and Pearlman, 1983). However, conduction velocities in carnivores and primates may be considerably higher than those seen in rodents and rabbit. Stimulation and recording methods virtually identical to those used in rabbit were applied to callosal axons of the
Information flow along neocortical axons
161
macaque prelunate gyrus (Swadlow et al., 1978a). Figures 1a and 2a (asterisks) show, respectively, the conduction velocities and conduction times (following stimulation near the midline) for these axons. Conduction velocities were considerably higher than those seen in rabbit (median=9 m/s, range=5–27 m/s), and no conduction velocities consistent with unmyelinated fibres were seen. In cat visual cortex, several authors have reported mean or median conduction velocities of 10–15 m/sec (Harvey, 1980; Innocenti, 1980; McCourt et al., 1990; Toyama et al., 1974), although a few neurones with very slowly conducting axons were reported (up to 15 ms; Innocenti, 1980). A higher proportion of very slowly conducting axons has been found in sensorimotor cortex of the cat. Miller (1975) reported antidromic latencies of <2 ms to >30 ms to stimulation near the callosal midline, and estimated axonal conduction velocities to be from <1 m/s to 10 m/s. Zarzecki (1989) reported a mean antidromic latency of 6.7 ms (range=1.9–15.2 ms) between motor cortices of the cat. Assuming a conduction distance of 30 mm, this implies a mean conduction velocity of 4–5 m/s, considerably lower than that reported in visual cortex. The question of whether interhemispheric conduction times are invariant over a large range of species and brain sizes has been recently addressed by Ringo et al. (1994). These authors argue, both from measurements of callosal axon diameter in a broad range of species and from allometric considerations, that conduction times cannot be held constant across this axonal system by compensatory variations in mean axonal diameter. They point out that to equalize the interhemispheric delay in mouse and man would require human callosal axons to be 15 times the diameter of those of the mouse, but that a diameter difference of only 26% is actually observed. They note, however, that extremely rare “giant” axons are seen in the corpus callosum of large brains. For example, Lamantia and Rakic (1990 [Figure 6]) presented an electron micrograph of one such axon in the monkey corpus callosum, with a diameter of ~9 µm. This axon was believed to originate in somatosensory cortex. Given an interhemispheric conduction distance of about 54 mm in this species (Swadlow et al., 1978a), the conduction time along this axon would be ~1 ms (Waxman and Bennett, 1972). Thus, although the mean axonal conduction time of the callosal system of different species is not scale-invariant, the high-velocity end of the frequency distribution may well be. 4.2. Ipsilateral Corticocortical Efferent Systems 4.2.1. Anatomy Each region of the neocortex projects to numerous cortical areas within the ipsilateral hemisphere, and hundreds of retrograde and anterograde tracing studies have characterized the laminar distribution and morphology of the cells of origin of these systems, their terminal arborization patterns and their sites of synaptic contact. There is considerably less information available regarding the axons of specific ipsilateral corticocortical systems. Some degeneration and/or labelling studies have characterized the apparent diameter of axons coursing towards a particular target (e.g. Rockland, 1989,
Time and the brain
162
1992, 1997; Zeki, 1978). In a recent review, Rockland (1997) concluded that most feedback as well as feedforward corticocortical axons of monkey V1 were about 1 µm in diameter. The one exception to this were the axons projecting from V1 to area MT/V5, which were commonly 2–3 µm in diameter. Such observations are very useful in characterizing the medium and large-diameter myelinated axons of a particular system. However, most central unmyelinated axons as well as the finest myelinated axons will be excluded from detection when using light microscopy. 4.2.2. Conduction velocities and conduction delays Visual cortical projections. Bullier et al. (1988) used collision and other methods to identify neurones projecting from area 17 to areas 18 and 19. They found both rapidly conducting (antidromic latencies of <2 ms) and slowly conducting elements (antidromic latencies of >8 ms), with median antidromic latencies of 2–4 ms and 6–8 ms for the projections to area 18 and 19, respectively. Assuming conduction distances of 5 mm for area 18, and 10 mm for area 19, the authors concluded that “the conduction velocities in fibres in each pathway and its reciprocal are of the order of 1 m/s.” Thus, the conduction properties of C-IC axons in cat V1 and rabbit V1 (above, Swadlow, 1988) appear quite similar. Considerably higher axon conduction velocities are seen in the projection from V1 of monkey to area MT (all latencies were <2 ms, Movshon and Newsome, 1996), and from the 17/18 border to area 19 of the cat (conduction velocities of 9–21 m/s; Toyama et al., 1974). However, this latter study employed intracellular recordings from C-IC somata, and also required a “fixed” latency (total range of 0.1 ms) for antidromic identification. Together, these methods would create a strong bias towards recording from neurones with rapidly conducting axons (see above). Somatosensory cortical projections. Neurones projecting from SI to S2 have been studied in cat. Whereas Miller (1975) reported both short (<2 ms) and very long (>30 ms) antidromic latencies, Manzoni et al (1979) found all antidromic latencies to be <2 ms. However, the methods employed in this latter study guaranteed that only rapidly conducting elements would be obtained because a “short latency” was a defining requirement for antidromic identification. A reluctance to consider long latencies as being antidromically mediated is not uncommon in studies of cortical projection neurones. In a pioneering study of corticotectal and corticogeniculate neurons of the cat (Gilbert, 1977) a “short and fixed latency…” was required for antidromic identification and no slowly conducting elements were reported. Later studies of the corticogeniculate system did not require a short and constant latency, and concluded that many corticogeniculate axons were very slowly conducting (Ferster and Lindstrom, 1983; Harvey, 1980; Tsumoto and Suda, 1980). The projection from SI to motor cortex has also been studied in the cat (Waters et al., 1982; Zarzecki et al., 1983; Zarzecki and Wiggin, 1982). Although most latencies were short (means=1.6–2.5 ms), some long antidromic latencies were reported in each of these studies. Waters et al. (1982), for example, found 87% of antidromic latencies to lie between 1–2.2 ms, corresponding to conduction velocities of 3–7 m/s. However, the longest antidromic latency was 6.8 ms (corresponding to a conduction velocity of ~1
Information flow along neocortical axons
163
m/s). A larger number of very slowly conducting elements may have been undetected in the above studies of cat S1 neurones, because these authors took great care to ensure that stimulus current did not spread beyond the motor cortex, using stimulus currents that were limited to <40 µA. Such low currents would favour the activation of large diameter axons, which have lower thresholds to electrical stimulation than do fine diameter axons (Nowak and Bullier, 1996; Ranck, 1975; see Figure 3 of Swadlow, 1998). Motor cortical projections. A mixed population of fast and very slowly conducting axons has also been reported in the motor cortical projection to SI in the cat (Deschênes, 1977). Although most such neurones had rapidly conducting axons (antidromic latencies of <2 ms), ~10% had antidromic latencies of 7–16 ms. 4.3. Descending Corticofugal Axons of Layer 5 4.3.1. Anatomy Layer 5 is the origin of virtually all cortical efferent systems that descend beyond the thalamus (Jones, 1984), as well as many that terminate within the thalamus (e.g. Deschênes et al., 1994; Ojima, 1994; Raczkowski and Diamond, 1978). For some of these systems, single axons branch to terminate in multiple destinations (e.g. Baker et al, 1983; Deschênes et al., 1994). Although the axons of many of these systems intermingle with those of other cortical efferent and ascending axons, some stay segregated for considerable distances. The pyramidal tract is a prime example of this, and this pathway has been subject to detailed morphological analysis in a wide range of species. In each, myelinated fibres show a characteristic skewed diameter distribution, with a peak at ~1 µm and a few fibres as large as 5–10 µm (e.g. Beidenbach et al., 1986; Doetsch and Towe, 1981; von Keyserlingk and Schramm, 1984; Harding and Towe, 1985; Leenen et al., 1985). Unmyelinated pyramidal tract fibres are rare in monkey, forming <1% of the population (Ralston et al., 1987), but are common in both cat (12%, Biedenbach et al., 1986) and rat (30–60%; Harding and Towe, 1985; Leenen et al., 1985). Tracing studies using electron microscopy have shown that the unmyelinated pyramidal tract axons of the rat are of cortical origin (Joosten and Gribnau, 1988). 4.3.2. Conduction velocities and conduction delays The high axonal conduction velocities seen in CF-5 axons of the rabbit are consistent with values reported for two classes of CF-5 axon that have been carefully studied in a number of species: (1) Corticotectal axons have been studied in many cortical areas and in several species (e.g. Finlay et al., 1976; Lemmon and Pearlman, 1981; Weyand et al., 1986, 1991). Medium-high conduction velocities (mean or median values of 8–10 m/s) were reported in all species, and no conduction velocities indicative of unmyelinated axons
Time and the brain
164
were reported in any of these studies. (2) Pyramidal tract neurones (PTNs) have been differentiated into fast and slowly conducting groups, with different membrane and other characteristics (e.g. Baranyi et al, 1993; Sakai and Woody, 1988; Takahashi, 1965). However, both “fast” and “slow” PTNs have conduction velocities indicative of myelinated axons (11–18 m/s and >21 m/s for slow and fast, respectively; Takahashi, 1965). In monkey, PTNs display a similar wide range of antidromic latencies, but all conduction velocities are characteristic of myelinated axons (e.g. Evarts, 1965). Similarly, no conduction velocities indicative of unmyelinated axons were seen amongst antidromically activated PTNs of rat (conduction velocities 5–19 m/s: Mediratta and Nicoll, 1983). This latter finding is puzzling in light of the high numbers of unmyelinated axons in the pyramidal tract of this species (above), and their demonstrated cortical origin (Joosten and Gribnau, 1988). 4.4. Descending Corticofugal Axons of Layer 6 4.4.1. Anatomy Throughout the neocortex, layer 6 is the origin of an extensive descending corticothalamic pathway (e.g. Jones, 1984). Early Golgi studies described fine fibre systems within the LGNd of cat (Guillery, 1966; Szentagothai, 1963) and monkey (Guillery, 1967) and suggested that these were the terminals of corticogeniculate neurones. Although most such fibres were “thin”, some “thick” axons were also observed (Guillery, 1966). Fine-diameter corticothalamic preterminal axons have also been reported using both retrograde and orthograde tracing methods (Boyapati and Henry, 1984; Hoogland et al., 1987; Robson, 1984). Recent single-cell labelling studies (Zhang and Deschenes, 1997) traced rat layer-6 corticothalamic axons from cortex to thalamus, and reported axon diameters of 0.5–0.8 µm. However, most unmyelinated fibres would not have been detected in this light-microscopic analysis and nearly 1/2 the axons were too faintly labelled to follow to thalamic targets. 4.4.2. Conduction velocities and conduction delays The wide spectrum of antidromic latencies seen in CF-6 neurones of rabbit is in agreement with similar values obtained for corticogeniculate neurones of the cat V1 (Ferster and Lindstrom, 1983; Tsumoto and Suda, 1980) and corticothalamic neurones of cat SI (Landry and Dykes, 1985). For example, Ferster and Lindstrom reported antidromic latencies of 2.5–45 ms for corticogeniculate neurones of cat, with >25% of this sample having antidromic latencies of >20 ms.
Information flow along neocortical axons
165
4.5. Specific Thalamocortical Axons Virtually all specific thalamic afferents to the neocortex have axonal conduction velocities characteristic of myelinated axons. In the cat, geniculo-cortical neurones may be separated into numerous sub-groups “x”, “y”, “w”, “lagged” and “non-lagged”) with different conduction times and velocities (e.g. Cleland et al., 1976; Humphrey and Weller, 1988; Wilson et al., 1976). Whereas thalamocortical conduction times for most LGNd neurones are <2 ms, a very small number show latencies of 5–10 ms. Thus, assuming a conduction distance of 20 mm (Humphrey and Weller, 1988), even the slowest of these axons conduct at velocities of >2 m/s, a value consistent with very fine diameter myelinated axons (Waxman and Bennett, 1972). Similarly, geniculo-cortical conduction times of monkey (Marrocco, 1976; Schiller and Malpeli, 1978), tree shrew (Sherman et al., 1975 and rat (Fukuda, 1973) are indicative of a myelinated axonal population.
5. CONDUCTION VELOCITIES OF MANY NEOCORTICAL AXONS ARE NOT STABLE Discussions of cortical axonal conduction time often make the implicit assumption that conduction time between soma and axon terminals is constant, at least over a time scale of minutes. As noted above, some experiments have imposed a constant latency (usually ±0.1 or 0.2 ms) as a defining requirement of antidromic invasion. However, many fine diameter axonal systems demonstrate significant variations in both axonal conduction velocity and excitability, that are dependent on prior impulse activity. Thus, a “supernormal” period of increased axonal conduction velocity and excitability (lasting from 4→100 ms) follows a single prior impulse, and this is followed by a “subnormal” period of decreased conduction velocity and excitability. The depth of the subnormal period increases with number of prior impulses, and can last for several minutes (Ferreyra Moyano and Molina, 1980; Raymond et al., 1990; Segal et al., 1983; Swadlow et al., 1978a). Such behaviour is seen in many central axonal systems composed of small diameter myelinated axons and/or unmyelinated axons (see Swadlow et al., 1980 for review). A pronounced supernormal period is seen in most CC, C-IC and CF-6 axons of rabbit V1, S1, S2 and motor cortex (Swadlow, 1985, 1988, 1990, 1994). However, minimal or no supernormal conduction is seen in CF-5 axons of these regions. Supernormal impulse conduction is also seen in visual callosal axons of the monkey (Swadlow et al., 1978a) and in some callosal (Innocenti, 1980) and ipsilateral corticocortical axons of the cat (Zarzecki and Wiggin, 1982). Interestingly, it appears that no critical fibre diameter or axonal conduction velocity determines the presence or absence of supernormal conduction (Swadlow, 1985), and this phenomenon is seen in myelinated as well as unmyelinated cortical axons (Waxman and Swadlow, 1976a). The variations in axonal conduction time that result from supernormal and subnormal effects may, in some cases,
Time and the brain
166
exceed 15% of the baseline conduction time (taken in the absence of prior impulse activity). The effects of such variations in conduction velocity on impulse conduction time will depend on the baseline activity pattern of the neurone in question. If the firing rate is very low (as is often the case in very slowly conducting axons: Swadlow, 1988, 1990, 1991, 1994), a single impulse occurring every few seconds will have a relatively constant conduction time. However, when the firing rate is high and encompasses a wide range of interspike intervals, axonal conduction velocity of different impulses will vary considerably depending on the temporal relationship of each impulse to prior impulse activity. Thus, an interspike interval distribution measured near the axon terminals of a cortical neurone may differ considerably from the distribution as measured at the site of impulse generation near the soma.
6. AXONAL PROPERTIES AND SHARPLY SYNCHRONOUS ACTIVITY 6.1. Thalamocortical Axons and “Sharp” Synchrony (±1 ms) between Neighbouring Cortical Neurones Synchronous activity between cortical neurones occurs over a wide range of time scales with differing putative underlying mechanisms (Kruger and Aiple, 1988; Kirkwood et al., 1982a; Moore et al., 1970; Nowak et al., 1995; Sears and Stagg, 1976; Swadlow et al., 1998; see chapters by Villa, by Nowak and Bullier, by Eckhorn and by Wennekers and Palm in this volume). Sharply synchronous activity at near “0” delays is sometimes observed in cross-correlograms (CCGs) obtained from closely spaced cortical (e.g. Gochin et al., 1991; Kruger and Aiple, 1988) or subcortical (Alonso et al., 1996; Sears and Stagg, 1976) neurones. One example of such sharply synchronous activity was recently demonstrated between all members of a class of putative GABAergic inhibitory interneurones (suspected inhibitory interneurones, SINs) within individual S1 “barrels” of the rabbit (Swadlow et al., 1998). Sharp synchrony (±1 ms) was seen between all of the SINs within a barrel that received monosynaptic input from ventrobasal thalamus. The above “sharp” synchrony was predicted based on the properties of a richly divergent/convergent network proposed to link nearly all of the SINs of an S1 barrel with neurones of the topographically aligned ventrobasal thalamic “barreloid” (Swadlow, 1995; Swadlow et al., 1998). Figure 3a presents a schematic illustration of the proposed network, and figure 3b shows a CCG obtained from two SINs within an SI barrel recorded via independent microelectrodes separated by ~200 µm. Such synchrony was not seen when SINs were paired with neurones of the same barrel column that did not receive monosynaptic thalamic input or when SINs were paired with other SINs located in neighbouring barrel columns. Control procedures showed that the observed sharp synchrony was not dependent upon peripheral stimulation, was not oscillatory, and survived general anaesthesia (see chapter by Nowak and Bullier, 1999, this volume, for interpretation of cross-correlation histograms). The above thalamocortical network linking SINs of an S1 barrel with ventrobasal
Information flow along neocortical axons
167
thalamic afferents of the aligned thalamic barreloid approaches the description of a “complete transmission line” (Griffith, 1963; Abeles, 1991), where each element in one node of a network makes synaptic contact with all of the members of the successive node. Sharp synchrony among postsynaptic elements of such networks is expected because of near-simultaneous EPSPs generated by the common inputs (cf. Moore et al., 1970). For this reason, and based on findings (Mendell and Henneman, 1971) showing that each muscle spindle 1A afferent fibre branches profusely before contacting nearly all homonymous motoneurones, Sears and Stagg (1976) predicted that a “short-term” synchrony should be observable among the recipient motorneurones, and that the duration of this synchrony should be comparable to the rise time of an EPSP. A very brief synchrony was, in fact, found between intercostal motoneurones, and was subject to considerable subsequent analysis (Kirkwood and Sears, 1978; Kirkwood et al., 1982a, b; Sears and Stagg, 1976).
Figure 3 . a. Highly divergent and convergent network proposed to link a population of VB thalamic barreloid neurones with SINs of the topographically aligned SI barrel (from Swadlow, 1995). b. Crosscorrelogram of spike trains of two SINs located within a single barrel. Each SIN was shown to receive monosynaptic input from VB thalamus. Note the sharply synchronous, near-coincident (±1 ms) responses in the CCG. Responses were generated from “spontaneous” activity, in the absence of peridpheral stimulation (from Swadlow et al., 1998).
The presence and degree of sharp synchrony between two post-synaptic neurones of such divergent and convergent networks will depend on the number of inputs shared by the two neurones and on the “contribution” and “efficacy” (cf., Levick et al., 1972) of these inputs (see Nowak and Bullier, this volume, for definitions of “contribution” and “efficacy”). Figure 4a illustrates a simple schematic illustration, where two neurones (“a”
Time and the brain
168
and “b”) are each presynaptic to neurones “x” and “y” and receive most of their input from these neurones. The hypothetical CCG between the spike trains of neurones “x” and “y” is illustrated in Figure 4b. The central peak occurs at near zero delay. This represents the near-simultaneous spikes in “x” and “y” that result from near-simultaneous EPSPs. This CCG will not be influenced by differences between cell “a” and “b” in conduction velocities along the main axonal trunks. So long as impulses from the two branches of each presynaptic axon reach their targets at the same time, a central peak will result. However, the CCG will be strongly influenced by differences in conduction times along the two branches of neurone “a” or “b”. Figure 4c-d illustrates this point. Here, the preterminal branches of neurone “a” that innervates neurones “x” and “y” are rapidly, and slowly conducting ones, respectively. Similarly, the branches of neurone “b” that
Figure 4 . a. Schematic illustration of a fully divergent/convergent network linking two presynaptic neurones (“a” and “b”) with two postsynaptic neurones (“x” and “y”). Each presynaptic neurone makes a potent “contribution” to the two postsynaptic neurones. Terminal branches of neurone “a” have identical conduction times, as do the terminal branches of neurone “b”. b. Schematic of CCG between spike trains of neurones “x” and “y” shown in Figure 4a c. Situation is identical to that seen in A except one of the branches of each of the presynaptic neurones (dashed lines) is slowly conducting, d. Schematic of CCG between spike trains of neurones “x” and “y” shown in Figure 4c.
innervate neurones “x” and “y” are slowly, and rapidly conducting, respectively. Figure
Information flow along neocortical axons
169
4d presents the schematic CCG between neurones “x and “y” that will result. Here, two peaks are seen. The displaced peaks reflect the different arrival times of impulses generated by each of the presynaptic neurones. When additional presynaptic branches are added to the circuit, the temporal dispersion of the central peak resulting from differing preterminal conduction times in the individual presynaptic neurones will dilute any sharply synchronous responses. 1 When sharp synchrony (±1 ms) does result from a highly divergent and convergent presynaptic network, the above considerations suggest that the presynaptic branches are likely to be rapidly conducting and myelinated until very near the region of synaptic contact. This will ensure near-simultaneous arrival of impulses from each presynaptic neurone at their post-synaptic targets. Since branching of thalamocortical afferents may occur at distances of >2 mm from sites of synaptic termination, unmyelinated terminal branches (which can conduct at velocities as low as 0.2–0.3 m/s) would result in a temporal dispersion of EPSPs in the postsynaptic elements and a resultant broadening of any central peak in the CCGs. In fact, thalamocortical afferents do retain their myelin following considerable branching within the cortex (Deschênes and Landry, 1980; Ferster and LeVay, 1978; Freund et al., 1989), as do retinal ganglion cells (e.g. Famiglietti and Peters; 1972) and 1A afferents to spinal motor neurones (Brown and Fyffe, 1978). Divergence of preterminal branches of each of these three axonal systems is believed to mediate a potent sharp synchrony of postsynaptic neurones (Alonso et al., 1996; Sears and Stagg, 1978; Swadlow et al., 1998). If rapidly conducting, myelinated terminal branches are necessary to achieve sharp synchrony between post-synaptic elements, then it is likely that the main axon trunk must also be myelinated. This conclusion follows from both empirical observations (axons generally decrease in diameter distal to branch points and few, if any, instances of increased diameter have been reported), and from theoretical considerations indicating that axons must decrease in diameter distal to branch points if impulses are to faithfully invade terminal arbors (e.g. Khodorov and Timin, 1975; Parnas et al., 1979 Swadlow, el al., 1980; Waxman, 1975 for reviews). 6.2. Corticocortical Axons and Synchrony between Distant Cortical Neurones The “sharp” synchrony described above, is very brief and non-oscillatory (Swadlow et al., 1998). In contrast, synchrony between distant cortical neurones generally has a longer time course and is often oscillatory (e.g. Eckhorn et al., 1988; Engel et al., 1991; Eckhorn, this volume). However, some types of distant synchrony may be mediated by mechanisms similar to those proposed above for sharp synchrony. For example, “T coupling” (Munk et al., 1995; Nowak et al., 1995; Nowak and Bullier, this volume) occurs between visual cortical neurones in opposite hemispheres, and has a time course of a few ms. In the simplest case, such mterhemispheric synchrony between two distant neurones could result if each received divergent inputs from a set of axons originating in only a single hemisphere. The 1. This analysis assumes that more slowly-conducting branches are randomly distributed to
Time and the brain
170
postsynaptic elements. A sharp peak at various delays can, occur if slowly conducting branches are selectively distributed to one of the postsynaptic neurons under study.
origin of these diverging axons cannot be extrastriate cortex because T coupling survives destruction of these regions (Munk et al., 1995; Nowak and Bullier, this volume). However, rich sources of such divergence are provided by the local recurrent collateral branches of callosal efferent neurones (e.g. White and Czeiger, 1991). In this scheme, synchrony between neurones (located in separate hemispheres) could be mediated by a population of callosal projection neurones. The main interhemispheric axon is envisaged to contact one element, while the recurrent collaterals contact the other. When conduction time along the main axon is very rapid (i.e. 1–2 ms), arrival times of impulses in the two hemispheres may be very similar. Synchrony between the spike trains of these distant neurones would result if they shared a significant number of such common inputs (above). According to this notion, slowly conducting recurrent collaterals may be acting as delay lines (cf., Waxman, 1975), synchronized with main axonal trunks and resulting in near-coincident arrival of impulses at post-synaptic targets in separate hemispheres. If conduction along the main axon was somewhat longer than that along the recurrent collaterals, a commensurate delay in the CCG peak would result. However, the corpus callosum is a reciprocal pathway, and if the two recipient neurones received divergent input from axons originating in both hemispheres, a symmetry about time 0 would result. Thus, the time course of synchrony resulting from such a mechanism would depend on the difference between interhemispheric and intrahemispheric conduction times, and on the proportion of inputs to each of the correlated neurones that originated from callosal efferent neurones in each of the two hemispheres. A predictable consequence of this proposal is that each of the neurones that demonstrate “T coupling” (in opposite hemispheres) should be a member of a relatively small subpopulation of visual cortical neurones that respond robustly to electrical stimulation of the corpus callosum (cf., Harvey, 1980; Innocenti, 1980). In addition, differences between the two neurones in latency to midline stimulation should be correlated with differences in the timing of the peak in the CCG (cf., Swadlow et al., 1998 [Figure 7c]).
Information flow along neocortical axons
171
7. CONDUCTION DELAYS AND CORTICAL PROCESSING Neocortical white matter is composed of axons that span a vast spectrum of diameters and the conduction times along such axons are consequently diverse. The proportion of the very finest, and the very largest axons may differ among both efferent systems and animal species. The finest unmyelinated cortical axons conduct at velocities of <0.5 m/s and conduction times for these axons may be measured in tens of milliseconds. As noted above, these elements sometimes form functionally distinct subpopulations within cortical efferent systems, showing little or no spontaneous impulse activity and only subthreshold or no responses to peripheral stimulation (e.g. Swadlow and Hicks, 1996, 1997; Tsumoto and Suda, 1980). As yet we know little about the role of such slowly conducting axons in cortical processing. Each of the neocortical axonal systems reviewed above had a subpopulation of rapidly conducting axons that mediate conduction times of a few milliseconds or less. The cost, in brain volume, of such rapid communication is quite high (cf., Ringo et al., 1994). A myelinated axon of 1 µm in diameter occupies >100 times the volume of the finest unmyelinated axon of equal length, and this ratio increases to >1:10,000 for the largest cortical axons. It would seem likely, therefore, that the uses of such biologically expensive processes are limited to special situations. Fast conducting axons will, of course, be found in systems that mediate tasks such as sensation and action that require speed, per se. Such axons may also be required when information from widely separated neurone pools must converge to produce coincident input on a post-synaptic target. Thus, for example, binocular responses of some neurones with receptive fields lying near the vertical meridian are mediated by a convergence of inputs from LGNd and the corpus callosum (Berlucchi and Rizzolatti, 1968; Payne et al., 1984), and binocular input to some neurones in the superior colliculus is mediated by a convergence of cortical (fastconducting corticotectal neurones) and retinal input (Palmer and Rosenquist, 1974). Considerations discussed in the previous two sections suggest that large diameter axons may be required to mediate sharply synchronous activity between the spike trains of either closely spaced or distant cortical neurones. Rapidly conducting axons may also be required to achieve synchronous, gamma-band oscillatory activity between neurones in distant cortical areas. Models of interacting non-linear oscillators (Schuster and Wagner, 1989) or neural networks coupled via excitatory (Sompolinsky et al., 1990) or inhibitory (Bush and Sejnowski, 1996) connections have shown that zero phase-lag synchrony between distant areas is dependent upon relatively short delays (i.e. <5 ms or <1/4 the period of oscillation). Thus, whereas unmyelinated axons could mediate such synchrony between closely spaced neural ensembles, myelinated axons would be required when greater distances are involved. In this regard, it is important to determine whether callosal and ipsilateral cortico-cortical projection neurones do take part in such stimulus-dependent oscillatory behaviour and whether fast-or slowly conducting elements are differentially involved. ACKNOWLEDGMENTS: Supported by grants from the National Institute of Neurological Diorders and Stroke (NS-32021) and the National Science Foundation
Time and the brain
172
(IBN-9723967)
REFERENCES Abeles, M. (1991) Corticonics: Neural circuits of the cerebral cortex . Cambridge University Press, Cambridge. Aboitiz, F., Zaidel, E., and Scheibel, A.B. (1989) Variability in fiber composition in different regions of the posterior corpus callosum in humans. Anatomical Record , 223:6A. Aboitiz, F., Scheibel, A.B., Fisher, R.S., and Zaidel, E. Fiber composition of the human corpus callosum. Brain Behavior and Evolution , 143–153. Alonso, J.-M., Usrey, W.M., and Reid, R.C. (1996) Precisely correlated firing in cells of the lateral geniculate nucleus. Nature, London , 383, 815–819. Baker, J., Gibson, A., Mower, G., Robinson, F., and Glickstein, M. (1983) Cat visual corticopontine cells project to the superior colliculus. Brain Research , 265 , 227–232. Baranyi, A., Szente, M.B., and Woody, C.D. (1993) Electrophysiological: characterization of different types of neurons recorded in vivo in the motor cortex of the cat. II. Membrane parameters, action potentials, current induced voltage responses and electrotonic structures. Journal of Neurophysiology , 69 , 1865–1879. Berlucchi, G., and Rizzolatti, G. (1968) Binocularly driven neurons in visual cortex of split-chiasm cats. Science, New York , 159 , 308–310. Bester, H., Menendez, L., Besson, J.M., and Bernard, J.F. (1995) Spino (trigemino) parabrachiohypothalamic pathway: electrophysiological evidence for an involvement in pain processes. Journal of Neurophysiology , 73, 568–585, 1995. Biedenbach, M.A., De Vito, J.L., and Brown, A.C. (1986) Pyramidal tract of the cat: axon size and morphology. Experimental Brain Research , 61 , 303–310. Bishop, G.H., and Smith, J.M. (1964) The sizes of nerve fibers supplying cerebral cortex. Experimental Neurology , 9, 483–501. Bishop, P.O., Burke, W., and Davis, R. (1962) The identification of single units in central visual pathways. Journal of Physiology (London) , 162, 409–431. Boyapati, J., and Henry, G. Corticofugal axons in the lateral geniculate nucleus of the cat. Experimental Brain Research , 53, 335–340. Boyd, I.A., and Kalu, K.U. (1979) Scaling factor relating conduction velocity and diameter for myelinated afferent nerve fibres in the cat hind limb. Journal of Physiology (London) , 289, 277–291. Brown, A.G., and Fyffe, R.E.W. (1978) The morphology of group la afferent fibre collaterals in the spinal cord of the cat. Journal of Physiology (London) , 274, 111– 127. Bullier, J., McCourt, M.E., and Henry, G.H. (1988) Physiological studies on the feedback connection to the striate cortex from cortical areas 1B and 19 of the cat. Experimental Brain Research , 70, 90–98. Bush, P., and Sejnowski, T. (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. Journal of Computational Neuroscience , 3, 91–110. Catsman-Befrevoets, C.E., Lemon, R.N., Verburgh, C.A., Bentivoglio, M., and Kuypers, H.G.J.M. (1980) Absence of callosal collaterals derived from rat corticospinal neurons: a study using fluorescent retrograde tracing and electrophysiological techniques. Experimental Brain Research , 39, 433–440.
Information flow along neocortical axons
173
Cleland, E.G., Levick, R.W., Morstyn, R., and Wagner, H.G. (1976) Lateral geniculate relay of slowly conducting retinal afferents to cat visual cortex. Journal of Physiology, (London) , 255, 299–320. Deschênes, M. (1977) Dual origin of fibers projecting from motor cortex to S1 in cat. Brain Research , 132, 159–162. Deschênes, M., Bourassa, J., and Pinault, D. (1994) Corticothalamic projections from layer V cells in rat are collaterals of long-range corticofugal axons. Brain Research , 664 , 215–219. Deschênes, M., and Landry, P. (1980) Axonal branch diameter and spacing of nodes in the terminal arborization of identified thalamic and cortical neurons. Brain Research , 191, 538–544. Doetsch, G.S., and Towe, A.L. (1981) The pyramidal system of the woodchuck. Brain Behavior and Evolution , 19, 37–55. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H.J. (1988) Coherent oscillations: a mechanism of feature linking iri the visual cortex? Biological Cybernetics , 60 :121–130. Engel, A.K., Konig, P., Kreiter, A.K., and Singer, W. Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, New York , 252, 1177– 1179. Evarts, E.V. (1965) Relation of discharge frequency to conduction velocity in pyramidal tract neurons. Journal ofNeurophysiology , 28, 216–228. Famiglietti, E.V.Jr., and Peters, A. (1972) The synaptic glornerulus and the intrinsic neuron in the dorsal lateral geniculate nucleus of the cat. Journal of Comparative Neurology , 144, 285–334. Ferreyra Moyano, H., and Molina, J.C. (1980) Axonal projections and conduction properties of olfactory peduncle neurons in the rat. Experimental Brain Research , 39, 241–248. Ferster, D., and LeVay, S. (1978) The axonal arborizations of lateral geniculate neurons in the striate cortex of the cat. Journal of Comparative Neurology , 182, 923–944. Ferster, D., and Lindstrom, S. (1983) An intracellular analysis of geniculo-cortical connectivity in area 17 of the cat. Journal of Physiology, (London) , 342, 181–215, 1983. Finlay, B.L., Schiller, P.H., and Volman, S.F. (1976) Quantitative studies of single cell properties in monkey striate cortex. IV. Corticotectal cells. Journal of Neurophysjology , 39, 1352–1361. Fleischhauer, K., and Wartenburg, H. (1967) Elektronenmikroskopische untersuchungen uber das wachstum der nervenfasern and Uber das auftreten von markscheiden im corpus callosum der katze. Zeitschrift fur Zellforschung und Mikroskopische Anatomie , 83, 568–581. Freund, T.F., Martin, K.A.C., Soltesz, I., Somogyi, P., and Whitteridge, D. (1989) Arborization pattern and postsynaptic targets of physiologically identified thalamocortical afferents in striate cortex of the macaque monkey. Journal of Comparative Neurology , 289, 315–336. Fukuda, Y. (1973) Differentiation of principal cells of the rat lateral geniculate nucleus into two groups: fast and slow cells. Experimental Brain Research , 17, 242–260. Fuller, J.H., and Schlag, J. (1976) Determination of antidromic excitation by the collision test: Problems of interpretation. Brain Research , 122, 283–298. Gibson, A., Baker, J. Mower, G., and Glickstein, M. Corticopontine cells in area 18 of the cat. Journal of Neurophysiology , 41, 494–495.
Time and the brain
174
Gilbert, C.D. (1977) Laminar differences in receptive field properties of cells in cat primary visual cortex. Journal of Physiology (London) , 268, 391–421, 1977. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of the awake macaque. Experimental Brain Research , 84 , 505–516. Goldman, L., and Albus, S. (1968) Computation of impulse conduction in myelinated fibres: theoretical basis of the velocity-diameter relation. Biophysical Journal , 8, 596– 607. Griffith, J.S. (1963) On the stability of brain-like structures. Biophysical Journal , 3, 299–308. Guillery, R.W. (1967) A study of Golgi preparations from the dorsal lateral geniculate nucleus of the adult cat. Journal of Comparative Neurology , 128, 21–50, 1966. Guillery, R.W. (1967) Patterns of fiber degeneration in the dorsal lateral geniculate nucleus of the cat following lesions in the visual cortex. Journal of Comparative Neurology , 130, 197–222. Harding, G.W., and Towe, A.L. (1985) Fiber analysis of the pyramidal tract of the laboratory rat. Experimental Neurology , 87, 503–518, 1985. Harvey, A.R. (1980) A physiological analysis of subcortical and commissural projections of areas 17 and 18 of the cat. Journal of Physiology, (London) , 302, 507–534. Hoogland, P.V., Welker, E., and Van der Loos, H. (1987) Organization of the projections from barrel cortex to thalamus in mice studied with Phaseolus vulgaris-leucoagglutinin and HRP. Experimental Brain Research , 68, 73–87. Humphrey, A.L., and Weller, R.E. (1988a) Functionally distinct groups of X-cells in the lateral geniculate nucleus of the cat. Journal of Comparative Neurology , 268, 429– 441. Innocenti, G.M. (1980) The primary visual pathway through the corpus callosum: morphological and functional aspects in the cat. Archives Italiennes de Biologie , 118, 124–188. Innocenti, G.M. (1986) General organization of callosal connections in the cerebral cortex. In: E.G.Jones and A.Peters (eds.) Cerebral Cortex, Vol. 5 , Plenum Press, New York. Jones, E.G. (1984) Laminar distribution of cortical efferent cells. In: A.Peters and E.G.Jones (eds) Cerebral cortex. Vol. 1. Cellular components of the cerebral cortex , Plenum Press, New York. Joosten, E.A., and Gribnau, A.A. (1988) Unmyelinated corticospinal axons in adult rat pyramidal tract. An electron microscopic tracer study. Brain Research , 459 , 173–177. Kaba, H., Saito, H., Otsuka, K., and Seto, K. (1986) Ventrolateral medullary neurons projecting to the medial preoptic/anterior hypothalamic area through the medial forebrain bundle: an electrophysiological study in the rat. Experimental Brain Research , 63, 369–374. Khodorov, B.I., and Timin, E.N. (1975) Nerve impulse propagation along nonuniform fibres. Progress in Biophysics and Molecular Biology , 30, 145–184. Kirkwood, P.A., and Sears, T.A. (1978) The synaptic connexions to intercostal motoneurones as revealed by the average common excitation potential. Journal of Physiology (London) , 275, 103–134. Kirkwood, P.A., Sears, T.A., Tuck, D.L., and Westgaard, R.H. (1982a) Variations in the time course of the synchronization of intercostal motoneurones in the cat. Journal of Physiology (London) , 327, 105–135. Kirkwood, P.A., Sears, T.A., Stagg, D., and Westgaard, R.H. (1982b) The spatial
Information flow along neocortical axons
175
distribution of synchronization of intercostal motoneurones in the cat. Journal of Physiology (London) , 327, 137–155. Kruger, J., and Aiple, F. (1988) Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. Journal of Neurophysiology , 60, 798–828. LaMantia, A.S., and Rakic, P. (1990) Cytological and Quantitative characteristics of four cerebral commissures in the rhesus monkey. Journal of Comparative Neurology , 291, 520–537. Landry, P., and Dykes, R.W. (1985) Identification of two populations of corticothalamic neurons in cat primary somatosensory cortex. Experimental Brain Research , 60, 289– 298. Lesnen, L.P., Masd, J., Posthuma, P.R., and Nieuwenhuys, R. (1985) A detailed morphometrical analysis of the pyramidal tract of the rat. Brain Research , 359, 65–80. Lemmon, V., and Pearlman, A.L. (1981) Does laminar position determine the receptive field properties of cortical neurons? A study of corticotectal cells in area 17 of the normal mouse and the reeler mutant. Journal of Neuro science , 1, 83–93. Levick, W.R., Cleland, G.G., and Dubin, M.W. (1972) Lateral geniculate neurons of the cat: retinal inputs and physiology. Investigative Ophthalmology , 11 , 302–311. Manis, P.B. (1989) Responses to parallel fiber stimulation in the guinea pig dorsal cochlaar nucleus in vitro. Journal of Neurophysiology , 61 , 149–161. Marrocco, R.T. (1976) Sustained and transient cells in monkey lateral geniculate nucleus: conduction velocities and response properties. Journal of Neurophysiology , 39, 340– 353. Manzoni, T., Caminiti, R., Spidalieri, G., and Morelli, E. (1979) Anatomical and functional aspects of the associative projections from somatic areas S1 to S11. Experimental Brain Research , 34, 453–470. Matsumoto, G., and Tasaki, I. (1977) A study of conduction velocity in nonmyelinated nerve fibers. Biophysical Journal , 20, 1–13. McCourt, M.E., Thalluri, J., and Henry, G.H. (1990) Properties of area 17/18 border neurons contributing to the visual transcallosal pathway in the cat. Visual Nsuroscience , 5, 83–98. Mediratta, N.K., and Nicoll, J.A. (1983) Conduction velocities of corticospinal axons in the rat studied by recording cortical antidromic responses. Journal of Physiology (London) , 336, 545–561. Mendell, L.M., and Henneman, E. (1971) Terminals of single la fibres: location, density and distribution within a pool of 300 homonymous motoneurons. Journal of Neurophysiology , 34, 171–187. Miller, R. (1975) Distribution and properties of commissural and other neurons in cat sensorimotor cortex. Journal of Comparative Neurology , 164, 361–374. Moore, G.P., Segundo, J.P., Perkel, D.H., and Levitan, H. (1970) Statistical signs of synaptic interaction in neurons. Biophysical Journal , 10 , 876–900. Movshon, J.A., and Newsome, W.T. (1996) Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. Journal of Neuroscience , 16, 7733–7741. Munk, M.H.J., Nowak, L.G., Nelson, J.H.I., and Bullier, J. (1995) Structural basis of cortical synchronization 11. Effects of cortical lesions. Journal of Neurophysiology , 74, 2401–2414. Nowak, L.G., and Bullier, J. (1996) Spread of stimulating current in the cortical gray matter of rat visual cortex studied on a new in vitro slice preparation. Journal of
Time and the brain
176
Neuroscience Methods , 67 , 237–248. Nowak, L.G., Munk, M.H.J., Nelson, J.I., James, A.C., and Bullier, J. (1995) Structural basis of cortical synchronization. 1. Three types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–2400. Ojima, H. (1994) Terminal morphology and distribution of corticothalamic fibers originating from layers 5 and 6 of cat primary auditory cortex. Cerebral Cortex , 4, 646–663. Palmer, L.A., and Rosenquist, A.L. (1974) Visual receptive fields of single striate cortical units projecting to the superior colliculus of the cat. Brain Research , 67, 27–42. Parnas, I., and Segev, I.A. (1979) Mathematical model for conduction of action potentials along bifurcating axons. Journal of Physiology (London) , 295, 323–343, 1979. Payne, B.R., Pearson, H.E., and Berman, N. (1984) Role of corpus callosum in functional organization of cat striate cortex. Journal of Neurophysiology , 52, 570–593. Raczkowski, D., and Diamond, I.T. (1978) Connections of striate cortex of Galago senegalensis. Brain Research , 144, 383–388. Ralston, D.D., Milroy, A.M., and Ralston, H.J. 3d. (1987) Non-myelinated axons are rare in the medullary pyramids of the macaque monkey. Neuroscience Letters , 73, 215– 219. Ranck, James B. (1975) Which elements are excited in electrical stimulation of mammalian central nervous system: A review. Brain Research , 98, 417–440. Raymond, S.A., Thalhammer, J.G., Popitz-Bergez, F., and Strichartz, G.R. (1990) Changes in axonal impulse conduction correlate with sensory modality in primary afferent fibers in the rat. Brain Research , 526, 318–321. Ringo, J.L., Doty, R.W., Demeter, S., and Simard, P.Y. (1994) Time is of the essence: a conjecture that hemispheric specialization arises from interhemispheric conduction delay. Cerebral Cortex , 4, 331–343. Robson, J.A. (1984) Reconstructions of corticogeniculate axons in the cat. Journal of Comparative Neurology , 225, 193–200. Rockland, K.S. (1989) Bistratified distribution of terminal arbors of individual axons projecting from area V1 to middle temporal area (MT) in the macaque monkey. Visual Neuroscience , 3, 155–170. Rockland, K.S. (1992) Configuration, in serial reconstruction, of individual axons projecting from area V2 to V4 in the macaque monkey. Cerebral Cortex , 2, 353–374. Rockland, K.S. (1997) Elements of cortical architecture: Hierarchy Revisited. Cerebral Cortex , 12, 243–293. Rushton, W.A.H. (1951) A theory of the effects of fibre size in medullated nerve. Journal of Physiology (London) , 115, 101–122. Sakai, H., and Woody, C.D. (1988) Relationships between axonal diameter, soma size, and axonal conduction velocity of HRP-filled, pyramidal tract cells of awake cats. Brain Research , 460, 1–7. Schiller, Ph.H., and Malpeli, J.G. (1978) Functional specificity of lateral geniculate nucleus laminae of the rhesus monkey. Journal of Neurophysiology , 41, 788–797. Schuster, H.G., and Wagner, P. (1989) Mutual entrainment of two limit cycle oscillators with time delayed coupling. Progress in Theoretical Physics , 81 , 939–945. Schwartz, M.L., and Goldman-Rakic, P.S. (1982) Single cortical neurons have axon collaterals to ipsilateral and contralateral cortex in fetal and adult primates. Nature, London , 299 , 154–156, 1982. Sears, T.A., and Stagg, D. (1976) Short-term synchronization of intercostal motoneurone activity. Journal of Physiology (London) , 263, 357–387.
Information flow along neocortical axons
177
Segal, M., Foote, S.L., and Aston-Jones, G. (1983) Physiological properties of ascending locus coeruleus axons in the squirrel monkey. Brain Research , 274, 381–387. Seggie, J., and Berry, M. (1972) Ontogeny of interhemispheric evoked potentials in the rat: significance of myelination of the corpus callosum. Experimental Neurology , 35, 215–232. Sherman, S.M., Norton, T.T., and Casagrande, V.A. (1975) X- and Y-cells in the dorsal lateral geniculate nucleus of the tree shrew (Tupaia glis). Brain Research , 93 , 152– 157. Simmons, P.A., and Pearlman, A.L. (1983) Receptive field properties of transcallosal visual cortical neurons in the normal and reeler mouse. Journal of Neurophysiology , 50, 838–848. Sompolinsky, H., Golomb, D., and Kleinfeld, D. (1990) Global processing of visual stimuli in a neural network of coupled oscillators. Proceedings of the National Academy of Sciences, U.S.A. , 87, 7200–7204. Stone, J. (1973) Sampling properties of microelectrodes assessed in the cat’s retina. Journal of Neurophysiology , 36, 1071–1079. Swadlow, H.A. (1985) Physiological properties of individual cerebral axons studied in vivo for as long as one year. Journal of Neurophysiology , 54, 1346–1362. Swadlow, H.A. (1988) Efferent neurons and suspected interneurons in binocular visual cortex of the awake rabbit: receptive fields and binocular properties. Journal of Neurophysiology , 59, 1162–1187. Swadlow, H.A. (1990) Efferent neurons and suspected interneurons in S-1 forelirnb representation of the awake rabbit: receptive fields and axonal properties. Journal of Neurophysiology , 63, 1477–1498. Swadlow, H.A. (1991) Efferent neurons and suspected interneurons in second somatosensory cortex of the awake rabbit: receptive fields and axonal properties. Journal of Neurophysiology , 66 , 1392–1409. Swadlow, H.A. (1994) Efferent neurons and suspected interneurons in motor cortex of the awake rabbit: axonal properties, sensory receptive fields and subthreshold synaptic inputs. Journal of Neurophysiology , 71 , 437–453. Swadlow, H.A. (1995) Influence of VPM thalamic afferents on putative inhibitory interneurons in SI of the awake rabbit: Evidence from: cross-correlation, microstimulation and latencies to peripheral sensory stimulation. Journal of Neurophysiology , 73, 1584–1599. Swadlow, H.A. (1998) Neocortical efferent neurons with very slowly conducting axons: strategies for reliable antidromic identification. Journal of Neuro science Methods , 79, 131–141. Swadlow, H.A., Beloozerova, I., and Sirota, M. (1998) Sharp, local synchrony among putative feed-forward inhibitory interneurons of rabbit somatosensory cortex. Journal of Neurophysiology , 79, 567–582. Swadlow, H.A., Geschwind, N., and Waxman, S.G. (1979) Commissural transmission in humans. Science, New York , 204 , 530–531. Swadlow, H.A., and Hicks, T.P. (1996) Somatosensory cortical efferent neurons of the awake rabbit: latencies to activation via supra- and subthreshold receptive fields. Journal of Neurophysiology , 75, 1753–1759. Swadlow, H.A., and Hicks, T.P. (1997) Subthreshold receptive fields and baseline excitability of “silent” S1 callosal neurons in awake rabbits: contributions of AMPA/kainate and NMDA receptors. Experimental Brain Research , 115, 403–409. Swadlow, W.A., Kocsis, J.D., and Waxman, S.G. (1980) Modulation of impulse
Time and the brain
178
conduction along the axonal tree. Annual Reviews of Biophysics and Bioengineering , 9, 143–179. Swadlow, H.A., Rosene, D.L., and Waxman, S.G. (1978a) Characteristics of interhemispheric impulse conduction between prelunate gyri of the rhesus monkey. Experimental Brain Research , 33, 455–467. Swadlow, H.A., Waxman, S.G., and Geschwind, N. (1980) Small diameter nonmyelinated axons in the primate corpus callosum. Archives of Neurology , 37, 114– 115. 1980. Swadlow, H.A., Waxman, S.G., and Rosene, D.L. (1978b) Latency variability and the identification of antidromically activated units in mammalian brain. Experimental Brain Research , 32, 439–443. Swadlow, H.A., and Weyand, T.G. (1985) Receptive-field and axonal properties of neurons in the dorsal lateral geniculate nucleus of awake unparalyzed rabbits. Journal of Neurophysiology , 54, 168–183. Szentagothai, J. (1963) The structure of the synapse in the lateral geniculate body. Acta Anatomica , 55, 166–185. Takahashi, K. (1965) Slow and fast pyramidal tract cells and their respective membrane properties. Journal of Neurophysiology , 28 , 908–924. Tomasch, J. (1954) Size, distribution, and number of fibers in the human corpus callosum. Anatomical Record , 119, 119–135, 1954. Tomasch, J. (1957) The number of fibers in the corpus callosum of the white mouse. Journal of Comparative Neurology , 107, 165–168. Towe, A.L., and Harding, G.W. (1970) Extracellular microelectrode sampling bias. Experimental Neurology , 29 , 366–38. Toyama, K., Matsunami, K., Ohno, T., and Tokashiki, S. (1974) An intracallular study of neuronal organization in the visual cortex. Experimental Brain Research , 21, 46–66. Tsumoto, T., and Suda, K. (1980) Three groups of cortico-geniculate neurons and their distribution in binocular and monocular segments of cat striate cortex. Journal of Comparative Neurology , 193, 223–236. von Keyserlingk, D.G., and Schramm, U. (1984) Diameter of axons and thickness of myelin sheaths of the pyramidal tract fibres in the adult human medullary pyramid. Anatomischer Anzeiger , 2, 97–111. Waters, R.S., Favorov, O., and Asanuma, H. (1982) Physiological properties and patterns of projection of cortico-cortical connections from the anterior bank of the ansate sulcus to the motor cortex, area 4 (gamma), in the cat. Experimental Brain Research , 46 , 403–412. Waxman, S.G. (1975) Integrative properties and design principles of axons. International Reviews of Neurobiology , 18, 1–40. Waxman, S.G., and Bennett, M.V.L. (1972) Relative conduction velocities of small myelinated and non-myelinated fibres in the central nervous system. Nature, New Biology , 238, 217–219. Waxman, S.G., and Swadlow, H.A. (1976a) Morphology and physiology of visual callosal axons: Evidence for a supernormal period in central myelinated axons. Brain Research , 113, 179–187. Waxman, S.G., and Swadlow, H.A. (1976b) Ultrastructure of visual callosal axons in the rabbit. Experimental Neurology , 53, 115–128. Waxman, S.G., and Swadlow, H.A. (1977) The conduction properties of axons in central white matter. Progress in Neurobiology , 8, 297–324, 1977. Weyand, T.G., Malpeli, J.G., Lee, C., and Schwark, H.D. (1986) Cat area 17. III
Information flow along neocortical axons
179
Response properties and orientation anisotropies of corticotectal cells. Journal of Neurophysiology , 56, 1088–1101. Weyand, T.G., Malpeli, J.G., and Lee, C. (1991) Area 18 corticotectal cells: response properties and identification of sustaining geniculate inputs. Journal of Neurophysiology , 65, 1078–1088. White, E. (1979) Thalamocortical synaptic relations: a review with emphasis on the projections of specific thalamic nuclei to the primary sensory areas of the neocortex. Brain Research Reviews , 1, 275–311, 1979. White, E.L., and Czeiger, D. (1991) Synapses made by axons of callosal projection neurons in mouse somatosensory cortex: emphasis on intrinsic connections. Journal of Comparative Neurology , 303, 233–244, 1991. Woodward, W.R., and Coull, B.M. (1984) Localization and organization of geniculocortical and corticofugal fiber tracts within the subcortical white matter. Neuroscience , 12 , 1089–1099. Wilson, P.D., Rowe, M.H., and Stone, J. (1976) Properties of relay cells in cat’s lateral geniculate nucleus: a comparison of W-cells with X- and Y-cells Journal of Neurophysiology , 39, 1193–1209. Zhang, Z.-W., and Deschenes, M. (1997) Intracortical axonal projections of lamina V1 cells of the primary somatosensory cortex in the rat: a single- cell labeling study. Journal of Neuroscience , 17 , 6365–6379. Zarzecki, P. (1989) Influence of somatosensory cortex on different classes of cat motor cortex output neuron. Journal of Neurophysiology , 62 , 487–94, 1989. Zarzecki, P., Blum, P.S., Bakker, D.A., and Herman, D. (1983) Convergence of sensory inputs upon projection neurons of somatosensory cortex: vestibular, neck, head, and forelimb inputs. Experimental Brain Research , 50, 408–14. Zarzecki, P., and Wiggin, D.M. (1982) Convergence of sensory inputs upon projection neurons of somato-sensory cortex. Experimental Brain Research , 48, 28–42. Zeki, S.M. (1978) The cortical projections of foveal striate cortex in the rhesus monkey. Journal of Physiology (London) , 277, 227–244, 1978.
5 Psychophysics of Human Timing Thomas H.Rammsayer 1 and Simon Grondin 2 1 Georg
Elias Mueller Institute of Psychology, University of Goettingen, Gosslerstrasse 14, D-37073, Goettingen, Germany Tel: (0049) 551–393611; FAX: (0049) 551–393662 e-mail: [email protected] 2 Ecole de Psychologie, Universite Laval, Quebec, Qc, Canada G1K 7P4 Tel: (001) 418–656–2131; FAX: (001) 418–656–3646 e-mail: [email protected]
This chapter introduces the main methods and findings in the field of psychophysics of human timing. Furthermore, two theoretical views of human timing are proposed. It has been shown that, for temporal intervals below 1 s, the Weber fraction generally decreases with longer interval durations. A generalized form of Weber’s law is often reported to hold for the temporal domain and, thus, supports the notion of a single timing mechanism underlying temporal processing. On the other hand, a series of behavioural and pharmacopsychological studies is presented that suggests the presence of two different timing mechanisms for temporal processing. One mechanism, involved in processing of very brief durations, appears to be located at a subcortical level, while the other mechanism, involved in processing of longer intervals (>1 s), is based on memory processes. KEYWORDS: Time estimation; Time perception; Psychophysical methods; Timing mechanisms This chapter reviews major recent findings in the field of human temporal information processing. After providing the reader with an overview of the methods for addressing the problem of temporal processing, several studies are reviewed, in order to compare the relative accuracy for estimating short and long time intervals.
1. PSYCHOPHYSICAL METHODS FOR ASSESSING PERFORMANCE OF TEMPORAL PROCESSING Traditionally, there are four major methods for assessing accuracy of temporal processing (Bindra and Waksberg, 1956; Clausen, 1950; Doob, 1971; Hicks et al., 1976; Wallace and Rabin, 1960; Zakay, 1990). The main characteristics of each method can be briefly summarized as follows. In a verbal estimation task, the duration of the target interval is
Time and the brain
182
estimated verbally in terms of temporal units. A production task involves producing an interval equal to a duration that is verbally indicated. With the method of reproduction, after presentation of a target interval, an attempt is made to reproduce its duration by means of some operation. Finally, the method of comparison involves the decision, after the presentation of two intervals, which one appeared to be longer. Although none of the methods has shown general superiority to the others (Allan, 1979; Carlson and Feinberg, 1970; McConchie and Rutschmann, 1971; Zakay, 1993), several studies suggest that verbal estimation and production are less reliable than operative reproduction (e.g. Block, 1989; Zakay, 1990). In verbal estimation, subjects give a verbal estimate in conventional temporal units, i.e. clock time such as seconds or minutes, whereas in production subjects produce a duration stated by the experimenter in clock time. Obviously, both methods use a translation of duration into socially learnt time units and, thus, depend on the relation of subjective time to clock time (Block, 1989; Clausen, 1950). In contrast, the method of reproduction as well as the method of comparison do not require such a translation (Block, 1989) and, therefore, results obtained by both these latter methods appear to be freer from such a possible confound (Eisler and Eisler, 1994; Rammsayer, 1997a). With the method of reproduction, subjects activate and stop some measurement device. While this appears to be an appropriate procedure for estimation of longer durations, motor response latencies are much too long for reliable assessment of timing of extremely brief durations. Therefore, for quantification of performance on temporal processing of brief durations in the range of milliseconds, the method of comparison has to be used. Another important factor in evaluating the existing literature on human timing performance is the methodological distinction between prospective and retrospective experimental paradigms (Block, 1989; Block and Zakay, 1997; Boltz, 1995; Brown, 1985; Brown and Stubbs, 1988; Gilliland et al., 1946; James, 1890; Zakay, 1990). In the prospective paradigm, subjects know in advance that they will be required to make a time judgement and, will therefore, be able to activate specific timing processes and attend to any available temporal information (Doob, 1971; Zakay, 1990). In the retrospective paradigm, on the contrary, subjects are unexpectedly asked to judge the duration of a temporal interval after it has already passed by. Therefore, in the retrospective paradigm, subjects incidentally encode temporal information which may be retrieved from memory some time later (Block and Zakay, 1997). Hence, it is plausible to assume that retrospective time judgements are more incidental and more variable than are prospective ones (Block and Zakay, 1997; Brown, 1985; Zakay, 1990). For this reason, the present review of psychophysical data on temporal processing in humans will focus exclusively on studies applying a prospective paradigm. On the strict psychophysical ground, one can distinguish two main methods for studying time (Allan, 1979): duration scaling and duration discrimination. With the scaling method, one can use verbal estimation, interval production, or interval reproduction. The purpose of this method is to relate the magnitude of a psychological or subjective impression ( Ψ ) to the magnitude of a physical stimulus ( S ) under investigation. This relationship between Ψ and S is exponential, linear, or logarithmic. Stevens (1975) reports that this relationship obeys a power law:
Psychophysics of human timing
183
(1) where k is a constant whose value depends on physical units used, and n is an exponent specific to the given continuum. When plotted on log-log co-ordinates, the functions are straight lines with n being equal to 1 if the relation is linear (constant steepness), greater than 1 if the relation is one of increasing steepness, and lower than 1 if the relation is one of decreasing steepness. Thus, n is the main characteristic of each sensory continuum and is sometimes referred to as the signature of a continuum. For the temporal continuum, the value of n is often reported to range around .9 (Eisler, 1976), with some authors arguing that psychological time tends to correspond to real time and, hence, n is assigned a value of 1 (see Allan, 1979). Even more important for the sections below is the understanding of results observed with duration discrimination tasks. In such tasks, the method of comparison is employed, and the sensitivity of subjects can be measured by determining the difference threshold as a psychophysical indicator of performance. Traditionally, there are three methods for threshold estimation in psychophysics: a) method of constant stimuli; b) method of limits; and c) method of adjustment (Gescheider, 1985). For investigating temporal processing in humans, the first two methods are employed most frequently. With the method of constant stimuli, the difference threshold is estimated on the basis of a psychometric function where the probability of judging a comparison value as longer than a standard is plotted as a function of this comparison value. The difference threshold, or just noticeable difference, is most often defined as half the interquartile range (Luce and Galanter, 1963). In this case, the interquartile range is represented by the difference between the x values corresponding to 75% and 25% of long responses. In other cases, difference threshold has been defined as one standard deviation of the psychometric function (Getty, 1975; Killeen and Weiss, 1987). With the method of limits, there are series of trials where a comparison value is gradually increased or decreased in fixed steps. Difference thresholds are estimated on the basis of transition points where a standard and a comparison value start or stop to be perceived as equal. A variation of the method of limits for estimation of difference thresholds represents adaptive psychophysical procedures (Kaernbach, 1991; Pentland, 1980; Rammsayer, 1992a). “Adaptive” means that the duration of the comparison interval is adjusted from trial to trial in order to reach a given probability of responding correctly. Depending on the rule chosen to change the duration of the comparison interval, adaptive procedures converge to a specific duration of the comparison interval required for a specific level of performance.
Time and the brain
184
2. ACCURACY OF TEMPORAL PROCESSING AS A FUNCTION OF BASE DURATION For more than a century, psychophysicists have been concerned with the question of whether accuracy of timing processes in humans is affected by the duration of a target interval (e.g. Vierordt, 1968). The answer to this question depends largely on the psychophysical measure chosen for quantifying timing accuracy. If, using the method of comparison, difference threshold is the criterion, then smaller threshold values, indicating better performance, will generally be observed for shorter base durations. Similar results will be obtained with difference scores, i.e. actual duration of the target interval minus estimated duration, when the methods of verbal estimation, production, or reproduction are applied. However, both difference thresholds and difference scores cannot be considered appropriate psychophysical indicators of performance since they are highly misleading when used for a wide range of different base durations (Hornstein and Rotter, 1969). For example, a 100 ms error in time judgement would be treated as being equally inaccurate regardless of whether the target duration was 500 ms or 5 s. The so-called Weber fraction represents a much more valid, and widely accepted, approach for quantification of timing accuracy as a function of base duration (e.g. Killeen and Weiss, 1987). In the case of verbal estimation, production, or reproduction, Weber fractions can be obtained by dividing the standard deviation of the time estimates by the base duration. However, typically, Weber fractions are obtained with duration discrimination tasks and are calculated by dividing the difference threshold (∆S) by the standard value of the stimulus (S) under investigation. This relation guides thinking in psychophysics. An ubiquitous finding in psychophysics is that difference threshold increases with the magnitude of the stimuli to be discriminated. When this relation is a linear function, it is referred to as Weber’s law:
(2) What is frequently observed in psychophysics is that Weber’s law holds for a given range of a sensory continuum, but, with relatively small values on this continuum, the difference threshold is larger than would be expected from a linear relationship. To account for this increase with smaller values, a constant value, c, is added to the above relationship:
(3) This relationship is referred to as the generalized form of Weber’s law (Getty, 1975). A large amount of empirical work has been devoted to test the robustness of some form of Weber’s law for duration (e.g. Killeen and Weiss, 1987; Getty, 1975, 1976; Treisman,
Psychophysics of human timing
185
1963). Getty (1975) offered a systematic analysis of this question for a duration discrimination task. Basically, the total variance in a timing task is assumed to be composed of a duration-dependent source, and of a duration-independent source. Depending on the assumptions made for the timing component, the form of the relationship between total variance and the duration of the target interval varies. Thus, total variance grows linearly in relation to base duration if timing depends on a Poisson process:
(4) where T is the duration of the target interval, and k and c are constants representing the rate of growth of the duration-dependent variability and the variability due to the duration-independent component, respectively. Note, that what was S in Equations 2 and 3 is now referred to as T, for physical time, in Equations 4 and 5. If variability of the timing process conforms to the generalized form of Weber’s law, then total variance of timing could be described as follows:
(5) Getty’s (1975) study was designed to provide a direct test of these two models. As could be shown, the generalized form of Weber’s law provides the best description of the relationship between total variability and the duration of the target interval. Interestingly, this demonstration of Getty was shown to hold for durations ranging from 200 ms to 2 s (see Table 1). With durations below or above this range of base durations, Weber fractions increased rapidly. Indeed, most data collected with human subjects support this generalized form of Weber’s law, at least for a restricted range of base durations (see Table 1). This conclusion was shown to hold with an interval production task (Grondin, 1992), with both time production and duration discrimination tasks (Ivry and Hazeltine, 1995), and with several other psychophysical methods applied to both human and animal subjects (Fetterman and Killeen, 1992). Literature on animal behaviour also provides support for the view that timing obeys some form of Weber’s law. One of the most popular theories in animal timing in the past 20 years, the scalar timing theory (Gibbon, 1991), also predicts Weber’s law. In addition to the empirical support received from experiments with animals, properties of scalar timing, which include Weber’s law, were shown to apply to human subjects as well (Allan and Gibbon, 1991; Wearden, 1991). Table 1 shows the changes in Weber fraction for different base durations (standard intervals) from various studies. Although the outcome of most studies suggests that accuracy of temporal processing increases with increasing base durations, as indicated by decreasing Weber fractions, this seems to be true only for a limited range of base durations. Rather, there is some evidence that decreasing Weber fractions are most likely to occur with increasing base durations. This effect becomes evident especially at the lower end of the series of base durations applied in an experiment. Many illustrations of
Time and the brain
186
this relationship between Weber fraction and base duration are also reported in Killeen and Weiss (1987). This relationship is indeed exactly what is predicted by the generalized form of Weber’s law. Increased values of the Weber fraction for shorter base durations are associated with nontemporal sources of variability. These sources can, for instance, be used to mark sensory mode (auditory or visual) or the structure (filled or empty)
Table 1 Changes in Weber fractions for different base durations from various psychophysical studies on temporal processing. Psychophysical method: C=comparison, P=production, R=reproduction, VE=verbal estimation. Change of Weber fractions: dec=decrease, inc=increase, ( )=range of base durations for dec/inc.
Authors(s)
Psychophysical Method
Range of Base Durations
Change of Weber Fractions
Abel (1972a)
C
0.1 6–960 ms
dec (0.1 6–10 ms)
Elbert et al., (1991)
R
1–8 s
dec (1–4 s)
Fetterman & Killeen (1992)
C
25 ms–5 s
dec (25–100 ms)
Getty (1975)
C
50 ms–3.2 s
dec (50–200 ms) inc (2.4–3.2 s)
Grondin (1993)
C
125 ms–4 s
dec (125 ms–1 s)
Grondin (1996b)
C
125 ms–4 s
dec (125 ms-2 s)
Hancock et al., (1994) P
1–20 s
dec (1–7 s)
Henry (1948)
C
32–480 ms
dec (32–480 ms)
Kinchla (1972)
C
1–8 s
dec (1–4 s)
Rammsayer (1992b)
C
50 ms vs. 1 s
dec
Rammsayer (1996)
R
5–40 s
dec (15–40 s)
Rammsayer & Vogel (1992)
C
50 ms vs. 1 s
dec
Rousseau et al., (1983) C
600ms–1.8 s
dec (600ms–1.2 s)
Smith (1969)
VE
15–1 20 s
dec (60–1 20 s)
Thomas & Brown (1974)
R
750 ms–5.5 s
dec (750ms–1.5 s)
Treisman (1963)
R, P
500 ms–9 s
dec (500 ms–2 s)
5–1 20s
dec (30–60 s)
Williams et al., (1989) VE, R
temporal intervals (Grondin, 1993; Grondin et al, 1996; Rammsayer, 1994c; Rammsayer and Lima, 1991; Skrandies and Rammsayer, 1995). On the other hand, whether or not Weber fractions increase with base durations longer
Psychophysics of human timing
187
than approximately 2 seconds, is closely related to the strategy adopted by the subject. When subjects refrain from counting, Weber fractions consistently increase with increasing base duration (Getty, 1975), whereas explicit counting improves performance on discrimination of intervals with base durations as short as 1.5 seconds (Grondin, 1996a). It should be noted, however, that there are some instances where neither Weber’s law, nor its generalized form, seem to apply to time judgements. One classical report of violation of Weber’s law is that of Kristofferson (1980). In his study, the subject received extensive training. When only the last 3 of 20 consecutive sessions with given base durations ranging from 100 to 1,480 msec were analysed, the function relating difference threshold to base duration was not continuous, but showed a series of steps. Threshold remained the same for a range of base durations until some point at which it doubled and then stayed constant for another range of base durations. There are also some studies of rhythm perception that call into question the general validity of Weber’s law for timing processes in humans. In a recent experiment, ten Hoopen et al. (1995) presented their subjects with isochronic intervals marked by a series of sounds, with the last sound creating anisochrony. With target intervals ranging from 50 to 200 ms, difference thresholds rather than Weber fractions remained constant indicating better performance with increasing base durations. Concurrently, these authors found a marked decrease in perceived duration of the last interval of the sequence, a phenomenon they called “time shrinking” (see also Nakajima et al., 1991; ten Hoopen et al., 1993); Drake and Botte (1993) also showed that Weber fractions are not constant for a wide range of base durations in temporal discrimination tasks. Their subjects had to discriminate tempi of two auditory sequences presented with inter-onset intervals (IOIs) ranging from 100 to 1,500 ms. Weber fractions were smallest for IOIs ranging from 300 to 800 ms and markedly increased for IOIs beyond this range. In conclusion, psychophysical data do not support the notion that temporal processing is more accurate for short than for longer intervals. Furthermore, there is no definitive answer to the question of whether there is a certain point on the time continuum where timing is most accurate. The answer to this question depends on the criterion adopted for quantifying timing performance. In general, variability is lower for shorter base durations, a finding that may be interpreted in terms of better temporal accuracy for brief intervals. However, within the framework of Weber-fraction analysis, shorter base durations do not provide better accuracy.
3. IS THERE EVIDENCE FOR DIFFERENT TIMING MECHANISMS? The finding that Weber’s law holds, at least for a given range of base durations, suggests that there is a single central mechanism underlying temporal processing for this range. This timing mechanism is often referred to as the “internal clock”, and most psychophysical models assume that such a clock is based on a neural counting process (e.g. Allan and Kristofferson, 1974; Church, 1984; Creelman, 1962; Treisman, 1963; Treisman et al., 1990). More specifically, the internal-clock mechanism is basically
Time and the brain
188
characterized by a neural pacemaker and an accumulator. The pacemaker generates pulses and the number of pulses relating to a physical time interval is recorded by the accumulator. Thus, the number of pulses counted during a given time interval is the internal representation of this interval. Hence, the higher the clock rate, the finer the temporal resolution of the internal clock will be, which is equivalent to more accuracy and better performance on time perception tasks. However, some authors propose that temporal processing involves more than one timing mechanism. Many of these views are related to the idea that time judgements may depend directly on memory processes. Indeed, in general, the role of cognitive mediation on time estimation has been amply documented (for reviews see Block, 1990; Levin and Zakay, 1989; Macar et al., 1992; Nichelli, 1993). Within the framework of prospective temporal processing, a distinction is sometimes made between estimation of time and perception of time (Fraisse, 1984; Rammsayer, 1994a). According to Fraisse (1984), perception of time is assumed to be mediated by experiential processes such as the psychological present (Fraisse, 1963; James, 1890; Michon, 1978). Estimation of time, on the other hand, is based on memory processes and refers to temporal processing of longer intervals in the range of seconds and above. From an information-processing point of view, Michon (1985) argued that, while temporal processing of shorter intervals is highly perceptual in nature and not accessible to cognitive control, temporal processing of intervals above 500 ms involves working memory. Indeed, the critical role of working memory on time estimation was recently reported by Fortin et al. (1993) who showed that concurrent non-temporal processing in working memory systematically lengthens temporal production of a 2-second interval. Likewise, Rammsayer and Lima (1991) found that temporal discrimination of 1-second auditory intervals is markedly impaired by an increased working-memory load while temporal discrimination of 50-millisecond intervals is unaffected by the same cognitive load. The notion of different mechanisms underlying temporal information processing is also supported by the outcome of pharmacopsychological studies. While a moderate dose of 0.65 g/kg ethanol does not interfere with temporal processing of one-second time intervals, the same dose of ethanol adversely affects processing of intervals in the range of milliseconds (Rammsayer, 1995; Rammsayer and Vogel, 1992). Additional evidence converging on this conclusion is provided by Mitrani et al. (1977). These authors showed that LSD and mescaline, both being substances that strongly affect time estimation of longer intervals (Cohen, 1966; Fischer, 1966), did not alter temporal discrimination of intervals shorter than one second. Both LSD and mescaline are well known for their cognitively mediated psychoactive effects, e.g. hallucinations (Barr et al., 1966; Longo, 1972). Therefore, Mitrani et al. (1977) concluded that long time intervals are processed at higher levels of the central nervous system, whereas brief intervals are processed almost automatically at some lower, subcortical, brain level. Finally, in a pharmacopsychological study on temporal information processing and memory (Rammsayer, 1994b), the benzodiazepine midazolam significantly reduced accuracy of temporal discrimination of intervals with a base duration of 1 s. Since midazolam produces marked deterioration of memory functions (Curran and Birch, 1991; File et al., 1992; Hennessy et al., 1991) and, furthermore, a highly significant correlation was found between performance on free recall and performance on time estimation under midazolam, decreasing task accuracy
Psychophysics of human timing
189
was attributed to midazolam-induced impairment of memory functions. Most interestingly, however, temporal processing of intervals in the range of milliseconds with a base duration of 50 ms has been shown to be resistant to the memory impairing effects of midazolam. The data presented so far, suggest that temporal processing of extremely brief intervals may depend on neural counting mechanisms rather than on cognitive processes. Most interestingly, Abel (1972a, b) showed that temporal discrimination of brief intervals below approximately 100 ms can be explained best by the assumption of neural counting mechanisms. Although the concept of an internal clock underlying temporal processing has been a central feature of many theoretical accounts of time perception (Allan, 1992), the neuroanatomical substrate of the assumed neural timing mechanism is still unknown. There is some evidence, however, that two subcortical structures, the cerebellum (Braitenberg, 1967; Ivry, 1993; Ivry and Keele, 1989; Keele and Ivry, 1991) and the basal ganglia (Artieda et al., 1992; Obeso et al., 1987; O’Boyle et al., 1996; Rammsayer, 1993, 1994a, 1997b; Rammsayer and Classen, 1997) may primarily be involved in timing functions. Finally, a more recent approach also provides converging evidence for the existence of neural networks in the brain mediating temporal information processing in the range of tens to hundreds of milliseconds. Simulation studies (Buonomano and Merzenich, 1995) applying a neural network composed of integrate-and-fire elements with realistic properties showed that these networks can represent any given interval between 30 and 300 ms. Taken together, there is convincing evidence for two distinct timing mechanisms involved in temporal information processing: One mechanism is based on workingmemory processes for timing of intervals in the range of seconds or above, and another one, which appears to be beyond cognitive control and, most likely, located at the subcortical level, is used for timing of extremely brief intervals in the range of milliseconds.
REFERENCES Abel, S.M. (1972a) Duration discrimination of noise and tone bursts. Journal of the Acoustical Society of America, 52, 1219–1223. Abel, S.M. (1972b) Discrimination of temporal gaps. Journal of the Acoustical Society of America, 52, 519–524. Allan, L.G. (1979) The perception of time. Perception and Psychophysics, 26, 340–354. Allan, L.G. (1992) News on the internal clock. International Journal of Psychology, 27, 21. Allan, L.G., and Kristofferson, A.B. (1974) Psychophysical theories of duration discrimination. Perception and Psychophysics, 16, 26–34. Allan, L.G., and Gibbon, J. (1991) Human bisection at the geometric mean. Learning and motivation, 22, 39–58. Artieda, J., Pastor, A., Lacruz, F., and Obeso, J.A. (1992) Temporal discrimination is abnormal in Parkinson’s disease. Brain, 115, 199–210. Barr, H.L., Langs, R.J., Holt, R.R., Goldberger, L., and Klein, G.S. (1972) Personality and Experience. New York: Wiley.
Time and the brain
190
Bindra, D., and Waksberg, H. (1956) Methods and terminology in the studies of time estimation. Psychological Bulletin, 53, 155–159. Block, R.A. (1989) Experiencing and remembering time: affordances, context, and cognition. In: I.Levin and D.Zakay (eds), Time and Human Cognition: A Life Span Perspective, Amsterdam: North-Holland, pp. 333–363. Block, R.A. (1990) Cognitive Models of Psychological Time. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Block, R.A., and Zakay, D. (1997) Prospective and retrospective duration judgments: a meta-analytic review. Psychonomic Bulletin and Review, 4, 184–197. Boltz, M.G. (1995) Effects of event structure on retrospective duration judgments. Perception and Psychophysics , 57, 1080–1096. Braitenberg, V. (1967) Is the cerebellar cortex a biological clock in the millisecond range? In: C.A. Fox and R.S.Snider (eds) The Cerebellum (Progress in Brain Research, vol. 25), Elsevier, Amsterdam, pp. 334–346. Brown, S.W. (1985) Time perception and attention: the effects of prospective versus retrospective paradigms and task demands on perceived duration. Perception and Psychophysics, 38, 115–124. Brown, S.W., and Stubbs, D.A. (1988) The psychophysics of retrospective and prospective timing. Perception, 17, 297–310. Buonomano, D.V., and Merzenich, M. (1995) Temporal information transformed into a spatial code by a neural network with realistic properties. Science, New York, 267, 1028–1030. Carlson, V.R., and Feinberg, Z. (1970) Time judgement as a function of method, practice, and sex. Journal of Experimental Psychology, 85, 171–180. Church, R.M. (1984) Properties of the internal clock. In: J.Gibbon and L.G.Allan (eds) Timing and Time Perception, New York: New York Academy of Sciences, pp. 566– 582. Clausen, J. (1950) An evaluation of experimental methods of time judgment. Journal of Experimental Psychology, 40, 756–762. Cohen, J. (1966) Subjective time. In: J.T.Fraser (ed), The Voices of Time. New York: Braziller, pp. 257–275. Creelman, C.D. (1962) Human discrimination of auditory duration. Journal of the Acoustical Society of America, 34, 582–593. Curran, H.V., and Birch, B. (1991) Differentiating the sedative, psychomotor and amnesic effects of benzodiazepines: a study with midazolam and the benzodiazepine antagonist, flumazenil. Psychopharmacology, 103, 519–523. Doob, L.W. (1971) Patterning of Time. New Haven, CT: Yale University Press. Drake, C., and Botte, M.-C. (1993) Tempo sensitivity in auditory sequences: Evidence for a multiple-look model. Perception and Psychophysics, 54, 277–286. Eisler, A.D., and Eisler, H. (1994) Subjective time scaling: influence of age, gender, and Type A and Type B behavior. Chronobiologia, 21, 185–200. Eisler, H. (1976) Experiments on subjective duration 1868–1975: A collection of power function exponents. Psychological Bulletin, 83, 1154–1171. Elbert, T., Ulrich, R., Rockstroh, B., and Lutzenberger, W. (1991) The processing of temporal intervals reflected by CNV-like brain potentials. Psychophysiology, 28, 648– 665. Fetterman, J.G., and Killeen, P.R. (1992) Time discrimination in Columbia livia and Homo sapiens. Journal of Experimental Psychology: Animal Behavior Processes, 18, 80–94.
Psychophysics of human timing
191
File, S.E., Skelly, A.M., and Girdler, N.M. (1992) Midazolam-induced retrieval impairments revealed by the use of flumazenil: a study in surgical dental patients. Journal of Psychopharmacology, 61, 81–87. Fischer, R. (1966) Biological time. In: J.T.Fraser (ed) The Voices of Time. New York: George Braziller, pp. 357–382. Fortin, C., Rousseau, R., Bourque, P., and Kirouac, E. (1993) Time estimation and concurrent nontemporal processing: specific interference from short-term-memory demands. Perception and Psychophysics, 53, 536–548. Fraisse, P. (1963) The Psychology of Time. New York: Harper and Row. Fraisse, P. (1984) Perception and estimation of time. Annual Review of Psychology, 35, 1–36. Gescheider, G.A. (1985) Psychophysics: Method, Theory, and Application. Hillsdale, NJ: Lawrence Erlbaum Associates. Getty, D.J. (1975) Discrimination of short temporal intervals: a comparison of two models. Perception and Psychophysics, 18, 1–8. Getty, D.J. (1976) Counting processes in human timing. Perception and Psychophysics, 20, 191–197. Gibbon, J. (1991) Origins of scalar timing. Learning and Motivation, 22, 3–38. Gilliland, A.R., Hofeld, J., and Eckstrand, G. (1946) Studies in time perception. Psychological Bulletin, 43, 162–176. Grondin, S. (1992) Production of time intervals from segmented and nonsegmented inputs. Perception and Psychophysics, 52, 345–350. Grondin, S. (1993) Duration discrimination of empty and filled intervals marked by auditory and visual signals. Perception and Psychophysics, 54, 383–394. Grondin, S. (1996a) When should one start explicit counting in time estimation? Paper presented at the 37th Annual Meeting of the Psychonomic Society. Chicago, Illinois. Grondin, S. (1996b) Discriminating intermodal time intervals. In: S.Masin (ed.) Fechner Day 96: Proceedings of the Twelth Annual Meeting of the International Society for Psychophysics. Padova, Italy: The International Society for Psychophysics, pp. 275– 280. Grondin, S., Ivry, R.B., Franz, E., Perreault, L., and Metthé, L. (1996) Markers’ influence on the duration discrimination of intermodal intervals. Perception and Psychophysics, 58, 424–433. Hancock, P.A., Arthur, E.J., Chrysler, S.T., and Lee, J. (1994) The effect of sex, target duration, and illumination on the production of time intervals. Acta Psychologica, 86, 57–67. Hennessy, M.J., Kirkby, K.C., and Montgomery, I.M. (1991) Comparison of the amnesic effects of midazolam and diazepam. Psychopharmacology, 103, 545–550. Henry, P.M. (1948) Discrimination of the duration of sound. Journal of Experimental Psychology, 38, 734–743. Hicks, R.E., Miller, G.W., and Kinsbourne, M. (1976) Prospective and retrospective judgements of time as a function of the amount of information processed. American Journal of Psychology, 89, 719–730. Hornstein, A.D., and Rotter, G.S. (1969) Research methodology in temporal perception. Journal of Experimental Psychology, 79, 561–564. Ivry, R.B. (1993) Cerebellar involvement in the explicit representation of temporal information. In: P.Tallal, A.M.Galaburda, R.R.Llinás and C. von Euler (eds) Temporal Information Processing in the Nervous System, New York: Annals of the New York Academy of Sciences, pp. 214–230.
Time and the brain
192
Ivry, R.B., and Hazeltine, R.E. (1995) The perception and production of temporal intervals across a range of durations: Evidence for a common timing mechanism. Journal of Experimental Psychology: Human Perception and Performance, 21, 3–18. Ivry, R.B., and Keele, S.W. (1989) Timing functions of the cerebellum. Journal of Cognitive Neuroscience, 1, 136–152. James, W. (1890) The Principles of Psychology (Vol. 1). New York: Henry Holt. Kaernbach, C. (1991) Simple adaptive testing with the weighted up-down method. Perception and Psychophysics, 49, 227–229. Keele, S.W., and Ivry, R.B. (1991) Does the cerebellum provide a common computation for diverse tasks: a timing hypothesis. In: A.Diamond (ed.) Development and Neural Basis of Higher Cognitive Function. New York: Annals of the New York Academy of Sciences, pp. 179–211. Killeen, P.R., and Weiss, N.A. (1987) Optimal timing and the Weber function. Psychological Review, 94, 455–468. Kinchla, J. (1972) Duration discrimination of acoustically defined intervals in the 1- to 8sec range. Perception and Psychophysics, 12, 318–320. Kristofferson, A.B. (1980) A quantal step function in duration discrimination. Perception and Psychophysics, 27, 300–306. Levin, I., and Zakay, D. (1989) Time and Human Cognition: A Life Span Perspective. Amsterdam: North Holland. Longo, V.G. (1972) Neuropharmacology and Behavior. San Francisco: Freeman. Luce, R.D., and Galanter, E. (1963) Discrimination. In: R.D.Luce, R.R.Bush and E.Galanter (ed.) Handbook of Mathematical Psychology. Vol. I. New York: Wiley, pp. 191–243. Macar, F., Pouthas, V., and Friedman, W.J. (1992) Time, Action and Cognition. Towards Bridging the Gap. Dordrecht: Kluwer Academic Publishers. McConchie, R.D., and Rutschmann, J. (1971) Human time estimation: on differences between methods. Perceptual and Motor Skills, 32, 319–336. Michon, J.A. (1978) The making of the present. In: J.Requin (ed.) Attention and Performance, Vol. 7. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 90–111. Michon, J.A. (1985) The compleat time experiencer. In: J.A. Michon and J.L. Jackson (eds) Time, Mind, and Behavior. Berlin: Springer, pp. 21–52. Mitrani, L., Shekerdjiiski, S., Gourevitch, A., and Yanev, S. (1977) Identification of short time intervals under LSD25 and mescaline. Activitas Nervosa Superior, 19, 103–104. Nakajima, Y., ten Hoopen, G.H., and van der Wilk, R.G.H. (1991) A new illusion of time perception. Music Perception, 8, 431–448. Nichelli, P. (1993) The neuropsychology of human temporal information processing. In: F.Boller and J.Grafman (eds), Handbook of Neuropsychology, Vol. 8. Amsterdam: Elsevier Science Publishers, pp. 339–371. O’Boyle, D.J., Freeman, J.S., and Cody, F.W. (1996) The accuracy and precision of selfpaced, repetitive movements in subjects with Parkinson’s disease. Brain, 119, 51–70. Obeso, J.A., Lacruz, F., Artieda, J., Pastor, A., and Martinez-Lage, J.M. (1987) Temporal discrimination of paired electrical stimuli is abnormal in Parkinson’s disease. Neurology, 37, (Suppl. 1), 281–282. Pentland, A. (1980) Maximum likelihood estimation: The best PEST. Perception and Psychophysics, 28, 377–379. Rammsayer, T.H. (1992a) An experimental comparison of the weighted up-down method and the transformed up-down method. Bulletin of the Psychonomic Society, 30, 425– 427.
Psychophysics of human timing
193
Rammsayer, T. (1992b) Effects of benzodiazepine-induced sedation on temporal processing. Human Psychopharmacology, 7, 311–318. Rammsayer, T.H. (1993) On dopaminergic modulation of temporal information processing. Biological Psychology, 36, 209–222. Rammsayer, T. (1994a) A cognitive-neuroscience approach for elucidation of mechanisms underlying temporal information processing. International Journal of Neuro science, 77, 61–76. Rammsayer, T.H. (1994b) Temporal information processing and memory. In: L.M. Ward, Fechner Day 94. Proceedings of the Tenth Annual Meeting of the International Society for Psychophysics, Vancouver, Canada: The International Society for Psychophysics, pp. 48–53. Rammsayer, T. (1994c) Effects of practice and signal energy on duration discrimination of brief auditory intervals. Perception and Psychophysics, 55, 454–464. Rammsayer, T. (1995) Extraversion and alcohol: Eysenck’s drug postulate revisited. Neuropsychobiology, 32, 197–207. Rammsayer, T.H. (1996) Why does time pass more quickly for women than men? Sexrelated differences in temporal information processing. In: I.Mervielde (ed.) 8th European Conference on Personality-Abstracts, p. 158. Ghent, Belgium: Academia Press. Rammsayer, T.H. (1997a) On the relationship between personality and time estimation. Personality and Individual Differences, 23, 739–744. Rammsayer, T.H. (1997b) Are there dissociable roles of the mesostriatal and mesolimbocortical dopamine systems on temporal information processing in humans. Neuropsychobiology, 35, 36–45. Rammsayer, T.H., and Classen, W. (1997). Impaired temporal discrimination in Parkinson’s disease: temporal processing of brief durations as an indicator of degeneration of dopaminergic neurons in the basal ganglia. International Journal of Neuroscience, 91, 45–55. Rammsayer, T.H., and Lima, S.D. (1991) Duration discrimination of filled and empty auditory intervals: Cognitive and perceptual factors. Perception and Psychophysics, 50, 565–574. Rammsayer, T., and Vogel, W.H. (1992) Pharmacologic properties of the internal clock underlying time perception in humans. Neuropsychobiology, 26, 71–80. Rousseau, R., Poirier, J., and Lemyre, L. (1983) Duration discrimination of empty time intervals marked by intermodal pulses. Perception and Psychophysics, 34, 541–548. Skrandies, W., and Rammsayer, T. (1995) The perception of temporal structure and auditory evoked brain activity. Biological Psychology, 40, 267–280. Smith, N.C. (1969) The effect on time estimation of increasing the complexity of a cognitive task. Journal of General Psychology, 81, 231–235. Stevens, S.S. (1975) Psychophysics: Introduction to Its Perceptual, Neural and Social Prospects. New York: Wiley. ten Hoopen, G.H., Hilkhuysen, G., Vis, G.Nakajima, Y., Yamauchi, F., and Sasaki, T. (1993) A new illusion of time perception—II. Music Perception, 11, 15–38. ten Hoopen, G., Hartsuiker, R., Sasaki, T., Nakajima, Y., Tanaka, M., and Tsumura, T. (1995) Auditory isochrony: time shrinking and temporal patterns. Perception, 24, 577– 593. Thomas, E.A.C., and Brown, I. (1974) Time perception and the filled-duration illusion. Perception and Psychophysics, 16, 449–458. Treisman, M. (1963) Temporal discrimination and the indifference interval: implications
Time and the brain
194
for a model of the “internal clock”. Psychological Monographs, 77, 1–31. Treisman, M., Faulkner, A., Naish, P.L.N., and Brogan, D. (1990) The internal clock: evidence for a temporal oscillator underlying time perception with some estimates of its characteristic frequency. Perception, 19, 705–743. Vierordt, K. (1868) Der Zeitsinn nach Versuchen. Tübingen: Laupp. Wallace, M., and Rabin, A. (1960) Temporal experience. Psychological Bulletin, 57, 213–235. Wearden, J.H. (1991) Do humans possess an internal clock with scalar timing properties? Learning and Motivation, 22, 59–83. Williams, J.M., Medwedeff, C.H., and Haban, G. (1989) Memory disorder and subjective time estimation. Journal of Clinical and Experimental Neuropsychology, 11, 713–723. Zakay, D. (1990) The evasive art of subjective time measurement: some methodological dilemmas. In: R.A.Block (ed.) Cognitive Models of Psychological Time, Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 59–84. Zakay, D. (1993) Time estimation methods—do they influence prospective duration estimates? Perception, 22, 91–101.
6 Cortical Processing by Fast Synchronization: High Frequency Rhythmic and Non-rhythmic Signals in the Visual Cortex Point to General Principles of Spatiotemporal Coding Reinhard Eckhorn Department of Physics, NeuroPhysics Group, Philipps-University, Renthof 7. D35032 Marburg, Germany Tel: (0049) 6421–2824–164 FAX: (0049) 6421–2827–034 e-mail: [email protected]
Synchronized signals are functionally important because they can activate a neurone more effectively than an uncorrelated input, especially if the input’s correlation is over a time span not broader than the temporal window of the neurone’s integration. In active cortices, integration times can be in the range of milliseconds, so that precisely synchronized signals seem important for cortical processing. We analysed such high frequency activities in visual cortex of cats and monkeys by multiple microelectrode recordings. Transiently changing scene segments were represented by synchronized population activities that were often non-rhythmic and phase-coupled to visual stimuli. During sustained activation, as in periods of stable retinal input, activities were more oscillatory (30–100 Hz) and dominated by cortical dynamics. Such oscillations were synchronized within a vertical column across cortical layers, among different columns of the same cortical area within some millimeters (“synchronization field”), and between different cortical areas amongst neurones with neighbouring receptive fields. Oscillation frequencies were highly variable, while average phase-delays, even in separate locations, were narrowly distributed around zero. Experimental data and related computer simulations support a comparably simple explanation: Fast oscillations are generated during states of sustained activation in local populations via local feedback inhibition, while distributed populations are synchronized via mutual facilitatory coupling. Synchronization at near-zero phase can be explained by common input from shared cortical and subcortical sources. The width of cortical synchronization fields with oscillations at zero-phase delay can be explained by temporal properties of cortical circuits shaped by Hebbian learning. Since the synchronization of fast cortical oscillations depends on specific grouping of visual features, its role in scene segmentation, object definition, and other more general association processes are discussed. Finally, it is argued how basic visual processing operations might be carried
Cortical processing by fast synchronization
197
out in a stepwise fashion by a modular circuit derived from interactions among simple and complex cells in primary visual cortex. KEYWORDS: Cortex; Gamma oscillations; Cortico-cortical interactions; Population coding; Perceptual grouping; Association field
1. INTRODUCTION 1.1. Synchronized Input Excites Neurones Strongly and Precisely 1.1.1. Definition of “Integration time” and “Synchronization” in the present context The term synchronization will be used with respect to the duration of a single neurone’s integration time, which is mainly given by the time course of postsynaptic influences in response to an input spike. To a first approximation we might think of the integration time as the half-height duration of an excitatory postsynaptic potential (EPSP) at the neurone’s spike encoder. In cortical neurones these durations span a broad range from about 2 up to 100 milliseconds. Integration times depend on the site and type of synapses, on the different dendro-somatic membranes in different types of neurones, but also on dynamic changes in membrane time constants of single neurones (Häusser and Roth, 1997). One example of variability in integration time is the prolongation of EPSPs from NMDA synapses with increasing levels of membrane depolarization (Fox and Daw, 1992), while another example is the shortening of PSPs with increase in a neurone’s average synaptic activation (Agmon-Snir and Segev, 1993; Nelson, 1994). In the activated awake cortex in which we are interested in this paper dominant integration times are estimated in the range 2 to 10 ms (e.g. König et al., 1995a), where the shorter time constants will be found in “low impedance locations” with several neighbouring synapses activated in parallel, while longer ones will occur at “high impedance locations” when only a single synapse is activated. Neural signal components covarying, with the same sign, during such integration windows will be called here “synchronized”. Synchronized events may occur singly, or repetitively with a more stochastic or rhythmic character. With respect to the processing of behavioural output, this synchronization range of 2 to 20 ms is rather short and we will therefore use the term “fast” for it. Accordingly, neural signals which are correlated positively in this range will be called “fast synchronized”. In this sense, rhythmic signals with half cycle durations of 5 to 10 ms (100 to 20 Hz) will be termed synchronized fast cortical oscillations (FCOs).
Time and the brain
198
1.1.2. Influence of synchronized input on the spike encoder The spike encoder of a neurone, including its threshold and refractory dynamics, favours synchronized compared to unsynchronized inputs, because the former produce higher and more steeply depolarized membrane excursions at equal numbers of input spikes. Output spikes will be generated at higher probability if a neurone’s excitatory inputs are activated in concert within its preferred integration window (e.g. Abeles 1991; Murthy and Fetz 1994; Bernander et al., 1995). If input spike patterns are even precisely coinciding, a spike encoder with short refractory period and appropriate threshold can extract this pattern and send it to other targets. In contrast, more dispersed input spike patterns, including statistically independent ones with the same number of spikes, will result in lower momentary excursions of membrane potentials at the spike encoder and hence, in a lower influence on spike firing (Abeles, 1991). 1.2. Potential and Effective Connectivity The brain is a highly connected structure, and it is generally agreed that its rich connectivity and special topography are the main reasons for its astonishing capabilities in sensory, motor, and cognitive processing (e.g. Braitenberg and Schüz, 1991). In recent decades, systems neuroscientists have therefore concentrated on the connectionistic properties of the brain. However, the brain’s wiring provides only a potential network of processing, while an actual task dynamically recruits a rather low percentage of neurones and connections, characterized by their effective connectivity. This is explained by the fact that any connection is functionally present only while a presynaptic spike influences the postsynaptic cell. This period may be about 10 to 100 ms at low activation levels but may be even shorter with stronger inputs (Agmon-Snir and Segev, 1993; Nelson, 1994). In addition, spike rates of cortical neurones are low without special activations, ranging from silence in many neurones to 1–3 per second in spontaneously active cells. These low rates are often suppressed by nearby activated neurones. Hence, it is argued that cortical processing relies, at each moment, on a small percentage of active neurones, and on the fact that the effective network configuration can change dynamically within milliseconds, due to specific activations. Indeed, rapidly-changing effective connectivity has been shown repeatedly in the past on the basis of multiple microelectrode recordings combined with cross-correlation techniques (e.g. Aertsen et al., 1989; Schneider et al., 1983; Eckhorn and Obermueller, 1993). In the present chapter we focus on the causes, mechanisms, and possible functional relevance of cortical synchronization dynamics. We will report on our observations of temporally precise correlations of cortical population signals, in particular of synchronized fast oscillations (FCOs) in the gamma range (30 to 100 Hz) and on high frequency stimulus-locked non-rhythmic synchronization. This includes discussion about neural codes—from precise spike pattern coding to spike rate coding—and the formation of neural assemblies defined by simultaneous activation of interacting neurones. As our experience is based on multiple microelectrode recordings from the visual cortex of
Cortical processing by fast synchronization
199
anaesthetized cats and awake monkeys, we will discuss the potential relevance of fast synchronized signals for visual perception, but generalizations to other systems will also be developed. Finally, we will try to explain the observed cortical dynamics with simulations including simple neural mechanisms and circuits.
2. FAST SYNCHRONIZED SIGNALS IN THE VISUAL CORTEX– RECORDED BY MULTIPLE MICROELECTRODES 2.1 Synchronized Components in Population Activities Our empirical evidence is based on multiple microelectrode recordings from the visual cortex of anaesthetized cats and awake monkeys. We have used local population activities (multiple unit spike activity (MUA) and local slow wave field potentials (1 to 150 Hz) (LFP)) for the analysis of cortical synchrony, because they comprise the synchronized components of local populations (e.g. Eckhorn, 1992). In particular, LFPs are a local weighted average of the dendro-somatic postsynaptic signals, reflecting mainly the synchronized components at the inputs of the population within (approximately) 0.5 mm of the electrode tip (Mitzdorf, 1987). MUAs, on the other hand, comprise, in their amplitudes, the simultaneity of spikes occurring at the outputs of a local population within (approximately) 0.05 mm of an electrode tip (Legatt et al., 1980; Gray et al., 1995). Both population signals are more suitable for detecting correlations among dispersed cortical recording locations than are single unit spike trains (Eckhorn et al., 1988, 1990; Eckhorn, 1992; Gray et al., 1989, 1990; Engel et al., 1990, 1991). Higher numbers of neurones contribute to LFP than to MUA, due to their shallower spatial decay (Legatt et al., 1980; Mitzdorf, 1987). In addition, the cortical ranges where FCOs occur with near zero phase delay is some millimeters, so that LFP data consist of superimposed signals from large numbers of neurones participating in the same oscillatory state. In the visual cortex, the main contribution to LFPs is from neurones involved in the same functional task, because neurones with functional similarity, activated by the same stimulus feature, are grouped in clusters (e.g. Gilbert, 1993). In conclusion, this superposition results in lower variability of LFP-FCOs for identical stimulus repetitions, and a higher number of significantly correlated LFP pair recordings from separate positions, compared to MUA. 2.2. Sustained Activation is Required for the Generation of FCOs A general condition for the occurrence of FCOs was sustained activation, with average spike rates typically larger than 10 impulses per second, and the absence of any fast excitatory or inhibitory response transients (Kruse and Eckhorn, 1996). This condition was particularly given with features fitting the classical receptive field (cRF) properties of the respective neurones. For example, stimulation with an oriented, one-dimensional, luminance grating, activated in a sustained manner those neurones preferring
Time and the brain
200
approximately the spatial frequency, orientation and contrast of that grating. Slow temporal modulations (below 5 Hz) also led to FCOs, which were modulated in their probability of occurrence and amplitude, by the slow temporal modulation. Such states, with sustained activation, are typical for situations in which gaze is nearly stabilized on a visual object (for more details see section 3.). Model investigations (Juergens and Eckhorn, 1997; and below) supported the experimental observation that the generation of FCOs depends on a sustained activation of a local network of coupled neurones. 2.3. Single Neurones are Differently Involved in Synchronized FCOs Three types of coupling dynamics were observed between single cell spikes and oscillatory population activity (LFP and MUA) in cat (Eckhorn and Obermueller, 1993) and monkey visual cortex (Frien et al., 1994) as well as in related neural network simulations (Juergens and Eckhorn, 1997): 1. In rhythmic states, single cell spike patterns had significant rhythmic modulation, and spikes were significantly correlated with oscillatory population activity; 2. In lock-in states, rhythmic modulation was not present in single cell spike patterns, while spikes were significantly correlated with oscillatory population activity; and 3. In non-participation states, rhythmic modulation was absent in spike trains, and in addition the spikes were not correlated with the actually present oscillatory population activity (Figure 1). State transitions of single cell coupling with oscillatory population activities depended on stimulation and receptive field properties. For example, a weak lock-in state changed into a strong synchronized rhythmic state in a cortical (directionally sensitive) neurone when reversing the movement direction of an oriented contour stimulus (Eckhorn and
Cortical processing by fast synchronization
201
Figure 1. Three different states of single cell couplings with oscillatory population activities in the primary visual cortex, a: rhythmic; b: lock-in, and c: non-participation states of three different neurones. AC: auto-correlation histograms of single cell spikes (SUA), of multiple unit activity (MUA), and of local field potential (LFP). STA denotes spike-triggered averages of multiple unit activity or local field potentials. According to the classification, STAs have oscillatory modulations in the rhythmic and lock-in states, and lack such modulation in the non-participation state. Note that in the rhythmic state (a) the single cell correlogram (top) is clearly modulated at 44 Hz, while in the lock-in (b) and the non-participation states (c) rhythmic modulations in the range 35–80 Hz are not visible (by definition). Lowest row of panels: power spectra for the above row of correlograms. (Figure modified from Eckhorn and Obermueller, 1993).
Obermueller, 1993). Stimulation in the preferred direction caused higher levels of average sustained activation (also in other neurones of the same directional preference), and this resulted in rhythmic synchronized states. 2.4. Frequency and Amplitude of FCOs are Highly Variable The oscillation frequencies and amplitudes of FCOs are probably not coding sensory features, because they are highly variable. Variability of the dominant frequency of FCOs is typically varying in the range over 20 Hz with constant stable stimuli. Such variations were found in several visual cortical areas of anaesthetized cats, as well as in awake monkeys, during the execution of perceptual tasks (Eckhorn et al., 1988; Gray et al., 1989; Eckhorn and Schanze, 1991; Kreiter and Singer, 1996; Frien et al., 1994; review in Gray, 1994). A general observation was that the frequency within a single oscillation spindle (typical duration being 80 to 200 ms) was more constant than in successive
Time and the brain
202
spindles. However, we observed some systematic changes in the average oscillation frequency with stimulation, ranging from 20 to 100 Hz, which is described in the next paragraph. 2.5. Average Oscillation Frequency and Amplitude of FCOs Depend on Visual Stimulation Even though the variability of FCO frequencies and amplitudes was high during stationary visual stimuli, they showed some systematic dependency of their average values on stimulation. In general, with increasing amplitudes of cortical activation, we found increasing FCO amplitudes accompanied by decreasing frequencies. When neurones were slightly activated above their “spontaneous” rate, incoherent broad band stochastic signals predominated, with some short and low amplitude FCOs of high frequency (<70 Hz). These oscillatory epochs seldom occurred coherently across distances of more than 1 mm cortex. With intermediate activation levels, domains of synchronized FCOs were on average broader, their amplitudes higher and their frequencies lower (50 to 60 Hz). Strong sustained activation resulted in high amplitudes of population activity at low frequencies (30 to 40 Hz) and the synchronized domains became larger, spanning 4 to 6 mm in V1 and V2 cortex. Figure 2c shows the continuous transition between the different signal types in a recording from monkey visual cortex, in which the contrast of a visual stimulus was continuously increased. The velocity and spatial extent of a visual stimulus also influenced the average oscillation frequency of FCOs (Eckhorn et al., 1989; Eckhorn and Frien, 1995). When visual stimuli moved slower, or were larger in extent, FCOs in the visual cortex of awake monkeys had on average lower oscillation frequencies and higher amplitudes (Figure 2a,b). We can explain the lower frequencies observed with larger stimuli by the longer average activation delays within the larger populations (for details see section 3.8). However, lower frequencies may also be due to the increased activation levels, with more extended stimuli leading to increased inhibition periods in the local populations. The influence of stimulus velocity on FCOs might have corresponding reasons. Slow velocities allow the formation of larger assemblies engaged in lower frequency activity than for faster movements. 2.6. Zero Average Phase Delay Among Synchronized FCOs Average phase differences between FCOs at separate locations of the same visual cortical area (e.g. V1 or V2), and among different areas, were found to be narrowly distributed around zero (Eckhorn et al., 1988; Gray et al., 1989; Engel et al., 1991b). Zero phase shift was commonly found, even among V1 and V2 of monkeys (Frien et al., 1994) where we expected this even less, because these areas are known as serially arranged in the visual processing stream, and thus suggesting a delay of V2 against V1 oscillations. This result seemed at first counterintuitive because the average conduction velocity of cortical axons is relatively slow (Swadlow, 1983; Nelson et al., 1992; Miller, 1994; Nowak et al. 1995b). In V1 and V2 of cat and monkey the velocity in horizontally
Cortical processing by fast synchronization
203
projecting axons can be estimated roughly at 1 m/s which is a value that can lead to clearly measurable phase shifts within the cortical distances measured by us (1 to 6 mm; Jordan, 1989; Brosch et al., 1995; Juergens et al., 1996). To find a plausible explanation we want to mention that non-rhythmic signals, occurring in epochs between FCOs, and before visual activation at the same electrode positions, can also show peaks in their cross-correlograms at zero-delay (Frien et al., 1994). Zero delay non-rhythmic correlation was also observed in V1 and V2 of anaesthetized cats (Nelson et al., 1992; Nowak et al., 1995a,b). We argue that single central peaks in cross-correlograms are indicative of inputs from a common source, with the same activation delay between source and cortical recording positions, because such common inputs guarantee zero delay independent of the type of signal (stochastic or rhythmic) and its frequency range (low or high)(see also Nowak and Bullier, this volume).
Figure 2. a-d. Dependencies of average oscillation frequencies on stimulus size, velocity and contrast in the primary visual cortex of an awake monkey, a: The average oscillation frequencies of LFPs shifted to lower values accompanied by higher amplitudes if the length of a moving stimulus (light bar) was elongated or if its width was broadened, or b: if its velocity was decreased. Frequency changes with size were restricted to the spatial range in which significant values of coherence were observed, c: Average oscillation frequency of FCOs decreased with stimulus contrast (20 consecutive MUA recordings from the same experiment), d: Linear regression fits to oscillation frequencies of LFP-data from the same experiment (c) during increase in stimulus contrast. (Figure 2a ,b and d modified from Woelbern et al., 1994; Figure 2c: unpublished from a.)
Time and the brain
204
2.7. Cortical Synchronization Range of FCOs The average range in which synchronized FCOs occurred in primary and secondary visual cortex areas V1 and V2 of cat and monkey was 4 to 6 millimeters (Jordan, 1989; Brosch et al., 1995; Frien et al., 1996; Juergens et al., 1996; König et al., 1995b). This was established by recordings with linearly arranged arrays of equally spaced microelectrodes and the calculation of coherence among FCOs at different separations. Spatial decline of coherence was less steep in positions where neurones represented similar visual features (for example for similar local orientation of a contrast border) compared to those with dissimilar features. Within the synchronization range, phase delays of local FCOs were narrowly distributed around zero (with a standard deviation of typically 1 ms). Beyond that range, covariance declined to noise levels (Figure 3). 2.8. Are Adjacent Visual Objects Represented by Phase-Shifted or by Uncorrelated FCOs? A visual scene generally consists of many segments that can be discriminated perceptually. If we follow the above mentioned “synchronization hypothesis”, adjacent or overlapping segments of different visual objects should be coded by temporally separable FCOs (or, more generally, by any other type of uncorrelated signals). Discrimination would be possible if, for example, the neurones representing adjacent objects are
Cortical processing by fast synchronization
205
activated at different relative phase shifts at the same frequency or at different uncorrelated frequencies. To our knowledge, there are only few experimental examples of uncorrelated oscillations mat were induced by two adjacent stimulus objects (Gray et al., 1990; Kreiter and Singer,
Time and the brain
206
Figure 3. Coherence of FCOs declines with cortical distance. Local field potentials (LFP, 35–80 Hz) recorded in primary visual cortex of awake monkey with linear multiple µ-electrode array from upper cortical layers. Horizontal bars are proportional to relative frequency of occurrence. Continuous line indicates mean, dotted standard deviation, of coherence. High values of coherence were obtained with similar, low with different, receptive field properties in the recording locations. (Figure modified from Juergens et al., 1996).
1996; Brosch et al., 1997) and not a single example of a phase difference at the same oscillation frequency. We have obtained preliminary experimental evidence that adjacent objects (or an object against a background) do not lead to phase-shifted FCOs (at the same frequency) in their cortical representations, but to statistically independent FCOs (Figure 4). We could show that the same local populations became engaged in synchronized FCOs if they were activated by a single coherent stimulus, but decoupled their FCOs when an object’s boundary separated the populations. In this case, the frequencies became different (typically higher in the neurones representing the smaller of two stationary objects; for influences on average FCO frequency see section 2.5). 2.9. Suppression of FCOs by Fast Stimulus-Locked Activations FCOs of high amplitude were generally induced in visual cortex during epochs of slow drift rates of retinal images, when cortical cells were activated by sustained thalamic inputs. In contrast, fast transient components were primarily evoked by stimulus epochs of changing high velocity and direction. This was extensively studied in the visual cortical areas V1 and V2 of anaesthetized cats (Kruse and Eckhorn, 1996). With
Cortical processing by fast synchronization
207
increasing amplitudes of fast transient stimulus movements, FCOs were gradually reduced in amplitude, and were finally fully suppressed (Figure 5). Related observations were recently made in awake monkeys (Guettler et al., 1996). Rapid changes of the retinal image due to (micro) saccades or sudden changes of an object in its contrast or position could immediately disturb ongoing oscillations. The latter were, as in the cat, generated only during states of stable or slowly moving retinal images.
Figure 4. Figure-ground segregation defined by the reduction of the coherence of FCOs across object contours. LFP recordings (35–80 Hz) in 3 mm lateral separation from upper layers of V1, while the monkey kept fixation within 0.5°. The measure of signal coupling utilized was similar to spectral coherence, which was calculated with the dominant oscillations in the range 35–80 Hz. The stimulus, a sinusoidal grating (the “background”: spatial period 0.7° visual angle), was presented stationary for 0.8 s and then moved perpendicular to its orientation at a velocity of 0.65°/s while a patch of it (the “object”: 1° x 4° visual angle) remained stationary. RF1 and RF2 (left upper display) indicate the cRF positions of the recording pairs in- and outside the “object”. Since the contour of the object was only visible when the gratings were out of phase, we were able to examine a dynamical figure-ground separation. (Modified from Guettler et al., 1997).
The occurrence of strong stimulus-locked components in cortical responses per se, is not a sufficient explanation for the observed partial or full suppression of fast oscillations. In principle, stimulus-locked responses and oscillations might superimpose and coexist independently without major interactions. Instead, we found strong suppression of oscillations by stimulus-locked responses (Figure 5). What are probable explanations of
Time and the brain
208
this suppression? (1) Inhibition of oscillation circuits by transient afferent activities. This possibility is supported by the finding that the fast stimulus-locked components, being preferentially transmitted via the magnocellular (transient type) afferents, are particularly effective in evoking fast inhibitory responses in the visual cortex (Douglas and Martin, 1991). (2) Disturbance of oscillations by pushing the phase of oscillatory circuits. From theories (Kuramoto, 1991), and simulations of loosely coupled oscillators (Eckhorn et al., 1990; Schuster and Wagner, 1990; Schillen and König, 1994; Sompolinsky et al., 1990; Gerstner et al., 1993) it is known that oscillations can only be maintained, if strong out-of-phase inputs are prevented. However, such out-of-phase disturbances were delivered to the cortex in the investigation of Kruse and
Figure 5. Perturbation of synchronized fast cortical oscillations by stimuluslocked synchronized signals. Note that stimulus-locked responses inhibit synchronized oscillations. Local field potential recordings from cat visual cortex. Stimulus: grating, moving with a random component superimposed on a constant slow velocity. Open symbols: average normalized response power of the high-frequency oscillatory components; filled symbols: normalized power of the stimulus-locked components. Amplitudes (standard deviation) of the random stimulus movement are indicated at the abscissa. Each value is the average of N=104 single curves (modified from Kruse and Eckhorn, 1996).
Eckhorn (1996) by the stimulus-locked afferent input, because its transient response components generally occurred at random times relative to the phases of the oscillations.
Cortical processing by fast synchronization
209
With intermediate levels of stimulus-locked activity, oscillations were only partially suppressed. This can also be explained by deteriorations of oscillatory processes by random out-of-phase inputs. Theoretically one would expect a broadening and reduction of the dominant frequency peak of oscillations in the power spectrum due to an increased phase jitter and a reduced amplitude of local oscillatory processes. This has, indeed, been observed (Eckhorn and Schanze, 1991; Kruse and Eckhorn, 1996). (3) Suppression of oscillations by transient reductions of membrane resistances due to strong stimulus locked activations. If strong phase-locked input activates a considerable number of excitatory and inhibitory synapses of “oscillating” neurones nearly simultaneously, this may transiently lead to massive reductions in membrane resistances of dendrites and somata (Agmon-Snir and Segev, 1993; Nelson, 1994). Consequently, the decay of postsynaptic potentials becomes faster, leading to a lower level of temporal superposition so that the membrane potentials are reduced in amplitude. In conclusion, our observations support the assumption that FCOs are generated as long as the afferent visual activation is sustained and lacks fast transients. Suppression of oscillations by random stimulus-locked responses may well be due to transient activations of inhibitory interneurones, or to out-of-phase disturbances or reductions of membrane potentials in neurones participating in the oscillations. However, the presently available knowledge does not provide enough information to decide whether these or other still unknown mechanisms can explain the observed effects. 2.10. Synchronized FCOs Occur Primarily During States of “Desynchronized” EEG FCOs were extensively recorded by several researchers with intracortical microelectrodes in the visual system (cited above). In addition, much work has recently been published about recordings of FCOs in other sensory modalities, as well as in the motor system (Adrian, 1950; Barth and McDonald, 1996; Baker et al., 1997; Bragin et al., 1995; McKay and Mendoca, 1995; Murthy et al., 1994, 1997a,b; Rougeul et al., 1979; Sanes and Donoghue, 1993; Steriade et al., 1996; Traub et al., 1996). This growing body of experimental data supports the hypothesis that the occurrence of synchronized FCOs is associated with cognitive processing requiring awake attentive states. In particular, if the reticular formation was stimulated electrically, mimicking arousal, FCOs occurred at an increased probability even in lightly anaesthetized cat (Munk et al., 1996). However, EEG recordings resembled generally “desynchronized” activity in such states characterized by broad band random signals of low amplitudes (compared to the high amplitude low frequency activity during sleeping stages). Hence, it seems as if FCOs either do not show up in EEGs, or that humans do not generate synchronized FCOs (Juergens et al., 1999; but see Tallon Baudry et al., 1997). We believe that the former is the case, mainly because the cortical field over which synchronized FCOs occur in anaesthetized cats and awake monkeys was restricted to less than 6 mm. However, EEG signals recorded by standard electrodes on the skull are superpositions of cortical activity
Time and the brain
210
over a range of at least 40 mm. In this range several pools of uncorrelated FCOs superimpose their activities. It seems probable that they largely average out because of their statistical independence.
3. POTENTIAL RELEVANCE OF SYNCHRONIZED FCOs FOR VISUAL PROCESSING 3.1. Preattentive Scene Segmentation is a Prerequisite for Object Recognition Stimuli composed of coherent features are integrated by our sensory systems into perceptual entities, even if the features are dispersed among different sensory modalities. We can perceive a sensory object as a perceptual whole even if various aspects of the object are occluded, obscured by the background, or are not present at all. The visual system can easily detect coherencies in an object’s local stimulus features, and is able to link, intensify and isolate them. These capabilities of grouping, mutual facilitation, and figure-ground separation require neural mechanisms for self-organization that are able to construct reliable and unique percepts out of ambiguous sensory signals. Early processing of these relations is highly important, because the infinite number of possibilities in which even simple “toy scenes” can be arranged has to be reduced substantially before visual associative memories can solve tasks of recognition. 3.2. Binding of Spatially Dispersed Visual Features by Sychronized FCOs? It has been proposed that binding of spatially distributed features is based on synchronized fast excitations of those neurones belonging to an object’s neural representation. More precisely, neurones participating in the processing and representation of a visual object engage in a common synchronized state in response to stimulation by that object (Milner, 1974; Reitboeck, 1983). This hypothesis attracted attention when stimulusspecific synchronized oscillations of 35–90 Hz were found in the visual cortex of anaesthetized cats (Eckhorn et al., 1988; Gray et al., 1989) and awake monkeys (Kreiter and Singer, 1992; Eckhorn et al., 1993). In addition, synchronized non-oscillatory signals were also proposed to support feature binding, including synchronization due to phase-locking to transient stimulus events. The latter effects were investigated in cortical recordings (Frien et al., 1994; Kruse and Eckhorn, 1996) and have been modelled (Eckhorn et al., 1990). Finally, the first experimental support for the validity of the feature linking hypothesis was recently obtained from the occurrence of precisely synchronized signals correlated with perceptual processes of monkeys and cats, respectively (spike coherence: Vaadia et al., 1995;—FCO coherence: Kottmann and Eckhorn, 1996; Fries et al., 1997).
Cortical processing by fast synchronization
211
3.3. Sharper Orientation Tuning with Amplitudes of Synchronized FCOs Compared to Spike Rates In recent years the debate about neural codes has intensified with the finding of FCOs and the hypothesis that fine temporal structure might code important information, in addition to rate coding. However, quantitative comparisons between temporal and rate coding have seldom been made for the same set of data. We have recently obtained recordings in which the sharpness of orientation tuning was measured in V1 of awake monkeys, using the common method of spike rates that were evoked by stimulus contours of different orientations, this being compared with FCO amplitudes of the same responses. We found that the width of orientation tuning with FCOs was considerably sharper compared to the spike count measure (Figure 6). Perception of contour orientation is much more sensitive than the orientation characteristics with rate codes would predict. In this case, at least, the use of FCOs for orientation discrimination seems therefore to be more effective than the use of a rate code. In addition, we have studied the coupling strength among FCOs generated in pairs of spatially separate groups of neurones, according to their cortical distance and the angle between their preferred stimulus orientations (Figure 7a; Frien et al., 1996). We found that the degree of coherence (synchronization) of spike activities (MUA) depended strongly on stimulus orientation relative to the preferred orientations at the recording sites. If stimulus orientation and orientation preference at the two sites were similar, the strongest and most far-reaching coupling was found. Synchronization was less strong if the preferred orientations differed. In these cases the strongest coupling values were obtained with stimulation at half the angle between the preferred ones at the two sites. It is worth mentioning that the low frequency signals simultaneously present in these recordings did not show the orientation specific synchronization effects (Figure 7b). In conclusion, neurones coding the local visual feature “contour orientation” mutually synchronize by a fast mechanism, which probably means by using fast axonal links.
Time and the brain
212
Figure 6. Orientation tuning with fast cortical oscillations is sharper than with spike rates of identical neurones. OI indicates the orientation index, resembling the “sharpness” of orientation. Microelectrode recordings of multiple unit activity (MUA, 1–10 kHz) from the upper layers of the primary visual cortex of an awake monkey.
3.4. Synchronized FCOs and Visual Perception During Binocular Rivalry The most convincing argument in favour of the proposed functional role of synchronized FCOs in visual perception would be a direct demonstration of their specific occurrence correlated with perception. We began experiments with trained monkeys that can directly show such correlation (Kottmann and Eckhorn, 1996). This is possible with stimuli that can be perceived alternatively in two different manners with the monkey giving an indication of the actual percept (by pressing a key). We used a paradigm of binocular rivalry in which either a horizontally or vertically oriented Gabor grating (visual stimulus of sinusoidal luminance distribution, windowed by a concentric Gaussian) was perceived while both were always presented (e.g. the visual stimulation remained the same in each trial). Our hypothesis of “feature-linking-by-synchronization” would gain support if synchronized FCOs would occur in those neurones representing the stimulus presently perceived, and not in the neurones representing the other possible percept. Indeed, our preliminary data support our hypothesis: neurones representing the orientation of the
Cortical processing by fast synchronization
213
presently perceived grating synchronized their FCOs at a high probability, while synchronization of neurones representing the orientation at right angles was seldom observed.
Figure 7a,b. Synchronized signals among cortical orientation detectors. Normalized cross-spectra. Left: Orientation-specific synchronization at high oscillation frequencies (35–90 Hz). Right: In the low frequency range (1–35 Hz) of the same recordings no orientation specificity is present. Recordings from sites with similar orientation preferences from primary visual cortex (V1) of an awake monkey. The correlation (grey level) at different cortical distances with respect to a reference position (distance from centre of polar plot) is plotted for various orientations (angle of polar plot) of a large oriented sinusoidal grating stimulus. The recordings were obtained with a onedimensional linear array of micoelectrodes, its orientation of penetration was randomly selected; 0.75 mm inter-electrode separation. (Figure modified from Frien et al., 1996).
3.5. Feature Representation by Classical Receptive Fields (cRFs) and Feature Relations by Association Fields (AFs) We are extending the concept of classical receptive fields (cRFs) to association fields (AFs), in order to bring perceptual feature-linking into correspondence with the local representation of basic sensory features (Eckhorn et al., 1990). In the retina, the visual thalamus, and the first visual cortical areas basic visual features are characterized by the cRFs of single neurones, which are represented in retinotopic maps (for example the local orientation of contrast edges in V1). The definition of AFs is based on our experimental observations. Spatial coherence of FCOs generally covered larger areas in visual cortical representations than the classical receptive field (cRF) of single neurones. However, coherence was laterally confined to a few millimeters of cortical surface (Figure 3;
Time and the brain
214
Eckhorn et al., 1988; Jordan, 1989; Gray et al., 1989; Eckhorn et al., 1992; Juergens et al., 1996) which means that synchronized cortical regions span only the representational range of small visual objects or parts of larger ones. The concept of AF extends that of cRF by including feature context. AFs of visual (cortical) neurones were defined by that range in visual space that is covered by the RFs of those neurones engaging in common synchronized FCOs. The AF of visual space is directly related to the cortical synchronization field by the cortical magnification factor (relating retinotopic to cortical distances of corresponding parts). Consequently, AFs span a larger range in the visual field than cRFs because they represent the spatial correlation structure among cRFs for an actual stimulus situation. We argue that spatial continuity of an object might be coded by a continuum of overlapping AFs, i.e. by overlapping synchronized regions, spanning the entire object representation. Alternatively, it might be coded by a hierarchy of AFs in which the assemblies engage in phase-locked oscillations at different frequencies: large objects (or larger parts) are represented by larger linking fields, defined by synchronization at lower frequencies, and smaller subparts are represented by smaller linking fields at higher frequencies (Schanze and Eckhorn, 1997). Our experiments have shown the presence of both types of synchronized states, including phase-locking among different frequencies. However, we do not yet know whether such codes are used for visual processing and perception. 3.6. Coding of Spatial and Temporal Feature Separation Feature binding is one of several operations required for visual scene segmentation. Generally, many objects are present in a current scene and they often overlap and partially occlude one another. Hence, spatial separation of features from different objects is required. While spatial binding may be coded by a short population burst during which neurones representing the same object fire within a few milliseconds together (a “population spike”), spatial separation may be characterized by population spikes occurring asynchronously in neurones representing different objects. This may occur either with a constant phase due to a delay circuit, or by decoupling the synchronization networks among the different representations. An additional requirement of visual processing is that of sharp contour coding, which seems necessary because the retinal images move continuously due to movements of the eyes and of visual objects. Hence, retinal image movements should cause “smearing” in perception, but this is generally not observed. Below we propose that the temporal “chopping” of neural activities into synchronized population spikes can provide the visual system with precise (“sharp”) internal representations of a scene’s spatiotemporal feature relations (see section 3.7.).
Cortical processing by fast synchronization
215
3.7. Suppression of FCOs by Stimulus-Locked Responses May Support Temporal Scene Segmentation Temporal segmentation of the continuous stream of visual information seems necessary for clear percepts, because retinal images are continuously moving. Even under conditions of rest and ocular fixation microsaccades and drifts of the eyes are always present. Cortical neurones, on the other hand, can persist in their responses over several hundreds of milliseconds, even to very short visual stimuli of 10–20 ms duration (e.g. Rolls and Tovee, 1994). However, if a figure with sharp contours has evoked cortical activations, which persist while the retinal image of that figure shifts across the retina, the contour’s perception might be smeared, because it activates successively many other neurones with neighbouring receptive fields. Smearing might be prevented if the continuously varying stream of afferent visual information were to be “chopped” by the cortex into short “image frames”. There are three modes with temporal interruptions: 1) Chopping in the representation range of an object, evoked by the sudden displacement of its retinal image; 2) chopping over a broader range of the visual representation by large and small ocular saccades; and 3) repetitive interruptions by synchronized FCOs. In the latter case optimal “frame rates” should be adapted to the rate of retinal image displacement (velocity) and the retinal size of the object’s image. Faster framing would be desirable with higher movement velocities and smaller size (according to the sampling theorem of signal theory see Fig. 2). Stimulus-dominated synchronizations were found to suppress ongoing oscillations partly or even fully. This makes sense in the framework of the “association-bysynchronization” hypothesis, because fast oscillations are generated during slow retinal image drifts, including periods of ocular fixation and smooth pursuit and, hence, may allow repetitive interactions with visual memories. Stimulus-dominated synchronization, on the other hand, is short-lasting and typically occurs after sudden object movements, and then suppresses oscillatory states (simulations to this topic in Gabriel et al., 1997). This may prevent super-position of contradictory scene segmentations by cutting down the persistence of oscillatory states. 3.8. Potential Influence of Oscillation Frequency on Visual Scene Segmentation In anaesthetized cats and especially in awake monkeys FCOs had on average higher oscillation frequencies when the stimulus moved faster or was smaller, compared with slower stimulus movements or larger size (Figure 2; Woelbern et al., 1994). This dependency of frequency suggests that it might be advantageous for precise spatiotemporal representations. A simple calculation shows that the spatial precision of perception based on series of such internal image representations is well in the range of optimal perceptual resolutions. Take the following simple calculation: in a typical recording experiment a stimulus object of 2° (visual angle) moving at l°/s induced a dominant oscillation frequency of about 60 Hz in the visual cortex of an awake monkey. This corresponded to a retinal image displacement of one minute of arc of visual angle
Time and the brain
216
during a single activation-inhibition cycle (17 ms). An even higher precision in spatiotemporal representation would exist during periods of ocular fixation. Assume residual drift movements of the eyes of about 0.1 °/s during ocular fixation tasks (in intervals between µ-saccades), and typical cortical oscillation frequencies around 40 Hz during fixation: this results in image displacements of 9 seconds of arc for a single oscillation cycle (25 ms “frame rate”). This is well in the range of the highest perceptual resolution for line displacements. In conclusion, changing visual scenes may be represented dynamically by synchronized FCOs, which offers the opportunity for an updated precise spatiotemporal representation in each FCO cycle. 3.9. Summary of Section 3 We can say that many experimental results favour the hypothesis of “feature-binding-bysynchronized FCOs”. However, additional recording experiments from awake monkeys, including their perceptual responses, will be necessary for a proof.
4. POTENTIAL MECHANISMS OF GENERATION AND SYNCHRONIZATION OF FCOs The question has often been asked whether FCOs are due to single neurones acting as local oscillators or (alternatively) occur as a consequence of network properties. We will show that this cannot yet be finally answered. However, recent experimental and modelling work has given many indirect hints how FCOs might be generated. In the next paragraphs the discussion of FCO generation is developed from oscillator properties of single neurones, to distributed synchronized assemblies generating FCOs as a network property. The latter may vary from local to wide distribution over the cortex. 4.1. Single Neurone Oscillators? 4.1.1. Spike encoder The analysis of spike trains shows the ability of neurones to generate rhythmic signals. Rhythmically discharging neurones might therefore be called “local oscillators”. Intracellular recordings with current injection has confirmed rhythmicity as a property of the spike encoder, which generally shows a monotonic increase of discharge rate with current or depolarizing potential (e.g. Llinás et al., 1991). Keeping the driving potential at a constant level results (after adaptation) in highly regular spike trains in many neurones. With strong depolarization, spike patterns can change into rhythmic burst firing (e.g. Gray and McCormick, 1996), especially in adapting spike encoders as found in cortical
Cortical processing by fast synchronization
217
pyramidal neurones. Regular spiking behaviour is well understood and was included in models of spike generating mechanisms (e.g. Fohlmeister and Miller, 1997). In conclusion, spike encoders are potential generators of FCOs. 4.1.2. Chattering cells A recently stated proposal for FCO generation by single neurones relies on a special subgroup of supragranular cells—the so called “chattering cells” (Gray and McCormick, 1996). When FCOs occurred in the visual cortex, chattering cells contributed to these synchronous discharges by firing high frequency spike bursts, recurring at intervals of 15–50 ms, i.e. at gamma frequencies. These properties were also apparent when injected with suprathreshold depolarizing current, showing that fast burst rates were a property of the single cell’s spike encoder (Gray and McCormick, 1996). These findings indicated, at least for visually induced FCOs, that a major source of their generation is of cortical origin, and not mainly due to thalamic or other extracortical sources, as has been argued by others (Ghose and Freeman, 1992). Cells receiving presynaptic input from chattering cells are expected to show membrane potential oscillations consistent with a synaptic rather than an intrinsic mechanism (Jagadeesh et al., 1992). In addition, FCOs synchronized across larger cortical populations may well be dominated by the influence of chattering cells, which may couple their bursts via lateral network connections. 4.1.3. Intrinsic membrane oscillations In addition to the spike encoder of a neurone, its input structures, including membranes of dendrites and soma, can favour certain frequency bands, as has recently been characterized by measuring their signal transfer properties with current injection (Connors and Amitai, 1997). The transfer function of cortical pyramidal cells showed multiple peaks, comprising high frequency ranges in which signals are less damped compared to other ranges. However, generation of subthreshold fast oscillations by dendro-somatic membranes themselves (distinguished from synaptically “injected” membrane oscillations) has not been convincingly shown in cortical neurones. In contrast, intracellular recordings indicated the synaptic origin of fast subthreshold oscillations in visual cortical neurones (Jagadeesh et al., 1992). In conclusion, even though single neurones have the properties of “local oscillators” the generation of FCOs is probably dominated by cortical network properties, because oscillatory signals in cortical neurones were synaptically supplied. This leaves the possibility that FCOs are selectively amplified by membrane resonances, so that the neurone’s spike encoder generates impulse sequences in which this frequency band is dominant.
Time and the brain
218
4.2. Local Network Oscillators As FCOs are probably supplied via synaptic inputs, they can be generated by either one or several remote oscillators (with intra- or extra-cortical locations) or they might emerge as a result of local cortical feedback circuitry introducing phase shifts due to delays and inhibition. There are several indirect arguments favouring, in my opinion, the generation of FCOs by cortical network properties instead of single neurone oscillators. Excitatory and inhibitory cortical neurones are present everywhere in the cortex. They establish local feedback circuits (McGuire et al., 1991) introducing phase shifts by sign inversion and transmission delays, and hence, are suitable for the generation of local FCOs (Freeman, 1959). On average, there are four times as many excitatory compared to inhibitory cortical neurones (Braitenberg and Schüz, 1991). It is therefore assumed that small groups of excitatory cells establish negative feedback via common inhibitory neurones (Douglas and Martin, 1991). Hence, such local groups of excitatory cells can be partially synchronized when they become activated (e.g. van Vreeswijk et al., 1994; Bush and Sejnowski, 1996). As their discharges are simultaneously chopped by their common interneurones they discharge well co-ordinated population bursts (Figure 8). Such bursts are a common observation in multiple unit (MUA) recordings in the visual cortex (e.g. Eckhorn et al., 1988; Gray et al., 1989). However, synchronization in such local groups may be sharpened by lateral coupling connections, which resemble local cRF relations as expected from Gestalt laws (see section 3; and Wertheimer, 1923; for models see e.g. Eckhorn et al., 1990; Juergens and Eckhorn, 1997; Wennekers and Palm, this volume). We argued in recent work on model networks for visual feature linking that the lateral linking connections should be symmetric and of a fast modulatory type (which means facilitatory, implemented by multiplicative interaction with feeding inputs: Eckhorn et al., 1990; Stoecker et al., 1996). Such linking connections can synchronize spikes of coupled neurones very quickly, and do not change the spike rates, while they avoid deterioration of cRF-properties by superposition (as would be the case with additively superimposed synaptic inputs used by most other models of spiking neurones). 4.3. Circuits for Synchronizing FCOs at Zero Correlation Delay We found FCOs’ phase differences narrowly distributed around zero, which means single central peaks in cross-correlograms. They can be explained by inputs from common sources, with approximately the same activation delays between these sources and cortical recording positions. This explanation is plausible because such common inputs guarantee zero delay
Cortical processing by fast synchronization
219
Figure 8. Common inhibition provides synchronized chopping of neurones discharging at different rates. E: excitatory, I: inhibitory neurones
independent of the type of signal (stochastic or rhythmic) and its frequency range (low or high). Relevant evidence was obtained in many experiments, for locations in several cortical areas of cat and monkey (Ts’o et al., 1986; Eckhorn et al., 1990; Engel, 1991a,b; Munk et al., 1992; Nelson et al., 1992; Frien et al., 1994). As most cortico-cortically projecting neurones only have a single remote target (Braitenberg and Schüz, 1991) it is probable that other neurones from the same synchronized population, in another layer of the same cortical column, supply the common input. Alternative or additional possibilities for establishing zero-phase FCOs are symmetric excitatory or modulatory feedback interactions among the cortical areas (Eckhorn and Frien, 1995; Saam and Eckhorn, 1998). Theoretical considerations and simulation models revealed that zero-delay phase locking requires special dynamic constraints, including activation delays around such loops to be shorter than about a third of the oscillation period (Gerstner et al., 1993; Ritz et al., 1994; Hansel et al., 1995; Crook et al., 1997), because this is the range in which constructive superposition of excitatory signals is possible. In particular, if the symmetric connectivity consists of inter- and intra-areal projections from the same local populations that activate their postsynaptic targets at about the same delay, the zero delay peaks in cross-correlograms can be explained for oscillatory and stochastic modes of activity (Eckhorn and Frien, 1995). Symmetric inhibitory connections among the inhibitory pools of spatially separate local population oscillators have also been proposed for establishing zero delay fast rhythms (Schillen and König, 1994). Even though this is a probable mechanism for synchronizing FCOs, it cannot explain the observed, central correlation peaks in states with stochastic activity. However, it cannot be ruled out on the basis of present experimental knowledge that mutual inhibition of inhibitory pools is an important mechanism for synchronizing FCOs.
Time and the brain
220
Common input from the visual thalamus (and the retina) is probably not the main source of FCOs, even though in anaesthetized cats oscillations were found there (Ghose and Freeman, 1992; Ito et al., 1994; Sillito et al., 1995; Neuenschwander and Singer, 1996). The first reason why such oscillations probably do not play a central role is that their oscillation frequencies were generally below those recorded in visual cortex (Neuenschwander and Singer, 1996). In addition, the distribution and phase relations of field potentials across the cortical layers is fundamentally different for signals coming from the visual thalamus and those of FCOs. The thalamic inputs always cause field distributions with a dipole character, showing phase reversal between upper and lower layers (zero at layer 4) independent of the frequency range (0–150 Hz), and with both electrical and visual stimulation (Creutzfeldt et al., 1966; Cracco and Cracco, 1978; Kraut et al., 1985; Mitzdorf, 1987). FCOs, in contrast, did not show phase shifts across cortical layers, their phases being narrowly distributed around zero (Jordan, 1989; Gray et al., 1989; Eckhorn, 1992). Coupling of cortical with other thalamic nuclei at FCO frequencies has also been found (Steriade, 1993; Steriade et al., 1996). However there are no indications yet whether these nuclei are involved in generating the FCOs. 4.4. Establishing Cortical Synchronization Fields by Hebbian Correlation Learning? We developed a biologically plausible network of spiking neurones, explaining the dependence of the width of cortical synchronization fields on lateral spike velocity, temporal dispersion of correlated input, duration of Hebbian learning window, and the period of the synchronized oscillatory driving input (Saam and Eckhorn, 1998). Spiking neurones were retinotopically organized in a map of feature detectors, and each neurone was laterally connected to all others by constant velocity axons and facilitatory synapses (Eckhorn et al., 1990). Systematic dependencies of synchronization field width on simulation parameters were derived (Figure 9): lower spike velocities led to narrower fields, because at larger distances the evoked PSPs of laterally travelling spikes came too late to coincide with spikes of the same oscillation period. Increasing the relative temporal jitter of the spike patterns in the neurone’s feeding inputs, from fully correlated and rhythmic to higher temporal dispersion (according to broader peaks in the pairwise cross-correlograms), increased the width of synchronization fields. This is because many input spikes occurred before the highest average spike correlation in the inputs so that longer lateral spike transmission delays could fulfil the Hebbian condition of simultaneity. If the input spike jitter exceeded a critical fraction of the rhythm’s period (about 1/3) the weight distributions lost their spatial structure. Conversely, reduction of the oscillation period narrowed the width of the synchronization field due to the restricted time available for lateral spike transmission and near-coincidence. In conclusion, our model implies that the width of cortical synchronization fields can become larger if lateral spike velocities are higher, correlation width of input signals broader, and periods of common oscillations longer. This model generated synchronization fields of the same width (3 to 5 mm) as found in visual cortex V1 and V2
Cortical processing by fast synchronization
221
of cat and monkey if biologically plausible values for lateral spike velocities (estimated average velocity 1 m/s) and oscillation frequencies (40 Hz) were used for the simulations. In
Figure 9. Width of association (synchronization) field (AF) in topographic neural map depends on lateral spike velocity and temporal dispersion of correlated input: a model with spiking neurones and Hebbian learning. a: Spatial distribution of correlation strength after learning (fixed axonal velocity), b: Dependence of association field width on axonal velocity, c: Dependence of AF width on temporal dispersion of correlated spikes. (Figure modified from Saam and Eckhorn, 1998).
addition, as in real cortices, the model produced (on average) zero phase delay among any neurones within the synchronization range. 4.5. Network Properties Explaining Influence of Visual Stimulation on Average FCO Frequency and Amplitude FCOs occur predominantly when cortical neurones are activated by visual stimulation in a sustained manner. Even with constant stimulation, oscillation frequency, amplitudes, and the cortical extent of synchronization fields vary over a broad range. However, their average values change in a systematic way with changes in stimulation (see section 3). These effects are compatible with a process of dynamic self-organization among coupled neurones receiving sustained activation by a common stimulus. With an increasing level of activation, local populations, having similar cRF-properties, start FCOs at high frequency due to their short, and hence fast, internal feedback loops. Synchronization among distant local populations does not occur because of the irregular nature of these FCOs, their low amplitude and the relatively long activation delays between populations more than a hypercolumn (1 mm) apart. At higher levels of activation, however, local populations activate their common inhibitory feedback circuits discharging longer bursts with stronger activation, and hence, inhibit the population over a longer period causing a lower frequency of the rhythm (Wennekers et al., 1994). At the lower FCO frequency mutual synchronization among neighbouring populations becomes possible via bidirectional connections, because the longer cycle period provides enough time for zerophase locking. In the context of this argument, a given local oscillation frequency limits the cortical coherence range due to lateral activation velocities (including axonal velocity and synapto-somatic activation delays) (Saam and Eckhorn, 1998). In monkey primary
Time and the brain
222
visual cortex the coherence range was 4 to 6 mm at about 40 Hz which is compatible with lateral activation velocities of 1 m/s (a biologically plausible value). 4.6. Mechanisms of Generation, Enhancement and Suppression of Signal Correlations by Local Populations of Coupled Neurones In order to understand the action of partially synchronized signals on a local group of laterally coupled neurones in different states of global activation, we developed a neural
Figure 10 A-C. Three Different Modes of Simulated Network Dynamics. They emerged by changing the lateral coupling strength, when the average
Cortical processing by fast synchronization
223
amplitudes of the random Gaussian white noise inputs were low, and a high constant value of the membrane potential caused relatively high sustained discharge patterns. A: Alternating stochastic and oscillatory activity (weak lateral coupling). B: Synchronized short oscillations, with random frequencies in successive events (medium coupling). C: Rhythmical bursts (stronger coupling). a: Feeding-, b: linking-, c: membrane potential, d: threshold potential, e: spike output patterns, f: multiple unit activity. ACF: auto-coincidence correlogram of output spikes, CCF: cross-coincidence correlogram of output spikes from two different model neurones. (Figure modified from Juergens and Eckhorn, 1997).
network model of spiking neurones (Juergens and Eckhorn, 1997). As in the visual cortex, in this network “FCOs” occurred only during enhanced sustained activations of the neurones (either by increasing the average input drive or by changing the threshold properties of all neurones equally). Three different patterns of synchronized FCOs emerged: enhancing the sustained drive sufficiently above threshold, so that medium frequencies of rather “regular” patterns were discharged by the neurones, led to relatively high frequency FCOs (Figure 10a). However, further increase in the sustained input level forced the neurones to discharge short high-frequency bursts, that occurred at lower repetition rates, and hence, lower FCO frequencies (Figure 10c) compared to the former oscillations (Figure 10a). At intermediate driving levels, spike trains were well synchronized, but these events occurred non-rhythmically (Figure 10b). However, low level inputs diminished the degree of synchronization, and FCOs were generally absent. In this state the neurones discharged only few action potentials, and partial synchronization of the input signals could be enhanced or suppressed gradually, depending on the lateral coupling strength among the neurones. These results fit our experimental observations quite well, even though the model was rather simple. 4.7. Networks Leading to Phase-Shift and Decoupling With Stimulation by Overlapping Visual Objects The situation in which several visual objects overlap, or are adjacent, has been analysed with neural network simulations by several groups (e.g. Schillen and König, 1994; Sompolinsky et al., 1990). For this situation, the synchronization hypothesis predicts separable oscillatory processes. However, this can be accomplished either by phase shift of the FCOs at the same common frequency, or by statistically independent FCOs. Nearly all models generated phase-shifted oscillations at the same frequency in neural representations of different scene segments. This was either due to a fast inhibitory feedback loop, that was common to all excitatory neurones (Schott, [1995], in Eckhorn, 1998 and see Fig. 8), and/or it was caused by mutual inhibition among spatially separate groups of excitatory neurones, as proposed by other model work. However, in the visual cortex this behaviour of phase-shifted coding at a common frequency has never been demonstrated, as far as the author knows the literature. Instead, several works have shown statistical decorrelation in their figures even though the
Time and the brain
224
authors do not explicitly state it. We recently designed an experiment in which this question was asked directly. We obtained statistically independent FCOs with two objects at two recording positions, while the FCOs synchronized if the neurones at each of these positions were stimulated by the same object. This demonstrates the potential for a functional disconnection of the lateral coupling. As synchronization was probably mediated via monosynaptic connections during these experiments, we propose that the neurones detecting the contrast separating two scene segments exert modulatory inhibition on the lateral coupling synapses and, by this means, functionally disconnect the coupling.
5. DO THE RELATIONS AMONG RECEPTIVE- AND ASSOCIATIONFIELDS GIVE HINTS TO A BASIC CORTICAL PROCESSING UNIT? We still do not know how most of the basic operations of vision are performed by the visual cortex. These operations include: (i) extraction and representation of relevant—and suppression of irrelevant—visual features; (ii) feature grouping into scene segments and objects; (iii) transformations into invariant representations, so that comparison with stored objects, and learning of new objects is possible; (iv) associative processing and memory; and (v) generation of stable and unambiguous percepts. It is not clear whether these operations each require special “smart” circuits, nor whether they are carried out more in a serial manner or more in a distributed way— stepwise by many similar modules. The latter possibility is supported by anatomical observations of similar local circuitry in the different areas of visual cortex. If this is actually the case we should already be able to identify basic modules in the primary visual cortex. The present approach tries to start with relations among simple-and complex-cell cRFs and AFs. 5.1. Interactions Among Simple and Complex Cells as an Example A simple cell in V1 responds well to an oriented contour if it is placed at a certain position, precisely defined by the locations and polarities of its cRF-subfields. The cRFs of complex neurones, on the other hand, show about the same width of orientation tuning, but the exact position of the contours and their polarity play a minor role for the cell’s activation. This inspired Hubel and Wiesel (1962) to propose that complex cells receive excitatory feeding inputs from a number of simple cells with similar orientation preferences, but with cRF positions spatially dispersed across the cRFs of the complex cell. In addition, simple cells should be driven by parvocellular neurones with aligned cRFs of appropriate position and polarity (ON- and OFF-centre). This connectivity seems convincing and is included in our scheme in Figure 11 a. Later experiments found direct and fast inputs from the magno-cellular pathway to complex cells (Hoffmann and von Seelen, 1978; Henry et al., 1979), while they and others confirmed the parvocellular input to simple cells (review in Henry et al., 1994). We therefore added excitatory feeding input from the magnocellular afferents to complex cells.
Cortical processing by fast synchronization
225
As FCOs are generated only in states of sustained input, it seems plausible that tonic parvocellular input to simple cells dominates in this situation, while the phasic magnocellular input is silent. With this and the above schema in mind, we predicted that FCOs would occur in simple cells, with a phase lead of a few milliseconds with respect to those in complex cells according to this activation delay. However, recordings from V1 of awake monkeys always revealed that FCO phase differences among simple and complex cells were narrowly distributed around zero (S.D. about 1 ms: Eckhorn et al, 1995). If we accept that complex cells receive activation by simple cells in this situation, we have to explain the zero phase delay by an additional mechanism. For this we propose a feedback loop, via an excitatory drive of simple and complex cells to a common inhibitory neurone (or pool of coupled neurones) that provides feedback inhibition to its input cells (circuit included in Figure 11 a). This circuit suppresses (simultaneously and repetitively) the activities of simple and complex cells by a fast inhibitory component (“chopped” synchronization as suggested in section 4.2). A longer-lasting inhibitory component supplied via the same connections may reduce activations of simple cells that are weakly driven by the stimulus, or are spontaneously active, and thus enhances contrast and signal-to-noise-ratio of an actual contour’s representation.
Figure 11, ab Proposed interactions among simple (S) and complex (C) cell; I: inhibitory cell; P: input from retino-thalamic parvocellular neurones, providing sustained activation during ocular fixation; M: input from retino-thalamic magnocellular neurones, signalling transient changes in local retinal luminance; S→C feeding: feeding connections from simple to complex cells, generating the cRF properties of complex cells by superimposing simple cRFs; S↔C linking: coupling connections providing fast spike synchronization, b: Range of
Time and the brain
226
synchronized simple cells association field (AF) influences classical receptive field (cRF).
This circuit also explains the suppression of static retinal information (supplied by the parvocellular pathway) by fast stimulus transients (transmitted by the magnocellular pathway; Lennie, 1980). The magnocellular input transiently activates complex cells, and these excite the inhibitory loop suppressing simple cell activities. In addition, this mechanism explains the suppression of FCOs in simple and complex cells by transient inputs (Kruse and Eckhorn, 1996) because any input transient will disturb and inhibit ongoing FCOs. Chopping by a common feedback inhibition also accounts for the zero phase delay among simple cells of similar orientation preference and overlapping cRFs. However, zero phase coupling among FCOs of simple cells with more separate cRFs cannot be explained by “local chopping”. For this we assume fast facilitatory connections, that can synchronize spike activities in remote populations without compromising their cRF properties (experimental and model results in Eckhorn et al., 1990). These “linking connections” are assumed to reflect Gestalt properties (as argued in section 3). Here in our example this is “similarity” (of orientation preference) and “proximity” (overlapping and neighbouring cRFs). Fast modulatory feedback connections from complex to simple cells are proposed, based on theoretical arguments and neural network simulations (Eckhorn et al., 1990). In the “transient mode” of stimulation they would sensitize, via the activated complex cell, the simple cells’ inputs (for example when a visual object suddenly moves). This can facilitate parvocellular inputs because they arrive delayed, and could explain involuntary attention and pop-out of transient stimuli. In the “sustained mode” of stimulation, when FCOs are generated by this circuit, complex cells would average the FCO phases of their inputs from simple cells and feed back this average signal for modulating and gating the simple cells’ feeding inputs from the previous stage. This operation would support spikes occurring synchronized at the FCOs rhythm, while out-of-phase activity, including spontaneous spikes, will be suppressed and therefore excluded as an input to the complex cell. It is probable from theoretical considerations and modelling studies that the width of the simple cells’ association field (defined by the superimposed cRFs of simple cells that can engage in a synchronized FCO) mainly determines the cRF width of complex fields (Figure 11b). This seems probable, because a synchronized input would be particularly successful with Hebbian learning, and it fits the observation that cRFs become larger in successive processing steps (e.g. from V1 to V2). 5.2. Generalized Functions of the Proposed Circuit If this circuit of simple/complex cell interactions is present in a similar way throughout other cortical areas, the following more general processing might be carried out by it. Extraction and representation of relevant features is performed by the convergence of feeding connections (with concentric cRFs) to the lower layer (simple cells) which are
Cortical processing by fast synchronization
227
superimposed to form new cRFs (preferring oriented contour segments). Feature grouping is supported in several ways: 1) by lateral linking connections synchronising (simple) cell spikes that were activated by the same stimulus object; 2) by feedback linking connections from the upper (complex) to the lower layer (simple) cells so that some averaged input is fed back supporting the nearly synchronized lower level cells; and 3) by the negative feedback loop that generates a rhythm (FCO) common to all lower level cells and suppresses dynamic feature representations occurring out-of-phase to it. Associative memory is incorporated in this coupling network by the distribution of synaptic weights. These weights are assumed to be shaped by Hebbian learning so that knowledge of relevant feature combinations is stored here (in our example the Gestalt rules of similarity and proximity of contour elements). “Learning” is supported by the repetitive high amplitudes of synchronized FCOs. “Recall” is executed when an input fits the requirements of activating a subset of cells in a synchronized manner (a partitioned oriented contour in a certain retinal position). “Associative completion” is made with partial or disturbed inputs, activating the same subset of units that is activated by the undisturbed input. Invariance and generalization can be assigned to the feedforward connections from lower to upper layer, by accepting only inputs with high similarity in some cRFproperties while other cRF properties are less strictly controlled but collected over a certain coding range and hence, are generalized (in our example, simple cells of equal orientation but distributed cRF positions converge on complex cells). Classification is made in the upper layer by a set of complex cells (with mutual synchronization and common inhibition?) with specific feeding inputs from simple cells, e.g. either sets of simple inputs with coaxially aligned cRFs (for contour-form representation) or sets of inputs with parallel cRFs (for grating-texture representation). In conclusion it has been argued how visual processing for object recognition might be carried out by a rather simple modular circuit, derived from interactions among simple and com-plex cells in the primary visual cortex. This module can, in principle, carry out stepwise basic visual operations, including feature extraction, associative grouping, processing of invariance and generalization, classification and associative memory and recall. These operations are also carried out by other sensory systems. However, more elaborate experimental and model investigations will be necessary before a generally acceptable module can be proposed, or before concepts of cortical modularity can be rejected.
ACKNOWLEDGEMENTS This article could not have been written without the numerous discussions with my colleagues and without their extensive help in experiments, data acquisition and processing, as credited in the text and figure captions. I want to thank Guenther Palm, Thomas Wennekers and Andrea Bibbig for years of fruitful discussions about the concepts proposed in this chapter. In addition, Robert Miller has considerably improved my work by pointing to unclear pathways of argumentation, by correction of my English
Time and the brain
228
and by his thorough editorial work. Expert help in care and preparation of our cats and monkeys, and help in experimental techniques came from U.Thomas, J.H.Wagner and W.Lenz. The financial support by the Deutsche Forschungsgemeinschaft is also greatly acknowledged (Ec 53/6, EC 53/7 and Ro 529/12, all to R.E.).
REFERENCES Abeles, M. (1991) Corticonics: Neuronal circuits of the cerebral cortex. Cambridge University Press. Adrian, E.D. (1950) The electrical activity of the mammalian olfactory bulb. Electroencephalography and Clinical Neurophysiology, 2, 377–388. Aertsen, A.M., Gerstein, G.L., Habib, M.K., and Palm, G. (1989) Dynamics of neuronal firing correlation: modulation of “effective connectivity”. Journal of Neurophysiology, 61, 900–917. Agmon-Snir, H., and Segev, I. (1993) Signal delay and input synchronization in passive dendritic structures. Journal of Neurophysiology, 70, 2066–2085. Baker, S.N., Olivier, E., and Lemon, R.N. (1997) Coherent oscillations in monkey motor cortex and hand muscle EMG show task-dependent modulation. Journal of Physiology (London), 501, 225–241. Earth, D.S., and MacDonald, K.D. (1996) Thalamic modulation of high-frequency oscillating potentials in auditory cortex. Nature, London, 383, 78–81. Bernander, O., Koch, C., and Usher, M. (1994) The effect of synchronized inputs at the single neuron level. Neural Computing, 6, 622–641. Bragin, A., Jando, G., Nadasdy, Z., Hetky, J., Wise, K., and Buzsaki, G. (1995) Gamma (40–100 Hz) oscillations in the hippocampus of the behaving rat. Journal of Neuroscience, 15:47–60. Braitenberg, V., and Schüz, A. (1991) Anatomy of the Cortex. Statistics and Geometry. Springer, Berlin. Brosch, M., Bauer, R., and Eckhorn, R. (1995) Synchronous high-frequency oscillations in cat area 18. European Journal of Neuroscience,7, 86–95. Brosch, M., Bauer, R., and Eckhorn, R. (1997) Stimulus-dependent modulations of correlated high-frequency oscillations in cat visual cortex. Cerebral Cortex, 7, 70–76. Bush, P.C., and Sejnowski, T.J. (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. Journal of Computational Neuroscience, 3, 91–110. Connors, B.W., and Amitai, Y. (1997) Making waves in the neocortex. Neuron, 18, 347– 349. Cracco, R.Q., and Cracco, J.B. (1978) Visual evoked potentials in man: early oscillatory potentials. Electroencephalography and Clinical Neurophysiology, 45, 731–739. Creutzfeldt, O.D., Watanabe, S., and Lux, H.D. (1966) Relation between EEGphenomena and potentials of single cells. Part I and II. Electroencephalography and Clinical Neurophysiology, 20, 1–37. Crook, S.M., Ermentrout, G.B., Vanier, M.C., and Bower, J.M. (1997) The role of axonal delay in the synchronization of networks of coupled cortical oscillators. J Computational Neuroscience, 4, 161–172. Douglas, R.J., and Martin, K.A.C. (1991) A functional microcircuit for cat visual cortex. Journal of Physiology (London), 440, 735–769.
Cortical processing by fast synchronization
229
Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboeck, H.J. (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biological Cybernetics, 60, 121– 130. Eckhorn, R., Reitboeck, H.J., Arndt, M., and Dicke, P. (1990) Feature linking among distributed assemblies: Simulations and results from cat visual cortex. Neural Computation, 2, 293–306. Eckhorn, R., and Schanze, T. (1991) Possible neural mechanisms of feature linking in the visual system: Stimulus-locked and stimulus-induced synchronizations. In: A.Babloyantz (ed.), Self-Organization, Emerging Properties and Learning. Plenum Press, New York, pp. 63–80. Eckhorn, R. (1992) Principles of global visual processing of local features can be investigated with parallel single-cell- and group-recordings from the visual cortex. In: A.Aertsen and V.Braitenberg (eds) Information Processing in the Cortex, Experiments and Theory, Springer-Verlag, Berlin Heidelberg, New York, pp. 385–20. Eckhorn, R., Schanze, T., Brosch, M., Salem, W., and Bauer, R. (1992) Stimulus-specific synchronizations in cat visual cortex: Multiple microelectrode and correlation studies from several cortical areas. In: E.Basar, and T.H.Bullock (eds), Induced Rhythms in the Brain. Brain Dynamics Series. Birkhäuser, Boston, Basel Berlin, pp. 47–82. Eckhorn, R., Frien, A., Bauer, R., Woelbern, T., and Kehr, H. (1993) High frequency (60–90 Hz) oscillations in primary visual cortex of awake monkey. NeuroReport, 4, 243–246. Eckhorn, R., and Obermueller, A. (1993) Single neurons are differently involved in stimulus-specific oscillations in cat visual cortex. Experimental Brain Research, 95, 177–182. Eckhorn, R., Rutschmann, R., Woelbern, T., Frien, A., and Bauer, R. (1995) Neurons responding with different latencies and time courses to a visual stimulus engage in near-zero delay mutual correlation in the visual cortex of awake monkey. Society for Neuroscience Abstracts, 21, 648.17. Eckhorn, R., and Frien, A. (1995) Neural signals as indicators of spatial and temporal segmentation coding in the visual system. Proceedings of the International Conference on Brain Processes, Theories and Models. J.Mira-Mira (ed). MIT-Press, Cambridge MA, pp. 1–10. Eckhorn, R. (1998) Neural mechanisms of scene segmentation: Recordings from the visual cortex suggest basic circuits for linking field models. IEEE Neural Networks. Engel, A.K.,König, P., Gray, C.M., and Singer, W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: Inter-columnar interaction as determined by crosscorrelation analysis. European Journal of Neuroscience, 2, 588–606. Engel, A.K., König, P., Kreiter, A.K., and Singer, W. (1991a) Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, New York, 252, 1177–1179. Engel, A.K., Kreiter, A.K., König, P., and Singer, W. (1991) Synchronization of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat. Proceedings of the National Academy of Sciences, U.S.A., 88, 6048–6052. Fohlmeister, J.F., and Miller, R.F. (1997) Mechanisms by which cell geometry controls repetitive impulse firing in retinal ganglion cells. Journal of Neurophysiology, 78, 1984–1964. Fox, K., and Daw, N. (1992) A model of the action of NMDA conductances in the visual cortex. Neural Computation, 4, 59–83.
Time and the brain
230
Freeman, W.J. (1959) Distribution in space and time of prepyriform electrical activity. Journal of Neurophysiology, 22, 644–666. Frien, A., Eckhorn, R., Bauer, R., Woelbern, T., and Kehr, H. (1994) Stimulus-specific fast oscillations at zero phase between visual areas V1 and V2 of awake monkey. NeuroReport, 5, 2273–2277. Frien, A., Eckhorn, R., and Reitboeck, H.J. (1996) Fast oscillations in V1 of awake monkey: synchronization depends on cortical distance, on angle between preferred orientations and on stimulus orientation. Society for Neuroscience Abstracts, 22, 255.5. Fries, P., Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997) Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proceedings of the National Academy of Sciences, U.S.A., 94, 12699–12704. Gabriel, A., Kopecz, K., and Eckhorn, R. (1997) Enhancement of stimulus-locked spatiotemporal contrast for scene segmentation in a computational model of spiking neurons. In: N.Elsner and H.Wässle (eds) Göttingen Neurobiology Report 1997. Thieme, Stuttgart New York, p. 1024. Gerstner, W., Ritz, R., and van Hemmen, J.J. (1993) A biologically motivated and analytically soluble model of collective oscillations in the cortex. I. Theory of weak locking. Biological Cybernetics, 68, 363–374. Ghose, G.M., and Freeman, R.D. (1992) Oscillatory discharge in the visual system: Does it have a functional role? Journal of Neurophysiology, 68, 1558–1574. Gilbert, C.D. (1993) Circuitry, architecture and functional dynamics of visual cortex. Cerebral Cortex, 3, 373–386. Gray, C.M., König, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, London, 338, 334–337. Gray, C.M., Engel, A.K., König, P., and Singer, W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: receptive field properties and feature dependence. European Journal of Neuroscience, 2, 607–619. Gray, C.M. (1994) Synchronous oscillations in neuronal systems: Mechanisms and functions. Journal of Computational Neuroscience, 1, 11–38. Gray, C.M., and Viana Di Prisco, G. (1997) Stimulus-dependent neuronal oscillations and local synchronization in striate cortex of the alert cat. Journal of Neuroscience, 17, 3239–3253. Gray, C.M., and McCormick, D.A. (1996) Chattering cells: superficial pyramidal neurons contributing to the generation of synchronous oscillations in visual cortex. Science, New York, 274, 109–113. Gray, C.M., Maldonado, P.E., Wilson, M., and McNaughton, B. (1995) Tetrodes markedly improve the reliability and yield of multiple single-unit isolation from multiunit recordings in cat striate cortex. Journal of Neuroscience Methods, 63, 43–54. Guettler, A., Eckhorn, R., Frien, A., and Woelbern, T. (1996) Influence of microsaccades on fast oscillations in monkey striate cortex. In: N.Elsner and H.-U.Schnitzler (eds) Brain and Evolution, (eds). Thieme, Berlin, New York, p. 421. Guettler, A., Eckhorn, R., Juergens, E., and Frien, A. (1997) Neural correlation contrast in visual cortex of monkey changes with stimulus contrast across an object-background border. In: N.Elsner and H.Wässle (eds) Göttingen Neurobiology Report 1997. Thieme, Stuttgart, New York, p. 551. Hansel, D., Mato, G., and Meunier, C. (1995) Synchrony in excitatory neural networks. Neural Computation, 7, 192–210.
Cortical processing by fast synchronization
231
Hāusser, M., and Roth, A. (1997) Estimating the time course of the excitatory synaptic conductance in neocortical pyramidal cells using a novel voltage jump method. Journal of Neuroscience, 17, 7606–7625. Henry, G.H., Harvey, A.R., and Lund, J.S. (1979) The afferent connections and laminar distributions of cells in the cat striate cortex. Journal of Comparative Neurology, 178, 725–744. Henry, G.H., Michalski, A., Wimborne, B.M., and McCart, R.J. (1994) The nature and origin of orientation specificity in neurons of the visual pathways. Progress in Neurobiology, 43, 381–437. Hoffmann, K.P., and von Seelen, W. (1978) Analysis of neuronal networks in the visual system of the cat using statistical signals—simple and complex cells. Biological Cybernetics, 31, 175–185. Hubel, D.H., and Wiesel, T.N. (1962) Receptive fields, binocular interaction, and functional architecture in the cat’s visual cortex. Journal of Physiology (London), 160, 106–154. Ito, H., Gray, C.M., and Viana Di Prisco, G. (1994) Can oscillatory activity in the LGN account for the occurrence of synchronous oscillations in the visual cortex? Society for Neuroscience Abstracts, 20.61.7. Jagadeesh, B., Gray, C.M., and Ferster, D. (1992) Visually-evoked oscillations of membrane potential in neurons of cat striate cortex studied with in vivo whole cell patch recording. Science, New York, 257, 552–554. Jordan, W. (1989) A model of global coding of feature relations in the visual cortex. Dissertation, Neurophysics Grp, Philipps-University Marburg. Juergens, E., Eckhorn, R., Frien, A., and Woelbern, T. (1996) Restricted coupling range of fast oscillations in striate cortex of awake monkey. In: N.Elsner and H.-U.Schnitzler (eds) Brain and Evolution. Thieme, Berlin, New York, p. 418. Juergens, E., Guettler, A., and Eckhorn, R., (1999) Visual stimulation elicits locked and induced gamma oscillations in monkey intracortical- and EEC-potentials, but not in human EEG. Exp. Brain Res., 129, 247–259. Juergens, E., and Eckhorn, R. (1997) Parallel processing by a homogenous group of coupled model neurons can enhance, reduce and generate signal correlations. Biological Cybernetics, 76, 217–227. König, P., Engel, A.K., Roelfsema, P.R., and Singer, W. (1995) How precise is neuronal synchronization? Neural Computation, 7, 469–485. König, P., Engel, A.K., and Singer, W. (1995) Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A., 92, 290–294. Kottmann, M., and Eckhorn, R. (1996) Perception related synchronized oscillations in monkey striate cortex in a binocular rivalry task. Society for Neuroscience Abstracts, 22, 255.10. Kraut, M.A., Arezzo, J.C., and Vaughan, H.G. (1985) Intracortical generators of the flash VEP in monkeys. Electroencephalography and Clinical Neurophysiology, 62, 300– 312. Kreiter, A.K., and Singer, W. (1996) Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. Journal of Neurophysiology, 16, 2381–2396. Kruse, W., and Eckhorn, R. (1996) Inhibition of sustained gamma oscillations (35–80 Hz) by fast transient responses in cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A., 93, 6112–6117.
Time and the brain
232
Kuramoto, Y. (1991) Collective synchronization of pulse-coupled oscillators snd excitable units. Physica D, 50, 15–30. Legatt, A.D., Arezzo. J., and Vaughan, Jr H.G. (1980) Averaged multiple unit activity as an estimate of phasic changes in local neuronal activity. Journal of Neuroscience Methods, 2, 203–217. Lennie, P. (1980) Perceptual signs of parallel pathways. Philosophical Transactions of the Royal Society, London, series B , 290, 23–37. Llinás, R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10–50 Hz frequency range. Proceedings of the National Academy of Sciences, U.S.A., 88, 879–901. MacKay, W.A., and Mendonca, A.J. (1995) Field potential oscillatory bursts in parietal cortex before and during reach. Brain Research, 704, 167–174. McGuire, B.A., Gilbert, C.D., Rivlin, P.K., and Wiesel, T.N. (1991) Targets of horizontal connections in macaque primary visual cortex. Journal of Comparative Neurology, 305, 370–392. Miller, R. (1994) What is the contribution of axonal conduction delay to temporal structure in brain dynamics? In: C.Pantev, T.Elbert, B.Lütkenhöner (eds) Oscillatory Event-Related Brain Dynamics. NATO ASI-Series A, Vol 271, Plenum Press London, pp. 53–58. Milner, P.M. (1974) A model for visual shape recognition. Psychological Review, 81, 521–535. Mitzdorf, U. (1987) Properties of the evoked potential generators: current source-density analysis of visually evoked potentials in the cat cortex. International Journal of Neuroscience, 33, 33–59. Munk, M.H.J., Nowak, L.G., Chouvet, G., Nelson, J.I., and Bullier, J. (1992) The structural basis of cortical synchronization. European Journal of Neuroscience, Suppl 5, 21. Munk, M.H.J., Roelfsema, P.R., König, P., Engel, A.K., and Singer, W. (1996) Role of reticular activation in the modulation of intracortical synchronization. Science, New York, 272, 271–274. Murthy, V.N., and Fetz, E.E. (1994) Effects of input synchrony on the firing rate of a 3conductance cortical neuron model. Neural Computation, 6, 1111–1126. Murthy, V.N., Aoki, F., and Fetz, E. (1994) Synchronous oscillations in sensorimotor cortex of awake monkeys and humans. In: C.Pantev (ed) Oscillatory Event Related Brain Dynamics. Plenum, New York, pp. 343–356. Murthy, V.N., and Fetz, E.E. (1997a) Oscillatory activity in sensorimotor cortex of awake monkeys: synchronization of local field potentials and relation to behavior. Journal of Neurophysiology, 76, 3949–3967. Murthy, V.N., and Fetz, E.E. (1997b) Synchronization of neurons during local field potential oscillations in sensorimotor cortex of awake monkeys. Journal of Neurophysiology, 76, 3968–3982. Nelson, M. (1994) A mechanism for neuronal gain control by descending pathways. Neural Computation, 6, 242–254. Neuenschwander, S., and Singer, W. (1996) Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature (London), 379, 728–732. Nowak, L.G., Munk, M.H.J., Nelson, J.I., James, A.C., and Bullier, J. (1995a) The structural basis of cortical synchronization. I. Three types of interhemispheric coupling. Journal of Neurophysiology, 74, 2379–2400.
Cortical processing by fast synchronization
233
Nowak, L.G., Munk, M.H.J., Girard, P., and Bullier, J. (1995b) Visual latencies in areas V1 and V2 of the macaque monkey. Visual Neuroscience, 12, 371–384. Nowak, L.G., and Bullier, J. (2000) Cross-correlograms for neuronal spike trains: Different types of temporal correlation in neocortex, their origin and significance. In: R.Miller (ed) Time and the brain. Harwood Academic Publishers, Reading, in press. Reitboeck, H.J. (1983) A multi-electrode matrix for studies of temporal signal correlations within neural assemblies. In E.Başar, (ed), Synergetics of the Brain, pp. 174–182. Ritz, R., Gerstner, W., Fuentes, U., and van Hemmen, J.L. (1994) A biologically motivated and analytically soluble model of collective oscillations in the cortex. II: Application to binding and pattern segmentation. Biological Cybernetics, 71, 349–358. Rolls, E.T., and Tovee, M.J. (1994) Processing speed in the cerebral cortex and the neurophysiology of visual masking. Proceedings of the Royal Society of London, Series B, 257, 9–15. Rougeul, A., Bouyer, J.J., Dedet, L., and Debray, O. (1979) Fast somato-parietal rhythms during combined focal attention and immobility in baboon and squirrel monkey. Electroencephalography and Clinical Neurophysiology, 46, 310–319. Saam, M., and Eckhorn, R. (1998) Width of association field in topographic neural map depends on lateral spike velocity and temporal dispersion of correlated input: a learning model with spiking neurons. In: N.Elsner and R.Wehner (eds) New Neuroethology on the Move. Thieme, Stuttgart, p. 767. Sanes, J.N., and Donoghue, J.P. (1993) Oscillations in local field-potentials of the primate motor cortex during voluntary movement. Proceedings of the National Academy of Sciences, U.S.A., 90, 4470–4474. Schanze, T., and Eckhorn, R. (1997) Phase correlation among rhythms present at different frequencies: Spectral methods, application to microelectrode recordings from visual cortex and functional implications. International Journal of Psychophysiology, 26, 171–189. Schillen, T.B., and König, P. (1994) Binding by temporal structure in multiple feature domains of an oscillatory neuronal network. Biological Cybernetics, 70, 397–405. Schneider, J., Eckhorn, R., and Reitböck, H.J. (1983) Evaluation of neuronal coupling dynamics. Biological Cyberneticsetics, 46, 129–134. Schott, U. (1995) Simultaneous unsupervised learning of phase-separated patterns in feed-back associative memories . Masters Thesis, Philipps University, Depart ment of Neurophysics, Marburg, Germany. Schuster, H.G., and Wagner, P. (1990) A model for neuronal oscillations in the visual cortex. Biological Cybernetics, 64, 72–82. Sillito, A.M., Jones, H.E., Gerstein, G., and West, D.C. (1995) Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London, 369, 479–482. Sompolinsky, H., Golomb, D., and Kleinfeld, D. (1990) Global processing of visual stimuli in a neural network of coupled oscillators. Proceedings of the National Academy of Sciences, U.S.A., 87, 7200–7204. Steriade, M. (1993) Central core modulation of spontaneous oscillations and sensory transmission in thalamo-cortical systems. Current Opinion in Neurobiology, 3, 619– 625. Steriade, M., Amzica, F., and Contreras, D. (1996) Synchronization of fast (30–40 Hz) spontaneous cortical rhythms during brain activation. Journal of Neuroscience, 16, 392–417.
Time and the brain
234
Stoecker, M., Reitboeck, H.J., and Eckhorn, R. (1996) A neural network for scene segmentation by temporal coding. Neurocomputing, 11, 123–134. Swadlow, H.A. (1983) Efferent systems of primary visual cortex: A review of structure and function. Brain Research Reviews, 6, 1–24. Tallon-Baudry, C, Bertrand, O., Wienbruch, C., Ross, B., and Pantev, C. (1997) Combined EEG and MEG recordings of visual 40 Hz responses to illusory triangles in human. NeuroReport, 8, 1103–1107. Traub R.D., Whittington M.A., Colling S.B., Buzsaki G., and Jefferys J.G.R. (1996) Analysis of gamma rhythms in the rat hippocampus in vitro and in vivo. Journal of Physiology (London), 493, 471–484. Traub R.D., Whittington M.A., Stanford I.M., and Jefferys J.G.R. (1996) A mechanism for generation of long-range synchronous fast oscillations in the cortex. Nature, London, 383, 621–624. Ts’o, D.Y., Gilbert, C.D., and Wiesel, T.N. (1986) Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by crosscorrelation analysis. Journal of Neuroscience, 6, 1160–1170. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London, 373, 515–518. van Vreeswijk, C., Abbott, L.F., and Ermentrout, G.B. (1994) When inhibition not excitation synchronizes neural firing. Journal of Computational Neuroscience, 1, 313– 321. Wennekers, T., Erb, M., Palm, G., and Eckhorn, R, (1994) Dependence of oscillation frequency on stimulus parameters in primary visual cortex of monkeys: simulations and experimental results. European Journal of Neuroscience, Suppl 7, 66.03. Wennekers, T., and Palm, G. (2000) Cell assemblies, associative memory and temporal structure in brain signals. In: R. Miller (ed) Time and the brain. Harwood Academic Publishers, Reading, In Press. Wertheimer, M. (1923) Untersuchungen zur Lehre von der Gestalt: II. Psychologische Forschung, 4, 301–350. Woelbern, T., Frien, A., Eckhorn, R., Bauer, R., and Kehr, H. (1994) Distributions of oscillation frequencies depend on stimulus size and velocity in monkey visual cortex. In: N. Elsner and H. Breer (eds), Proceedings of the 22nd Göttingen Neurobiology Meeting. Thieme Verlag, Stuttgart, New York, p. 518.
7 EEG Alpha and Cognitive Processes Wolfgang Klimesch Department of Physiological Psychology, Institute of Psychology, University of Salzburg, Hellbrunnerstr. 34, A-5020 SALZBURG Austria Tel: 0043 662 8044 5120 or 5100; Fax 0043 662 8044 5126 e-mail: [email protected]
The suppression of the alpha rhythm, which can be observed in response to a variety of different tasks, is the best known EEG phenomenon. Because the alpha rhythm is most evident in relaxed wakefulness, the implication is that alpha reflects mental inactivity rather than active cognitive processing. However, more recent evidence suggests that activity in the alpha frequency range reflects active information processing, and that different frequencies within the alpha frequency band have quite different functions. Upper alpha, which is a 2 Hz band above the individually determined alpha frequency (IAF), reflects the processing of sensory-semantic information, whereas lower-1 alpha, in the range of IAF-4 Hz to IAF-2 Hz, is associated with attentional processes. The lower-2 alpha band, which falls just below IAF (IAF-2Hz to IAF), seems to reflect expectancy. The general conclusion is that different EEG frequencies reflect different types of cognitive processes. KEYWORDS: Alpha; Brain oscillations; Pace maker; Attention; Memory
1. INTRODUCTION Alpha is an oscillatory EEG activity, within a range of about 8–13 Hz, which has been known since the early days of EEG research. It can best be observed over posterior regions of the head during relaxed wakefulness with closed eyes (cf. Niedermeyer, 1993a). As compared to other EEG phenomena it is characterized by the following outstanding features: (i) spectral analysis reveals that alpha is the dominant frequency in the human EEG, and with the exception of extremely low frequencies (below about 2.5 Hz) shows maximal power; (ii) it is well documented that alpha desynchronizes (becomes suppressed) in response to a variety of different tasks; and (iii) Alpha is the only frequency in the human scalp EEG that clearly and undoubtedly reflects an oscillatory process. Because of its outstanding features, many researchers have assumed that alpha may play a key role in understanding other EEG phenomena (Başar, 1997; Başar et al., 1997a,b). Before considering the functional meaning of alpha in more detail, we first
EEG Alpha and cognitive processes
237
review some of its most important properties.
2. ALPHA POWER AND FREQUENCY: BASIC FINDINGS Unlike frequency, power measurements are strongly effected by a variety of morphological factors of the head (such as the thickness of the skull or the volume of cerebrospinal fluid) and also by methodological and technical factors (such as interelectrode distance, or type of montage). If all of these factors could be perfectly controlled, or kept constant between individuals and different studies, alpha power would be an ideal measure to reflect the number of neurones that oscillate synchronously within the alpha frequency range. One way to compensate for the influence of these factors is to use relative power measurements (e.g. percentages, z-scores, differences or ratios between different conditions). Another way is to use large samples of subjects. In addition to these distorting influences, alpha power is well known to change with sensory, motor and cognitive task demands (see below) but also with age. Age-related changes in absolute and relative alpha power are not linear, but occur in several stages (Epstein, 1980; Hudspeth and Pribram, 1990). On the average, however, young children show an age related increase in alpha power up to about 9 years (Gasser et al., 1988; John et al., 1980; Katada et al., 1981; Marthis et al., 1980; Niedermeyer, 1993b) or even 15 years (John et al., 1980). For older subjects (about 50 years and older) a variety of studies have found age-related changes in EEG power (e.g. Obrist, 1954; Christian, 1984; review in: Van Sweden et al., 1993). These studies show a general slowing of the EEG, with a pronounced power increase in the slow frequency ranges of about 7 Hz and below, and a decrease in higher frequencies of about 7 Hz and higher (e.g. Obrist, 1954; Christian, 1984; Markand, 1990). A very similar pattern of results is obtained if cognitively impaired or demented subjects are compared with age-matched controls (Coben et al., 1985; Brenner et al., 1986; John and Prichep, 1993, [p. 995]). It should be noted that a few studies have reported that demented (Brenner et al., 1986) or old subjects (Könönen and Partanen, 1993) even show an increase in alpha power. In the study of Brenner et al. (1986) this effect was restricted to the lower alpha band. More recent studies have shown repeatedly that a selective increase in lower alpha power reflects a state of decreased attention (Klimesch et al., 1992, 1993; Crawford et al., 1995). Könönen and Partanen (1993) used a bipolar montage. Thus their findings reflect effects that are topographically restricted, and may therefore not be related to changes in absolute or relative power as reported in other studies. In summary, the reported findings suggest that alpha power (in the broad range of 8–13 Hz) changes with age in a nonlinear way. There is a strong increase from early childhood to adulthood. During adulthood alpha power remains stable, but beyond the age of 50 or 60 years (Breslau et al., 1989) a period of a decrease in alpha power can be observed, which may well be due to age-related neurological disorders, and not to age per se (Hubbard et al., 1976). In addition to alpha power, alpha reactivity or suppression is also known to change with age (e.g. Duffy et al., 1984). Many studies (including those reviewed below in the
Time and the brain
238
section about the functional meaning of alpha power) have shown that the extent of event-related alpha power changes (or event-related variability in alpha power) is indeed related to cognitive performance (Klimesch et al., 1997a,b). As an example, Sheridan et al. (1988) have reported that Alzheimer demented subjects, with normal alpha that is suppressed during eye opening, have significantly higher WAIS Performance IQ scores than patients with irregular alpha that does not change (or changes only weakly) during eye opening. Fuller (1978) has found that learning disabled children show less taskrelated alpha attenuation than an age-matched control group. Age-related changes in alpha frequency are well documented. During childhood, alpha frequency increases, from about 7 Hz at an age of about 2 years, to about 10 Hz during adulthood (e.g. Somsen et al., 1997; see also summary in Niedermeyer, 1993c). Then, with increasing age, alpha frequency starts to decline, down to as low as 7 Hz at an age of about 90 years. Köpruner et al. (1984) have found a linear relationship (alpha peak frequency=11.95–0.053 x age) within the age range of adult subjects. According to this relationship, a young adult (e.g. 20 years) has an expected peak frequency of 10.89 Hz, whereas a 70 year old subject shows a drop of 2.65 Hz, down to a frequency of 8.24 Hz. Even in a sample of age-matched subjects, interindividual differences are about as large as age-related differences. When interindividal variability is described in terms of a normal distribution (Klimesch, 1996), about a third of subjects are expected to show a difference in alpha frequency of more than 2 Hz.
3. THE FUNCTIONAL MEANING OF ALPHA FREQUENCY Several experiments reported by Surwillo (Surwillo, 1961, 1963a,b, 1964a,b, 1971) indicate that alpha frequency is significantly correlated with the speed of information processing, as measured by reaction times (RT). Subjects with high alpha frequency show fast RTs, whereas slow subjects have low alpha frequency. These findings are in good agreement with the results from a variety of experiments from our laboratory, which have revealed that alpha frequency of good memory performers is about 1 Hz higher than that of age-matched samples of bad performers (Klimesch et al., 1990a; Klimesch et al., 1990b; Klimesch et al., 1993a,b; Klimesch, 1995, 1996, 1997). Because good performers are faster in retrieving information from memory than bad performers (Klimesch, 1994), these data also indicate that alpha frequency is related to the speed of information processing or reaction time. It is important to note that all of these findings are based on the interindividual variability of alpha frequency, which was found to be significantly related to interindividual differences in the speed of information processing. It is well known that alpha frequency varies not only between individuals, but also within individuals. These intraindividual (or task-related frequency changes) may be as large as interindividual differences in alpha frequency (Klimesch et al., 1990b, Klimesch et al., 1993a; Köpruner et al., 1984).
EEG Alpha and cognitive processes
239
3.1. Task-Related Shifts in Alpha Frequency The interesting question is whether task-related or intraindividual differences in alpha frequency are also related to the speed of information processing. We have addressed this question in an experiment (Klimesch et al., 1996a) in which subjects had to perform a standard recognition task. The EEG was recorded, (i) during the study phase, in which the target words (to be remembered) were presented, (ii) during the recognition performance of the randomly presented targets and distractors and (iii) during a resting condition. RTs were measured in the recognition task. The time course of a single trial, and the intervals used for measuring alpha power are depicted in Figure 1. Because short intervals of only 1 sec were selected for detecting even fast intraindividual shifts in alpha frequency, “gravity” instead of peak frequency was measured. Gravity frequency has the disadvantage that it varies as a function of the selected frequency window. However, considering the large interindividual variability in alpha frequency, parts of the alpha power distribution will fall outside the relevant frequency range if a fixed frequency window (e.g. 8–13 Hz) were to be used. To avoid this problem, we have determined the frequency windows individually for each subject (i), by using peak frequency f(i) as an anchor point. As in studies performed earlier (Klimesch et al., 1990b), the measurement of alpha frequency was done in two steps. First, peak frequency f(i) was measured, during a period of 3 sec preceding the presentation of a word in the study phase (in which alpha power was comparatively large). Then, in a second step, mean (gravity) frequency was calculated within a frequency window which goes 4 Hz below and 3 Hz above f(i) for each subject. This method resulted in a frequency window from 6.4–13.4 Hz on the average. In order to avoid confusions with conventional measures we use the term “individual alpha frequency” (IAF). The variability in intraindividual differences was studied by comparing correctly- and incorrectly-identified targets and distractors. Interindividual differences were studied by correlating RT and alpha frequency over the sample of subjects. It is well known that correctly recognized words (targets or distractors) are identified much faster than incorrectly recognized words (e.g. Klimesch, 1994). Thus, the interesting question is whether alpha frequency is higher for correctly as compared to incorrectly identified words. The results have shown that this is not the case and, thus, lead to the rejection of the hypothesis that the speed of information processing is related to intraindividual, taskrelated shifts in alpha frequency. On the other hand, interindividual differences in alpha frequency were indeed related to RT, as the correlation coefficients in Table 1 indicate. In the resting phase, none of the correlations reached significance. Significant correlations were found for correctly identified words, and only at Pz and occipital recording sites over the left side of the scalp. It is interesting to see that the smallest correlations were obtained during the poststimulus interval of the recognition phase, when the degree of alpha power suppression reached a maximum. By comparing the spectra for the resting period with the reference,
Time and the brain
240
Figure 1. Structure of a typical trial for measuring event-related changes in band power. The use of a warning signal (WS) helps a subject to stay relaxed during a reference interval, and to increase his/her alertness before an imperative stimulus is presented. The time course (in sec) is indicated by numbers.
Table 1 Correlation coefficients between RT and IAF for remembered targets
Occipital, left hemisphere Study
Pz
Recognition
Study
Recognition
ref
pre
post
ref
pre
post
ref
pre
post
ref
pre
post
−.42*
−.43*
−.22
−.45*
−.31
−.11
−.39
−.42*
−.24
−.38
−.29
−.14
Note: *=significant beyond the 5%-level, one sided; ref, pre, post=reference, pre- and poststimulus interval. No significant correlations were found at frontal, central and parietal sites and over the right side of the scalp
pre- and poststimulus intervals, we have found that alpha power becomes increasingly suppressed or desynchronized if the task demands increase, a result that is known since Berger (e.g. Berger, 1929). Intraindividual or task-related shifts in alpha frequency may be due to an asymmetric desynchronization (suppression) of the alpha rhythm, as demonstrated in the two power spectra in Figure 2. Two different cases can be distingushed: desynchronization may favour either the lower (left spectra in Figure 2) or the upper (right spectra in Figure 2) alpha band. The first case leads to an increase, the second to a decrease in alpha frequency. Which of the two cases dominates depends on the type of task. As an example, if semantic task demands predominate, a selective desynchronization in the upper alpha band can be
EEG Alpha and cognitive processes
241
Figure 2. Different types of desynchronization (dotted line) during a test, in comparison to a reference interval (bold line). Asymmetric desynchronization, a very common finding, indicates that the lower and upper alpha band desynchronize independently from each other. The left spectrum shows a case where selective desynchronization of the lower alpha band leads to an increase in frequency. On the other hand, a selective desynchronization of the upper alpha band leads to a decrease in frequency as is shown in the right spectrum.
observed (see the section about the “functional meaning of the upper alpha band”, below). On the other hand, if attentional demands predominate, a selective desynchronization in the lower alpha band can be observed (see the section about the “functional meaning of the lower alpha band”, below). This selective desynchronization in the lower alpha band leads to an increase in frequency, which is frequently observed when task demands increase (e.g. Osaka, 1984; Klimesch et al., 1993a). When focusing on the relationship between RT and alpha frequency, the question arises, under which condition should alpha frequency be measured. Under specific task demands, alpha frequency may be distorted by a selective desynchronization of the lower or upper alpha band. During resting conditions, in which subjects are relaxed, the lower alpha band may even synchronize, showing an increase in band power, and consequently a decrease in alpha frequency. Thus, depending on the condition under which the EEG is measured, the frequency of the alpha rhythm may be distorted by a selective synchronization or desynchronization in the lower and or upper alpha band. We assume that a medium level of attentional (or task) demands shows a more symmetric desynchronization and, thus, gives a less distorted measure of alpha frequency as compared to a very difficult task (with a strong selective desynchronization of the lower
Time and the brain
242
alpha), or the resting condition in which subjects may already be drowsy (with a strong synchronization in the lower alpha band). In summarizing the results about alpha frequency and reaction time, we see that during a state of maximal desynchronization (which was found during the poststimulus interval of the recognition phase) the relationship between RT and alpha frequency is blurred. In a similar way, during a state of maximal alpha synchronization (which was found during the resting phase) no significant correlations between RT and alpha frequency were found. Only if alpha shows a medium level of desynchronization (which was found during the reference and prestimulus interval), did a significant relationship between alpha frequency and RT emerge (cf. Table 1). Our results are well in line with Surwillo’s findings (Surwillo, 1961, 1963a,b, 1964a,b, 1971) because he used only those epochs for measuring alpha frequency which preceded a subject’s response. Thus, his findings too indicate that alpha frequency is an indicator of processing speed only if it is measured during a prestimulus interval in which alpha oscillations are in a state of medium desynchronization. The important conclusion is that task related frequency shifts are not “real” and, thus, not related to the speed of information processing. Our findings contradict the view that there is a single alpha rhythm. They indicate (at least) that a lower and an upper alpha frequency component must be distinguished. It may even be hypothesized that there is an entire population of different alphas (Başar, 1997; Başar and Schürmann, 1997) which oscillate synchronously within a rather broad frequency range during a resting state. In a state of medium desynchronization reflecting increased alertness, the frequency window in which synchronous oscillations occur becomes smaller because lower and higher alpha frequency components already start to desynchronize. During this state of medium desynchronization, the dominant frequency of the alpha rhythm may reflect the activity of alpha pacemaker(s), possibly in the thalamus (e.g. Steriade et al., 1990). During strong desynchronization, different neural populations start to oscillate with different frequencies within the alpha band (cf. Klimesch, 1995, 1996). As a result, the dominant EEG alpha rhythm starts to disappear and the power spectrum becomes quite flat. 3.2. Brain Volume and Cognitive Maturity The findings just reported suggest that alpha frequency is an indicator of cognitive performance. This assumption is supported not only by the well documented relationship, with RT, but also by the fact that alpha frequency increases (although in a nonlinear way) from early childhood to adulthood, and then decreases with age over the remaining life span in a way which parellels that of brain volume and general cognitive performance (e.g. Bigler et al., 1995; Willerman et al., 1991). However, cognitive performance is not the only factor that affects alpha frequency, as the findings of Nunez et al. (1978) indicate. These authors have found that alpha frequency is negatively correlated with brain size. Nunez (1995) assumes that many different alpha waves travel from different places over the cortex. They may be generated in the cortex, or may be induced at least in part by thalamo-cortical feedback loops. The basic idea is that after substantial interference, standing waves are produced. An
EEG Alpha and cognitive processes
243
important prediction from his theory is that larger brains will, on the average, produce lower alpha frequencies. According to his theory (Nunez, 1995: p. 560ff.), the fundamental mode frequency, which may be interpreted as alpha frequency, can by estimated by:
where Ve is the velocity of action potentials (in cortico-cortical axons) and R is the brain radius of an idealized sphere (cf. Nunez, 1995: p. 82ff). Figure 3 illustrates this negative correlation between alpha frequency and brain size for different values of Ve. Nunez et al. (1978) report a correlation coefficient with head size (defined as the cube root of the product of three measures, ear to ear, nasion to inion, and circumference) of r =−.206 (p=.02, n=123).
4. THE FUNCTIONAL MEANING OF ALPHA POWER One of the most typical (and at the same time surprising) features of the alpha rhythm is its tendency to be reduced in amplitude (power) or even blocked when task demands increase. In some sense this behaviour of the alpha rhythm is the opposite of what one would expect. It would be plausible to assume a general increase in electrical brain activity with active mental processes. However, as far as the alpha frequency range is concerned an inverse relationship between band power and mental activity can be observed. This certainly holds true for more sophisticated measures of event-related alpha power changes (as we will see in the next section below). Another interesting finding is that alpha frequency is inversly related to alpha power. High alpha frequency is related to low alpha power, whereas low alpha frequency is related to high alpha power (Wieneke et al., 1980; Pigeau and Frame, 1992; see also review in Markand, 1990). High alpha frequency is related to good performance and is correlated with low alpha power if attentional demands are high (cf. Fig. 2). Thus, low alpha power should also be related to good performance. Studies measuring hemispheric differences in alpha power have demonstrated that alpha suppression is more pronounced over the task relevant hemisphere (Furst, 1976; Galin et al., 1982; Klimesch et al., 1990c, 1997b; but see Papanicolaou et al., 1986) and that over the right hemisphere alpha power is generally
Time and the brain
244
Figure 3. Hypothetical regression lines for head size and alpha frequency as predicted by Nunez (1995). Global (in contrast to local) alpha frequency is interpreted in terms of standing waves, and is estimated by f(alpha)=Ve/27πR, where Ve is the velocity of action potentials (in cortico-cortical axons) and R is the brain radius (in cm) of an idealized sphere (cf. Nunez, 1995, p. 82ff). A radius of R=14 cm represents a brain with a cortical surface of 2463 cm2- which is about the mean value for young adults. Because alpha frequency is about 10 Hz in young adults, the corresponding velocity Ve of action potentials is about 8.5 m/sec. Differences in the thickness of the myelin sheath play an important role in determining Ve, and thus have a strong effect on alpha frequency. A difference in Ve of 1 m/sec is related to a difference in alpha frequency of more than 1 Hz on the average. The lower alpha frequency in young children may be due to differences in the myelin sheath.
larger than over the left hemisphere (Markand, 1990; Niedermeyer, 1993a). Because even for left handed subjects, the left hemisphere has a probability of about p=.70 of being dominant (Kolb and Whishaw, 1985), these findings indicate that the dominant hemisphere is more active, and thus shows less alpha power. One important exception to this inverse relationship between mental activity (or cognitive performance) and alpha power refers to the finding that EEG power in the frequency range above 6 Hz decreases beyond an age of about 50 years, and that cognitively impaired or demented subjects also show a decrease in alpha band power. One likely interpretation is that not age per se but age-related neurological disorders (which may lead to a pronounced decrease in brain volume and loss of neurones) are the cause of this decline in alpha power see Klimesch, 1999 for a review. The validity of this
EEG Alpha and cognitive processes
245
assumption is supported by studies which report that healthy elderly subjects do not show a general slowing of the EEG (Hartikainen et al., 1992) or of the alpha rhythm (e.g. Hubbard et al., 1976; Torres et al., 1983) or changes in EEG power. 4.1. The Measurement of Event-Related Changes in Alpha Band Power Besides absolute alpha power, the measurement of alpha reactivity or alpha suppression is a promising method for studying the dynamics of event-related band power changes in selected frequency bands. A better understanding of these dynamics was provided by a new method, introduced by Pfurtscheller and Aranibar (1977), who coined the term ERD (event-related desynchronization). This method allows one to calculate the percentage of a event-related power change in different frequency bands. In more detail, the procedure is as follows. The EEG data for each epoch and each recording site are digitally bandpass filtered, squared (in order to obtain simple power estimates) and averaged separately for each lead, experimental condition and subject. The percentage of an event-related change in band power during a test interval is calculated by applying the following formula: ERD%={[(alpha band power, reference interval)—(alpha band power, test interval)]/(alpha band power, reference interval)}* 100. As reference, usually an interval of 1000 ms at the beginning of each trial is used. Test intervals usually are selected intervals (of 125 ms or more) preceding and following the presentation of a stimulus (cf. Figure 1). It should be noted that positive ERD-values indicate a state of desynchronization or power suppression, whereas negative ERD-values reflect a state of synchronization or increase in power. One critical question when using ERD is the selection of frequency bands. Due to large interindividual differences of alpha frequency, large portions of alpha power will fall outside a fixed frequency window and invite misleading interpretations. As an example, for a subject with a low mean alpha frequency of 8 Hz, the lower alpha band—defined as the frequency range below mean alpha frequency—already falls outside the frequency window of the fixed band (8–13 Hz; lower alpha: 8–10.5 Hz; upper alpha: 10.5–13 Hz). Thus, in this example the frequency band of 8–13 Hz would only cover the upper alpha and some portions of lower beta. As a consequence, changes in the (true) upper alpha band would be confused with those of the (fixed) lower alpha band. To avoid the problems of fixed-frequency windows, we (e.g. Klimesch et al., 1990c; Klimesch, 1997a,b,c) suggested the use of individually-adjusted frequency bands. Mean peak frequency f(i) of the dominant EEG frequency in the alpha band is used as an anchor point for the selection of frequency bands. Then, frequency bands with a width of e.g. 2 Hz can be defined: (f(i)–6) to (f(i)–4); to (f(i)−4) to (f(i)−2); (f(i)−2) to f(i); f(i) to (f(i)+2). For each subject, the ERD is calculated within these individually-determined frequency bands, which are termed theta, lower (1) alpha, lower (2) alpha, and upper alpha. Averaged over subjects, the cut-off points between the frequency bands for young adults yield the following typical values (e.g. Klimesch et al., 1996b): 4.7–6.7; 6.7–8.7; 8.7–10.7; 10.7–12.7. Adjusting frequency bands individually is important not only because alpha frequency varies as a function of several factors such as age, memory performance and attentional
Time and the brain
246
demands. What is even more important is that the frequency of the transition region between a task-related decrease in band power (as an indicator for the alpha band) and a task-related increase in band power (as an indicator for the theta band) also shows a tendency to vary as a function of alpha frequency (Klimesch et al., 1996b).
5. THE FUNCTIONAL MEANING OF THE LOWER ALPHA BAND It is important to note that alpha desynchronization is not a unitary phenomenon. If different frequency bands are distinguished within the range of the extended alpha band, two distinct patterns of desynchronization can be observed. Desynchronization of lower alpha (in the range of about 6–10 Hz) is obtained in response to a variety of (non-task specific) factors which may be best subsumed under the term “attention”. It is topographically widespread over the entire scalp, and probably reflects general task demands and attentional processes. Desynchronization of upper alpha (in the range of about 10–12 Hz) is topographically restricted and develops during the processing of sensory-semantic information, as recent evidence indicates (e.g. Klimesch et al., 1994; Klimesch et al., 1996c, 1997a,b). Since the early days of EEG research it has been well known (and has been shown repeatedly) that the alpha rhythm is sensitive to attentional demands (cf. Ray and Cole, 1985; Mulholland, 1969). Studies from our laboratory, however, indicate that it is particularly the lower alpha band which is responsible for this observation. Cognitive theories of attention distinguish between two basic aspects, divided attention and selective attention (for a review see e.g. Cowan, 1988; Kahneman and Treisman, 1984). These aspects refer to a subject’s processing capacity and the extent to which different tasks draw on the limited attentional resources. Besides these more cognitive aspects, Posner (e.g. Posner, 1975, 1995) considers alertness (a concept similar to arousal) another important component of attention. Alertness is further subdivided into phasic and tonic alertness. Phasic changes in alertness occur at a rapid rate, and are under volitional control, whereas tonic changes occur at a much slower rate and are not under direct volitional control. A good example of a phasic change is the increase in alertness after the presentation of a warning signal. The effects of phasic alertness and expectancy (as a special form of selective attention) were studied in two experiments reported by Klimesch et al. (1992). The basic design consisted of a warned stimulus presentation. In Experiment 1, subjects had to perform a reading task. Alertness was manipulated by randomly varying the interval between the appearance of the warning signal and the imperative stimulus. In Experiment 2, subjects had to perform two different category judgement tasks. In the semantic judgement task, they had to indicate to which category (animals versus tools) each of a series of words belonged. In the numerical judgement task, subjects had to judge whether the presented numbers were odd or even. Expectancy and alertness were varied in an orthogonal design. Whereas alertness was varied in the same manner as in Experiment 1, expectancy was manipulated by presenting the words and numbers either in blocks or in random order. Only under the blocked presentation did subjects know which type of task they had
EEG Alpha and cognitive processes
247
to perform on the next trial. The most important results of both experiments is that the effects of the warning signal showed up in the lower alpha band. Only in the lower alpha band was a pronounced desynchronization found in response to the appearance of the warning signal. This finding indicates that the lower alpha band reflects changes in phasic alertness. In contrast to the lower alpha band, the upper alpha band responds to changes in expectancy, and shows the most pronounced task-related power suppression from the preto the poststimulus interval. This indicates that the upper alpha band responds to the presentation of the imperative stimulus and to task-related cognitive processes. This interpretation agrees with the fact that changes in the upper alpha band are strictly localized, occurring at occipital, temporal and parietal recording sites. Because of the visual-verbal nature of the tasks, these brain areas would have been expected to exhibit the most significant increase of ERD. In more general terms, our results suggest that the lower alpha band reflects primarily non-task-specific processes (such alertness) whereas the upper alpha band reflects primarily task specific cognitive processes (such as perceptual processes and task related selective attention or expectancy). It is very important to emphasize that two different types of attention components (alertness and expectancy) are reflected in two different frequency bands (the lower and upper alpha band, respectively). However, in contrast to alertness, which is independent of the task type, expectancy represents a more cognitive aspect of attention, which is highly dependent on the type of cognitive operations a subject selects to prepare for a task. In everyday life the meaning of attention is much closer to the term “alertness” as compared to “expectancy”. For this reason we will prefer and use the term attention instead of alertness. The hypothesis that the lower alpha band reflects attention was supported by a variety of other studies. As an example, Klimesch et al. (1993a) have found that good memory performers (as compared to bad performers) show a significantly stronger desynchronization during encoding and retrieval in the lower alpha (for similar results see also Sterman et al., 1996). We have interpreted these findings by assuming that good memory performance may be due to a phasic increase in alertness, which is reflected by a strong desynchronization in the lower alpha band. Thus, a lack of desynchronization in the lower alpha band may be related to a lack of alertness or attention in general. In good agreement with this interpretation, Crawford et al. (1995) report that subjects with poor sustained attention, who have difficulty inhibiting distracting environmental stimuli, have significantly more lower alpha power than subjects not reporting attentional deficits. An even better analysis of attentional processes is achieved if an extended lower alpha band (divided in a lower-1, and lower-2 alpha with 2 Hz bandwidths) is used, as the findings of a more recent study (Klimesch et al., 1997d) indicate. (For details of method see the section about ERD measurement above.) The results of a visual oddball task, with a warned stimulus presentation (Klimesch et al., 1997d) demonstrate that the to-beattended stimulus (also termed the “target”) shows a particularly strong desynchronization in the lower-1 alpha band. What is even more important is the finding that a significant response (decrease in band power) to the warning signal could be observed only in the lower-1 alpha band, and when the warning signal preceded a target.
Time and the brain
248
Because targets are rare, and because no more than three targets or nontargets were allowed to occur in a row, it is likely that after a few trials subjects were well able to predict (expect) the occurrence of a target. Thus, as the upper two diagrams in Figure 4 demonstrate, the warning signal shows an alerting effect only when a target is expected. An interesting finding refers to the way the three alpha bands respond to the presentation of the warning signal. Only in the lower-1 alpha band, can a phasic response to the warning signal be observed (i.e. a decrease in band power that is interrupted and, thus, separable from the decrease in band power in response to the imperative stimulus). A tonic response (i.e. a steady decrease in band power, starting even before the onset of the imperative stimulus and lasting for the entire poststimulus period) was obtained in the lower-2 and upper alpha band. This tonic response may well be interpreted in terms of stimulus-related expectancy. According to this interpretation we can assume that the alerting effect of the warning signal should be particularly strong in those trials where a target is presented. In contrast to the lower-1 alpha band, the tonic decrease in power in the lower-2 and upper alpha band, starting as early as 1250 ms before the presentation of a
Figure 4. Event-related changes in induced band power (IBP; cf. Klimesch et al., 1998) for targets and nontargets at selected recording sites (Cz, Pz) and different frequency bands. IBP represents band power changes that are devoid of phase-locked EEG activity and are, thus, not influenced by event-related potentials (ERPs). In order to eliminate intersubject variations, band power was z-transformed. Negative z-values exceeding the dotted line (99% confidence interval with respect to the reference interval) represent a significant decrease in band power (desynchronization). It is important to note that in the upper alpha band a significant desynchronization can be observed
EEG Alpha and cognitive processes
249
primarily for targets, whereas the lower-1 and lower-2 alpha band also exhibit distinct responses to the warning signal.
target or nontarget, may reflect the way a subject prepares to respond to a particular type of stimulus. When expecting a nontarget, subjects prepare to ignore it, but when expecting a target subjects prepare to respond. Because expectancy plays a role during the prestimulus period for both types of stimulus, there is no reason to assume that the prestimulus period will differ between target and nontarget trials. This effect of expectancy seems to be reflected in the lower-2 alpha band (for the entire time course of the trial) and in the upper alpha band (but only up to the point when the imperative stimulus actually is processed). It is important to emphasize that the processing of targets and nontargets will be strikingly different. Because we have assumed that the upper alpha band reflects task specific effects, we expect that, particularly during the second half of the poststimulus interval (i.e. during the time the stimulus type is recognized), targets should show a significantly stronger decrease in upper alpha band power. This indeed is the case as inspection of the lower two diagrams in Figure 4 indicate. The interesting finding of this latter study is that the distinction of three bands within the extended alpha band proved useful for making a better distinction of different cognitive processes. The alerting effects of the warning signal is restricted to the lower-1 alpha band, whereas the processing of the task relevant stimulus shows up in the upper alpha band only. The effects of expectancy dominate in the lower-2 alpha band band, and can also be observed in the upper alpha band. Finally it should be mentioned that the reported findings are not due to (or distorted by) the influence of evoked potentials. By applying a method which is based on the “intertrial variance technique” (Kaufman et al., 1989; Kalcher and Pfurtscheller, 1995) we (Klimesch et al., 1997d) were ably to demonstrate that the above reported results are independent of phase-locked EEG activity and represent changes in “induced band power” or IBP. We assume that IBP reflects induced oscillations that are modulated by stimuli or events, and which (in contrast to evoked rhythms) do not respond in a phaselocked manner (for a similar but much broader definition see: Bullock, 1992).
6. THE FUNCTIONAL MEANING OF THE UPPER ALPHA BAND In the previous section we have suggested that the lower alpha band reflects attentional demands, whereas the upper alpha band is primarily associated with specific task demands such as the visual processing of the imperative stimulus. More recent experiments (e.g. Klimesch et al., 1994, 1997a,b,c) have shown that the upper alpha band may reflect the processing of sensory-semantic information. In Klimesch et al. (1994) subjects performed a semantic congruency task, in which they had to judge whether or not the sequentially presented words of concept-feature pairs (such as “eagle-claws” or “pea-huge”) are semantically congruent. Then, without prior warning, they were asked to perform an episodic recognition task. This was done to prevent subjects from using semantic encoding strategies, and thus to increase episodic memory demands. In the
Time and the brain
250
episodic task, the same concept-feature pairs were presented together with new distractors. Subjects had now to judge whether or not a particular concept-feature pair was one already presented during the semantic task. From reaction time experiments it is well known that episodic tasks of the kind used here are much more difficult than semantic tasks (see the review in Klimesch, 1994). This fact is important because we assume that upper alpha desynchronization selectively reflects semantic task demands regardless of general task demands, such as task difficulty. The results have shown that in spite of the fact that the semantic task is easier, the upper alpha band shows a significantly stronger desynchronization during the processing of the semantic task, as compared to the episodic task. In a similar study by Klimesch et al. (1997b) three experiments were carried out in which semantic processing demands were varied. The same sample of subjects was used in all of the three experiments. In Experiment 1 subjects had to judge whether pairs of sequentially presented words were semantically congruent (in a manner similar to that in the semantic task of Klimesch et al., 1994). In the following experiments subjects were presented the initial words of word-pairs from Experiment 1 and were asked to perform a free association task (in Experiment 2), and a cued recall task (in Experiment 3). The results of this study reveal that a significant increase in upper alpha desynchronization was found only during that time interval in which the semantic task actually was carried out (cf. Figure 5). Whereas the theta band does not respond to semantic task demands at all, the two lower alpha bands exhibit a stepwise increase in desynchronization that exceeds the level of significance even before a semantic relationship between the initial and second words of the word-pairs can be detected. Because the semantic task cannot be carried out until the second word of a pair is presented (during t4, in Figure 5), it becomes evident that only the upper alpha band responds selectively to semantic task demands, as the data in Figure 5 demonstrate. Furthermore, upper alpha desynchronization was significantly larger during t4 and t5 of Experiment 1, as compared to t2 and t3 of Experiment 2 and 3. Because semantic memory demands were highest in the congruency task, this finding provides further evidence for the hypothesis that the upper alpha band is associated with semantic memory processes. Upper alpha desynchronization during the semantic task (during t5) is strictly localized over the left hemisphere as Figure 6 shows. This finding is well in line with a variety of PET-studies and with Tulving’s HERA model (Tulving et al., 1994). The data also demonstrate that the upper alpha band responds not only to semantic but also to visual task demands. Topographical differences reveal that desynchronization is largest at occipital sites during t2 and t4 when the first and second words are presented. It seems plausible to assume that this highly selective increase in desynchronization reflects visual encoding processes. Comparing different recording sites shows an interesting dissociation between different brain areas. Whereas frontal, central, parietal and temporal regions do not respond to the presentation of a word, but only to semantic processing demands, occipital regions respond to both perceptual and semantic processing demands. Perceptual processes serve to extract the meaning from sensory information. It, thus, appears likely that upper alpha desynchronization responds not only to verbal-semantic but to any kind of sensory-semantic encoding processing. The two lower alpha bands show a flat topographical distribution of desynchronization,
EEG Alpha and cognitive processes
251
a generally higher level of desynchronization as compared to the upper alpha band, and a clear relationship to task difficulty. Thus, these findings support the view that the two lower alpha bands reflect rather unspecific cognitive processes such as general task demands and attention. We assume that the gradual increase in lower alpha desynchronization may reflect the gradual increase in attentional or general task demands from the beginning to the end of a trial.
Time and the brain
252
Figure 5. Event-related band power changes as measured by ERD in three different experiments (data from Klimesch et al., 1997b). Intervals t1-t5 represent consecutive intervals of 500 ms each. In Experiment 1, a feature word was presented during t2 whereas a concept word was presented during t4. In Experiment 2 and 3 only feature words were presented (during t2). In all of the three experiments, t1 represents the prestimulus period. In Experiment 1 subjects had to judge whether pairs of feature-concept words (e.g. “claws—eagle”, “blue—canary”) were semantically congruent. In Experiment 2, subjects were asked to respond with a free association to a feature word (presented during t2). In Experiment 3, the presentation of a feature word (during t2) served as cue to recall the respective concept word, presented in Experiment 1. Note that semantic memory demands are highest in Experiment 1, but only during t4 and t5 when the congruency judgment is carried out. Only the upper alpha band responds selectively to this type of task demand. Significant differences with respect to t1 are marked by asterisks.
EEG Alpha and cognitive processes
253
Figure 6. Hemispheric differences in upper alpha desynchronization (ERD) in the semantic task (Experiment 1 in Figure 5). A strong left hemispheric advantage (larger ERD-values over the left as compared to the right hemisphere) was found during semantic processing, particularly during t5.
Time and the brain
254
7. CONCLUDING REMARKS With respect to the functional meaning of the alpha rhythm, the easiest question to answer refers to the relationship between alpha frequency and the speed of information processing. Age-related differences, the slowing of the alpha rhythm as a function of neurological disorders, and studies documenting a significant relationship between alpha frequency and reaction time suggest that the frequency of the alpha rhythm is—in some way—related to the speed of information processing. According to Nunez’s “standing wave” theory (Nunez, 1995) we may assume that alpha waves, generated at many different locations, travel over the cortex, and, after interference, form a standing wave. The important prediction from this theory (supported by experimental results) is that brain size and alpha frequency are negatively correlated, which means that large brains tend to have a slower alpha frequency than small brains. This is paradoxical because several studies have shown that brain size is positively correlated with intelligence. More intelligent people process information faster than less intelligent people, and consequently also should have a higher frequency of the alpha rhythm. A possible solution of this paradox may come from reconsidering the distinction between a global alpha rhythm and local, or functional alphas (Başar, 1997). There is strong evidence for the view that there is no single alpha rhythm, but instead a population of different alpha rhythms (e.g. Lopes Da Silva et al., 1973, 1980; see Başar and Bullock, 1992, for a more recent review). A strong alpha that can be recorded with scalp electrodes documents the simple but important fact that a very large population of neurones oscillates synchronously with the same phase and with the same frequency, or at least within a narrow frequency window. During this type of “large scale” synchronous oscillation, which may also be termed “global” alpha (or “type 1” alpha oscillation reflecting a state of maximal synchronization: Klimesch, 1996), the brain is not actively processing information. During active information processing, alpha desynchronizes and the assumption is that different alpha rhythms (local alphas or functional alphas) start to oscillate with different frequencies. As a result, the dominant EEG alpha rhythm desynchronizes. Now, it is important to see that although the global alpha rhythm desynchronizes, each alpha subpopulation may still oscillate synchronously. This latter type of microscale synchronous oscillation is termed “type 2 oscillation”. Thus, the frequency of global alpha, as measured during a resting state, will not reflect properties of the working brain and should not be correlated with the speed of information processing. Global alpha is most pronounced during relaxed wakefulness, and studies trying to measure peak alpha frequency usually use a resting period in which the EEG is recorded. The measurement of alpha frequency by Nunez et al. (1978) was also obtained in this way. Thus, the relationship between head size and alpha frequency refers to global alpha. With respect to a resting situation it appears plausible to assume that the alpha rhythm needs more time to spread through a large (as compared to a small) brain. During actual information processing the global alpha (or type 1 large-scale alpha oscillation) disappears at least in part, and type 2 micro-scale alpha oscillations appear. In contrast to global alpha (reflecting maximal synchronization during a resting state), the frequency of
EEG Alpha and cognitive processes
255
local or functional alphas (type 2 micro-scale alpha oscillations) should be related to the speed of information processing. Thus, in a situation of medium desynchronization (possibly reflecting the interaction between global and local alphas) alpha frequency (as measured by scalp electrodes) should already be correlated with the speed of information processing. This is indeed the case as the results of Klimesch et al. (1996) indicate (cf. Table 1). As was expected, during a resting state with maximal alpha synchronization (reflecting global alpha activity) no significant correlation was obtained. However, during maximal desynchronization, in which type 2 micro-scale alpha oscillations predominate, no significant correlations were obtained either. This failure, however, most probably has a different explanation: because of technical and methodological limitations the frequency of type 2 micro-scale alpha oscillations can hardly be measured with scalp electrodes. Thus, only during a state of medium desynchronization, in which global alpha no longer dominates, and type 2 oscillations can still be detected by scalp electrodes, a significant correlation between alpha frequency and the speed of information processing emerges.
REFERENCES Başar, E. (1997) Towards a renaissance of “alphas”. International Journal of Psychophysiology, 26, 1–3. Başar, E., and Bullock, T.H. eds. (1992): Induced Rhythms in the Brain. Birkhäuser, Boston Basel Berlin. Başar, E., Hari, R., Lopes da Silva, F.H., and Schürmann, M. (1997a) Brain alpha acitvity—new aspects and functional correlates. International Journal of Psychophysiology, 26, 1–482. Başar, E., Schürmann, M., Başar-Eroglu, C., and Karakas, S. (1997b) Alpha oscillations in brain functioning: An integrative theory. International Journal of Psychophysiology, 26, 5–29. Başar, E., and Schürmann, M. (1997) Functional correlates of alphas: Panel discussion of the conference “Alpha Processes in the Brain”. International Journal of Psychophysiology, 26, 455–474. Berger, H. (1929) Über das Elektroenkephalogramm des Menschen. Archiv für Psychiatric und Nervenkrankheiten, 87, 527–570. Bigler, E.D., Johnson, S.C., Jackson, C., and Blatter, D.D. (1995) Aging, brain size, and IQ. Intelligence, 21, 109–119. Brenner, R.P., Ulrich, R.F., Spiker, D.G., Scalbassi, R.J., Reynolds, C.F., III, Marin, R.S., and Boller, F. (1986) Computerized EEG spectral analysis in elderly normal, demented and depressed subjects. Electroencephalography and Clinical Neurophysiology, 64, 483–492. Breslau, J., Starr, A., Sicotte, N., Higa, J., and Buchsbaum, M.S. (1989) Topographic EEG changes with normal aging and SDAT. Electroencephalography and Clinical Neurophysiology, 72, 281–289. Bullock, T.H. (1992) Introduction to induced rhythms: A widespread, heterogeneous class of oscillations. In: E. Başar and T.H. Bullock (eds) Induced rhythms in the brain. Boston: Birkhäuser, pp. 1–26. Christian, W. (1984) Das Elektroencephalogramm (EEG) im höheren Lebensalter. Nervenarzt, 55, 517–524.
Time and the brain
256
Coben, L.A., Danziger, W., and Storandt, M. (1985) A longitudinal EEG study of mild senile dementia of Alzheimer type: Changes at 1 year and at 2.5 years. Electroencephalography and Clinical Neurophysiology, 61, 101–112. Cowan, N. (1988) Evolving conceptions of memory storage, selective attention, and their mutual constraints within the human information-processing system. Psychological Bulletin, 104, 163–191. Crawford, H., Knebel, T., Vendemia, J., Kaplan, L., and Ratcliff, B. (1995) EEG activation patterns during tracking and decision-making tasks: Differences between low and high sustained attention. Paper presented at: 8th International Symposium on Aviation Psychology, Columbus, Ohio, 1995, April 24–27. Duffy, F.H., Albert, M.S., McAnulty, G., and Garvey, A.J. (1984) Age-related differences in brain electrical activity of healthy subjects. Annals of Neurology, 16, 430–438. Epstein, H.T. (1980) EEG developmental stages. Developmental Psychology, 13, 629– 631. Fuller, P.W. (1978) Attention and the EEG alpha rhythm in learning disabled children. Journal of Learning Disabilities, 11, 303–312. Furst, G.J. (1976) EEG alpha asymmetry and visuospatial performance. Nature, London, 260, 254–255. Galin, D., Ornstein, R., Herron, J., and Johnstone, J. (1982) Sex and handedness differences in EEG measures of hemispheric specialization. Brain and Language, 16, 19–55. Gasser, T., Verleger, R., Bächer, P., and Sroka, L. (1988) Development of EEG of school-age children and adolescents. I Analysis of band power. Electroencephalography and Clinical Neurophysiology, 69, 91–99. Hartikainen, P., Soininen, H., Partanen, J., and Helkala, E.L. (1992) Aging and spectral analysis of EEG in normal subjects: A link to memory and CSF AChE. Acta Neurologica Scandinavica, 86, 148–155. Hubbard, O., Sunde, D., and Goldensohn, E.S. (1976) The EEG in centenarians. Electroencephalography and Clinical Neurophysiology, 40, 407–417. Hudspeth, W.J., and Pribram, K.H. (1990) Stages of brain and cognitive maturation. Journal of Educational Psychology, 82, 881–884. John, E.R., Ann, H., Prichep, L., Trepetin, M., Brown, D., and Kaye, H. (1980) Developmental equations for the EEG. Science, New York, 210,1255–1258. John, E.R., and Prichep, L. (1993) Principles of neurometric analysis of EEG and evoked potentials. In: E. Niedermeyer and F.H. Lopes da Silva (eds) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn, Baltimore, Hong Kong, London, Munich, Sydney, Tokyo: Williams & Wilkins, pp. 989–1004. Kahneman, D., and Treisman, A. (1984) Changing views of attention and automaticity. In: R.Parasuraman and D.R.Davies (eds) Varieties of attention. London, Academic Press, pp. 29–61. Kalcher, J., and Pfurtscheller, G. (1995) Discrimination between phase-locked and nonphase locked event-related EEG activity. Electroencephalography and Clinical Neurophysiology, 94, 381–384. Katada, A., Ozaki, H., Suzuki, H., and Suhara, K. (1981) Developmental characteristics of normal and mentally retarded children’s EEG. Electroencephalography and Clinical Neurophysiology, 52, 192–201. Kaufman, L., Schwartz, B., Salustri, C., and Williamson, S.J. (1989) Modulation of spontaneous brain activity during mental imagery. Journal of Cognitive Neuroscience,
EEG Alpha and cognitive processes
257
2, 124–132. Klimesch, W. (1994) The structure of long-term memory: A connectivity model for semantic processing. Hillsdale, NJ: Lawrence Erlbaum. Klimesch, W. (1995) Memory processes described as brain oscillations in the EEG-alpha and theta band. Psycoloquy (electronic journal). Klimesch, W. (1996) Memory processes, brain oscillations and EEG sychronization. International Journal of Psychophysiology, 24, 61–100. Klimesch, W. (1997) EEG-alpha rhythms and memory processes. International Journal of Psychophysiology, 26, 319–340. Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Research Reviews. 29, 169–195. Klimesch, W., Schimke, H., and Pfurtscheller, G. (1990a) The topography of alpha frequency and memory performance. Journal of Psychophysiology, 4, 191–192. Klimesch, W., Schimke, H., Ladurner, G., and Pfurtscheller, G. (1990b) Alpha frequency and memory performance. Journal of Psychophysiology, 4, 381–390. Klimesch, W., Pfurtscheller, G., Mohl, W., and Schimke, H. (1990c) Event-related desynchronization, ERD-mapping and hemispheric differences for words and numbers. Journal of Psychophysiology, 8, 297–308. Klimesch, W., Pfurtscheller, G., and Schimke, H. (1992) Pre- and poststimulus processes in category judgement tasks as measured by event-related desynchronization (ERD). Journal of Psychophysiology, 6, 186–203. Klimesch, W., Schimke, H., and Pfurtscheller, G. (1993a) Alpha frequency, cognitive load, and memory performance. Brain Topography, 5, 241–251. Klimesch, W., Pfurtscheller, G., and Schimke, H. (1993b) ERD—attentional and cognitive processes in the upper and lower alpha band. Electroencephalography and Clinical Neurophysiology, 87, 133. Klimesch, W., Schimke, H., and Schwaiger, J. (1994) Episodic and semantic memory: An analysis in the EEG-theta and alpha band. Electroencephalography and Clinical Neurophysiology, 91, 428–441. Klimesch, W., Doppelmayr, M., Schimke, H., and Pachinger, T. (1996a) Alpha frequency, reaction time and the speed of processing information. Journal of Clinical Neurophysiology, 13, 511–518. Klimesch, W., Doppelmayr, M., Russegger, H., and Pachinger, T. (1996b) Theta band power in the human scalp EEG and the encoding of new information. NeuroReport, 7, 1235–1240. Klimesch, W., Schimke, H., Doppelmayr, M., Ripper, B., Schwaiger, J., and Pfurtscheller, G. (1996c) Event-related desynchronization (ERD) and the Dm-effect: Does alpha desychronization during encoding predict later recall performance. International Journal of Psychophysiology, 24, 47–60. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Ripper, B. (1997a) Brain oscillations and human memory: EEG correlates in the upper alpha and theta bands. Neuroscience Letters. 238, 9–12. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Russegger, H. (1997b) Event-related desynchronization in the alpha band and the processing of semantic information. Cognitive Brain Research, 6, 83–94. Klimesch, W., Doppelmayr, M., Schimke, H., and Ripper, B. (1997c) Theta synchronization in a memory task. Psychophysiology, 34, 169–176. Klimesch, W., Russegger, H., Doppelmayr, M., Pachinger, Th. (1998) A method for the calculation of induced band power: Implications for the significance of brain
Time and the brain
258
oscillations. Electroencephalography and clinical Neurophysiology, 108, 123–130. Kolb, B., and Whishaw, I.Q. (1985) Fundamentals of human neuropsychology. New York: Freeman. Könönen, M., and Partanen, J.V. (1993) Blocking of EEG alpha activity during visual performance in healthy adults. A quantitative study. Electroencephalography and Clinical Neurophysiology, 87, 164–166. Köpruner V., Pfurtscheller G., and Auer L. (1984): Quantitative EEG in normals and in patients with cerebral ischemia. In: G.Pfurtscheller, E.Jonkman, F.Lopes da Silva (eds) Brain Ischemia: Quantitative EEG and Imaging Techniques. Amsterdam, Elsevier, pp. 29–50. Lopes da Silva, F.H., Lierop, T.H., Schrijer, C.F., and Storm van Leeuwen, W. (1973) Organization of thalamic and cortical alpha rhythms: Spectra and coherences. Electroencephalography and Clinical Neurophysiology, 35, 627–639. Lopes Da Silva, F.H., Vos, I.E., and Van Rotterdam, A. (1980) Relative contributions of intracortical and thalamo-cortical processes in the generation of alpha rhythms, revealed by partial coherence analysis. Electroencephalography and Clinical Neurophysiology, 50, 449–456. Markand, O.N. (1990) Alpha rhythms. Journal of Clinical Neurophysiology, 7, 163–189. Marthis, P., Scheffner, D.M., Benninger, C., Lipinski, C., and Stolzis, L. (1980) Changes in the background activity of the electroencephalogram according to age. Electroencephalography and Clinical Neurophysiology, 49, 626–635. Mulholland, T.B. (1969) The concept of attention and the electroencephalographic alpha rhythm. In: C.R.Evans, & T.B., Mulholland (eds) Attention in Neurophysiology. London, Butterworths, pp. 100–127. Niedermeyer, E. (1993a) The normal EEG of the waking adult. In: E.Niedermeyer and F.H.Lopes da Silva (eds) Electroencephalography: basic principles, clinical applications, and related fields, 3rd edn. Baltimore: Williams & Wilkins, pp. 131–152. Niedermeyer, E. (1993b) Normal aging and transient cognitive disorders in the elderly. In: E.Niedermeyer and F.H.Lopes da Silva (eds) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn. Baltimore: Williams & Wilkins, pp. 329–338. Niedermeyer, E. (1993c) Maturation of the EEG: Development of waking and sleep patterns. In: E.Niedermeyer and F.H.Lopes da Silva (eds.) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn. Baltimore: Williams & Wilkins, pp. 167–191. Nunez, P.L., Reid, L., and Bickford, R.G. (1978) The relationship of head size to alpha frequency with implications to a brain wave model. Electroencephalography and Clinical Neurophysiology, 44, 344–352. Nunez, P.L. (1995) Toward a physics of neocortex. In: P.L. Nunez (ed.) Neocortical dynamics and human EEG rhythms. New York, Oxford: Oxford University Press, pp. 68–132. Obrist, W.D. (1954) The electroencephalogram of normal aged adults. Electroencephalography and Clinical Neurophysiology, 6, 235–244. Osaka, M. (1984) Peak alpha frequency of EEG during a mental task: Task difficulty and hemispheric differences. Psychophysiology, 21, 101–105. Papanicolaou, A.C., Loring, D.W., Deutsch, G., and Eisenberg, H.M. (1986) Task-related EEG asymmetries: A comparison of alpha blocking and beta enhancement. International Journal of Neuroscience, 30, 81–85. Pfurtscheller, G., and Aranibar, A. (1977) Event-related cortical synchronization detected
EEG Alpha and cognitive processes
259
by power measurements of scalp EEG. Electroencephalography and Clinical Neurophysiology, 42, 817–826. Pigeau, R.A., and Frame, A.M. (1992) Steady-state visual evoked responses in high and low alpha subjects. Electroencephalography and Clinical Neurophysiology, 84, 101– 109. Posner, M.I. (1975) Psychobiology of attention. In: M.S.Gazzaniga and C.Blakemore (eds) Handbook of Psychobiology. New York: Academic Press, pp. 441–480. Posner, M.I. (1995) Attention in cognitive neuroscience: An overview. In: M.S.Gazzaniga (ed.) The cognitive neurosciences. Cambridge, MA: Bradford, MIT Press, pp. 615–624. Ray, W.J., and Cole, H.W. (1985) EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes. Science, New York, 228, 750– 752. Sheridan, P.H., Sato, S., Foster, N., and Bruno, G. (1988) Relation of EEG alpha background to parietal lobe function in Alzheimer’s disease as measured by positron emission tomography and psychometry. Neurology, 38, 747–750. Somsen, R.J.M., Van-Klooster, B.J., Van-der-Molen, M.W., and Van-Leeuwen, H.M. (1997) Growth spurts in brain maturation during middle childhood as indexed by EEG power spectra. Biological Psychology, 44, 187–209. Steriade, M., Jones, E., and Llinás, R.R. (1990) Thalamic oscillations and signaling, New York, Chichester, Brisbane, Toronto, Singapore: John Wiley & Sons. Sterman, M.B., Kaiser, D.A., and Veigel, B. (1996) Spectral analysis of event-related EEG responses during short-term memory performance. Brain Topography, 9, 21–30. Surwillo, W. (1961) Frequency of the alpha rhythm, reaction time and age. Nature, London, 191, 823–824. Surwillo, W. (1963a) The relation of simple response time to brain-wave frequency and the effects of age. Electroencephalography and Clinical Neurophysiology, 15, 105– 114. Surwillo, W. (1963b) The relation of response-time variability to age and the influence of brain wave frequency. Electroencephalography and Clinical Neurophysiology, 15, 1029–1032. Surwillo, W. (1964a) Some observations of the relation of response speed to frequency of photic stimulation under conditions of EEG synchronization. Electroencephalography and Clinical Neurophysiology, 17, 194–198. Surwillo, W. (1964b) The relation of decision time to brain wave frequency and to age. Electroencephalography and Clinical Neurophysiology, 16, 510–514. Surwillo, W. (1971) Human reaction time and period of the EEG in relation to development. Psychophysiology, 8, 468–482. Torres, F., Faoro, A., Löwenson, R., and Johnson, E. (1983) The electroencephalogram of elderly subjects revisited. Electroencephalography and Clinical Neurophysiology, 56, 391–398. Tulving, E., Kapur, S., Craik, F.I.M., Moscovitch, M., and Houle, S. (1994) Hemispheric encoding/retrieval asymmetry in episodic memory: Positron emission tomography findings. Proceedings of the National Academy of Sciences U.S.A., 91, 2016–2020. Van Sweden, B., Wauquier, A., and Niedermeyer, E. (1993) Normal aging and transient cognitive disorders in the elderly. In: E. Niedermeyer and F.H. Lopes da Silva (eds) Electroencephalography: basic principles, clinical applications, and related fields, 3rd edn. Baltimore: Williams & Wilkins, pp. 329–338. Wieneke, G.H., Deinema, C.H.A., and Spoelstra, P. (1980) Normative spectral data on
Time and the brain
260
the alpha rhythm in male adults. Electroencephalography and Clinical Neurophysiology, 49, 636–645. Willerman, L., Schultz, R., Rutledge, J.N., and Bigler, E.D. (1991) In vivo brain size and intelligence. Intelligence, 15, 223–228.
8 Theta Frequency, Synchronization and Episodic Memory Performance Wolfgang Klimesch Department of Physiological Psychology, Institute of Psychology, University of Salzburg, Hellbrunnerstr. 34, A-5020 SALZBURG Austria Tel: 0043 662 8044 5120 or 5100; Fax 0043 662 8044 5126 e-mail: [email protected]
Based on findings from animal research, the hypothesis is examined that the encoding and retrieval of new information is reflected by a task related increase (synchronization) in theta power. The findings from several studies support this hypothesis, and show that only those items were correctly remembered that exhibited a significant increase in theta power during the encoding process. During the actual retrieval processes, a significant increase in band power was found only for correctly remembered words. In contrast to the theta band, remembered and not-remembered items revealed a different and more complex pattern of results in the lower and upper alpha band. These and similar findings support the view that synchronous neural activity within the theta frequency range reflects the encoding and retrieval of new (episodic) information, and is induced in the cortex via hippocampo-cortical feedback loops. KEYWORDS: Theta; Brain oscillations; Pace maker; Encoding; LTP; Episodic memory
1. INTRODUCTION In contrast to the alpha rhythm, most of what is known about the theta rhythm stems from animal research. Therefore, the most important properties of the hippocampal EEG will be reviewed first. Most interestingly, when studying results of experiments linking animal behaviour with the hippocampal EEG, it becomes evident that hippocampal theta power increases with increasing task demands just as does theta band power in the human scalp EEG. With respect to the functional meaning of the theta rhythm, evidence from divergent fields such as electrophysiology, anatomy, and neuropsychology point towards a specific relationship between hippocampal theta activity and the encoding of new information. Animal research shows that theta, as the dominant rhythm in the hippocampal formation, is closely related to the strength of long-term potentiation (LTP), which is generally considered a synaptic memory mechanism for the encoding of new
Theta frequency, synchronization and episodic memory performance
263
information. Scalp recordings from human subjects show that the encoding of new information, and the retrieval from short-term memory (STM), is accompanied by a significant increase in theta power. The reviewed evidence suggests that theta oscillations may provide the basis for the encoding of new information. It may even be suggested that—in a functional sense—memory traces are actually stored in the form of nested oscillations within the theta frequency range (Lisman and Idiart, 1995; Klimesch, 1996).
2. FINDINGS FROM ANIMAL RESEARCH In animal research, the frequency range of the theta rhythm recorded from the hippocampus of lower mammals is about 3 to 12 Hz (e.g. Lopes da Silva, 1992) and, thus, shows a much wider frequency range than in humans, where the theta frequency range lies between about 4 and 7 or 8 Hz. Its wide frequency range and large power make observation of its frequency and power changes easy in animals. This is in sharp contrast to the human scalp EEG, where—without the help of sophisticated methods—changes in theta frequency are very difficult or almost impossible to detect. The regular oscillatory pattern of the hippocampal EEG which also is termed Rhythmic Slow Activity (RSA) can be observed if animals make voluntary movements (e.g. Vanderwolf and Robinson, 1981; Vanderwolf, 1992) during exploratory behaviour (Buzsaki et al., 1994). In the absence of exploratory behaviour (e.g. in slow wave sleep, SWS, or during alert immobility) the hippocampal EEG shows Irregular Slow Activity (ISA) which also is called Large Irregular Activity (LIA) or Sharp Waves (SPW; Buzsaki et al., 1994). In simplifying terminology we will use the term SPW (instead of ISA and LIA) in what follows. RSA or theta is usually described in terms of local field potentials that are recorded from microelectrodes implanted in field CA1 or in the dentate layers of the hippocampus. It is induced from the septum which serves as a primary pacemaker (Petsche et al., 1962; cf. the reviews in Buzsaki et al., 1983, and Miller, 1991). The extracellular currents that underly theta activity reflect synchronous fluctuations in the membrane potentials of pyramidal and granule cells (e.g. Bland 1986; Buzsaki et al., 1983). Most interestingly, during theta activity, the majority of pyramidal and granule cells are silent, whereas interneurones exhibit a pronounced increase in their firing rate which is phase-locked to the theta rhythm. At the first glance this appears paradoxical, because theta activity is associated with exploratory behaviour and, thus, most likely with increased mental activity. However, when comparing the local field potentials with multi unit recordings, it can be observed that during high frequency (“type-1”: see below) theta the number of unit discharges is much higher than during low frequency (“type-2”: see below) theta, and that the number of discharges increases linearly with increasing theta frequency in both types of theta (Bland, 1986; Sinclair et al., 1982). Several authors have reported that unit discharges are phase-related, and occur during the negative peaks of theta waves (e.g. Buzsaki et al., 1983; Fox et al., 1986; Buzsaki et al., 1992). Particularly interesting is the finding from Sinclair et al. (1982) that discharges occur briefly after (i.e. about 30 degrees after) the theta wave has reached its negative
Time and the brain
264
peak (at 180 degrees). This result suggests that the negative peak of the theta wave may take an active part in triggering action potentials. 2.1. Theta Frequency, Phase and Behaviour Voluntary movements (e.g. walking, jumping, etc.) which can be observed during exploratory behaviour, are characterized by a highly regular theta oscillation which is termed type-1 theta. In rodents, type-1 theta ranges from about 6.5 to 12 Hz. A somewhat slower and more irregular theta frequency can be observed during a state of immobile alertness, if sensory stimuli are presented while animals are immobile but alert and, thus, in a state of arousal (Montoya et al., 1989). This type-2 theta frequency varies within a range of about 4–9 Hz. A third type of behaviour which is characterized by intermittent SPWs (of 40 to 120 ms duration) can be observed during sleep and “automatic” motor patterns. If the frequency of SPWs is measured, the following interesting relationship with behaviour emerges: with increasing frequency (ranging from about 0.02 to 3 Hz) SPWs occur during slow wave sleep (SWS), awake immobility, drinking, eating, face washing and grooming (Buzsaki et al., 1983). If we assume that attentional demands increase from SWS, awake immobility, automatic motor patterns, high alertness during immobility and finally to voluntary movements during exploratory behaviour, a perfect association between the frequency of the hippocampal EEG (within a range of 0.02–12 Hz) and attentional (or task demands) can be detected. With respect to the theta frequency range, it is important to note that with increasing task demands the hippocampal EEG becomes more regular, increases its frequency and power. Thus, as a result of increasing task demands, EEG-theta becomes synchronized within the small range of peak theta frequency (c.f. Leung et al., 1982; Buzsaki et al., 1983). Not only theta power and frequency, but also theta phase, is related to behaviour. Within the pyramidal cell layer of the rat hippocampus, cells have been found that, unlike theta cells, respond with a more irregular pattern of spikes. O’Keefe and Dostrovsky (1971) observed that this type of cell responds to the animal’s location in the environment. Phase-locked to the concurrent EEG-theta, these “place cells” fire several bursts of spikes as the rat runs through a particular location in its environment, which is called the place field of that cell. The most important result is that the firing of place cells began at a particular phase of theta frequency as the rat entered the place field. Within a particular place field the angle of theta phase was related to a particular spatial location (O’Keefe and Recce; 1993; see also review in O’Keefe, 1993). Thus, when considering the place field as a specific stimulus, theta frequency is phase-locked to the appearance of this stimulus. 2.2. The Functional Meaning of Hippocampal Theta The first evidence for the view that the hippocampal theta rhythm reflects memory processes, or the encoding of new information in particular, came from different research
Theta frequency, synchronization and episodic memory performance
265
fields. An important discovery was reported by Scoville and Milner (1957), who observed a severe anterograde amnesia for patient H.M., who had undergone a bilateral temporal lobectomy including the hippocampal formation. (For a review of related work see e.g. Squire, 1992). A few years earlier, Green and Arduini (1954) had found a dominant rhythmic electrical activity within the theta band in the hippocampus of rats (see the excellent review in Miller, 1991). Thus, it was plausible to assume that the hippocampal theta rhythm might be related to the encoding and/or retrieval of new information. Additional evidence came from more recent studies which have documented that long-term potentiation (LTP) occur preferentially in the hippocampal formation, and that theta activity induces, or at least enhances LTP (e.g. Larson et al., 1986; Greenstein et al., 1988; Maren et al., 1994). LTP is considered the most important electrophysiological memory mechanism for the encoding of new information. If it can be demonstrated that LTP is related in a specific way to some properties of the theta rhythm, an important argument would be at hand for considering theta to be an electrophysiological correlate of working memory. Such a relationship is supported by the finding that the induction of LTP is optimal with stimulation patterns that mimic the theta rhythm (Larson et al., 1986), whereas stimulations which are not repeated and short are usually ineffective. Another important result is that the induction of LTP depends on the phase of theta rhythm. Furthermore, the strength of the induced LTP was found to increase linearly with increasing theta power (Maren et al., 1994 [especially Figure 6 on p. 50]). These findings provide strong support for the view that hippocampal theta is related to the encoding of new information, just as is LTP. Comprehensive theories about hipocampal theta and memory can be found in Miller (1991), Buzsaki et al. (1994) and Klimesch (1996). Miller’s theory of resonant loops is of particular importance, because it suggests a rhythmic interplay between the hippocampus and cortex. The human scalp EEG primarily reflects cortical activity. Biophysically, theta frequency in the hippocampus, deep inside the brain, would be difficult to detect with scalp electrodes. If, however, hippocampal theta is transmitted to the cortex, it should be possible to detect memory related changes in theta activity even in the human scalp EEG.
3. FINDINGS FROM THE HUMAN SCALP EEG In the previous chapter (about the measurement of event-related changes in the alpha frequency range) it was emphasized that the individual determination of frequency bands is essential. The reason was that alpha frequency shows a large interindividual variability, and that, as a consequence, fixed frequency bands may include only parts of the “true” alpha frequency range. Will we face similar problems when attempting to measure bandpower changes in the theta frequency range?
Time and the brain
266
3.1. Theta Frequency The traditional view holds that theta frequency lies within a fixed range of about 4 to 8 Hz just below alpha frequency (of about 8–13 Hz). A fixed frequency band would be justified only if it can be demonstrated that theta does not vary between subjects. Thus, before considering findings from the human scalp EEG in more detail, it is of crucial importance to address the following two questions: (i) is there a physiological criterion that allows one to decide which frequency marks the transition between alpha and theta?; and (ii) does theta frequency vary in a manner similar to alpha frequency? The answer to the first question is surprisingly easy. It is a well established fact that, with increasing task demands, theta synchronizes whereas alpha desynchronizes. If EEG power in a resting condition is compared with a test condition, alpha power will become suppressed whereas theta power will increase. As Figure 1 illustrates, that frequency in the power spectra which marks the transition from theta synchronization to alpha
Theta frequency, synchronization and episodic memory performance
267
Figure 1. Power spectra for reference and test intervals intersect at the transition frequency (TF) between the theta and alpha band. During a reference interval (bold line) subjects are alert but not performing a task. In a test interval (dashed line), in which a subject performs some kind of task, alpha power decreases, but theta power increases. The intersection of the two power spectra, where theta synchronization gives way to alpha desynchronization, may be considered a physiological criterion for the transition region between the alpha and theta band.
Time and the brain
268
desynchronization may be considered the individual transition frequency (TF) between the alpha and theta band for each subject. When adopting this method to estimate TF, the second question can also be answered. Results document that TF shows a large interindividual variability (ranging from about 4 to 7 Hz), which is linked to the interindividual variability of alpha peak frequency. Preliminary evidence for a covariation between theta and alpha frequency was already found by Klimesch et al. (1994). By using the following method, further evidence for such a covariation was obtained in a more recent study (Klimesch et al., 1996). First, power spectra for the reference and test intervals were calculated for each subject and averaged over all trials and all leads. Then, the frequency of the transition region between the theta and alpha band was determined within a fixed frequency window of 3.5–7.5 Hz. For those few subjects who showed an asymmetric alpha desynchronization (with no desynchronization in the lower alpha), and thus failed to show an intersection (in the range of 3.5–7.5 Hz), the transition between the theta and alpha band was considered to be that frequency where the difference between the test and the reference interval reached a minimum. In Klimesch et al. (1996) Spearman’s rank correlation between alpha peak frequency and TF yielded a significant value of p=.64 (one sided: p < .02; two sided: p < .04). Similar results were reported by Doppelmayr et al. (1998a). These findings document the fact that theta frequency varies as a function of alpha frequency, and suggest the use of alpha frequency as a common reference point for adjusting different frequency bands, not only for the alpha, but in the theta range as well. For theta, the individual determination of frequency bands may be even more important, because otherwise the effects of theta synchronization are masked by alpha desynchronization, particularly in the range of TF. Results from our laboratory indicate that TF lies about 4 Hz below the individuallydetermined alpha frequency. As an example, in a sample of 10 subjects (Klimesch et al., 1996) with a mean alpha frequency of 10.7 Hz, mean TF is about 6.7 Hz. If we assume a bandwidth of 4 Hz, theta frequency would lie in a range of 2.7–6.7 Hz, which gives an estimate that appears much too low with respect to the lower frequency boundary of 2.7 Hz: frequencies below 4 Hz are considered to belong to the delta frequency range. To avoid an overlap with delta, we defined the theta frequency range as that band with a width of 2 Hz that falls below TF. All of the findings reported below are based on individually adjusted theta bands of 2 Hz width. These findings indicate that the theta frequency range in humans may be much smaller than originally assumed. 3.2. Event-Related Changes in Theta Power Measurements of event-related changes in theta power are based on the ERD-method which is described in more detail in the previous chapter on the alpha frequency range (Klimesch, this volume). Originally proposed by Pfurtscheller and Aranibar (1977), event-related desynchronization (ERD) is defined as the percentage decrease (positive ERD) or increase (negative ERD) in band power during a test, with respect to a reference interval. A negative ERD is also termed “event-related synchronization” or ERS
Theta frequency, synchronization and episodic memory performance
269
(Pfurtscheller, 1992). All of the experiments reported below were designed to test the hypothesis that a taskrelated increase in theta power reflects selectively the successful encoding and/or retrieval of new information. In reviews focusing on findings in functional neuroanatomy, electrophysiology, amnesia and memory research, Klimesch (1995, 1996) has proposed that the hippocampal theta rhythm reflects primarily the encoding and retrieval of episodic memory. It was suggested that the hippocampal theta rhythm may be induced in the cortex via hippocampo-cortical feedback loops, and thus may even be detected by scalp electrodes in the human EEG. The well known finding that theta power increases in a large variety of different tasks (Schacter, 1977; Arnolds et al., 1980) seems to contradict the suggested hypothesis of a specific relationship between theta and the processing of new information. However, if we consider the fact that the processing of new information is—in some way and to some extent—necessary for the performance of almost any type of task, it becomes obvious that specially designed experiments are needed to test the proposed hypothesis. Of particular importance is the experimental control of unspecific factors (such as attentional demands, task difficulty and cognitive load) which usually will go together—and, thus, will be confounded, with the actual processing of new information. 3.3. Theta Synchronization and Episodic Memory In most every-day situations, the processing of “new” information is synonymous with “episodic” information. According to Tulving (e.g. Tulving, 1984), episodic information is that type of contextual information which keeps an individual autobiographically oriented within space and time. In contrast to a sensory code which primarily reflects physical information, an episodic code which is created through the action of working memory processes (Baddeley, 1992), and reflects primarily subjective information, such as context, expectancy, emotion and autobiographic aspects. Because time changes the context permanently, there is a permanent and vital need to update and store new (or episodic) information. In a first study (Klimesch et al., 1994) the relationship between theta synchronization, alpha desynchronization and episodic memory demands was investigated. The experimental design consisted of two parts. Subjects first performed a semantic congruency task in which they had to judge whether the sequentially-presented words of concept-feature pairs (such as “eagle-claws” or “pea-huge”) are congruent. Then, without prior warning, they were asked to perform an episodic recognition task. This was done in an attempt to prevent subjects from using semantic encoding strategies, and to increase episodic memory demands. In the episodic task, the same word pairs were presented together with new distractors (generated by re-pairing known concept-feature words). In this task, subjects had to know whether or not a particular concept-feature pair had already been presented during the semantic task. Because distractors were semantically similar and generated by re-pairing already-presented words, subjects were able to give a correct response only if new episodic information (represented by a specifc combination of a concept and feature word) was actually stored in memory. The results of a reaction
Time and the brain
270
time study with the same experimental paradigma (Kroll and Klimesch, 1992, Experiment 4) indicated that semantic features speeded up semantic but slowed down episodic decision times (for an extensive review on this topic see also Klimesch, 1994). This finding demonstrates that semantic and episodic memory processes can be differentiated well by using the proposed design. Because pairs of items are presented, the episodic and semantic task can be performed only after the second item of a pair (i.e. the feature) is presented. Thus, the critical issue is to compare the amount of theta synchronization during the presentation of the concept and feature word in the episodic and semantic task. Only correctly identified conceptfeature pairs were analyzed. The results show that only in the episodic task, and only when the feature word is processed can the expected increase in theta power be observed. During the processing of the concept words theta power shows almost identical values in the semantic as in the episodic task. Because the same words were presented in both tasks, and all other variables (such as exposure time, length of interstimulus interval, etc.) were kept constant, the findings support the hypothesis of a specific relationship between memory (for new episodic information) and theta synchronization. The fact that the theta band responds selectively to episodic task demands is also demonstrated by the finding that the lower and the upper alpha band show quite different results. Whereas event-related changes in the lower alpha band were comparatively small, the upper alpha band shows a much larger degree of desynchronization during the semantic as compared with the episodic task. Thus, there is a dissociation between theta synchronization, which is maximal during the processing of new information, and upper alpha desynchronization, which is maximal during the processing of semantic information (see also the previous chapter about the alpha frequency range). In confirming and extending these findings, in a more recent study (Klimesch et al., 1997a) we were able to demonstrate that the extent of upper alpha desynchronization is significantly correlated with semantic memory performance, whereas the extent of theta desynchronization is significantly but negatively correlated with episodic memory performance. Thus, in the theta band, episodic memory performance is correlated with an event-related increase in power whereas the opposite holds true for semantic memory and the upper alpha band. The topography and time course of event-related power changes in the theta band during encoding are depicted in Figure 2. Theta synchronization occurs in two phases. In the first third of the poststimulus interval (of 1000 ms) occipital leads show maximal synchronization, whereas frontal leads show maximal synchronization during the last third. This typical topography and time course of theta synchronization did not vary as a function of semantic or episodic task demands. However, the overall extent of synchronization was stronger during the episodic task. It should be noted that in the lower and upper alpha band the topography and time course of desynchronization is strikingly different. In contrast to theta, alpha desynchronization gradually increases after the presentation of a stimulus and (depending on the type and difficulty of a task) reaches a maximum about 300 ms after the stimulus. In a study with a series of three experiments (Klimesch et al., 1997b) we obtained similar results. In all of the three experiments, a shortlasting synchronization was
Theta frequency, synchronization and episodic memory performance
271
observed, but only at occipital sites, and only during the first 500 ms after the stimulus when a word was presented. This increase in theta power which is particularly strong at occipital areas is due, most probably, to lambda waves, which are known to exert maximal power in the theta band (Billings, 1989a,b). According to Billings (1989b), lambda waves result from retinal afferents and are observed particularly at occipital regions during the processing of visual information. It appears plausible to assume that lambda waves are phase-locked theta activity that represents the encoding of new visual information. 3.4. Theta Synchronization during Successful Encoding and Retrieval The purpose of more recent studies was to investigate the functional meaning of theta synchronization with respect to the formation of a memory code. An interesting prediction of our hypothesis is that, during encoding, those words that will later be remembered (e.g. in a recognition task) should exhibit a significantly stronger theta synchronization than words
Figure 2. Time course and topography of theta synchronization during the presentation of a word in an episodic memory paradigm (data from Klimesch et al., 1994). Occipital recording sites show an early phase of pronounced synchronization (from 125 to 375 ms poststimulus), whereas frontal sites exhibit a late phase of maximal synchronization (from 750 to 1000 ms poststimulus). Intervals t1-t8 represent consecutive time periods of 125 ms each. Note that negative ERD reflects an increase in band power.
that cannot be remembered later. Another prediction is that during the actual recognition process correctly recognized targets will show a significantly stronger increase in theta power than targets which are not recognized. Furthermore, because an episodic memory
Time and the brain
272
trace is available for remembered targets only, distractors are not expected to show a significant increase in theta power. In a study of Klimesch et al. (1997c), a set of 96 words was used as targets. Subsamples of 16 words each belonged to one of 6 categories (birds, fruits, vegetables, vehicles, clothes and weapons). The 96 distractors were selected in a way that, for each target (e.g. robin), a semantically similar distractor (e.g. sparrow) was obtained. Thus, as for the targets, the 96 distractors are subdivided into the same 6 semantic categories, each comprising 16 words. This similarity between distractors and targets guarantees that subjects have to rely on episodic information (and not for example, on semantic familiarity) in order to make a correct decision in the recognition task. If, for instance, a subject has to distinguish the target word “robin” from the distractor “sparrow”, semantic information representing the meaning of the words will not be helpful. For a correct response, the subject has to remember which of the two words was presented in the context of the study list. The results of the study and recognition task show clearly that the theta band responds with a task-related increase in band power (synchronization), whereas the alpha bands respond with a task-related decrease in band power (desynchronization). However, theta synchronization was significant only under those conditions in which a word was either successfully encoded or retrieved. Thus, in the study phase, only those words show a significant synchronization in the theta band which can be remembered in the later performed recognition task (cf. Figure 3). In comparison to Figure 2, inspection of Figure 3 reveals a similar time course and topography of theta synchronization during encoding.
Theta frequency, synchronization and episodic memory performance
273
Figure 3. Successful encoding in an explicit memory paradigm: theta synchronization is plotted during encoding in an experiment where subjects knew that their memory would be tested later (data from Klimesch et al., 1997). Words that could be remembered in a later recognition test (bold lines) show a significant stronger synchronization during encoding than words which could not be remembered later (dashed line). As in Figure 2, frontal sites exhibit a late phase of maximal synchronization (from 750 to 1000 ms poststimulus). Intervals t1-t4 represent consecutive time periods of 250 ms each.
Whereas at frontal leads theta synchronization is largest at the end of the poststimulus interval, the opposite holds true for occipital sites (cf. Figure 2). During the actual recognition process, only the remembered words exhibit a significant increase in theta power, as Figure 4 indicates. Not-remembered words, as well as distractors show no increase in theta power. The findings obtained support the proposed hypothesis that theta synchronization is related selectively to the encoding and retrieval of episodic information. Comparing the topography and time course of theta synchronization during successful encoding (Figure 3) and retrieval (Figure 4) again shows that, during the end of the poststimulus interval, synchronization is maximal at frontal leads, but minimal at occipital sites. Thus, frontal regions play an important role in the explicit formation and retrieval of a memory trace. Because subjects knew that their memory would be tested later, the finding that theta synchronization is significantly stronger during the encoding of those words which can later be rememberd could be due to specific encoding strategies. If this were true, theta synchronization during encoding would reflect a rather nonspecific factor, and not the actual encoding of new information. One way to avoid the possible influence of encoding strategies is to use an incidental memory paradigm (instead of an intentional one). During the encoding phase of an incidental memory paradigm, subjects do not know that
Time and the brain
274
memory performance will be tested later, because they are performing some type of distractor task. Therefore, specific mnemonic techniques and specific attentional factors will not play a significant role during encoding. In the study of Klimesch et al. (1996), we
Figure 4. Successful retrieval in an explicit memory paradigm: theta synchronization is plotted during the actual recognition process, in an experiment where subjects knew that their memory would be tested (data from Klimesch et al., 1997). Words that were correctly remembered (bold lines) show a highly significantly stronger synchronization during actual recognition than words which could not be remembered later (dashed line). Note the large difference in the strength of synchronization between occipital and frontal recording sites and the late maximum of synchronization (during 750 to 1000 ms poststimulus). Intervals t1–t4 represent consecutive time periods of 250 ms each.
used an incidental memory paradigm that consisted of two parts. In the first part subjects were asked to categorize a series of words, and to respond with “yes” if a word belonged to the category “living”, and with “no” if it belonged to the category “nonliving”. At this point of the experiment, subjects did not know that in the second part of the experiment they would have to recall the words which were presented in the judgement task. Band power values during the encoding stage were calculated, and words which could be remembered later were compared with those words which could not be remembered later. The results, which are depicted in Figure 5, indicate again that theta synchronization reflects the actual encoding of new information. However, as compared to the results depicted in Figures 2, 3 and 4, differences in the time course and topography of theta synchronization can be observed during the implicit encoding processes in an incidental memory paradigm: Early time intervals show a particularly strong synchronization for words which can later be remembered. Possibly, this finding reflects the readiness of the “theta system” to encode new information (during the beginning of the early poststimulus
Theta frequency, synchronization and episodic memory performance
275
period) which obviously fluctuates between trials.
4. CONCLUDING REMARKS The reported findings from animal research and the human scalp EEG clearly support the hypothesis that theta synchronization is related to the encoding and retrieval of new episodic information. In an attempt to explain the possible functional signifi-
Figure 5. Successful encoding in an implicit memory paradigm: theta synchronization is plotted during encoding, in an experiment where subjects did not know that their memory would be tested later (data from Klimesch et al., 1996). Words that were correctly remembered in a later recall test (bold lines) show a highly significant stronger synchronization during implicit encoding than words which could not be remembered later (dashed line). Note the early maximum of synchronization (during 0 to 500 ms poststimulus) and the differences in topography as compared to Figures 2, 3 and 4. Intervals t1–14 represent consecutive time periods of 250 ms each.
cance of the theta rhythm in the human EEG, we assume that synchronized bursts of a small set of hippocampal pyramidal cells induce theta activity in selected (but distributed) cortical regions, which are relevant for performing a particular task. Experimental findings support this view, and indicate that theta band power increases with increasing (episodic) task demands (Klimesch et al., 1994, 1996, 1997 a,c). Evidence for the view that only a small percentage of hippocampo-cortical feedback loops is synchronized comes from a re-examination of the pacemaker role of the septum in the production of the hippocampal theta rhythm (Petsche et al., 1962; Stewart and Fox, 1990). A large fraction of the septo-hippocampal projections terminate on inhibitory (GABA-ergic) hippocampal interneurones (Freund and Antal, 1988; see also the reviews
Time and the brain
276
in Lopes da Silva, 1992). Stewart and Fox (1990) assume that the septal input might organize the hippocampal theta activity, via rhythmic inhibition of hippocampal interneurones. Thus, interneurones are particularly likely to behave as theta cells (Fox and Ranck, 1981; Traub et al., 1989). The finding that only a small percentage of the pyramidal neurones displays synchrony, agrees with the idea that hippocampo-cortical feedback loops induce theta activity into selected cortical areas, where new information is encoded. Considering the fact that theta frequency induces, or at least enhances LTP (see also Lopes da Silva, 1992), it seems tempting to assume that theta activity, induced into selected cortical areas, reflects a process of encoding or retrieving new information, by keeping or putting selected cortical areas into a state of resonance (cf. Miller, 1999; Klimesch, 1996). Finally, an apparent contradiction with well known results should be discussed. Many studies have reported that absolute power in the theta frequency range increases with age (e.g. Christian, 1984; Niedermeyer, 1993b) and is increased in demented subjetcts (Brenner et al., 1986; Coben et al., 1985). Because memory performance decreases with age (and is decreased in demented subjetcs), these findings seem to contradict our hypothesis that an event-related increase in theta power reflects episodic memory performance. However, in a study by Doppelmayr et al. (1998b) it was demonstrated, that subjects with large absolute theta power show significantly smaller theta ERS than subjects with smaller theta power, who show large ERS. Thus, absolute theta power and the extent of event-related changes in theta power are negatively correlated. Consequently, the finding of an age related increase in theta power is in good agreement with our results, which are based on relative or event-related theta power.
REFERENCES Arnolds, D.E.A.T., Lopes Da Silva, F.H., Aitink, J.W., and Boeijinga, W. (1980) The spectral properties of hippocampal EEG related to behaviour in man. Electroencephalography and Clinical Neurophysiology, 50, 324–328. Baddeley, A. (1992) Working Memory. Science, New York, 255, 556–559. Billings, R.J. (1989a) The origin of the initial negative component of the averaged lambda potential recorded from midline electrodes. Electroencephalography and Clinical Neurophysiology, 72, 114–117. Billings, R.J. (1989b) The origin of the occipital lambda wave in man. Electroencephalography and Clinical Neurophysiology, 72, 95–113. Bland, B.H. (1986) The physiology and pharmacology of hippocampal formation theta rhythms. Progress in Neurobiology, 26, 1–54. Brenner, R.P., Ulrich, R.F., Spiker, D.G., Scalbassi, R.J., Reynolds, C.F., III, Marin, R.S., and Boller, F. (1986) Computerized EEG spectral analysis in elderly normal, demented and depressed subjects. Electroencephalography and Clinical Neurophysiology, 64, 483–492. Buzsaki, G., Horvath, Z., Urioste, R., Hetke, J., and Wise, K. (1992) High-frequency network oscillation in the hippocampus. Science, New York, 256, 1025–1027. Buzsaki, G., Leung, L., and Vanderwolf, C. (1983) Cellular bases of hippocampal EEG in the behaving rat. Brain Research Reviews, 6, 139–171. Buzsaki, G., Bragin, A., Chrobak, J.J., Nadasdy, Z., Sik, A., Hsu, M., and Ylinen, A.
Theta frequency, synchronization and episodic memory performance
277
(1994) Oscillatory and intermittent synchrony in the hippocampus: Relevance to memory trace formation. In: G. Buzsaki, R. Llinás, W. Singer, A. Berthoz and Y. Christen (eds) Temporal coding in the brain. Berlin, Heidelberg: Springer-Verlag, pp. 145–172. Coben, L.A., Danziger, W., and Storandt, M. (1985) A longitudinal EEG study of mild senile dementia of Alzheimer type: Changes at 1 year and at 2.5 years. Electroencephalography and Clinical Neurophysiology, 61, 101–112. Christian, W. (1984) Das Elektroencephalogramm (EEG) im höheren Lebensalter. Nervenarzt, 55, 517–524. Doppelmayr, M., Klimesch, W., Pachinger, Th., and Ripper, B. (1998a) Individual differencies in brain dynamics: Important implications for the calculation of eventrelated band power measures. Biological Cybernetics, (in press). Doppelmayr, M., Klimesch, W., Pachinger, Th., and Ripper, B. (1998b) The functional significance of absolute power with respect to event-related desynchronization. International Journal of Psychophysiology, (in press). Fox, S.E., and Ranck, Jr., J.B. (1981) Electrophysiological characteristics of hippocampal complex-spike cells and theta cells. Experimental Brain Research, 41, 399–410. Fox, S.E., Wolfson, S., and Ranck, J.B.Jr. (1986) Hippocampal theta rhythm and the firing of neurons in walking and urethane anesthetized rats. Experimental Brain Research, 62, 495–508. Freund, T., and Antal, M. (1988) Hippocampal interneurons. Nature, London, 336, 170– 173. Green, J.D., and Arduini, A.A. (1954) Hippocampal electrical activity in arousal. Journal of Neurophysiology, 17, 533–547. Greenstein, Y.J., Pavlides, C., and Winson, J. (1988) Long-term potentiation in the dentate gyrus is preferentially induced at theta rhythm periodicity. Brain Research, 438, 331–334. Klimesch, W. (1994) The structure of long-term memory: A connectivity model for semantic processing. Hillsdale, NJ: Lawrence Erlbaum. Klimesch, W. (1995) Memory processes described as brain oscillations in the EEG-alpha and theta band. Psycoloquy (electronic journal). Klimesch, W. (1996) Memory processes, brain oscillations and EEG synchronization. International Journal of Psychophysiology, 24, 61–100. Klimesch, W., Schimke, H., and Schwaiger, J. (1994) Episodic and semantic memory: An analysis in the EEG-theta and alpha band. Electroencephalography and Clinical Neurophysiology, 91, 428–441. Klimesch, W., Doppelmayr, M., Russegger, H., and Pachinger, Th. (1996) Theta band power in the human scalp EEG and the encoding of new information. NeuroReport, 7, 1235–1240. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Ripper, B. (1997a) Brain oscillations and human memory performance: EEG correlates in the upper alpha and theta bands. Neuroscience Letters, 238, 9–12. Klimesch, W., Doppelmayr, M., Pachinger, Th., and Russegger, H. (1997b) Event-related desynchronization in the alpha band and the processing of semantic information. Cognitive Brain Research, 6, 83–94. Klimesch, W., Doppelmayr, M., Schimke, H., and Ripper, B. (1997c) Theta synchronization in a memory task. Psychophysiology, 34, 169–176. Kroll, N.E.A., and Klimesch, W. (1992) Semantic memory. Complexity or connectivity? Memory and Cognition, 20, 192–210. Larson, J., Wong, D., and Lynch, G. (1986) Patterned stimulation at the theta frequency
Time and the brain
278
is optimal for the induction of hippocampal long-term potentiation. Brain Research, 368, 347–350. Leung, L., Lopes da Silva, F., and Wadman, W. (1982) Spectral characteristics of the hippocampal EEG in the freely mooving rat. Electroencephalography and Clinical Neurophysiology, 54, 203–219. Lisman, J.E., and Idiart, M.A.P. (1995) Storage of 7 +−2 short-term memories in oscillatory subcycles. Science, NewYork, 267, 1512–1515. Lopes da Silva, F.H. (1992) The rhythmic slow activity (theta) of the limbic cortex: An oscillation in search of a function. In: E. Başar and T.H. Bullock (eds) Induced rhythms in the brain. Boston: Birkhäuser, pp. 83–102. Maren, St., DeCola, J., Swain, R., Fanselow, M., and Thompson, R. (1994) Parallel Augmentation of hippocampal long-term potentiation, theta rhythm, and contextual fear conditioning in water-deprived rats. Behavioral Neuroscience, 108, 44–56. Miller, R. (1991) Cortico-hippocampal interplay and the representation of contexts in the brain. Berlin: Springer. Montoya, Ch., Heynen, A., Faris, P., and Sainsbury, R. (1989) Modality specific type 2 theta production in the immobile rat. Behavioral Neuroscience, 103, 106–111. Niedermeyer, E. (1993) Normal aging and transient cognitive disorders in the elderly. In: E. Niedermeyer and F.H. Lopes da Silva (eds) Electroencephalography: Basic principles, clinical applications, and related fields, 3 edn. Baltimore: Williams & Wilkins, pp. 329–338. O’Keefe, J. (1993) Hippocampus, theta and spatial memory. Current Opinion in Neurobiology, 3, 917–924. O’Keefe, J., and Dostrovsky, J. (1971) The hippocampus as a spatial map: preliminary evidence from unit activity in freely moving rats. Brain Research, 34, 171–175. O’Keefe, J., and Recce, M. (1993) Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus, 3, 317–330. Petsche, H., Stumpf, C., and Gogolak, G. (1962) The significance of the rabbit’s septum as a relay station between the midbrain and the hippocampus. Electroencephalography and Clinical Neurophysiology, 19, 25–33. Pfurtscheller, G., and Aranibar, A. (1977) Event-related cortical synchronization detected by power measurements of scalp EEG. Electroencephalography and Clinical Neurophysiology, 42, 817–826. Pfurtscheller, G. (1992) Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest. Electroencephalography and Clinical Neurophysiology, 83, 62–69. Schacter, D. (1977) EEG theta waves and psychological phenomena: A review and analysis. Biological Psychology, 5, 47–82. Scoville, W., and Milner, B. (1957) Loss of recent memory after bilateral hippocampal lesions. Journal of Neurology, Neurosurgery and Psychiatry, 20, 11–21. Sinclair, B.R., Seto, M.G., and Bland, B.H. (1982) Theta-cells in CA1 and dentate layers of hippocampal formation: Relations to slow-wave activity and motor behavior in the freely moving rabbit. Journal of Neurophysiology, 48, 1214–1225. Squire, L.R. (1992) Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans. Psychological Review, 99, 195–231. Stewart, M., and Fox, S.E. (1990) Do septal neurons pace the hippocampal theta rhythm? Trends in Neuroscience, 13, 163–168. Traub, R., Miles, R., and Wong, R. (1989) Model of the origin of rhythmic population oscillations in the hippocampal slice. Science, New York, 243, 1319–1325.
Theta frequency, synchronization and episodic memory performance
279
Tulving, E. (1984) Precis of elements of episodic memory. The Behavioral and Brain Sciences, 7, 223–268. Vanderwolf, C.H. (1992) The electrocorticogram in relation to physiology and behavior: a new analysis. Electroencephalography and Clinical Neurophysiology, 82, 165–175. Vanderwolf, C., and Robinson, T. (1981) Reticulo-cortical activity and behavior: A critique of the arousal theory and a new synthesis. Behavioral and Brain Sciences, 4, 459–514.
9 Distributed Assemblies, High Frequencies and the Significance of EEG/MEG Recordings Friedemann Pulvermüller Department of Psychology, Universität of Konstanz, PO Box D23, 78457 Konstanz, Germany and Cognition & Brain Sciences Unit Medical Research Council, 15 Chaucer Road, Cambridge CB2 2EF, England e-mail: [email protected]
According to recent proposals, coherent high-frequency activity in cortical neurones may be a specific indicator of cognitive processes in the brain, including those related to the perception of objects, the programming of actions, and the processing of language. One may therefore speculate that cell assemblies involved in higher cortical functions primarily generate high-frequency rhythms. In contrast, it has been argued that such neuronal assemblies may also generate equally strong spectral activity in various frequency bands, high and low, when active. It is proposed here that the controversy can be decided on the basis of knowledge about axonal conduction times inferred from fibre calibres measured in the human cortex. These data suggest that conduction times of most long axons are around 5–10 m/s, and even loops between neurones in distant areas can be travelled in a few hundredths of a second. Reverberation of excitation in assemblies would thus primarily lead to high-frequency cortical activity above 20 Hz. This leads to an explanation of recent findings from large-scale neurophysiological recordings during cognitive processing in humans. KEYWORDS: MEG; EEG; Cell assembly; Axonal conduction time; 30 Hz; 40 Hz; Gamma; Cognition; Language Cortical neurones usually fire a few spikes per second, that is, with frequencies of a few Hertz (see, for example, Hubel [1988] and Fuster [1995]). However, much faster activity, ranging from tens to hundreds of Hertz, has been found not only in cortical neurones, but in subcortical structures as well (Bressler and Freeman, 1980; Llinás and Ribary, 1993; Nuñez et al., 1992; Singer and Gray, 1995; Steriade, 1993). There are several types of high-frequency activity, including rhythms generated by individual neurones (Llinás et al., 1991; Gray and McCormick, 1996), by subcortico-cortical loops involving many neurones, which force them to fire synchronously (Llinás and Ribary, 1993; Steriade, 1993), and by interactions between excitatory and/or inhibitory cortical cells which also evoke coherent activity in these neurones (Singer and Gray, 1995; Traub et al., 1996). Apparently, several types of rhythms can be distinguished that play different
Time and the brain
282
roles in arousal and attention, perception, and higher cognitive processing (Pulvermüller et al., 1997). One possible role is the maintenance of active memory in the neocortex. Accordingly, a specific engram would correspond to a network of neurones in which activity reverberates, and thereby evokes a complex spatiotemporal activity pattern in its many neurons (Abeles, 1991; Fuster, 1995; Hebb, 1949). Figure 1 illustrates Donald Hebb’s idea about an assembly consisting of small groups of neurones connected in such a way that a wave of activity takes a complex path through the network, producing multiple reverberations. The loops in the assembly give rise to repeated well-timed activity in many neurones, and if multiple reverberations are allowed in each of these loops, several activation-deactivation cycles with similar frequencies will be generated when the assembly is active. This would be interesting for cognitive brain research, because if cell assemblies are sufficiently large and “influential” their high-frequency dynamics may be observable by large-scale non-invasive recording techniques, that is, by EEG and MEG, making it possible to scrutinize an important correlate of cognitive processes in humans. However, this experimental strategy depends crucially on the processes one envisages to occur during reverberatory activity of an assembly. If, for example, an assembly includes many neurones with slowly conducting axons connected in sequence, reverberations may occur only once a second or even less frequently. In contrast, if the assembly looks like the diagram in Figure 1 and therefore includes loops formed by only three smaller neurone populations (see the cycle 1–2-3), and if axonal conduction times in these neurones are rather fast, one may even expect 100 Hz or faster activity to be generated by the network, when it is active. Robert Miller (1994; 1999) has recently argued that cell assemblies should produce neuronal activity at various frequencies, and that no specific predictions can therefore be drawn about the frequency range primarily affected when an assembly becomes activated. However, one may ask whether there are indications that cell assembly activation affects certain frequency bands more than others, and whether there is a range where investigation appears to be particularly fruitful. Let us take the Hebbian idea illustrated in Figure 1 as a starting point: cell assemblies consist of neuronal subsets connected by orderly connections, which include loops. Thus, the subsets whose neurones fire synchronously become active in a fixed order, and some of the subsets (those included in loops) can become active repeatedly. For determining the frequency of these repetitions, three characteristics of the assemblies are of particular importance: (i) the distances to be bridged and, thus, the distributedness of the assemblies; (ii) the number of subsets making up an internal loop within an assembly; and (iii) the conduction times of the axons involved. Ad (i): It has been argued that the neurones of one assembly may well be spread out over distant areas in one hemisphere, and even over both hemispheres (Miller, 1996; Pulvermüller and Mohr, 1996). However, assembly neurones probably are not dispersed over the entire cortex, but include neurones in a small set of well-defined areas (Posner and Raichle, 1994). Figure 2 shows a sketch of an assembly, assumed to represent a word referring to an action, for example “to write”, whose neurones are distributed over three cortical loci, the anterior and the posterior language areas and motor cortices relevant for
Distributed assemblies, high frequencies and the significance of EEG/MEG
283
hand movements. In humans, the distances from the anterior to the posterior language region, from dorsal motor cortices to sites close to the Sylvian fissure, or between homologous areas of the hemispheres are in the order of 10 cm.
Figure 1. Hebb’s idea of a reverberatory cell assembly is illustrated. Arrows represent neurone sets and numbers indicate the sequence in which they become active.
Ad (ii): Although Hebb’s diagram suggests that only a few subsets form a loop within the assembly, there is no a priori argument against assuming longer loops. The smallest loops possible would be a neurone set projecting onto itself, and two sets with reciprocal connections. However, longer loops including 10 or even 20 neurone sets are possible. When estimating “round trip” times and frequencies, several options must be taken into account here. The time for travelling a loop also varies as a function of the number of steps, because synaptic transmission and the spreading of the postsynaptic voltage change towards the axon hillock, (where action potentials are generated) take time. About 1 ms after the action potential arrives at the presynapse, the excitatory postsynaptic potential reaches its maximum at the postsynaptic membrane, and it may take about another
Time and the brain
284
millisecond before voltage changes start to be detectable at the axon hillock. These synaptic and dendritic delays will be assumed to lie around 2 ms (1 ms synaptic and 1 ms dendritic delay). Accordingly, if all neurones of a subset A of an assembly fire, so that action potentials arrive synchronously at the neurones included in the next subset B, the B neurones will synchronously fire 2 ms later. If a loop includes n subsets, the synapticdendritic delay can be estimated to lie around 2n ms.
Figure 2. Hypothetical cell assembly representing a word referring to an action (e.g. to write). The cell assembly includes neurones in the anterior and posterior language regions and in motor and premotor areas. Circles indicate local neurone clusters, the short lines are called “within-area” connections, and the long lines are between-area connections.
Ad (in): Precise calculations of round trip times are possible based on actual data about axon calibres (Aboitiz et al., 1992; LaMantia and Rakic, 1990): The larger the diameter of an axon, the faster it conducts action potentials. The fibre calibres allow one to estimate conduction times (see below). Figure 3 presents data from macaca (left) and from homo sapiens (right). In both studies, fibres from the Corpus callosum were evaluated. It must be emphasized that the interpretation of these data rests on the assumption that the fibre composition of the callosum also gives an adequate picture of the axons connecting neurones within one hemisphere. This assumption may turn out to be wrong, in which case the estimations below would have to be revised. The histograms obtained from ape and monkey do not differ in their general shape and the most numerous fibre type differs between species by only a few tenths of a µm (ca. 0.5 µm in macaca vs. 0.8–0.9 µm in homo). This relatively good agreement may be taken as an argument in favour of the reliability of the results from humans. Both columns of histograms in Figure 3 plot the number (or percentage) of myelinated
Distributed assemblies, high frequencies and the significance of EEG/MEG
285
axons as a function of their calibres. The slow unmyelinated fibers were not included. They represented only around 10% of the total number of fibres, or even less. In addition, calibres, and therefore conduction times, of these few unmyelinated fibres vary greatly (see Table 1). Therefore, loops involving these fibres cannot generate predominant rhythms. In contrast to this view, Robert Miller (1994; 1996) has argued that the data from humans (Aboitiz et al., 1992) are questionable, because fixation may have been a problem and the number of unmyelinated axons may thus be underestimated. Miller therefore advocates interpretation of data from cat, rabbit and rat revealing higher percentages of unmyelinated fibers. However, consistent with the results in humans
Figure 3. Histograms from two studies of fibre calibres in the corpus callosum. Data from macaca are presented on the left (adapted from LaMantia
Time and the brain
286
and Rakic, 1990), and data from humans are on the right (adapted from Aboitiz et al., 1992). Normalized numbers of axons or their percentage is plotted for different calibers of myelinated axons.
(Aboitiz et al., 1992), the study by LaMantia and Rakic (1990) yielded small percentages of unmyelinated fibers in macaca indicating that in monkeys or even in higher mammals the slow fibres play a minor role. This would make good sense, in particular because the brains of most higher mammals are relatively large. While unmyelinated fibres certainly also participate in generating physiological responses in the human brain, the predominant physiological activity is probably related to the predominant anatomical substrate, that is, to the myelinated fibres slightly less than 1 µm in diameter. It is important to note that more than 60% of the axons have diameters in the range between 0.5 and 1 µm. Loops in the human cortex are therefore most likely to be formed by neurones with these frequent axon types. How rapidly would these axons conduct their action potentials? Table 1 presents estimated conduction times for fibres with different diameters. These estimates are based on measurements of peripheral nerves (e.g., Lee et al., 1986). (The fact that tissue shrinks during fixation has been taken into account.) Obviously, there are fibres with very fast conduction velocities which could bridge the gap between cortical areas 10 cm apart in less than 2 ms. On the other hand, there are very slow unmyelinated fibres taking almost half a second for this distance, and also thin myelinated axons along which an action potential would take around 20 ms. However, the histograms in Figure 3 indicate that both very fast and very slow fibers are relatively rare. The conduction velocities of the frequent types—with diameters between 0.5 and 1 µm—are between 5 and 10 m/s. Therefore, a 10 cm distance would be travelled in around 10 to 20 ms. When estimating predominant activity dynamics in cortical networks, it makes sense to consider two types of common axons, the fast common type (velocity 10 m/s) and the slow common type (velocity 20 m/s), to obtain an upper and a lower boundary for the estimates. Based on the numbers discussed above, it is easy to calculate round trip times in loops formed by various numbers of neurone sets connected by fibres with different calibres. However, it is adequate to restrict the considerations regarding the number of subsets connected in loops. In particular, for neurone populations located far apart, it appears likely that the loops they form do not include too many steps. Investigations in macaca indicate that the likelihood of two distant areas being connected is only around 30% (Young et al., 1994, 1995). However, connections are usually topographic and reciprocal. This means that if two neurone populations are located in two distant cortical areas, respectively, and if these subsets exhibit connections in one direction, the probability of the reciprocal connections existing is high (Young et al., 1995). On the basis of this reciprocity principle, two-step reverberations between distant subsets are likely. If one local population is connected to two other ones, the probability of the latter two exhibiting strong direct connections is low, and reverberations would again occur between the connected pairs. In contrast to the long-range connections, local con
Distributed assemblies, high frequencies and the significance of EEG/MEG
287
Table 1 True minimal diameter, corrected diameter, estimated conduction velocity and estimated travelling time for a distance of 100 to 130 mm listed for different fiber populations. Adapted from Aboitiz et al., (1992).
fibre group
corrected diameter (µm)
velocity axonal m/s
conduction delay 10–13 cm (ms)
unmyelinated
0.1–1
0.3–3.2
50–433
0.4 µm
0.6
5.2
19.2–24.9
1 µm
1.5
13.1
7.6–9.9
3 µm
4.6
40
2.5–3.2
5 µm
7.7
67
1.5–1.9
Table 2 Estimated “round trip” times and frequencies produced by three types of loops of neurones exhibiting conduction times around 10m/s (fast frequent type).
length of individual axon branches involved and conduction delays
overall conduction time (sum)
synaptic and dendritic delays (sum)
round trip time
round trip frequency
(mm|ms)
(ms)
(ms)
(ms)
(Hz)
local, 2 steps
0.1|0.01
0.02
4
4
250
local, 10 steps
0.1|0.01
0.1
20
20
50
local, 20 steps
0.1|0.01
0.2
40
40
25
withinarea, 2 steps
10|1
2
4
6
167
withinarea, 10 steps
10|1
10
20
30
33
betweenarea, 2 steps
100|10
20
4
24
42
loop type
nections between sets of adjacent spiny cells, and between neurones in adjacent cortical areas have a higher probability of existing, and therefore the possibility of longer loops,
Time and the brain
288
including, for example, 10 or even 20 steps, should be considered. Tables 2 and 3 present estimates of “round-trip” times for neuronal activity in local loops and between-area loops, together with the corresponding frequencies generated by reverberating activity. In addition to local loops between adjacent neurone populations and between-area loops, semi-local reverberations in neurone populations within one area are examined (distances around 1 cm). In Figure 2, this would correspond to activity dynamics assumed to occur in the local neurone clusters depicted as small circles (local loops), in the groups of 4 adjacent local clusters (within-area loops), and in the reciprocal long range connections that would allow for two-step long-range loops (between-area loops). Axonal conduction times are calculated as the sum of axonal conduction times and synaptic and dendritic delays. The two tables present data separately for the slow and the fast common axon types, evidenced by the neuroanatomical study. Synaptic and dendritic delays are estimated, as detailed above. Surprisingly, for all the parameter choices considered the reverberation frequencies are above 20 Hz. This means that a coherent wave of neuronal activity travelling along these
Table 3 Estimated “round trip” times and frequencies produced by various loops of neurones with axons exhibiting conduction times around 5 m/s (slow frequent type).
length of individual axon branches involved and conduction delays
overall conduction time (sum)
synaptic and dendritic delays (sum)
round trip time
round trip frequency
(mm|ms)
(ms)
(ms)
(ms)
(Hz)
local, 2 steps
0.1|0.02
0.04
4
4
250
local, 10 steps
0.1|0.02
0.2
20
20
50
local, 20 steps
0.1|0.02
0.4
40
40
25
withinarea, 2 steps
10|2
4
4
8
125
withinarea, 10 steps
10|2
20
20
40
25
betweenarea, 2 steps
100|20
40
4
44
23
loop type
loops takes less than 50 ms for one cycle. The predominant rhythms generated by
Distributed assemblies, high frequencies and the significance of EEG/MEG
289
between-area loops lie between 23 and 42 Hz, and for more local loops the frequency range extends towards even higher frequencies. One may argue that it is necessary to consider longer loops involving many more steps, and these would undoubtedly generate longer “round-trip” times and lower frequencies. However, as already mentioned, the not-so-frequent but reciprocal and topographic connections between-areas argue in favour of two-step loops, and against a larger number of steps. The local neurone clusters are linked through local axon collaterals synapsing on basal dendrites of adjacent cells. But both axon collaterals and dendritic trees have only a very limited range, so that after more than 10 or 20 steps the neurones activated probably cannot project back through their local connections. This also argues against longer local loops. Another point one may want to make is that, in many cases, local axon collaterals are not myelinated while the axon branches reaching the white matter are. This would lead to longer conduction times for the axons involved in local loops, so that the respective numbers in Tables 2 and 3 should be increased, probably by a factor of 10. However, because estimated conduction times are very low (below half a millisecond), such an increase would only lead to a slight increase of round-trip times, and to a minimal reduction of the frequencies. (Notice that the present estimates of round-trip times in local loops are entirely determined by synaptic and dendritic delays.) This adjustment would not change the overall conclusion. From these calculations, one may develop an optimistic perspective on the investigation of cognitive brain processes by means of non-invasive physiological recordings: if there are cell assemblies, including subsets of neurones connected in sequence and in loops, so that coherent reverberatory activity in many neurones is generated by the loops whenever the assembly is active, it may be possible to capture the gist of these reverberations at the surface of the head. In this case, the frequency range to which one may want to be particularly attentive is above 20 Hz and, if one is specifically interested in widely distributed neurone populations, in the 20–40 Hz range. Recent results from investigations of high-frequency dynamics during cognitive processing indicate that spectral dynamics between 20 and 40 Hz indeed co-occur with processes such as perception of a Gestalt-like figures or access to a word form or to its meaning (Pulvermüller et al., 1997). Interestingly, specific topographic patterns can sometimes be observed that may allow for conclusions on assembly topographies. (Pulvermüller, 1999) Let me finally stress again that the estimates above were made for the most frequent fibre types. Without any doubt, there are faster and slower fibres and, therefore, an assembly may well generate various rhythms, high and low, when active. However, the fast rhythms relying on the most common fibres probably predominate.
ACKNOWLEDGEMENTS I am grateful to Valentino Braitenberg and Robert Miller for ample discussion of ideas related to this article. Special thanks go to Detlef Heck and Almut Schüz who gave me advice on the neurophysiological and neuroanatomical parameters discussed. However, not they but the author is to be blamed for any remaining errors or inappropriatenesses.
Time and the brain
290
Supported by grants Pu 97/2, Pu 97/5, Pu 97/10 and Pu 97/11 of the Deutsche Forschungsgemeinschaft (DFG).
REFERENCES Abeles M. (1991) Corticonics—Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Aboitiz, F., Scheibel, A.B., Fisher, R.S., and Zaidel, E. (1992) Fiber composition of the human corpus callosum. Brain Research, 598, 143–153. Bressler, S.L., and Freeman, W.J. (1980) Frequency analysis of olfactory system EEG in cat, rabbit and rat. Electroencephalography and Clinical Neurophysiology, 50, 19–24. Fuster, J.M. (1995) Memory in the cerebral cortex. An empirical approach to neural networks in the human and nonhuman primate. Cambridge, MA: MIT Press. Gray, C.M., and McCormick, D.A. (1996) Chattering cells: superficial pyramidal neurns contributing to the generation of synchronous oscillations in the visual cortex. Science, New York, 274, 109–113. Hebb, D.O. (1949) The organization of behavior. A neuropsychological theory. New York: John Wiley. Hubel, D. (1988) Eye, brain, and vision. New York: Freeman. LaMantia, A.S., and Rakic, P. (1990) Cytological and quantitative characteristics of four cerebral commissures in the rhesus monkey. Journal of Comparative Neurology, 291, 520–537. Lee, K.H., Chung, K., Chung, J.M., and Coggeshall, R.E. (1986) Correlates of cell body size, axon size, and signal conduction velocity for individually labelled dorsal root ganglion cells in the cat. Journal of Comparative Neurology, 243, 335–346. Llinás, R.R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in mammalian cortex layer 4 exhibit intrinsic activity in the 10 to 50 Hz frequency range. Proceedings of the National Academy of Sciences, USA, 88, 897–901. Llinás, R.R., and Ribary, U. (1993) Coherent 40-Hz oscillation characterizes dream state in humans. Proceedings of the National Academy of Sciences, USA, 90, 2078–2081. Miller, R. (1994) Cognitive Processing, but not cell assembly ignition. Commentary on Pulvermüller et al. on brain-rhythms. Psycoloquy, 5 (50), 1–7. Miller, R. (1996) Axonal conduction times and human cerebral laterality. A psychobiological theory. Amsterdam: Harwood Academic Publishers. Miller, R. (1999) Unifying cell assembly theory with observations from brain dynamics. Behavioral and Brain Sciences, 22, 297–298. Nuñez, A., Amzica, F., and Steriade, M. (1992) Voltage-dependent fast (20–40 Hz) oscillations in long-axoned neocortical neurons. Neuroscience, 51, 7–10. Posner, M.I., and Raichle, M.E. (1994) Images of mind. New York: Scientific American Library. Pulvermüller, F., Birbaumer, N., Lutzenberger, W., and Mohr, B. (1997) High-frequency brain activity: Its possible role in attention, perception and language processing. Progress in Neurobiology, 52, 427–445. Pulvermüller, F., and Mohr, B. (1996) The concept of transcortical cell assemblies: a key to the understanding of cortical lateralization and interhemispheric interaction. Neuroscience and Biobehavioral Reviews, 20, 557–566. Pulvermüller, F., (1999) Words in the brain's language. Behavioral and Brain Sciences, 22, 253-366.
Distributed assemblies, high frequencies and the significance of EEG/MEG
291
Singer, W., and Gray, C.M. (1995) Visual feature integration and the temporal correlation hypothesis. Annual Review in Neuroscience, 18, 555–586. Steriade, M. (1993) Central core modulation of spontaneous oscillations and sensory transmission in thalamocortical systems. Current Opinion in Neurobiology, 3, 619– 625. Traub, R.D., Whittington, M.A., Stanford, I.M., and Jeffreys, J.G.R. (1996) A mechanism for generation of long-range synchronous fast oscillations in the cortex. Nature, London, 383, 621–624. Young, M.P., Scannell, J.W., Burns, G., and Blakemore, C. (1994) Analysis of connectivity: neural systems in the cerebral cortex. Reviews in Neuroscience, 5, 227– 249. Young, M.P., Scannell, J.W., and Burns, G. (1995) The analysis of cortical connectivity. Heidelberg: Springer.
Time and the brain
292
This page intentionally left blank.
10 Cell Assemblies, Associative Memory and Temporal Structure in Brain Signals Thomas Wennekers and Günther Palm Abteilung Neuroinformatik, Fakultät für Informatik, Universitat Ulm, Oberer Eselsberg, D-89069 Ulm, Germany Tel: (0049)–731-502–4151; FAX: (0049)–731–502–4156; e-mail: (thomas,palm)@neuro.informatik.uni-ulm.de
In this work we discuss Hebb’s old ideas about cell assemblies in the light of recent results concerning temporal structure and correlations in neural signals. We want to give a conceptual (and necessarily only rough) picture of how ideas about “binding by synchronization”, “synfire chains”, “local and global assemblies”, “short and long term memory” and “behaviour” might be integrated into a coherent model of brain functioning based on neuronal assemblies. KEYWORDS: Cell assemblies; Synchronization; Gamma-oscillations; Synfire chains; Memory; Behaviour
1. CELL ASSEMBLIES AND ASSOCIATIVE MEMORIES 1.1. Cell Assemblies Cell assemblies were introduced by Donald Hebb with the intention of providing a functional, and at the same time a structural model for cortical processes and neuronal representations of external events (Hebb, 1949). According to Hebb’s ideas, stimuli, objects, things, but also more abstract entities, like concepts, contextual relations, ideas (and so on) are considered to be represented in the brain by simultaneous activation of large groups of neurones, which are connected by relatively numerous and/or strong mutual excitatory synapses. Each single neurone may belong to many different cell assemblies. The determinant of an assembly is the connectivity structure between cells. This defines which cells lend support to each others’ firing, and hence have a higher probability of becoming coactivated in a reliable manner in response to different versions of the same stimulus. If an external stimulus excites a sufficiently large subset of cells of an assembly, then the whole assembly can “ignite” or “fire”, because recurrent activity,
Time and the brain
294
distributed via the specific mutual connections, can also raise above threshold those cells which are not (or are only weakly) stimulated externally. This can be viewed as an elementary associative process, where the firing of externally driven cells represents the key information, and triggers the firing of cells representing information addressed by, but not yet contained in the key. In a similar way, some kind of short term memory is also supported. If continuously changing sub-groups of cells produce maintained activation of other groups, the activation within the assembly may survive for some time after its ignition, even if the external stimulation has already vanished. The assembly concept furthermore proposes a mechanism for long term memory, that is, the formation of new assemblies in cortical tissue, under the influence of electrical activation. Learning is believed to be expressed in activity-dependent changes of synaptic efficacies, and the “Hebbian learning rule”, which is nowadays well known, states that synapses become strengthened when both connected cells are activated simultaneously within a certain time window. This hypothesis was motivated by the observation that events which repeatedly occur together should somehow belong together. Every time they appear in conjunction, they drive certain subgroups of cortical cells; the correlated firing of these sets of neurones (which are not yet assemblies!) should be learned and by that, the respective groups should become associatively connected. 1.2. Associative Memories The classical associative memory is an abstraction of the assembly concept (Willshaw et al., 1969; Palm, 1980, 1982), and provides a model based on single neurones as simple threshold elements. Figure 1 shows an example of an associative memory comprising 6 neurones with 6 inputs. Circles at the bottom, and vertical lines, represent threshold neurones and their dendrites, horizontal lines are axonal input fibres, and small filled circles are synapses, which are assumed to have an initial weight of zero. Neuronal excitation patterns are represented by binary {0,1} sequences reflecting whether or not cells fire in a given time window. A set of associations, that is, pairs of patterns (xi,yi), i=1,2,..P, is stored in the synaptic connectivity matrix. Special cases are the so-called auto-association, where xi equals yi for all pairs, or hetero-association, where xi and yi can be different (Palm, 1991). In Figure 1a two associations are stored: first the vector x1=(100101) activates the input lines, and y1=(110100) activates the neurones. All synapses, which simultaneously receive pre- and postsynaptic activity are set to a value of 1, indicated by the black filled synapses. Afterwards the second association, (x2,y2), is stored in the same way (grey synapses). Figure 1b displays an elementary retrieval process. The address pattern x=(111000) activates some of the previously strengthened synapses (squares), via the first three input lines. Each threshold-neurone sums over its activated synapses; this determines the “potentials” of the cells, shown as small numbers above the neurones. A neurone generates an output signal of 1, if its potential is larger than a certain threshold θ. Here θ=2, and thus, the output vector is y=(010011), which is obviously exactly the stored vector y2. The input pattern x during retrieval is more similar to x2 than to x1, because the number of common “ones” is 2 in the first, but only 1 in the second case. Hence this is a
Cell assemblies, associative memory and temporal structure in brain signals
295
correct association. In auto-associative networks the output vector can be fed back to the input for further recurrent iteration steps. For best performance in networks of realistic size the stored patterns should contain only a small fraction of active neurones, which is well in accordance with the situation in the cortex (“sparseness”, see Palm 1980). Further theoretical results on associative memories can be found in Willshaw (1967), Palm (1980, 1982, 1991), and Schwenker et al. (1996).
Figure 1. Simple associative memory model comprising 6 inputs and outputs, a) Learning of 2 associations (x1,y1) and (x2,y2). At first, the first pattern-pair is applied to input and output; then all synapses (crossings of vertical and horizontal lines) which are pre- and postsynaptically activated are set to a value of one (black circles). Afterwards the second pattern-pair is learned in the same way (grey synapses), b) Retrieval from an erroneous incomplete key, x= (111000). This input activates synapses that are drawn as black rectangles. The sum over activated synapses of a neurone determines its potential (small numbers above threshold neurones, (130122)); if this is larger than or equal to θ=2, the neurone generates an output of one, otherwise of zero. The output vector y is equal to the stored vector y2. Because the address pattern x has a larger overlap (common ones) with x2 than with x1 this is a proper association.
1.3. Spiking Neurones and Threshold Control The classical associative memory presumably covers basic aspects of pattern storage in real brains. Nonetheless, compared with biological neurones and synapses, the inclusion of further physiological details seems desirable, particularly, if we ask for temporal properties of neural systems. Conventional associative memory models employ simple threshold neurones and an iterative, time-quantized update scheme. Therefore they can give only a very rough insight into dynamical processes in cell assemblies. For this
Time and the brain
296
reason we add some features to the classical model, which mimic the spiking behaviour of real neurones as well as properties of spatiotemporal integration on dendrites. First, we choose a “continuous time” model in all simulations shown later.1 Single cells are modelled as “spiking neurones”: dendrites and soma of a cell are lumped into a single potential value. Every time this potential reaches a certain threshold θ a pulse-like “action-potential” is generated. Afterwards a suitable refractory mechanism prevents the cell from firing immediately afterwards. Action potentials, when they arrive at a target cell, evoke “post-synaptic potentials” of realistic form and time-constants, and the responses of different synapses are supposed to add up linearly. 1. We only give informal model descriptions in this paper, because the arguments and results in the sequel are to a large degree independent of implementational details. More precise descriptions and equations can be found in Wennekers et al. (1995) and Bibbig et al. (1995). For comparable spiking neurone models see Stein (1967) and Gerstner and v. Hemmen (1993)
Investigations of recurrent auto-associative memories by Schwenker et al. (1996) reveal the need for a proper continuous and global threshold control, in order to avoid the network being insensitive to any input (θ in Figure 1b being too large) or that its activity explodes (θ too small). In accordance with common ideas about cortical functioning (e.g. Braitenberg and Schüz, 1991), we assume that pyramidal cells and their excitatory connections carry out essential information processing tasks, like feature extraction, pattern storage, pattern retrieval, etc., and that inhibitory interneurones have mainly regulatory functions, one of which is probably some kind of activity control, similar to the threshold regulation required on theoretical grounds. Therefore, we start from an associative network of spiking neurones linked by excitatory connections, and embed interneurones within the network, which measure local firing rates and inhibit the excitatory cells accordingly. Networks of this architecture typically reveal collective oscillations when external excitation exceeds some critical level, and assuming excitatory and inhibitory synapses are strong and/or numerous enough. We have shown (Wennekers et al. 1995) that with reasonable assumptions concerning network parameters, firing can be sparse even in states of collective oscillations, that is, single neurones need not fire in every period (see also next section). This work also shows that the more realistic model neurones, with a threshold and refractory period, support synchronization more efficiently than models with a purely probabilistic spike encoder (e.g. rate modulated Poisson processes). This is because the threshold-crossing process determines precise firing times, whereas those are necessarily imprecise in modulated Poisson models.
2. LOCAL ASSEMBLIES Experiments on anaesthetised cats and alert behaving monkeys have shown that local populations of cells in primary visual areas often respond rhythmically with frequencies in the gamma-range (30–90 Hz) (cf. Eckhorn, this volume; experimental review in Singer
Cell assemblies, associative memory and temporal structure in brain signals
297
and Gray, 1995). Most interestingly, those “oscillations”, when observed at two distant cortical sites, can reveal a considerable amount of synchronization, which depends strongly on certain non-local stimulus properties, and seems to follow simple “Gestalt”rules like proximity, colinearity, etc., even though the recording sites may be located in different cortical areas or hemispheres. These findings have been taken as evidence for the so-called temporal correlation hypothesis of sensory integration in the mammalian cortex (von der Malsburg, 1981; Eckhorn et al., 1988; Singer and Gray, 1995), which states, that neurones which fire in response to stimulation by the same external entity should display correlated—in particular synchronized—firing. This way the cohesion of different parts of a single object can be signalled, even if those parts are processed over distributed regions of the brain (binding by synchronization). Many attempts have been made to model this “binding” process (c.f. chapters by Eckhorn, and Borisyuk, [this volume]; see also Wennekers and Palm [1997] for an overview and a discussion of some principal assumptions of similar models). In the sequel to this section we present our own simulations of oscillatory assemblies in localized patches of cortex, and discuss the possible role of fast gamma rhythms for information processing. 2.1. Local Dynamics in a Primary Area We consider a local patch of cortical tissue in a primary cortical area of (roughly) the size of a cortical column. This patch will contain cells tuned to different stimulus orientations, directions, velocities, etc. For simplicity we assume that cells can be ordered by their orientation preference and that similarly oriented cells have a higher probability of being connected. We neglect other tuning properties of cells which might be reflected by the local cortical circuitry, and also we do not model the laminar structure of real cortices. This means that, to a first approximation, we consider a one-dimensional topographically ordered model network of N=128 excitatory spiking neurones. Connectivities are restricted to some neighbourhood of a given cell (roughly N/4, with Gaussian spatial decay of probabilities of synapses). Here, topography is meant to represent orientation tuning, but note that a spatial interpretation is also possible. Embedded within the network of excitatory cells are inhibitory interneurones, which receive input from excitatory cells in a neighbourhood (of roughly N/8) and inhibit the excitatory cells accordingly. Inhibitory cells in the model have graded responses, and represent local pools of interneurones. All synaptic response functions have transmission delays of 1ms, a rise time of t,. and fall time of tf, where tr=1ms, tf =3ms for excitatory and tr=2ms, tf=5ms for inhibitory synapses. Axonal conduction delays are not included, since we only consider a localized patch of cortex. An external “bar” or “grating”-stimulus will excite cells in a local cortical network in different ways, depending strongly on their orientation (and maybe on other properties). This is represented as an external input current into the excitatory cells, which is centred at some “orientation” (say neurone N/2), and falls off in a Gaussian way. Furthermore each neurone also receives a certain amount of white noise background activity, corresponding to spontaneous spiking of background cells. Figure 2a displays the activation dynamics of our network for a “bar” stimulus, as
Time and the brain
298
explained above. Neurone N/2=64 receives the strongest input (e.g. is well tuned to the bar), and the input strength decays for smaller and larger neurone numbers (e.g. to less well-tuned cells). Shown as “LFP” in the figure is the ensemble average over excitatory postsynaptic potentials, averaged over all cells. This corresponds roughly to the physiologically derived local field potentials (LFP). In addition the spike trains of all excitatory cells are displayed as a raster plot over time (SUA, single unit activity). The LFP signal clearly shows oscillatory activity, although in a waxing and waning manner similar to physiological recordings. Diamonds above the LFP indicate spike-times of an arbitrary neurone (here neurone number 49, also indicated by the dashed horizontal line in the raster plot). Apparently the neurone is not rhythmic, but emits spikes only in some periods of the collective rhythm. A closer investigation of all single unit spike trains reveals that this is the generic case in our model. No neurone fires periodically. In particular we find the same categories of cells as reported by Eckhorn and Obermüller (1993) for experimental data: “locked” cells, whose spikes display auto-correlation histograms (ACH’s) with oscillatory side-peaks and which are also correlated with the collective rhythm, “lock-in” cells, which are coupled to the LFP but are not rhythmic by themselves (as revealed by the flatness of their ACH), and “non-participating” cells that are neither rhythmic nor coupled to the global oscillation, although they are driven by the stimulus and fire significantly! (see also the chapter by Eckhorn, this volume [especially Figure 1]).
Cell assemblies, associative memory and temporal structure in brain signals
Figure 2. Activation dynamics of a local cell assembly in response to a bar stimulus, a) Local field potentials (LFP; computed as a low-passed version of the total number of spikes per millisecond) reveal clear oscillations in the gamma-range. The raster plot of single unit activity of the 128 excitatory neurones (SUA) nonetheless shows that cells are only loosely synchronized, b) Firing rates of all single neurones (dashed curve; average taken over five seconds) reveal “tuning” properties, but most cells fire at rates well below the frequency of the LFP oscillation (51 Hz). Thick line in b): further spatial average over seven nearest neighbours, c) On average sub-optimally stimulated cells fire later in each LFP-cycle than optimally-stimulated ones. Here, lag is measured relative to maxima of LFP-oscillation by spiketriggered averaging.
299
Time and the brain
300
As a consequence of this irregular firing, single cells in our model can be tuned, which means that they change their firing frequencies when the stimulus orientation (and the corresponding centre of the Gaussian shaped input) changes. This can be seen in Figure 2b, where the firing rate of all neurones is displayed (dashed line; rates are averaged over a total sample length of five seconds) together with a further average over local neighbourhoods (thick line, roughly equivalent to multiple unit activity, MUA). The horizontal line at 51/s indicates the “oscillation” frequency derived from the power spectrum peak of LFP. Almost all cells have lower rates in accordance with for instance experimental results by Kreiter and Singer (1996). Furthermore the shape of the rate function in Figure 2b reflects the Gaussian input strength, and this in turn reflects the single cell tuning; deviations from the Gaussian shape are mainly due to random lateral connections. Although tuning is one of the most basic properties of cortical neurones, it cannot be observed in many network simulations, which aim to explain “binding by synchronization”. Those often operate in parameter-regimes, where single cells (or other kinds of simulated “units”) have very similar firing frequencies, equal to the oscillation frequency of the collective rhythm, and almost perfectly synchronize, when they are sufficiently strongly connected. As a consequence “tuning curves” are flat in the synchronized regime. In contrast to this “tight-binding” situation, in our simulations synchronization is only rather loose. Nonetheless, we should mention another property well in accordance with experimental results: König et al. (1995a) investigated precisely how sub-optimally driven cells lock into the collective rhythm. Measuring peak-shifts in cross-correlograms between two sites with orientation preference (ILM)1 and (ILM)2 and varying the orientation $ of a stimulus, they found, that sub-optimal cells reveal a systematic phaselag relative to optimally stimulated cells. This depends linearly on the stimulus orientation (ILM) and is, in addition, proportional to the difference (ILM)2–(ILM)1 between the preferred orientations of the recorded cells. This implies that the lag of cells with a particular orientation ψ relative to the best matching cells (or alternatively to some reference oscillation like the LFP in our simulation) must depend quadratically on ψ. This is clearly seen in Figure 2c, where phase-lags obtained from fitted peaks of spike triggered averages of the LFP signal are displayed together with a quadratic fit of the lags estimated from the same 5 second data set as before. Of course, the strong scatter in Figure 2a shows that, at least in our simulations, this lag is only an average property of cell firing, and by no means implies deterministic delays between firing times of different neurones. What actually happens in our network is that the best matching cells on average fire more often. They can ignite waves of activation spreading to less excited neurones, which have not yet fired. Therefore, these cells fire later. This may be used as a coding principle: the later a cell fires the less directly is it supported by the external stimulus, but instead may reveal properties of the cortical connectivity matrix. The inhibitory interneurones recognize the increase in activity and suppress further firing for some milliseconds. Due to different sources of randomness, these cycles of excitatory amplification and subsequent inhibition appear to be rather imprecise in amplitude and period duration. Nevertheless they reveal elementary properties, which cannot be explained by tight-binding theories. Two points are particularly important: First, there is a
Cell assemblies, associative memory and temporal structure in brain signals
301
pronounced global gamma-oscillation, but single cells show a broad spectrum of typically slower firing rates, and are coupled only loosely into the global rhythm. Second, in response to different stimulus conditions cells show systematic shifts in their firing times relative to other cells or local field potentials. These shifts, however, are observable only in cross-correlograms computed over long times; within single gamma-periods the firing of cells appears to be unreliable and the relative timing imprecise. After some reflection this suggests that it is less the “phase coupling” of oscillatory cortical activity in the gamma range that matters in our simulations, than the temporal synchronicity and fine spike timing shifts of sets of similarly tuned cells within single periods. Sets of cells are required for reasons of proper signalling since the single units fire unreliably. The oscillation appears as a by-product of repeated local processes, characterized by a fast spread of activity in excitatory subnetworks, followed by a subsequent inhibition phase, which is somewhat delayed, because inhibitory potentials are typically slower than excitatory ones, and the excitatory activity is itself needed to evoke the inhibition. Properties of the collective rhythm—amplitudes, frequencies—are rather imprecise. Therefore, it seems unlikely that they code for particular stimulus features. Similarly, due to the strong fluctuations, correlations decay quickly in time— typically exponentially on a time scale of several tens to less than 100 milliseconds—just as is found in experiments. This supports the idea that only short epochs of the signals are relevant for information processing, and long coherent wave trains (as used for example, in holography) are not necessary. In this context experiments by König et al. (1995b) are quite interesting: they show, that for electrode distances above roughly two millimetres (within or between areas) cross-correlograms have, almost exclusively oscillatory sidebands, whereas at shorter distances those can be completely missing. It seems that distant sites require (at least) two gamma-periods for effective mutual interactions. 2.2. Pattern Completion and Gamma-oscillations The previous section considered local assemblies in primary sensory areas. Higher association areas are usually small. Therefore, we may assume that those areas can be modelled reasonably well as fully connected associative memories (Palm, 1982). The above interpretation of gamma-oscillations viewed as fast excitatory amplification processes followed by inhibited phases also applies in this case. Here, the spread of activation proceeds from externally driven neurones (the address pattern) to associated neurones of the stored assembly or memory pattern. This may involve one or more steps of synaptic transmission, which might be identified with iterative feedback steps in the associative memory model. However, the growing inhibition suppresses the activation after a short time, interrupts the retrieval process, and restarts it after a short relaxation phase. This always keeps the network sensitive to changes in the input signal. Now, it is most notable that Schwenker et al. (1996) have shown that iterative retrieval in sparsely coded associative memories is extremely fast, provided that firing thresholds are adapted to the network activity at each step; then at most three feedback steps (and most often only one or two) suffice for perfect retrieval. In the current context this means the following: taking a few ms for a single associative feedback step in the cortex, which
Time and the brain
302
is determined by synaptic and axonal delays, perfect pattern completion can be performed in less than about 10ms, which well fits into the observable gamma-periods. Furthermore the work of Schwenker et al. (1996) shows that pattern completion is most efficient (in terms of storage capacity) when the number of ones in the address pattern is about half that of the stored patterns. In practice this means that the active input synapses to any relevant neurone that is about to be “addressed”, should not be less than half the synapses that could be activated by the complete pattern. This implies further, that spikes must be synchronous if efficiency is required. Since real neurones have an integration time of a few ms, all relevant information should be present during those short time intervals. These arguments show that the hypothesis that local information processing is essentially restricted to population bursts of cell pools in single gamma periods goes together very well with iterative retrieval in associative memories. Not only do synchronized spikes support a high memory capacity, but also frequencies in the gamma range are almost the fastest possible operation speed for rhythmic retrieval. A more thorough discussion of this and related topics can be found in Wennekers and Palm (1997).
Figure 3. Rhythmic associative retrieval in a fully connected memory, comprising 64 spiking neurones. Three input patterns (a-c)—coarse-
Cell assemblies, associative memory and temporal structure in brain signals
303
grained to a size of 8 times 8 bins—were used as input to the memory neurones. During a preceding learning phase, each pattern was individually presented to the network for 500 steps and learned, by means of a Hebbian coincidence rule (data not shown, cf. Figure 5). Test pattern (d) was applied during the displayed simulation run. Shown in e) are raster plots of only those neurones, which fire in response to the stimulus; these cells are ordered (as well as possible for three overlapping patterns) by their membership to the different patterns (roughly the upper third of units represents the triangle; middle part: square; lower part p