Time and the Brain
Conceptual Advances in Brain Research A series of books focusing on brain dynamics and information processing systems of the brain. Edited by Robert Miller, Otago Centre for Theoretical Studies in Psychiatry and Neuroscience, New Zealand (Editor-in-chief), Günther Palm, University of Ulm, Germany and Gordon Shaw, University of California at Irvine, USA.
Volume 1 Brain Dynamics and the Striatal Complex edited by R.Miller and J.R.Wickens Volume 2 Complex Brain Functions—Conceptual Advances in Russian Neuroscience edited by R.Miller, A.M.Ivanitsky and P.Balaban Volume 3 Time and the Brain edited by R.Miller Forthcoming Volume Sex Differences in Lateralization in the Animal Brain edited by V.L.Bianki and E.B.Filippova, translated by T.Ganf Volumes in Preparation Cortical Areas: Unity and Diversity The Female Brain Functional Memory and Brain Oscillations This book is part of a series. The publisher will accept continuation orders which may be cancelled at any time and which provide for automatic billing and shipping of each title in the series upon publication. Please write for details.
Time and the Brain Edited by
R.Miller Otago Centre for Theoretical Studies in Psychiatry and Neuroscience New Zealand
harwood academic publishers Australia • Canada • France • Germany • India • Japan Luxembourg • Malaysia • The Netherlands • Russia • Singapore Switzerland
This edition published in the Taylor & Francis e-Library, 2005. "To purchase your own copy of this or any of Taylor & Francis or Routledge's collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk." Copyright © 2000 OPA (Overseas Publishers Association) N.V. Published by license under the Harwood Academic Publishers imprint, part of The Gordon and Breach Publishing Group. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, without permission in writing from the publisher. Printed in Singapore. Amsteldijk 166 1st Floor 1079 LH Amsterdam The Netherlands British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library. ISBN 0-203-30457-8 Master e-book ISBN
ISBN 0-203-34338-7 (Adobe eReader Format) ISBN: 90-5823-060-0 (Print Edition) ISSN: 1029-2136
CONTENTS
Series Preface Preface List of Contributors 1 Empirical Evidence about Temporal Structure in Multi-unit Recordings Alessandro E.P.Villa 2 Cross-correlograms for Neuronal Spike Trains. Different Types of Temporal Correlation in Neocortex, their Origin and Significance Lionel G.Nowak and Jean Bullier 3 The Space-Time Continuum in Mammalian Sensory Pathways Asif A.Ghazanfar and Miguel A.L.Nicolelis 4 Information Flow along Neocortical Axons Harvey A.Swadlow 5 Psychophysics of Human Timing Thomas H.Rammsayer and Simon Grondin 6 Cortical Processing by Fast Synchronization: High Frequency Rhythmic and Non-rhythmic Signals in the Visual Cortex Point to General Principles of Spatiotemporal Coding Reinhard Eckhorn 7 EEG Alpha and Cognitive Processes Wolfgang Klimesch 8 Theta Frequency, Synchronization and Episodic Memory Performance Wolfgang Klimesch 9 Distributed Assemblies, High Frequencies and the Significance of EEG/MEG Recordings Friedemann Pulvermüller 10 Cell Assemblies, Associative Memory and Temporal Structure in Brain Signals Thomas Wennekers and Günther Palm 11 The Relation between EEG and Evoked Potentials Erol Başar , Sirel Karakas , Elke Rahn and Martin Schürmann 12 Coherence and Phase Relations between EEG Traces Recorded from Different Locations Peter Rappelsberger , S.Weiss and Baerbel Schack 13 Temporal Structure of Neural Activity and Models of Information Processing in the Brain Galina N.Borisyuk , Roman M.Borisyuk and Yakov B.Kazanovich
vii ix xi 1 63
112 150 181 196
236 262 281
293 320 346
389
Discussion Section POSTLUDE The Neuroanatomy of Time Valentino Braitenberg Index
410 454 460
SERIES PREFACE The workings of the brain, including the human brain are a source of endless fascination. In the last generation, experimental approaches to brain research have expanded massively, partly as a result of the development of powerful new techniques. However, the development of concepts which integrate and make sense of the wealth of available empirical data has lagged far behind the experimental investigation of the brain. This series of books entitled Conceptual Advances in Brain Research (CABR) is intended to provide a forum in which new and interesting conceptual advances can be presented to a wide readership in a coherent and lucid way. The series will encompass all aspects of the sciences of brain and behaviour, including anatomy, physiology, biochemistry and pharmacology, together with psychological approaches to defining the function of the intact brain. In particular, the series will emphasise modern attempts to forge links between the biological and the psychological levels of describing brain function. It will explore new cybernetic interpretations of the structure of nervous tissue; and it will consider the dynamics of brain activity, integrated across wide areas of the brain and involving vast numbers of nerve cells. These are all subjects which are expanding rapidly at present. Subjects relating to the human nervous system as well as clinical topics related to neurological or psychiatric illnesses will also make important contributions to the series. These volumes will be aimed at a wide readership within the neurosciences. However, brain research impinges on many other areas of knowledge. Therefore, some volumes may appeal to a readership, extending beyond the neurosciences. Books suitable for the series are monographs, edited multiauthor collections or books deriving from conferences, provided they have a clear underlying conceptual theme. In order to make these books widely accessible within the neurosciences and beyond, the style will emphasise broad scholarship comprehensible by readers in many fields, rather than descriptions in which technical detail of a particular speciality is dominant. The next decades promise to provide major new revelations about brain function, with far-reaching impact on the way we view ourselves. These great breakthroughs will require a broad interchange of ideas across many fields. We hope that the CABR series plays a significant part in the exploration of this important frontier of knowledge.
PREFACE Since the days of Galileo, time has been a fundamental variable in scientific attempts to understand the natural world. It has been clear since the first recordings of electrical activity in the brain, by Caton, and later by Danielewski, that electrical signals from the brain consist of very complex temporal patterns. This perspective has continued to the present day. It is now richly demonstrated by recordings of electrical activity at both the single unit level and by the methods of electroencephalography. When recording of the electrical activity from single nerve cells in the brain became possible, the emphasis was initially on events occurring on a more brief time scale than the fastest of psychological processes. However, in more recent years, single unit activity has also been widely studied on a larger time scale, corresponding to many psychological processes. In particular, in the last decade there has been a major shift in the way we view single unit activity in several brain structures: A generation ago, the emphasis was on mapping space (for instance in a sensory receptor surface) against spatial location in the brain. Nowadays we are more concerned with the temporal structure detectable in neuronal impulse trains. Temporal structure in impulse trains may thus be part of the way in which timing of sensory or motor events is represented in the brain; and indeed, it is possible that external events and inner cognitive processes may be represented in the brain in part using temporal coding, even when these events and processes, in themselves, lack detailed temporal structure. Analysis of temporal structure in the activity of large populations of nerve cells in the brain has also undergone major advances in the last ten or fifteen years. Functional brain imaging using metabolic activity gives good spatial resolution of brain processes. In contrast, mapping based on brain activity, based on the EEG or the MEG is superior in revealing the temporal structure of dynamic processes in the brain. For the EEG, the time scale needed to describe the basic rhythms of brain electrical activity corresponds roughly to that of the more rapid of psychological processes. That electrical activity is also known to correlate in many ways with psychological or behavioural events observed at the level of the whole organism. This book, the third in the series Conceptual Advances in Brain Research, explores modern approaches to these temporal aspects of brain electrical activity. The earlier chapters focus mainly on temporal structure revealed from trains of impulses recorded from single nerve cells, or from several such nerve cells recorded at the same time, and their possible relation to behaviour and psychological processes. In the later chapters, the emphasis shifts to temporal structure in the EEG, again with several essays exploring correlations with behaviour and psychology. There are also two chapters of a more theoretical nature. The final chapter forms an edited discussion between contributors, delving further into a variety of issues which arose during editing of the main chapters. In this, a significant question is also touched on—the relation (if any) between temporal structure discernible at the single unit level, and that seen in the EEG.
Study of the electrical activity of the brain by EEG recording and by the methods of single unit electrophysiology are often not well integrated: The two approaches tend to be adopted in different laboratories, and indeed in different countries. By bringing together within a single book modern information acquired using both approaches it is hoped that these two approaches can become better integrated as complementary windows on the information processing achieved by the brain. As editor of this book, I have to say that I am greatly honoured to have received such a fine set of chapters to edit, and I hope that my own contributions to the book are at the same standard as those of the chapter authors. R.Miller.Dunedin. December. 1999
LIST OF CONTRIBUTORS Erol Başar Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Galina N.Borisyuk Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region, 142292 Russia Roman M.Borisyuk Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region, 142292 Russia Valentino Braitenberg Max Planck Institute for Biological Cybernetics Spemann Strasse 38 D-7400 Tübingen Germany Jean Bullier INSERM U371 Cerveau et Vision 18 Avenue du Doyen Lépine 69675 Bron Cedex France Reinhard Eckhorn Department of Physics
Neurophysics Group Philipps-University Renthof 7 D-35032 Marburg Germany Asif A.Ghazanfar Department of Neurobiology Box 3209 Duke University Medical Centre 101 Research Drive Bryan Research Building Durham, NC 27710 USA Simon Grondin Institute of Psychology University of Goettingen Gosslerstrasse 14 D-37073 Goettingen Germany Sirel Karakas TÜBITAK Brain Dynamics Research Unit Ankara Turkey Yakov B.Kazanovich Institute of Mathematical Problems in Biology Russian Academy of Sciences Pushchino Moscow Region 142292 Russia Wolfgang Klimesch Department of Physiological Psychology Institute of Psychology University of Salzburg Hellbrunnerstrasse 34 A-5020 Salzburg Austria
Miguel A.L.Nicolelis Department of Neurobiology Box 3209 Duke University Medical Centre 101 Research Drive Bryan Research Building Durham, NC 27710 USA Lionel G.Nowak Section of Neurobiology Yale University School of Medicine C303 Sterling Hall of Medicine 333 Cedar Street New Haven, CT 06510 USA Günther Palm Abteilung Informatik Fakultät für Informatik Universität Ulm Oberer Eselberg D-7900 Ulm Germany Friedemann Pulvermüller Department of Psychology Fachgruppe Psychologic Sozialwissenschaftliche Fakultät Universität Konstanz Postfach D23 D-78457 Konstanz Germany Elke Rahn Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Thomas H.Rammsayer Institute of Psychology University of Goettingen
Gosslerstrasse 14 D-37073 Goettingen Germany Peter Rappelsberger Institute for Neurophysiology University of Vienna Waehringer Strasse 17 A-1090 Vienna Austria B.Schack Institute for Medical Statistics, Informatics and Documentation Friedrich Schiller University Jahn Strasse 3 D-07740 Jena Germany Martin Schürmann Institute of Physiology Medical University Lübeck Ratzeburger Allee 160 D-23538 Lübeck Germany Harvey A.Swadlow Department of Psychology (U-20) The University of Connecticut Storrs, CT-06269 USA Alessandro E.P.Villa Laboratory of Neuro-heuristics Institute of Physiology University of Lausanne 7 Rue du Bugnon CH-1005 Lausanne Switzerland S.Weiss Institute for Neurophysiology University of Vienna Waehringer Strasse 17 A-1090 Vienna
Austria Thomas Wennekers Abteilung Informatik Fakultät für Informatik Universität Ulm Oberer Eselberg D-7900 Ulm Germany
1 Empirical Evidence about Temporal Structure in Multi-unit Recordings Alessandro E.P.Villa Laboratory of Neuro-heuristics, Institute of Physiology, University of Lausanne, 7 Rue du Bugnon, CH-1005 Lausanne, Switzerland Tel: ++41+21–692–5532/X.5516/X.5500; FAX ++41–21–692.5532 e-mail:
[email protected] URL: www-lnh.unil.ch
The brain is a highly interconnected network of neurones, in which the activity in any neurone is necessarily related to the combined activity in the neurones that are afferent to it. Due to the widespread presence of reciprocal connections between brain areas, reentrant activity through chains of neurones is likely to occur. Certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements as a consequence of developmental and/or learning processes. In cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring, ordered, and precise (in the order of few ms) interspike interval relationships are referred to as “spatiotemporal patterns” of discharges. This term encompasses both their precision in time, and the fact that they can occur across different neurones, even recorded from separate electrodes. This chapter introduces the fundamental assumptions and algorithms that lead to the detection of complex patterns of neural discharges and introduces a way of interpreting this activity within the framework of non-linear dynamics. Empirical results of experimental and simulation studies are provided in different sections of the chapter. KEYWORDS: Spatiotemporal firing patterns; Neural dynamics; Brain theory; Time code; Frequency code; Multi-unit recordings; Non-linear dynamics; Sensorimotor association
1. BRIEF HISTORICAL INTRODUCTION In 1753–1755 the physiologist Albrecht von Haller published in Göttingen an historical essay, the “Dissertation on the Irritable and Sensitive Parts of Animals” (original title: De partium corporis humani sensibilius et irritabilus). This work was based on numerous experiments of vivisection, and on stimulation of organs using the new knowledge
Time and the brain
2
offered to physiology by physics, chemistry and natural history. With a rudimentary technique of stimulation, Von Haller classified the parts as irritable, sensible or elastic and noted that the reactions varied between different parts of the brain. The historical importance of the work by Von Haller is not so much related to the results obtained, but rather in systematically developing a transdisciplinary approach to brain research using the new technologies of his time. In 1791 the Italian physician Luigi Galvani started the publication of a remarkable series of studies that demonstrated muscle twitch in a frog by touching its nerves with electrostatically charged metal, and later using two dissimilar uncharged metals. These observations led Galvani to postulate that the circulation of a particular body fluid, that exists naturally in the nerves in a state of disequilibrium, provided the stimuli for the muscle fibres to contract. In addition, normal muscular contraction without a source of electrostatic electricity was, in Galvani’s view, evidence for the existence of an additional and “natural” form of electricity, that he called “animal electricity”. This latter statement set the scene for the famous Galvani-Volta controversy. Alessandro Volta, a friend of Galvani and an Italian physicist who, in 1775, invented the electrophorus—a device to generate static electricity—gave an opposed and experimentally valid explanation of Galvani’s experiment. The electricity did not come from the animal tissue but was generated by the contact of different metals, brass and iron, in a moist environment. The interest in this controversy resides in the confrontation of two basic, irreducible interpretations of the same observation, derived from each scientist’s different background: Galvani saw the frog phenomenon as the work of biological organs, Volta as that of a physical apparatus. The outcome of the controversy was exceptional. On the one hand the challenge of Volta’s opinion led Galvani to perform a new series of experiments that demonstrated muscular contraction by touching the exposed muscle of one frog with the nerve of another frog, thus showing for the first time the existence of bioelectric forces. On the other hand, Volta focused his research efforts upon the study of electric fluids between dissimilar metals, and in 1800 he presented the first electric battery, providing future researchers with a stable source of electricity not dependent on electrostatic forces. With the introduction of currents of “Galvanic fluids” into the brain, a way was opened to making new discoveries in physiology at the beginning of the XIXth century. Electricity was used not just as an experimental intervention, which formed the basis of electrophysiology (the study of the connection between living organisms and electricity), but in addition, the proper characteristics of propagation and generation of this type of energy became the basis of fertile hypotheses. In the last quarter of the XIXth century, Eduard Hitzig and Gustav Fritsch discovered the localization of cortical motor areas in the dog, using electrical stimulation, and Richard Caton was the first to record electrical activity from the brain. Electrophysiology started to develop rapidly, and Edgar D.Adrian published a seminal study suggesting the all-or-none principle in nerve (1912). In the late 1920s Hans Berger in Germany demonstrated the first human electroencephalogram and opened the way to clinical applications of electrophysiology. Nevertheless, the English School led investigations in electrophysiology in the first part of our century, and for his specific research on the function of neurones, Adrian shared the 1932 Nobel Prize for Medicine with Sir Charles Sherrington. Although most remembered for his scientific contributions to neurophysiology, Sherrington’s research focused on spinal reflexes as
Empirical evidence about temporal structure
3
well as on the physiology of perception, reaction and behaviour. His approach was transdisciplinary, in the sense that disciplines were used not only “one next to the other”, but were really intermingled in his protocols, an extraordinary example of a nonreductionist view of neurophysiology. At present, and still based upon the work performed by Adrian (1934), most neurophysiologists make their deductions by observing mean frequencies of nervous discharges (spikes), i.e. whether there are a lot or only a few spikes over a relatively prolonged time interval. Important concepts and findings have been clarified by using the overall mean rate as a measure of neuronal activity, at both peripheral and central levels of the nervous system. In particular it is important to remind ourselves of the influence of the findings of Mountcastle et al. (1963) indicating that the relationship between frequency of firing of selected neurones in the ventrobasal thalamus of deeply anaesthetized monkeys and the angle of extension of the contralateral knee was a perfect power function. Hence, the relationship between log frequency and log angle is virtually a perfect linear function. Although this point will not be developed further in the present chapter, it is important to note that the effect, and the type of anaesthesia determine dramatically the neuronal discharge pattern, especially at thalamic and cortical levels (e.g., see Mukhametov et al., 1970; Zurita et al., 1994). This effect has not always been adequately taken into account by investigators eager to develop theoretical models of “neuromimetic” circuits. The slow integration time of nerve cells, operating in the range of a few milliseconds, roughly a million times slower than presently available supercomputers, and the huge number of connections established by a single neurone (for review, see Braitenberg and Schüz, 1991) has suggested that information in the nervous system might be transmitted by simultaneous discharge of a large set of neurones. Multiple dimensions of stimuli relevant to sensory function and behaviour are processed by thousands of neurones distributed over many areas of the brain. Indeed, the hypothesis that neurones (both individually and jointly) process information over time, and following precise time relationships, has pervaded the neurosciences ever since the nervous system was conceptualized as a set of dynamic networks of interacting neurones (McCulloch and Pitts, 1943; for review, see MacGregor, 1987). That neurones convey a temporal code has been well-known since electrophysiological studies in the 1960s led to the recognition that spike trains—the time series formed by the sequences of time intervals between spikes—were related to meaningful physiological variables (Bullock, 1961; Segundo et al., 1963; Perkel and Bullock, 1965; Segundo et al., 1966; Nafe, 1968). Subsequent work made an important contribution, by recognizing temporal coding in different experimental circumstances and animal species, and by proposing worthwhile quantification procedures (Perkel et al., 1967a,b; Segundo and Perkel, 1969; Legéndy, 1975; Eckhorn et al., 1976; Brillinger et al., 1976; Klemm and Sherry, 1981; Abeles, 1982; Tsukada et al., 1982; Rosenberg et al., 1989; Bialek et al., 1991; Rapp et al., 1994; Friston, 1995; Martignon et al., 1995; Rieke et al., 1995; de Ruyter van Steveninck et al., 1997). This provided new insights into how the association between simultaneously recorded spike trains, in the time and frequency domains, may reflect the degree of coordinated activity within a cell assembly. The integrative capabilities of neurones (for review, see Segundo, 1986) on the one hand, and parallel processing and redundancy
Time and the brain
4
inherent to neural pathways on the other hand, have become more obvious during the past two decades, thanks also to the advances in neuroanatomical methods using retrograde and/or anterograde tracers (e.g. horseradish peroxidase, fluorescent dyes, biocytin, etc.). However, for most functional systems the available electrophysiological data demonstrate merely the presence of temporally organized neural activity, so that during recent years investigations have mainly focused on establishing causal relations between the occurrence of precise temporal relationships and cognitive or motor processes (Fetz, 1997). Therefore, two hundred years after the Galvani-Volta controversy, two main classes of theories explaining information processing in the brain have been proposed: In one, neurones convey a precise temporal code (Abeles, 1982; Abeles, 1991), while the other is based on noisy rate coding (Shadlen and Newsome, 1994). According to the rationale of a noisy rate code, the question has been raised whether the variability of spike intervals carries information. This variability may depend on the role of the decay of the post-synaptic potential (PSP) in determining the prevailing operating mode of network processing (König et al., 1996). Softky and Koch (1993) have shown that, in order to find the same degree of variability seen in vivo, coincidence detection with fast integration of small excitatory PSPs is required in models. Alternatively, Shadlen and Newsome (1994) have proposed that the balance between inhibition and excitation plays a critical role in the variability of network behaviour with membrane time constants of 8–20 ms. If an irregular interspike interval results from integration of excitatory and inhibitory postsynaptic potentials, then the timing of postsynaptic spikes is random and can no longer reflect the timing of presynaptic events. Precise patterns of spikes—their intervals and coincidences—would fail to propagate (Shadlen and Newsome, 1994). König et al., (1996) have provided a summary of these two viewpoints and proposed that the interplay of excitation and inhibition (E-I) effectively reduces the time constant for synaptic integration, thus providing coincidence detection in the cortex, although the actual synaptic decay is relatively long. Indeed, a balance condition of excitatory and inhibitory post-synaptic potentials plays an important role in the activity of many network models (Wilson and Cowan, 1974; Douglas and Martin, 1991; Tsodyks and Sejnowski, 1995; Usher and Stemmler, 1995; Xing and Gerstein, 1996) and has many complex spatial and temporal influences (Thomson and Deuchars, 1994). Changes in cellular excitability modify the time to firing, thus altering the E-I dynamics. Some specific examples of time-varying factors influencing the balance of E-I levels are the cell threshold potential, PSP kinetics, receptor activity such as NMDA, and calcium currents (Edmonds et al., 1995). The inability to preserve information about the time of a spike suggests that the discharge rate cannot be modulated in a time-locked fashion to specific inputs. In a random walk model (Gerstein and Mandelbrot, 1964) this assumption may be wrong because the discharge rate can follow the activity of inputs, but, given a base rate, the time to the next spike would appear as random. Then, the average instantaneous discharge rate through an ensemble of neurones belonging to a functional assembly would be capable of transmitting changes in spike rate with a precision of 10–50 ms (Shadlen and Newsome, 1998). Recordings performed in several cortical areas of behaving monkeys support the hypothesis of a connection between fluctuations in neural discharge rate and behaviour (Georgopoulos et al., 1993; Celebrini and Newsome, 1994). However, the existence of the rate code
Empirical evidence about temporal structure
5
mechanism does not imply that temporal codes do not also exist in the brain. The cerebral cortex is a highly interconnected network of neurones, in which the activity of each cell is necessarily related to the combined activity in the neurones that are afferent to it. Due to the presence of reciprocal connections between cortical areas, reentrant activity occurs, through chains of neurones. Furthermore, certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements, as a consequence of developmental and/or learning processes. According to the rationale of a precise temporal code, in cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring and ordered interspike intervals, with a precision of the order of a few milliseconds, are referred to as “spatiotemporal patterns” of discharges. For true demonstration, such temporal firing patterns must occur to a statistically significant level (see Figure 1). If functional correlates of spatiotemporal neural coding exist, one would expect that whenever the same information is presented, the same temporal pattern of firing would be observed. Several lines of evidence exist showing Spatiotemporal patterns in vivo (Bair and Koch, 1996) and in vitro (Mainen and Sejnowski, 1995). Recent studies on active propagation of action potentials in dendrites have provided additional results supporting the existence of precise neuronal timing (Stuart and Sakmann, 1994). A synaptic response increases if the presynaptic spike precedes the postsynaptic spike, but if the order is reversed, the synaptic response decreases. The window for synaptic
Time and the brain
6
Figure 1 . Outline of the general procedure followed by pattern detection algorithms, (a) Analysis of a set of simultaneously recorded spike trains. Three cells, labelled A, B, and C, participate in patterned activity. Three occurrences of a precise pattern are detected. Each occurrence of the pattern has been labelled by a specific marker in order to help the reader to identify the corresponding spikes, (b) Estimation of the statistical significance of the detected pattern, (c) Display of pattern occurrences as a raster plot aligned on the start of the pattern.
plasticity is 100 ms wide, and a difference in spike timing of only 10 ms near coincidence switches plasticity from potentiation to depression (Markram et al., 1997). Neurophysiological measurements in human subjects performing a visual go/no-go categorization task indicate that speed of processing of visual information in human is in
Empirical evidence about temporal structure
7
the order of 150 ms (Thorpe et al., 1996). Taking into account the firing rates of neurones in the visual pathway, only few spikes at most can be generated by each neurone in performing this highly demanding task, thus providing strong arguments for the importance of precise temporal coding. Indeed, electrophysiological experiments in primates have established that correlated firing between single neurones recorded simultaneously in the frontal cortex may evolve within tens of milliseconds, in systematic relation to behavioural events, without modulation of the firing rates (Vaadia et al., 1995). Furthermore, in multiple electrode recordings performed in the primary motor cortex of monkeys trained in a delay-pointing task, spike synchronization occurred in relation to purely internal events (stimulus expectancy), during which modulations of firing rate were distinctly absent (Riehle et al., 1997). These findings provide an important support for the general view that a stronger synaptic influence is exerted by multiple converging neurones firing in coincidence, thus making synchrony of firing ideally suited for highlighting responses and for expressing relations among neurones with high temporal precision.
2. DETECTION OF SPATIOTEMPORAL FIRING PATTERNS An influential and remarkable model based on the assumptions of high temporal precision in brain processing is the “synfire” chain hypothesis. This model suggests how precise timing can be sustained in the central nervous system by means of feed-forward chains of convergent/divergent links and re-entrant loops between interacting neurones forming an assembly (Abeles, 1982). Structures like synfire chains may exhibit attractors in which a group of neurones excite themselves, maintaining elevated firing rates for long periods and allowing the same neurone to participate in many different synfire chains. A fundamental prediction of such a model is that simultaneous recording of activity of cells belonging to the same assembly and involved repeatedly in the same process should be able to reveal repeated occurrences of such spatiotemporal firing patterns. Note that the term “firing pattern” encompasses both their precision in time, and the fact that they can occur across different neurones, even when recorded from separate electrodes. The following example (Figure 2) illustrates schematically the recurrence of patterns in a network connected according to the “synfire” model. A chain of 56 model neurones is formed by 8 sets of 7 neurones connected by diverging/converging links. This means one neurone in a set (in black in Figure 2a) receives 7 inputs from the previous set and projects to all 7 neurones of the next set. In the example shown in Figure 2a the links are subdivided into 5 excitatory (open circle) and 2 inhibitory connections (gray circles). This network is embedded into a larger network (200 neurones) with random connections. Note that the last link of the chain is connected to the first link. The model neurones follow simple integrate-and-fire dynamics (Villa and Tetko, 1995; Hill and Villa, 1997). Each neurone integrates all post-synaptic potentials. If the integrated depolarization passes the threshold, the neurone fires, becoming refractory, whereas if firing does not take place, the membrane potential tends to return to the resting level. Synaptic strengths and time constants have been selected in
Time and the brain
8
order to allow “spontaneous activity” to propagate but also to prevent “epileptic” spikes to self-generate in a simulation run. Figure 2b shows 28 successive frames of the neuronal chain activity, in arbitrary time units. Let us assume that three selected neurones, i.e. c10, c36, and c46, are recorded by virtual electrodes. The corresponding spike trains are shown in Figure 2c, and a firing pattern formed by four spikes was detected in these spike trains. More generally, let the list of cell labels that appear in a pattern of c spikes be noted as S c =(i 1 ,...,i j ,...i c ). In this, any label of the recorded neurones can be assigned to i j . In particular, for a 4-tuple pattern of spikes (quadruplet) obtained from a simultaneous recording of ten spike trains, let us assume that we find S 4 =(c36, c36, c10, c46), meaning all patterns of complexity c equal to 4, formed by a spike recorded from cell c36, followed by another event of cell c36, then a spike recorded from cell c10, which in turn is followed by a spike recorded from cell c46. One specific example of such patterns with delays of 6 time units between the first two spikes from
Figure 2 . Example of spatiotemporal pattern detected in a simulated chain of diverging/converging links. (a) Basic structure of one set of the chain, formed by 7 neurones. One neurone in a set (black circle) receives 5 excitatory (white circles and solid lines) and 2 inhibitory (gray circles and dashed lines) inputs from the previous chain. The output projections of one neurone on the cells of the next set follow the same pattern. (b) The results of the simulation of the activity in a network formed by eight successive sets from time 61 to 88 in arbitrary time steps. The large dots indicate that a neurone is firing.
Empirical evidence about temporal structure
9
The events belonging to the pattern detected in panel (c) are enclosed by squares, (c) A significant pattern formed by four cells is detected. The pattern was: cell 36; 6 time units (t.u.) later cell 36 again; 8 t.u. later cell 10 and 11 t.u. later cell 46.
c36, 2 time units between c36 and c10 and 5 time units between c10 and c46 is notated as
(see Figure 2). In addition, this notation assumes that the same time jitter was found for all events forming the pattern. An apparent weak point of the “synfire” theory is the requirement for neural mechanisms able to support a precise timing of spike patterns even after large time delays, and in the presence of various neuromodulators. For instance, a prominent effect of cholinergic modulation is to reduce adaptation of spike frequency. This results in an increase of neuronal excitability and a shortening of inter-spike intervals, so that the overall effect of such modulation can be a modification of spike timing. Differential actions of acetylcholine on the excitability of two subtypes of inhibitory cortical interneurones exist, so that cortical cholinergic activation may change the direction of information flow within cortical circuits (Xiang et al., 1998). Despite this, patch-clamp experiments in rat neocortical slices have shown that timing of spikes produced in response to fluctuating current injections may be preserved during cholinergic modulation (Tang et al., 1997). Transmission across neocortical synapses depends on the frequency of presynaptic activity and studies of synaptic depression have demonstrated that different aspects of the firing patterns of their afferents are transmitted depending on the average presynaptic frequency (Abbott et al., 1997; Tsodyks and Markram, 1997). When a single sensory stimulus drives many neurones to fire at elevated rates, the spikes of these neurones may become tightly synchronized, which could be involved in “binding” together individual firing-rate feature representations into a unified object percept. However, the response to elevated rates of firing may be weak, if any, and population coding based on relative spike timing can systematically signal stimulus features following the stimulus time course even where mean firing rate does not change (deCharms and Merzenich, 1996). These properties determine the fact that little information about steady-state frequency is transferred across synapses. This result is in contradiction with the rate code hypothesis, assuming that a higher input rate corresponds to a higher output rate. In this respect, information theory applied to the coding problem in the frog and house fly (Bialek and Rieke 1992; Rieke et al., 1995) also suggests that spike timing is important for reliable information transfer from receptors to brain, and may explain the subsequent behavioural events. In summary, several lines of evidence are in favour of precise temporal activity in the brain, but the development of analytical methods to detect reliably the transient temporal relations in sequences of spike intervals therefore represents a critical step to establish their function as a temporal code. One proposed technique aimed at detecting “favoured patterns”—i.e. patterns that occur more often than can reasonably be expected at random—consists of identifying template patterns even if the detailed timing of the patterns varies slightly (Dayhoff and Gerstein, 1983a). The advantages of this method, referred to as Favoured Pattern Detection (FPD), is that it can be used to detect other favoured patterns whose occurrences may have extra or missing spikes, and the firing patterns can be tested for
Time and the brain
10
significance by Monte Carlo algorithms. Unfortunately, this technique can be applied only for analysis of temporal patterns of activity generated by a single neurone, and for fixed jitter—i.e. the fluctuation in temporal precision allowed for each spike. Moreover, the choice for settings, e.g. the threshold level parameter, may have discouraged its use among electrophysiologists, and limited its application to experimental data (Dayhoff and Gerstein, 1983b). The use of multiple microelectrode recordings, and of large artificial neural networks of spiking neurones has challenged investigators to search for methods allowing the analysis of higher complexity firing patterns, formed by several spike trains and involving several events within a pattern. An approach to the exact calculation of the probability of randomly obtaining each individual recurring pattern was proposed for three and more distinct spike trains (Frostig et al., 1984, 1990). In this method the statistical evaluation is based on the use of a 2×2 contingency tables and the application of Fisher’s exact test. At the first step, the detection stage, all possible intervals between three spike trains are tested for significance, and at a second step, the expansion stage, all significant patterns found previously are checked for higher-order associations among spike trains. The main limitations of this method consist of its inability to detect recurring patterns involving less than three different spike trains and patterns of more than three spikes, if not all subpatterns formed by a triplet of spikes are statistically significant. Several investigators have pointed out that in most experimental preparations presently available the chance of observing patterns of discharges of higher-order complexity is very low, and specific analytical methods aimed at analyzing patterns formed by three spikes (triplets) were developed. The principles of three-fold correlation among spikes are illustrated in Figure 3. Suppose that the firing times of three neurones A, B, and C are recorded simultaneously (Figure 3a). One could ask whether the firing of cells B and C depends on the time elapsed from a spike of cell A. Let us consider the time delays tAB of the discharges of cell B after neurone A fired and similarly the time delays tAC. The joint distribution histogram may be constructed as shown in Figure 3b. At each discharge of cell A the spike train of neurone B is plotted in the X-axis and the spike train of neurone C is plotted in the Y-axis of the graph. The crossings of columns and rows, falling within a defined bin size, that correspond to the spike trains of neurones B and C recorded simultaneously are counted and a Z-axis, perpendicular to the plan of drawing, is plotted. The value of the compound rate plotted on this Z-axis may be coded in shades of gray (or on a colour scale). A graphic possibility for plotting all compound joint distributions between three neurones is based on a triangular coordinate system because the three ). The X-Y graph can be variables tAB, tAC, tCB, are not independent (since skewed by 60° (Figure 3b, right panel) so that all three joint distributions can be plotted together (Figure 3c). A practical application of this method in practice is shown in Figure 3d. In this example the data were recorded from the thalamus of an anaesthetized cat during spontaneous activity (Villa, 1988, 1990). Cell 1 (discharging at a rate of 2.8 spikes/s) was recorded in the posterior group of the thalamus, cell 3 (1.7 spikes/s) in the dorsal nucleus of the medial geniculate body and cell 7 (5.7 spikes/s) in the brachium of the inferior colliculus. A black triangular bin indicates a significant event corresponding to a spike of cell 3, then 8 ms later a spike from cell 7 and a spike of cell 1 occurring 116 ms after the onset of the pattern. The time accuracy of this estimation was limited to 15
Empirical evidence about temporal structure
11
ms so that spikes of cells 7 and 1 might be considered synchronous. Detection of triplets was first achieved using the “snowflake” triple-spike renewal histogram (Perkel et al., 1975; Abeles, 1983). However, triple-spike renewal histograms do not provide a reliable estimation of the significance of the detected patterns, and to solve this problem the method was developed further into the Joint PeriStimulus Time Histogram (JPSTH) (Aertsen et al., 1989). A number of applications of this technique to
Figure 3 . Outline of the general procedure followed to study three-fold correlation among spikes, (a) Three spike trains, labelled A, B and C, are recorded simultaneously. The time of occurrence of spikes A are
Time and the brain
12
successively taken as onset time for calculation of delays of simultaneously recorded spikes of cell B (tAB) and C (tAC). Three markers indicate three successive spikes of cell A. (b) A plot of all possible patterns formed by B and C spikes as a function of the A spikes marked on the top panel is presented in form of a X-Y plot where the abscissa represents the delays of B spikes after A firing and the ordinate the delays of C spikes after A firing. The left panel shows the same plot with the Y-axis skewed by 60°. (c) A three-cell correlogram is constructed by joining three graphs in order to represent all possible time relations among the firing of three neurones. The three intervals (tAB, tAC, tCB) associated with any point on the graph can be read by projecting the point onto the three external axes, (d) The activity of three neurones, labelled 1, 3 and 7, were recorded from three different electrodes in the cat thalamus. A significant pattern is detected as shown by the black triangular bin. The pattern was: cell 3; 8 ms later cell 7 and 108 ms later cell 1. The shades of gray correspond to the number of counts per bin on a relative scale.
the study of simultaneously recorded spike trains in behaving monkeys have been reported (Abeles et al., 1993b,c). Since then, the algorithm has been improved for statistical evaluation, in order to take into account the existence of significant crosscorrelation between cells. Analyses of multiple spike trains recorded in behaving monkeys using this most recent algorithm, referred to as Joint Triplet Histogram (JTH), have been published recently (Prut et al., 1998). A careful analysis of the compound activity of three-cells may suggest interesting hypotheses about functional connections between distinct nuclei. Let us consider the result illustrated in Figure 3d. One could argue that cells in the brachium of the inferior colliculus and cells in the dorsal nucleus of the medial geniculate body receive common afferent activation from the ascending auditory pathway, and synchronous activation of these nuclei is easy to accept. However, the cross-correlation between the neurones recorded in these nuclei was flat, as indicated by the absence of dark diagonal bands centered on time zero in the snowflake plot, and the quasi-synchronous activation of cells 3 and 7 was only detected when a spike in cell 1 occurred more than 100 ms later. This delay cannot be due to direct connections through the thalamus (according to the present knowledge of thalamic organization). The observation of a complex pattern of firing in different thalamic nuclei suggests that these anatomical regions may form distributed cell assemblies that are functionally connected by some kind of reverberating activity, probably involving re-entrant activity from the cerebral cortex. In chains of diverging/converging links there is no need for each of the synaptic connections to be particularly strong, since they become effective through coactivation with others. Then, several synapses can depolarize the postsynaptic cell to reach its firing threshold, by spatial and temporal summation. In the case of pure excitatory feed-forward chains of neurones (Abeles, 1982, 1991) it has been demonstrated that a strong stimulation (e.g. input fibres firing at a high asynchronous rate) applied to all the cells of a set in the chain would elicit a spike in all of them synchronously, so that the next set of neurones down the chain would also be activated synchronously. This synchronous
Empirical evidence about temporal structure
13
activity would then propagate along the whole chain, with a limited time jitter. Thus, interconversions from firing rate code to synchronous activity, and back, can be performed by a chain of diverging/converging links. This observation is related to the origin of the “synfire” chain model (Abeles, 1982). If these chains include excitatory and inhibitory links, the analysis of the propagating activity is not so simple. In addition when one considers that distinct sets of the chain can be activated independently , then the picture of a propagating wave becomes less clear. However, these chains will tend to produce unitary events, i.e. precise spike synchronization between neurones, which occur significantly more often than expected by chance. The temporal spacing between the spikes should not be regarded as corresponding to the relative location of the parent cells along the chain. Such characteristics of the model are illustrated by Figure 4. In this example the very same simulation results as in Figure 2 are considered, but different cells were sampled by the virtual electrodes. It is important to note that cells involved in unitary events would not necessarily show signs of synchronization in crosscorrelograms, because they do not receive either common excitatory inputs or common inhibitory inputs. In Figure 4a it can be seen that, due to the presence of other inputs in some occurrences of the temporal pattern, a spike belonging to the pattern may be missing because a cell is in a refractory state. However, due to the peculiar properties of diverging/converging links the pattern is sustained by the other neurones and may persist in the network. An increasing number of studies are now investigating unitary events and preliminary experimental evidence has been reported (Riehle et al., 1997; Prut et al., 1998). It is therefore possible that unitary events in the primary motor cortex are related to pattern generation, for instance as starting points for delayed synchronous patterned activity related to the precise activation of sequences of muscles involved in motor output.
3. EMPIRICAL EXAMPLES The Pattern Detection Algorithm (PDA) developed by Abeles and Gerstein (1988) allows one to perform a comprehensive search for firing patterns and to test if there is a statistical excess of patterns. According to the complexity of the patterns to be analyzed,
Time and the brain
14
Figure 4 . Example of a unitary event detected in a simulated chain of diverging/converging links, (a) Two events corresponding to the synchronous activation of cells 1, 17, 23 and 33 have been detected, plus an event corresponding to the synchronous activation of cells 1, 17 and 33. (b) Same simulation results of Figure 2b. The large dots indicate that a neurone is firing. The events belonging to the unitary events are enclosed by squares.
two methods (“ad-hoc” and “bootstrap” algorithms tested against simulated multineurone data) are used for estimation of the expected number of repeated patterns of spikes, regardless of their complexity. Improved and computationally effective versions for estimation of the significance of patterns of spikes in this algorithm, based on probabilistic and combinatorial algorithms, were recently elaborated (Tetko and Villa, 1997a). The main restriction of PDA when compared to the other methods is that it does not allow one to establish the statistical significance of an individual pattern which repeats a number of times, but only the existence of an overall excess of precise patterns beyond expectations. This limits its main applicability to cases when pattern occurrence is very rare. Assuming a Poisson distribution of point processes, the significance of the excess of detected patterns Z, of complexity c, and repeating exactly r times in the record,
Empirical evidence about temporal structure
15
over the expected number X is estimated by pr{Z,X}=P(Z,X) where P(Z,X) represents the incomplete gamma function (Press et al., 1994). This estimation is simplified to if only one firing pattern is detected. Let us fix the level of significance to be p 0 =0.001. For example, if the number of patterns detected by PDA is Z=6, and only one pattern (i.e., X=1) is expected by chance, then a significant excess of detected patterns is observed because . However, this estimation cannot indicate which ones out of the six detected patterns are significant. In some cases PDA allows one to detect an excess of patterns when all sequences of spike intervals are significant. Consider an example where the expected number of patterns is as low as X=0.001. If one pattern is detected, its significance is equal to thus indicating that the pattern is highly significant. If in the above example we detect six patterns instead of one, then any of these six can be considered as highly significant. (In such a case each of the six pattern can be considered significant by itself.) Routine studies using the classical PDA have demonstrated that this optimal case is exceptional. Most often, patterns that could be significant by themselves under appropriate values of jitter are detected by PDA as distinct patterns if the search is performed with maximum accuracy, i.e. with time jitter equal to 1 ms. The application of PDA to several experimental models, and to recordings from various brain areas, has revealed that complex Spatiotemporal firing patterns do indeed occur (Vaadia et al., 1989; Villa and Abeles, 1990; Abeles et al., 1993a; Villa and Fuster, 1992). In these studies the algorithm was usually set to find any repeating pattern of three or more spikes, provided that the entire pattern lasted not more than 900 ms at most, and was repeated with accuracy of 1 ms. Since high frequency bursts (200 spikes/s or more) may produce patterns which relate to intracellular processes, burst filtering should be performed. A filtering procedure was originally proposed by Abeles and Gerstein (1988). A good filtering algorithm should provide a minimal loss of information, for an acceptable rate of errors in the estimation method. This would require fine tuning of this procedure that includes several combinations of three independent parameters: the time window for burst detection, the time window and the number of counts in high-frequency filtering. An alternative filtering method based only on one parameter, the filtering frequency, has been recently proposed (Tetko and Villa, 1997a). This method, with filtering frequency ranging between 200 and 400 spikes/s, provides better estimates of the number of patterns, for a comparable loss of information. A record was considered as having an excess of repeating firing patterns only when the probability of finding as much (or more) repetitions by chance was less than the significance level p 0 . Significant patterns that repeated exactly, more than four times, may be used as templates for searching approximate patterns with jitter of ±5 ms. Figure 5 shows an example of the activity of six neurones recorded in the cat thalamus during spontaneous activity (Villa, 1990; Villa and Abeles, 1990). It illustrates a pattern of 3 spikes, repeating 25 times, starting with spike 2, then after 89 ms spike 4, and then after 305 ms spike 5. On the left panel (Figure 5a) the rasters are aligned on pattern start and on the right panel (Figure 5b) the corresponding histograms (we may call peri-pattern histograms) are computed. In addition to the pattern itself, lasting hundreds of ms and formed by spikes recorded from different electrodes, this example is important because it shows an additional feature. The histogram of Figure 5b shows that two bursty neurones, cells 1 and 6, do not
Time and the brain
16
discharge in correspondence to the last spike of the triplet (cell 5 at 394 ms), but both cells show a significant increased tendency to discharge, in bursts, near 410 ms after the pattern
Figure 5 . (a) Raster display of the activities of six neurones during spontaneous activity recorded in the auditory thalamus of the cat. The rasters are aligned by displaying the first spike in the pattern at time 0. The pattern was: cell 2; 89±5 ms later cell 4, and then 305±5 ms later cell 5. The units were recorded by four microelectrodes. The first, in the thalamic reticular nucleus, picked-up units 1–2; the second, in the medial geniculate body, unit 3; the third, also in the medial geniculate body, picked-up units 4–5 and the last, in thalamic reticular nucleus, unit 6. (b) Histograms of the activities of the units triggered by the pattern onset. The abscissa full scale is 800 ms and the ordinate full scale is 85 spikes/s. Note that units 1 and 6 had a significant higher firing rate 410 ms after the pattern onset. The dashed lines correspond to the 99% confidence levels and a solid line is plotted at the average firing level. The curves are smoothed by a moving Gaussian bin shape of 10 ms.
onset. Therefore we could say that Figure 5 illustrates in fact a very complex pattern that should be noted . This is true, but no algorithms are yet available to detect such a pattern because the firing of the bursty cells is only loosely,
Empirical evidence about temporal structure
17
yet significantly, synchronized with the pattern onset. Furthermore, the selective inhibition of cells 1 and 6 at the same time as the third event of the triplet represents per se a valuable piece of information, related to the highly precise temporal processing occurring in the thalamus. Unfortunately, complex temporal firing patterns mixing spike occurrences and “missing spikes” are not detected by the usual algorithms. This is partly due to the enormous increase in computations required by taking recursively the “negative” of one spike train with all other spike trains recorded simultaneously, and partly by the fact that it is not correct to simply take the “negative”. However, there is no doubt that selective absence of spikes at specific timings would contribute to a better understanding the temporal processing in the brain, and it is reasonable to forecast that in the next few years methods will appear to deal with this problem.
4. PATTERN RECURRENCE IN RELATION TO STIMULI AND RECORDING CONDITION The next two examples illustrate some relations of firing patterns elicited by sensory stimuli, in the sense that they would not be observed during spontaneous activity (Villa 1988; Villa et al., 1991). Recordings were made in the auditory thalamus of the cat. Figure 6 shows a triplet, occurring selectively during the presentation of a stimulus which was a white noise burst lasting 200 ms, delivered to both ears. The firing pattern always started after the stimulus onset, but it always ended before the stimulus offset. However, the pattern was not time locked with the stimulus onset, as clearly shown by the irregular alignment of the stimulus onset and the start of the pattern. Similarly, a pattern could be elicited by a stimulus, but occurring after the stimulus offset and without a tight locking to it. Such an example is illustrated by the triplet in Figure 7, repeating 12 times, starting by spike 3, then after 285 ms spike 1 and then after 189 ms spike 2. Note that spikes 1 and 2 were recorded from the same electrode, thus indicating that these cells lay within a distance of few tens of microns of each other. Nevertheless the delay between these correlated discharges was near to 200 ms, and suggested long reentrant loops from the cortex to the thalamic reticular nucleus or, alternatively, from the substantia nigra and/or from modulatory mesencephalic nuclei, that are known to project to this thalamic region. In order to establish the significance of firing patterns as signposts of distributed network activity related to specific brain processes, it is crucial to demonstrate that whenever the network is working in the same state, the same temporal pattern of firing is observed. Several above-mentioned examples have suggested that cortical activity may affect the recurrent activity in the thalamus, as shown by several corticofugal studies, either experimental (Villa et al., 1991; Payne et al., 1996; Villa et al., 1999b) or theoretical (Tetko and Villa, 1997b). The next examples report data recorded in the auditory thalamus of the rat during reversible cortical deactivation by cooling (Villa et al., 1999b). This type of study illustrates an additional problem that may arise in studies of temporal coding. The number of spikes necessary for a fair statistical evaluation of the
Time and the brain
18
Figure 6 . Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.0 mm. The lowermost raster shows the onset of the stimulus, as a thick tick, and the gray bars represent the time of stimulus duration. The rasters are aligned by displaying the first spike in the pattern <2,3,1; 49,83> at time 0. The pattern is repeated 19 times with a jitter of ±3 ms. The abscissa full scale is 600 ms.
significance of the patterns may not be met in all data-sets because the experimental protocol, in this case the cortical cooling, may produce a massive decrease in firing rates of selected single units, or alternatively a massive increase of discharges, and a major change in the pattern of discharges (i.e. from regular to bursty). In case of a decrease in firing rate a partial methodological compensation might be obtained by increasing the recording time. Figure 8 shows an example of a pattern formed by three spikes belonging to two single units recorded simultaneously from two different electrodes in the rat auditory thalamus. The pattern is formed by cell 7 firing, then after 350 ms by a discharge of cell 11 and then after 21 ms cell 11 firing again. This exact pattern was used as template for searching approximate matching patterns with a jitter of ±2ms. Note that several spikes may occur in between the spikes belonging to the pattern without affecting the precise intervals separating the occurrences of the cell discharges. In this example, the significant pattern was observed 9 times in 400 s of spontaneous activity recorded during the control condition. No patterns were observed in 300 s of recording during cooling, but the very
Empirical evidence about temporal structure
19
same precise pattern was observed 5 times in 300 s during the recovery period. Note that between the first and the last occurrences of the pattern more than 90 minutes had passed. Another example (Figure 9) illustrates that significant patterns observed during cortical cooling might be completely absent prior to and after cortical deactivation. This case
Figure 7 . Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.4 mm. The rasters are aligned by displaying the first spike in the pattern <3,1,2; 285,474> at time 0. The pattern is repeated 12 times with a jitter of ±2 ms. The abscissa full scale is 1000 ms.
shows a pattern of 3 spikes repeating 9 times (with a jitter of ±2ms) starting by a spike of cell 8, followed by a spike of cell 5 180 ms later, then after 280 ms by a spike of cell 6. Note that only cells 5 and 6 were recorded from the same electrode. This second example also illustrates the tendency observed in firing patterns during cortical cooling to exhibit a higher complexity than patterns observed during control and recovery conditions (Villa and Abeles, 1990).
5. PATTERNS IN RELATION TO BEHAVIOURAL STATES Although the previous examples have provided solid evidence for the existence of precise temporal structure in neural activity, this does not yet establish its function as a temporal code. What is needed is some demonstration that Spatiotemporal firing patterns occur
Time and the brain
20
reliably under particular behavioural conditions, and are related to cognitive activity. The following example shows experimental results recorded in the primate inferotemporal cortex, which is considered one of the last in an ascending hierarchy of cortical processing stages that begins in the striate cortex. Studies with behaving monkeys have led to the conclusion that the inferotemporal cortex is not only a higher order stimulusanalyzer, but is also involved in retention of visual information, and is modulated by attention (Fuster, 1990). In the data presented here, the monkey performed a visual delayed matching-to-sample task (Villa and Fuster, 1992). A trial in one such task consisted of the following:
Figure 8 . Spatiotemporal pattern of spikes detected in the rat auditory thalamus 9 times prior to and 5 times after recovery from cooling of the auditory cortex, but absent during cortical deactivation. The firing pattern was formed by three spikes, starting by spike 7, then after 359 ms spike 11 and then after 21 ms spike 11 again. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated by thick ticks in the raster displays (right panels) aligned by displaying the first spike in the pattern at time 0. Note that spikes 7 and 11 were recorded from different electrodes. The analyzed records corresponded to 400 s of spontaneous activity
Empirical evidence about temporal structure
21
during control condition, 300 s during cortical cooling and 300 s in the recovery condition. Each block of 100 s is represented by a thick mark along the time axis of the experimental protocol, and the dots indicate the start event of the pattern.
(1) sample stimulus (a colour or a geometric figure on a luminescent button with a diameter of 25 mm); (2) 10–20 seconds of delay (retention period); and (3) choice of one stimulus (among 2 or 4) that matched the sample by a given feature (colour or symbol). Correct responses were reinforced with a squirt of juice. About 40% of all single units recorded in the inferotemporal cortex showed a sustained elevation of firing during the delay after the sample stimulus (which was retained in short-term memory for performance of the task). This sample was further subdivided into selective units (14% of all units) showing an activation during the delay which was selectively and significantly higher after one particular tested stimulus, and non-selective units (26%) which were unspecifically activated by all tested stimuli. The distribution of firing rates for both
Figure 9 . Triplet of spikes detected 9 times in the rat auditory thalamus, exclusively during cortical deactivation, starting by spike 8, then after 180 ms spike 5 and then after 280 ms spike 6. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated as in Figure 8. Note that only spikes 5 and 6
Time and the brain
22
were recorded from the same electrode. The analyzed records corresponded to 300 s of spontaneous activity during control condition, 500 s during cortical cooling and 300 s in the recovery condition.
groups in spontaneous discharge was skewed and broad, but the median firing rate for non-selective units (4.4 spikes/s) was higher than that for selective units (2.5 spikes/s). In this study, only one spike train at a time was recorded and thus the temporal firing patterns were formed by multiple spikes of one neurone. Figure 10 illustrates two patterns formed by three spikes observed in two different neurones. The triplet of cell 14 repeated 6 times during the intertrial periods, which did not reach statistical significance, whereas its repetitions were significant (19 times out of 68 trials) during the delay period. The accuracy of the pattern was ±3 ms. The triplet of cell 47 repeated 7 and 19 times during the intertrial and delay periods, respectively, out of 75 trials. One pattern involved long intervals, of the order of hundreds of ms, whereas the other pattern involved relatively short intervals. However, the striking similarity between these two patterns is that both patterns appeared to a significant extent during the retention period of the sample stimulus, and that only one occurrence of the pattern (seldom two) was observed per trial (although in only 20–25% of all trials).
Figure 10 . Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of two non-selective units recorded in the
Empirical evidence about temporal structure
23
primate inferotemporal cortex during intertrial intervals (left panels) and during the delay (right panels). The measuring time was equal in both conditions. Cell 14 shows a pattern of 3 spikes repeating 6 and 19 times during the intertrial and delay periods, respectively: The second spike of the triplet occurred 165 ms after the pattern start and the third 195 ms after the second spike. Cell 47 shows a triplet repeating 7 and 19 times during the intertrial and delay periods, respectively. The first spike of cell 47 occurred at time 0 (aligning the rasters), the second 42 ms later, and the third 48 ms later. In these rasters the pattern is allowed an accuracy of ±3 ms.
In spike trains recorded in the inferotemporal cortex it was difficult to observe firing patterns composed of 4–6 spikes recurring more than twice within a record. Figure 11 shows an example of such a complex pattern formed by spikes of a selective unit during intertrial periods, and during the delay after the preferred stimulus (the only periods characterized by an increase in firing rate). The pattern was composed of 4 spikes: a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. This quadruplet repeated 6 times during the intertrial periods. The four subpatterns corresponding to the triplets included in the original quadruplet are also displayed in Figure 11. The incidence of the highly organized pattern decreased when the cell was activated by its preferred stimulus, irrespective of an increase in the rate of discharges during the delay. However, it is important to note that one specific subpattern, i.e. was detected at a statistically-significant level during the intertrial and delay periods. This triplet occurred 5 and 9 times during the intertrial and delay periods, respectively. The example of Figure 11 is important, because it illustrates several additional characteristics of precisely time-structured activity in the cortex. Firstly, it illustrates the contribution of subpatterns to a higher order pattern, thus suggesting the robustness of a complex pattern even if at some occurrences a spike is missing. Secondly, it illustrates that subpatterns may express information processing different from that involving the superpattern (as shown by the disappearance of the event at a delay of 95 ms in Figure 11, and at the same time a relative increase in occurrences of the triplet ). On a more general level, the examples of Figures 10–11 raise an interesting problem. Notably, the incidence of firing patterns tended to be inversely related to the selectivity of the units for the stimulus that the animal held in short-term memory (Villa and Fuster, 1992). During the delay periods, and in comparison with intertrial periods, selective units showed a decrease of firing patterns, whereas non-selective units showed an increase. On the one hand, the more patterned activity of non-selective units may reflect their involvement in wide networks representing multiple and general attributes of that particular stimulus held in memory. On the other hand the term “selective unit” shows the long-held belief and
Time and the brain
24
Figure 11 . Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of a selective unit (cell 61) recorded in the primate inferotemporal cortex during intertrial intervals (25 lines) and during the delay (17 lines). The pattern was composed of 4 spikes, a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. The occurrences of the exact subpatterns formed by 3 out of 4 spikes of the original pattern are also displayed on this raster. The quadruplet is allowed an accuracy of ±3 ms, whereas the subpatterns ±1 ms. Note that during the delay the average firing rate of the cell increases, while the quadruplet pattern disappears, although one particular subpattern remains visible.
prejudicial influence of the rate code on our assumption of what is “selective”. In fact, selective units were defined on the basis of an increased firing rate during the delay period that followed a particular stimulus of the tested set. It is not possible to rule out the possibility that in another paradigm or experimental condition the “selective” units would become “non-selective” and vice versa. Thus, one should bear in mind that the term “selective” should be restricted to a particular experimental situation. However, it is interesting to note that both populations of cells increased their rate of discharges during the retention period, but only the non-selective units increased the patterned activity. An attractive interpretation of these data could be the following: to retain all stimulus characteristics, some transfer of information occurs between the selective units, which encode the specific stimulus feature being retained, and the non-selective units, which are part of a wider network encoding contextual attributes of the stimulus that may be used for binding with other features. This model supports the hypothesis of conversion from asynchronous activity to synchronous activity (and back) through diverging/converging
Empirical evidence about temporal structure
25
connections, as proposed by Abeles (1982, 1991). One important limitation of the PDA method is its application to patterns significant by themselves, as explained above. An additional limitation is the use of a fixed jitter. A new method, called Pattern Grouping Algorithm (PGA), has recently been developed, aiming to identify and evaluate the statistical significance of temporal patterns of spikes formed by three or more different events (Villa and Tetko, 1999; Tetko and Villa, 1999). In particular, this new algorithm is able to recognize patterns of spikes with slight differences in spike timing, and to cluster them as one single group of patterns. Firstly, PGA considers those patterns detected by the classic pattern detection algorithm as templates for search of similar patterns in the spike train data. There are three adjustable parameters in PGA: (i) the maximal duration of the pattern, measured as a delay between the first and the last spike in the sequence of spikes (i.e. the window duration); (ii) the level of significance to be used for detection of significant groups; and (iii) the maximum allowed jitter for timing accuracy of time delays in a pattern group. Based on the idea developed in the template search method (Dayhoff and Gerstein, 1983a) and on selection of optimal jitter, PGA is able to minimize the jitters allowed for each spike in the template independently, according to the actual distribution of spikes. Thus, the probability of detection of significant patterns of spikes is optimized. PGA is not restricted to the analysis of patterns formed by only one cell, and may be applied to the identification of precise temporal patterns regardless of their complexity and number of different cells in a pattern. Therefore, PGA can estimate the significance of each spatiotemporal pattern of firing. The method (described in detail in Tetko and Villa, 1999) consists of an independent estimation by the previously developed PDA, FPD and JTH methods, that have been updated in order to consider the variable jitters. Only those patterns that are significant by all methods are considered for the final step of PGA. Eventually, the final output of PGA consists of a central template pattern with variable jitters corresponding to previously detected patterns grouped into one class. Several tests and validation analyses have demonstrated that PGA can detect true patterns and avoid the detection of spurious patterns, even in the presence of several non-stationarities in the spike trains. These non-stationarities are likely to occur in recordings from behaving and freely-moving animals. Even so, it must be kept in mind that for most of the functional systems the available data demonstrate the presence of precisely timed neural activity whose relationship with behaviour is not always evident. To prove that these temporal patterns do indeed serve for response selection, data are required that establish causal relations between the occurrence of precise temporal relationships and cognitive or motor processes. The next examples will provide solid evidence about these processes. Spike trains from up to 15 single neurones were recorded simultaneously in the temporal cortex of freely moving rats, while animals waited for acoustic cues in a Go/NoGo task (Villa et al., 1999c). The auditory stimulus contained two types of information: pitch (“high” or “low”), and position (“left” or “right”). During the first phase of training, the location was kept constant and the rats had to discriminate between tones of high and low pitch, with one signalling “Go” and the other “NoGo”. Reinforcement was given only after correct “Go” trials. In the second phase, pairs of tones were delivered (one tone from each location), with four possible tone-position combinations. We have shown that, despite the lack of reward for correct NoGo
Time and the brain
26
performance, and the lack of painful punishment for incorrect trials, all rats learn the task to a satisfactory level (performance in the range 70–90%). The paradigm allows analysis of activity during two functional epochs: the “wait” period prior to the stimulus, during which the animal must remain at the back of the cage, and restrain responding despite an increasing probability of imminent stimulus delivery; and the “processing” time, from stimulus delivery until beginning of movement, during which the information content of the signal is processed and motor output organized. Only the “wait” period is considered here for analysis. Significant patterns were detected using PGA from a data-set of 13 hours of recording, involving over one million spikes (Villa et al., 1998a, 1999a). “Go” responses resulted both from correct movements to the feeder in response to low pitch sound at the right speaker and from incorrect movements when a high pitch was delivered to this speaker. Conversely, “NoGo” responses could be correct (failure to move in response to the high pitch from the right speaker) or incorrect (failure to move in response to low pitch from right speaker). Of particular interest were the patterns that were significantly associated with the type of response the animal made later, independent of whether the response was that prompted by the cue. Figure 12 illustrates one such pattern, a triplet formed by the spikes of two cells recorded from distinct electrodes placed in the temporal cortex within the same hemisphere. Note that there were more than twice as many trials (n=27) with patterns that were followed by “Go” responses as there were trials (n=12) with patterns followed by “NoGo” responses, despite the fact that the total number of “Go” and “NoGo” responses was nearly the same (n=283 and 269, respectively). An alternative display of the spikes belonging to the triplets is shown in Figure 12b. In this illustration, the rasters are aligned to an external event (the stimulus onset) instead of being aligned to the start of the pattern. Of these “behaviour-predicting” patterns, half were associated with an enhanced tendency to “Go” in response to the stimulus, and for about 20% of these patterns, trials including the pattern were associated with a faster or slower reaction time than those lacking the pattern (Villa et al., 1998a, 1999a). Figure 13 illustrates one such pattern, and the corresponding reaction times. The triplet shown here is formed by three different neurones, two of which were recorded from the same electrode. It is important to notice that the patterns were associated in almost half (22/37) of the “Go” responses and that pattern occurrence during the wait period led to an accelerated reaction time, by 355 ms on average. Although the patterns could start at any time during the waiting period, we
Empirical evidence about temporal structure
27
Figure 12 . (a) Raster display of the activities of two neurones recorded from two different electrodes in the temporal cortex of a freely-moving behaving rat. The rasters are aligned by displaying the first spike in the pattern at time 0. The pattern was: cell 5; 320±3 ms later cell 11, then 342±3 ms later cell 5 again. The data shown include trials recorded on two consecutive days and in which both Go (n=27) and NoGo (n=12) responses occurred, (b) Raster display of the data corresponding only to the Go responses shown in (a) but on a compressed time scale, and aligned to the time of stimulus onset instead of to time of the first spike of the pattern. Spikes involved in generating instances of the pattern during the waiting period (time to the left of the stimulus onset) are displayed as bars instead of ticks. Three spikes constituting one instance of the pattern are picked out by empty circles. Note the stability of this pattern recorded over two consecutive days.
Time and the brain
28
observed that in several cases the time course of pattern appearance was related to the reaction time. Figure 14 illustrates this finding on a cumulative display of reaction time vs. timing of pattern occurrence. The reaction times of the subjects were normalized to 800 ms on average, in order to cumulate eleven examples on the same curves. Notice that the individual timing of each pattern was by itself significantly correlated with the reaction time on a quadratic regression analysis . The left
Figure 13 . Raster display of the activities of three rat cortical neurones participating in a spatiotemporal firing pattern, aligned by the occurrence of the first spike in the pattern. The time of stimulus presentation is marked by horizontal ticks in the lower window. The pattern was formed by spike #12, 867±2 ms later cell #11, and then after 29±1 ms cell #5. The timing of the three spikes forming the pattern is indicated by the arrows below the stimulus raster. Note that this pattern repeated 22 times (p<0.001) before the onset (i.e. during the waiting period) of a stimulus which triggered a Go response (either correct or incorrect). On average, the subject responded significantly faster after stimulus onset if a pattern was detected in the waiting period.
panel of Figure 14 shows that the earlier the pattern occurred in the waiting period, the slower the reaction time. A pattern occurring just before the stimulus presentation elicited a fast reaction time, thus suggesting a kind of matching operation. The right panel of Figure 14 illustrates the opposite case, where the occurrence of the pattern just before the stimulus provoked a slower reaction.
Empirical evidence about temporal structure
29
Figure 14 . Reaction time, measured as the delay between stimulus onset and the first infra-red beam crossing, vs. the timing of patterns before the stimulus onset. The timing occurrence of five different patterns (labelled 1–5) are cumulated on the left panel, showing a faster reaction time if the pattern occurred just before the stimulus. The right panel shows six other patterns (A-F) cumulated on the same curve and showing a slower reaction time when the pattern occurred just before the stimulus. For all patterns the reaction time of the subjects was normalized to 800 ms on average in order to cumulate the data.
The remaining behaviour-predicting patterns were associated with an enhanced “NoGo” tendency. Overall mean rates of activity did not vary significantly across trials, and the patterns could not be related to the prompting auditory cues, since these occurred later and were chosen randomly. Hence, these data demonstrate for the first time that particular spatiotemporal patterns predict future behavioural responses. The information coded by means of precise spike timings has predictive value, and can therefore allow forecasts of the animal’s future response, regardless of whether or not these responses are ones demanded by subsequent unpredictable cues. The impact of the presence of specific spatiotemporal patterns on the responses of the animal suggests that such patterns participate in set-related activities that prime a particular response, and thus bias the subject’s response choice, regardless of appropriateness. Alternative, and not mutually exclusive hypotheses, can be proposed, whereas the underlying dynamics can be only a matter of conjecture. Firstly, patterns may be considered as parts of “templates” generated by preparatory processes activated in order to extract the useful information from forthcoming sensory cues (auditory “objects” in cognitive contexts requiring difficult discriminations). Templates would be activated selectively prior to the stimulus—the “Go” or the “NoGo” sound—and allow faster match-mismatch recognition; consequently, they are comparable to the “preconceived ideas” that influence human perception. “Go patterns”, for instance, would reflect states wherein signals are more likely to be interpreted as “Go” prompts. Such fine structures of brain activity, coined in more precise terms than overall mean rates, could be related to thalamo-cortical information processing. Several lines of evidence indicate that the auditory cortex exerts
Time and the brain
30
a dynamic control over the functional segregation of signals transmitted via the thalamocortical loops, and activity in such loops may generate precisely repeating interspike intervals (Villa et al., 1991; McCormick and Bal, 1994; Miller, 1996; Tetko and Villa, 1997b).
6. PATTERNS IN RELATION TO MOTOR PROGRAMMING The next examples investigate a second hypothesis which cannot be ruled out, that patterns may be parts of motor programs, each pre-prepared for active performance of either a “Go” or a “NoGo” response. The activity of multiple neurones, recorded simultaneously from the monkey primary motor cortex during the performance of a delayed-pointing task, was analyzed by PDA and PGA (Villa et al., 1995, 1998b). Each trial was initiated by the monkey pressing a lever. Following a fixed interval (predelay period), a preparatory signal and then, after a second delay, an imperative (response) signal were presented (Figure 15a). This protocol was designed to manipulate the temporal aspects of preparation for visually guided pointing movements by varying the interval between these two signals. On any one trial, this interval (the preparatory period) could take one of 4 values (600, 900, 1200, or 1550 msec). Thus, during trials with the longest delay, there were three earlier times at which a response signal could have occurred, and which might have been expected by the animal. These moments are referred to as expected signal times. Spatiotemporal firing patterns were indeed detected preferentially in relation to the expectancy of behaviourally relevant events. The timing of several of these spatiotemporal patterns was found to be related to the preparatory signal. Figure 15b shows such an example, in which a pattern of three spikes occurred 20 times, started by neurone 4, then after 113±1.0 ms neurone 6 and then after
Empirical evidence about temporal structure
31
Figure 15 . (a) Diagram of the time course of a trial in the delayed-pointing task. Each trial was initiated by the monkey pressing a lever (“start”). Following a fixed interval (pre-delay period), a preparatory signal (PS) and then, after a second delay, an imperative (response) signal (RS) were presented. The preparatory periods were randomly selected out of four possible fixed durations (600,900, 1200, and 1500 ms). For the longest preparatory period, three expected signal delays (ESI at 600 ms, ES2 at 900 ms, and ES3 at 1200 ms) are identified, (b) Raster displays of spiking activities of a group of four neurones recorded in the primate motor cortex. In the last row of the panel, the markers indicate the time of occurrence of the corresponding events. The rasters are aligned by the first spike in the pattern. The scale bar corresponds to 500 ms. The pattern was: cell 4; 113±1.0 ms later cell 6, and then 294±2.0 ms later cell 6 again. In
Time and the brain
32
three trials this pattern occurred twice. These trials can be recognized by the labels 1 and 2 near the corresponding PS. Note that the majority of patterns precede the onset of the preparatory signal. PS: preparatory signal; RS: response signal; ES: expected occurrence of RS.
294±2.0 ms neurone 6 again. In 15 out of 20 occurrences this firing pattern ended a few hundreds of ms before the onset of the preparatory signal. This result suggests that neurones in the primary motor cortex can undergo temporally patterned activity in relation to the expectancy of signal occurrence.
Empirical evidence about temporal structure
33
Figure 16 . Raster displays of spiking activities a group of four neurones (different from Figure 15) recorded in the primate motor cortex. The rasters are aligned by the last spike in the pattern. The pattern was: cell 4 and cell 5 firing almost synchronously (within ± 1.0 ms); 55±3.5 ms later cell 5, and 223±4.5 ms later cell 4. All types of trials were pooled and the patterns were equally distributed among expected signal times (ES1: n=4; ES2: n=3) and response signals (RS1: n=3; RS2: n=2; RS3: n=1; RS4: n=1). Note that when the pattern was preceding an expected signal (labeled by empty solids) the time of occurrence of the corresponding response signal is also marked. The response signals are labeled by black dots. The question mark indicates the brain process associated with the expected signal.
Of particular interest are several patterns that occurred preferentially prior to the preparatory signal and/or expected signal times. A complex pattern formed by four spikes was observed fourteen times (each time in a different trial) and always ended over a
Time and the brain
34
narrow interval (20-230 ms) before the actual or expected signal time (Figure 16). For example, in the case of the longest preparatory period (i.e. 1500 ms) a response signal could have been presented at any of four moments. Since trials were presented at random, the animal could not know in advance the duration of PP. The timing of the pattern shown in Figure 16, which predicts with reasonable accuracy the times of possible responses signals, suggests that the occurrences of the pattern correspond to some brain process associated with expectancy of upcoming signals. Furthermore, patterned activity might be initiated by synchronous volleys of spike discharges, as shown by the two quasi-synchronous events shown in Figure 16, and ended before the occurrence of the response signal. This pattern is interesting also because it exemplifies a typical result of PGA, since the optimal time jitters were all unequal. An alternative display to show the relationship between the preparatory period and the pattern occurrence is based on a raster plot of the spike trains triggered by onset of the PS. In addition, this raster should be restricted to those trials characterized by an occurrence of a specific pattern during the preparatory period. Figure 17a shows an example of a brief pattern (lasting less than 50 ms) formed by three spikes from two different neurones: the first two spikes are synchronous, and the third occurs 48 ms later. The histogram of the 28 occurrences of the end-times of the pattern (Figure 17b) demonstrates that this distribution is not uniform. Half of the patterns (14/28) ended within 100 ms before the actual or expected end of the preparatory period. Both neurones involved in the pattern showed only a slight modulation in firing rate during the preparatory period compared to the predelay period, as shown in the peri-stimulus time histograms for the longest preparatory periods (Figure 17c). However, the most interesting feature of this pattern is its predictive value for the subsequent behaviour. The total response time, i.e. the interval between signal onset and touching the target, was 512±12 ms (mean and SEM) in the 22 trials containing a spiking pattern, whereas in the 101 trials without pattern, response time was 569±13 ms. In this particular example, reaction time, i.e. the interval between signal onset and the movement onset, was significantly faster as well (248±9 vs. 297±9 ms).
7. SUMMARY OF EMPIRICAL EXAMPLES In summary, experimental results of the coordinated timing of spikes, and the synchronous firing or network synchronization have provided several conceptual frameworks in which temporal coding in spiking activity is important. Temporal codes do not preclude a rate code being superimposed on it simultaneously. The presence of sensory stimuli or motor output often correlates with single-unit firing rates averaged over many trials and/or with asynchronous compound activity of a neuronal population (Riehle and Requin, 1995; Georgopoulos, 1995). In addition, it has been well established that hippocampal place cells signal by their rates whether the animal is within the place field, independently of whether the animal is entering or leaving that field (Wilson and McNaughton, 1993). However, as such studies increase in complexity, a simple rate code may be rendered inadequate as a predictor of behaviour. In this section a number of
Empirical evidence about temporal structure
35
bodies of experimental evidence for the existence of precise neuronal timing has been provided. Spatiotemporal patterns do in fact occur spontaneously, but occur recurrently, and in a more stable way, in particular temporal relationships to stimuli, or movement, and during the retention period of a delayed response task; and under particular experimental conditions “behaviour-predicting” patterns have been detected. It has been proposed (Ferster and Spruston, 1995) that clear proof for a
Figure 17 . (a) Raster display of the spiking activities of two neurones recorded simultaneously in the primate motor cortex. The spike trains are aligned on the onset of the preparatory signal (PS). Only trials characterized by the presence of a pattern are displayed. The pattern was: cell 1 and cell 6 firing almost synchronously (within ±1.0 ms) and 48±5.0 ms later cell 1 again. In one trial the three spikes participating in the pattern are indicated by circles. In the second row (cell 1) thick ticks indicate the last spiking event of each pattern. In
Time and the brain
36
the third row the ticks indicate the response stimuli (RS) associated with each pattern (i.e., RS1, RS2, RS3, and RS4 after 600, 900, 1200, and 1500 ms, respectively). Note that pattern occurrences were distributed in 22 trials, i.e. in few cases one pattern occurred in relation to the expected stimulus (ES) and one in relation to the response stimulus in the same trial, (b) Histogram of the end-times of pattern occurrences. The bin size is 100 ms. Note that the last spike of this pattern preceded either an expected stimulus (12 times, empty bins) or RS (16 times, filled bins), (c) Peri-stimulus time histograms of the activity of cells 1 and 6 for 31 trials characterized by preparatory periods lasting 1500 ms (PP4). The abscissa is time (ms) and the ordinate is scaled in rate units (spikes/s). The vertical full scale for cell 1 and cell 6 is 27 and 45 spikes/s, respectively. The curve is convoluted by a Gaussian shape bin of 22 ms.
temporal code would require that distinct stimuli could reliably produce different temporal spike patterns. However, this restrictive view, to assimilate the temporal code to a kind of “Morse alphabet”, cannot account for the complex dynamics existing in the brain described in the next section.
8. CHAOS AND ATTRACTORS An alternative way to investigate neural dynamics in spike trains assumes that the whole time series of spike occurrences is an expression of some fundamental process governing the activity of the neurones being recorded. When a specific input pattern activates a cell assembly, it is assumed that the neurones will be activated either in a timely organized mode or in an asynchronous mode. Then, a mode of activity defines how information is processed within a neural network and how it is associated with the output pattern of activity that is generated. This assumption does not exclude the possibility that the same cell would respond in an opposite mode to other stimuli. Furthermore, the same neural network may subserve several modes of activity through modulation of its connectivity (for instance during learning or pathological processes) or by modulation of its excitability, e.g. by modulation of the resting potential or of the synaptic time constants. In this framework the state of the neural network is defined by a set of parameters characterizing the neural network at a certain time. Then, the state of the network at any given time is represented by the values of these parameters, and a network state is fully determined if all parameters are known for each neurone. A state, in general, changes over time. Neurones are embedded in an inextricably large neural networks whose coordinated activities determine the timing of action potentials. Ab absurdo if we were able to set the same initial conditions for all elements of the neural networks we would obtain the same spike trains. Such a dynamical system as a whole is said to be deterministic, as it is possible to predict precisely the evolution of the system in time if one knows exactly the initial conditions. However, a slight change or incorrect measurement in the initial conditions results in a seemingly unpredictable evolution of
Empirical evidence about temporal structure
37
the system. Such sensitive dependence on initial conditions is a hallmark of chaotic systems. A process is defined by the passage of a state over time. The simplest definition of a chaotic process, or simply chaos, is that it is completely deterministic at each step of its temporal evolution, yet over the long term, its response is unpredictable. An equivalent definition of a process is a “path” over time, or trajectory, in the state space. The attractor is defined by the set of points approached by the trajectory as time increases to infinity. If the evolution in time of the system is described by a trajectory forming a closed loop (also referred to as a periodic orbit), then the system is said to have a limit cycle, also referred to as a periodic attractor. In general, brief initial perturbations applied to any combination of the governing set of parameters move such a system away from the closed loop, but, with passage of time, the trajectory collapses asymptotically to the same attractor. If, on the other hand, the system is deterministic, yet sensitive to small initial perturbations, the trajectory defining its dynamics is an aperiodic orbit, then the system is said to have a chaotic attractor, often referred to as a strange attractor.
9. DYNAMICAL SYSTEM ANALYSIS Let us consider a simple example of a point process, whose dynamics is characterized by discrete steps in time. Let be a time series with K points, where x represents the state of the system. In a dynamical system the subsequent state of the system is determined by its present state (Addison, 1997). The simplest expression would be to consider a map defined by where α is a control parameter. The biological systems, and the brain in particular, are often characterized by feedback mechanisms. The expression known as the logistic map, illustrates a simple dynamical system with a negative non-linear feedback, defined for x [0,l]. It is clear from this expression that the time arrow is non-reversible, because it is always possible for each x i to obtain a value x i+1 but there are two possible x i for each x i+1 . The behaviour of this equation is very interesting. For all values 0≤α≤1 the iterated series decay towards zero. Conversely, for some other values of α , e.g. 1.7 or 2.1, the series converges to a fixed point, equal to 0.52381 and 0.41176 respectively. Conversely, for values α equal to 3.2 and 3.52, the series converge to two alternating fixed points, i.e. and four alternating fixed points, i.e. respectively. The trajectories of the three last examples are periodic orbits, with periods equal to 1,2, and 4, respectively. However, it is possible to find other intersesting cases. With a control parameter equal to 4.0 and an initial condition x 0=0.6 the system tends to decay to zero, but with an initial condition x 0=0.4 the dynamics never produces a repeating sequence of states. This aperiodic behaviour is different from randomness, or white noise, because an iterated value x i can occur only once in the series, otherwise, due to the deterministic dynamics of the system, the next value must be also a repetition and so on for all subsequent values. In formal terms, let us define a function ƒ which describes how a state x is mapped into the space of state. Such a function is the probability function f of the mode of activity of
Time and the brain
38
the network. If the function is set by a control parameter µ we can write . A dynamical system x’ is a subset of the space of state and can be obtained by taking the gradient of the probability function with respect to the state variable, that is . Mathematically speaking, the space of state is a finite dimensional smooth manifold, assuming that ƒ is continuously differentiable and the system has a finite number of degrees of freedom (Smale, 1967). For periodic activity the set of all possible perturbations define the inset of the attractor or its basin of attraction. In case of the logistic map, for a=3.2 all initial conditions in the interval end up approaching the period 2 attractor. This interval is known as the basin of attraction for the period 2 attractor, whereas the value x 0=0.6875 is a fixed point. If the activity is generated by chaotic attractors, whose trajectories are not represented by a limit set either before or after the perturbations, the attracting set may be viewed through the geometry of the topological manifold in which the trajectories mix. The function ƒ corresponding to the logistic map, is the parabolic curve containing all the possible solutions for x . This function belongs to the single humped map functions which are smooth curves with single maxima (in here the single maximum is at x =0.5). It is easy to conceive that in large neural networks the complexity of the system is such that several attractors may appear, moving in space and time across different areas of the network. Such complex Spatiotemporal activity may be viewed more generally as an attracting state, instead of simply an attractor. In particular, simulation studies demonstrated that a neural circuit activated by the same initial pattern tends to stabilize into a temporally-organized mode or into an asynchronous mode if the excitability of the circuit elements is adjusted to the first order kinetics of the postsynaptic potentials (Villa and Tetko, 1995; Hill and Villa, 1997).
10. DETECTION OF CHAOS IN EXPERIMENTAL DATA SETS It is obvious that it is not possible to know all variables determining brain dynamics, yet the progress made by computational and statistical physics during the two past decades has brought a number of methods allowing one to differentiate between random (i.e. unpredictable) and chaotic (i.e. seemingly unpredictable) time series. It is difficult to prove that biological systems generate chaotic attractors, owing primarily to their shortlived and apparently nonstationary behaviour (Mpitsos, 1989). Several lines of experimental observation suggest that chaotic processes may be generated by brain activity (Babloyantz et al., 1985; Rapp et al., 1985). In particular, chaotic discrete dynamics observed in spike trains allow simple networks of neurones to perform complicated tasks that would require considerably more complex networks to perform if the signals were generated by non-chaotic discrete or continuous processes (Mpitsos and Burton, 1992). Among several methods available for classification of discrete time series the Grassberger-Procaccia algorithm (Grassberger and Procaccia, 1983) has been widely applied to theoretical and experimental cases. This algorithm is based on the following
Empirical evidence about temporal structure
39
idea. Let { x 1 ......x k } be a time series corresponding to interspike interval sequences generated by K +1 number of spikes (Cutler and Kaplan, 1997). Delay coordinates { y 1 ....y n } in a d -dimensional embedding space are defined by setting y 1 = ( x 1 .....x d ), y and Y≡ { y 1 ....y N } is the 2 = { x 2 ....x d+1),....y N ={ x N ....x K ), where set of delay vectors. One fixes an embedsding space with dimension d and computes the values of the so-called correlation integrals, corresponding to the average of the number of points falling inside an hypersphere (with arbitrary small radius) around each point of the original time series. These quantities are plotted against the radius for several values d of the dimension of the embedding space (Figure 18). When the curves are definitely parallel, the system is said to be deterministic and their slope defines the correlation dimension, often referred to as D 2 , of the attractor underlying the chaotic structure of the system (Figure 18b). The value of d at which the convergence starts is the embedding dimension. In practical applications, the correlation dimension D 2 must be calculated in a meaningful range of values of the radius. This interval, denoted as the scaling region, must have an amplitude of at least a decade (Eckmann and Ruelle, 1992). Moreover, the number K of points defining the time series should be significantly large in order to get reliable estimates of the correlation dimension. In particular, for typical experimental time series with 800≤ K≤10,000 points the embedding spaces should have a realistic dimension d≤6. Higher dimensional systems cannot be reliably detected over relatively
Figure 18 . Correlation integrals of two spike trains, (a) No attractor is detected in the embedding space of dimensions d up to d=8 for this spike train (n=1094 spikes), (b) This spike train (n=876 spikes) is characterized by a correlation dimension D 2=2.1 embedded in a 4-dimensional space. The thick black bars indicate the scaling region. Note that for time series with less than 900 points the computation of the correlation integrals is stopped at an embedding space of dimension d=7.
short time series and their dynamics cannot be distinguished from random noise Rapp et al., 1993). Furthermore, the embedding dimension of the interspike interval attractor
Time and the brain
40
reconstruction must be greater than or equal to the correlation dimension of the attractor in order to obtain a meaningful result (Castro and Sauer, 1997). Misapplications of the Grassberger-Procaccia algorithm led, in the 1980s, to unreliable estimations of deterministic structures for time series (see Rapp et al., 1993; Theiler and Rapp, 1996, for details). Nonetheless, its careful use in recent years has indicated that numerous neurones (up to 25% in some samples) recorded in the substantia nigra and thalamus of anaesthetized rats can be characterized by deterministic dynamics (Celletti and Villa, 1996; Celletti et al., 1997,; Celletti et al., 1999). The analysis of neuronal activity recorded in the primate motor cortex during the delayed-pointing task (see Section §6) revealed the existence of chaotic dynamics in and across paired spike trains (Villa et al., 1998b). The attractors were generally embedded in a space of low dimension, ranging from 2 to 5, and were characterized by a correlation dimension between 0.15 and 3.34 (Figure 19). In the intertrial periods, the dynamical system tended to be characterized by a correlation dimension near 1, and by an attractor embedded in a 4-dimensional space. Conversely, during the preparatory period the geometric scaling properties of the neural dynamics were broadly distributed in the sense that no clear clustering around a specific value of the correlation dimension could be observed (Figure 19). This might suggest the disruption of a common state of activity by the ongoing motor preparation. Attractors observed during the preparatory period were not related to the activity associated with anticipation of the response signal, but rather to preparatory motor processes. Thus, the question is raised whether such dynamics bear some message,
Figure 19 . Neuronal activity recorded in the primate motor cortex (see Figure 15a). Scattergram and marginal distributions of the embedding and
Empirical evidence about temporal structure
41
correlation dimensions of the attractors determined by the GrassbergerProcaccia algorithm in 34 spike trains. The filled symbols and the black bars refer to observations during the preparatory period whereas empty circles and light gray bars refer to the control period.
or may represent a “side effect” of some global self-organizing dynamics within the cerebral cortex. Indeed, deterministic dynamics in the cortex may be interpreted as a collective phenomenon arising from the expression of local attractor dynamics, sustained by a synaptic matrix formed in the process of learning (Herrmann et al., 1993; Amit et al., 1994; Amit, 1995; Amit et al., 1997). However, with respect to information processing in the brain, the body of observations derived from the application of these methods strongly depends upon the dynamics of the underlying neural circuitry. The detection of deterministic dynamics necessarily requires the stability of the generating processes over a relatively long period of time. Thus, significant deterministic dynamics across spike trains may reveal neuronal interactions involved in long-term processes (e.g. memory traces, learned motor programs), whereas the previously cited methods could reveal short-term operational processes at attentional or feature-binding level. Therefore, pattern detection algorithms and dynamical systems analysis may both be characterized by repetitive dynamics of neuronal activity. The question can be raised if there are some relations between these two approaches.
11. SPATIOTEMPORAL FIRING PATTERNS AND CHAOTIC ATTRACTORS PGA was applied to two records from a large scale simulated network described elsewhere (Amit and Brunel, 1997) with maximum allowed jitter equal to 7 ms. The network was composed of 15,000 integrate-and-fire cells, from which 12,000 were excitatory and 3,000 were inhibitory. Each cell in the network had a probability 0.15 of a direct contact with other cells and received, on average, 1,800 excitatory and 450 inhibitory synapses from neurones belonging to the network and 1,800 excitatory synapses from outside the network. In the first record, referred to as “random”, the strength of neuronal connections was initially set as random (Amit and Brunel, 1997) and was not modified. Thirty units were selected by chance, and their activity was recorded during 180 seconds with an accuracy of 1 ms. The activity in the “random” record corresponded to sustained spontaneous rates, in the range 0.1–11.7 spikes/s (mean=4.0 spikes/s and median=3.1 spikes/s). In the second record, referred to as “Hebbian”, the weights of specific populations of synapses were modified according to some type of learning described in detail elsewhere (Amit and Brunel, 1997). The activity of the same set of units was recorded following a stimulation correlated with the learned stimulus. Discharge rates of cells were increased after the Hebbian learning (mean and median equal to 6.5 and 4.2 spikes/s, respectively). No firing pattern was detected in the “random” record by PGA, whereas most cells (20/30 of this sample) formed patterns in the “Hebbian” record. Note that all detected patterns were formed by one cell. The firing rates (mean and median equal to 7.1 and 5.3
Time and the brain
42
spikes/s, respectively) of cells involved in significant group patterns were higher than the rates (mean and median equal to 5.3 and 3.9 spikes/s, respectively) of cells that never formed a temporal pattern. However, the firing rate or the shape of the autocorrelogram were not predictive of the presence of patterns. Most of the autocorrelograms were almost flat and indicated that the spike trains deviated only slightly from a Poisson renewal point process (Figure 20a). In this example, cell 16 was characterized by a firing rate of 17.6 spikes/s but no significant temporal pattern was detected. Cell 17 fired at a similar rate, 17.2 spikes/s, and its autocorrelogram was similar to that of cell 16 (Figure 20a), but 238 patterns of complexity 3–5 were detected. Figure 20b illustrates a significant pattern formed by 5 events repeating 10 times, i.e. <17, 17, 17, 17, 17; 31±2.0, 168±1.5, 186±3.5, 231±2.0.>. In this example the event at delay 186 ms from the pattern onset was detected only at the maximum allowed variability, whereas the other four events formed a significant subpattern at a much greater precision. The relation between spatiotemporal patterns of firing and attractors can be illustrated further by the following example. Let us consider a classical dynamical system, the 2dimensional Hénon mapping (Hénon, 1976). This map is characterized by chaotic behaviour within a certain range of input parameters. With a particular set of such initial parameters it is possible to generate a discrete time series, scaled to a base frequency of 3 spikes/s, thus simulating a possible cortical spike train recording (Tetko and Villa, 1997c). One thousand spikes were generated, corresponding approximately to 330 seconds of recording time. Figure 21a illustrates an example of a spike train generated in this way. In addition to the raster display the return map provides a useful tool to determine visually the presence of a structure. The X-Y plot of the return map is constructed by taking the value of an interval between two successive spikes as
Empirical evidence about temporal structure
43
Figure 20 . High complexity pattern elicited by an attractor mode of activity in a randomly connected network after “Hebbian” learning, (a) Autocorrelograms of three cells recorded simultaneously. The abscissa is lag (in ms) and the ordinate is scaled in rate units (spikes/s). The curves are smoothed by a 10 ms Gaussian shaped bin; the dashed lines indicate the 99% confidence level assuming a Poisson distribution. Note that the shapes of the curves are similar for the three cells, but cell #16 produced no significant patterns, (b) Ten repetitions of a pattern of complexity five, formed by repeated events of cell #17, are displayed as rasters aligned on pattern start. Note that the fourth event, at a delay of 186 ms, was detected on the limit of the maximum allowed accuracy (±3.5 ms).
X-coordinate and the immediately next interspike interval as Y-coordinate. Even if some
Time and the brain
44
regularity may be detected visually on the inspection of the raster display (Figure 21a, left panel), the geometric structure of the return map (Figure 21a, right panel) revealed deterministic dynamics more clearly. In particular, notice the values of the intervals in the range 190–260 ms. The dynamical systems can enter a chaotic regime corresponding to small perturbations of some parameters. The counterpart is that a chaotic regime can be perturbed to
Figure 21 . Raster display and return map of a spike train either generated directly following the Hénon mapping (a) or after deletion of 30% of the spikes and addition of 30% dynamical noise (b). The values of successive intervals In-1 and In are expressed in ms.
some extent, but remains chaotic because of its attractor structure. Let us consider the Hénon spike train perturbed by random deletion of some spikes, and with insertion of new spikes generated according to a uniform distribution on the actual interval. An example of such noisy Hénon spike train is illustrated on Figure 21b. The usual algorithmic methods used to determine if such noisy time series were deterministic may fail to demonstrate any significant structure in the dynamics (Tetko and Villa, 1997c) although in the present example the visual inspection of the return map suggests some regularities (Figure 21b, right panel). It is important to note that chaotic dynamics found in neurophysiological spike trains may be low-dimensional (Celletti and Villa, 1996; Celletti et al., 1998; Villa et al., 1998b), but still with attractors embedded in spaces of dimensions 3–5 whose observation is difficult on a 2-dimensional projection. The
Empirical evidence about temporal structure
45
application of pattern detection algorithms to noisy spike trains generated by attractor dynamics may indeed reveal the presence of recurrent temporal patterns. A typical example is illustrated by Figure 22a, showing a triplet characterized by spike intervals equal to 210 and 191 ms, that fall in the range of the original attractor (Figure 21a). An additional example illustrates an effect due to the noise in the pattern detection results of this type of data-set. The occurrence of 12 triplets <1, 1, 1; 60±4.0, 204±1.5>, characterized by the second event of the pattern having a much higher variability than the third, are shown in Figure 22b. In fact the delay of the third spike is equal to 204 ms, a value included in the range of the original attractor dynamics. Conversely the second event is contributed either by chance or by dynamical noise. In fact, an increase in firing rate loosely time-locked to the onset of the pattern can be observed at delays of approximately 448 and 1440 ms, thus suggesting a non-random origin. However, this example is interesting because it shows that events belonging to the same pattern may have been generated by different mechanisms, so that the temporal pattern might represent a signpost of a sort of binding between those mechanisms. Similar conclusions were discussed from experimental observations described above (Figure 11). The time series generated by chaotic attractors can produce patterns of spikes that recur significantly more often than expected by chance. This finding raises the question whether significant patterns of spikes detected in experimental data are the product of attractor behaviour of neural networks, or if they are produced solely by synfire chains. As mentioned above, the comparison between dynamics of primate motor cortical cells recorded during the period preceding the preparatory signal and during the preparatory period showed important differences of the attractors characteristics (Villa et al., 1998b). The intertrial period corresponds to an interval during which the monkey is quiet and waits for the occurrence of the preparatory signal. In such behavioural conditions, it is
Figure 22 . Raster display, aligned by displaying the first spike in the pattern at
Time and the brain
46
time 0, of the activity of a simulated neurone whose dynamics was generated according to the noisy Hénon mapping (Figure 21b). (a) The pattern repeated 5 times, and was composed of 3 spikes: a spike at time 0, the second 210 ms later, and the third spike 191 ms later. Six additional occurrences of the exact subpatterns formed by the two pair of spikes <1,1; 210> and <1,1; 191> are also displayed on this raster. The triplet was detected with a fixed accuracy of ±2 ms. (b) The pattern repeated 12 times and was composed of 3 spikes: a spike at time 0, the second 60±4.0 ms later, and the third spike 144±1.5 ms later. Notice the variable jitter. An increase in firing rate which is loosely time-locked with the pattern start is indicated by the asterisks. The abscissa full scale is 2000 ms.
possible that the primary motor cortex is not yet involved in motor preparation, because the actual salient cognitive activity is the expectancy of the preparatory signal. Then, the firing activities of motor cortical cells may be attracted in the same basin, a kind of “common attractor” as postulated theoretically for spontaneous activity (Villa, 1992; Amit, 1995; Villa and Tetko, 1995; Amit and Brunel, 1997). In the next section a model of neural dynamics is presented, based on formal definitions and on the above considerations of chaotic activity in spike trains.
Empirical evidence about temporal structure
47
12. A MODEL OF NEURAL DYNAMICS System identification rests upon a vast quantitative methodology (Nieman et al., 1971), but most of it refers to linear operations. In previous sections significant sensory, cognitive, behavioural, and simulation correlates of non-linear temporal patterns of firing have been provided. Spike trains are statistically expressed by point-like processes (Segundo et al., 1966). Thus, a crucial requirement for a theoretical framework is to identify these point process systems without any assumption as to whether or not they are linear. Specific methods for this purpose have been developed (Brillinger 1974, 1975; Rapp et al., 1994). Point process systems—systems whose input and output are point processes—are said to be identified when an acceptable model is found. The first step of the identification is to estimate certain conditional rate functions, called kernels, of the spike trains. The one of zero-order, i.e. a constant, simply measures the mean rate. The one of first-order, a function of a single time argument, relates to the average effect of a single trigger spike (presynaptic) on the spike train; the one of second-order, a function of two time arguments, relates to the interactions between pairs of spikes; and so forth for higher-order functions. Then successive models are constructed recursively and based on the kernel of zero-order, on the kernels of zero- and first-order, on the kernels of zero-, first- and second-order, and so on. The acceptance of a kernel estimate as plausible does not necessarily mean that the series up to the corresponding term is a good predictor. The predictability may be evaluated by means of coherence (Brillinger, 1975). On the basis of the above assumptions it is possible to identify the neural network system (see Appendix I) that fulfils the necessary requirement of the probability function f describing its mode of activity. With respect to the state variable µ, the system may be qualified as a dynamical system x’ where , as mentioned above (Section §9). The next step consists of assuming that the dynamical system is structurally stable. In terms of topology, that means that, for a dynamical system x’ there exists a neighbourhood N(x’) in the state space with the property that every is topologically equivalent to x’. This concept is extremely important, because a structurally stable dynamical system cannot degenerate. There is no need to know the exact equations of the dynamical system because qualitative, approximate equations—i.e. “in the neighbourhood”—show the same qualitative behaviour (Andronov and Pontryagin 1937) (Figure 23). In the case of two control parameters, the probability function ƒ is 2 defined as the points µ of IR with a structurally stable dynamics of (Peixoto, 1962). That means the qualitative dynamics x’ is defined in a neighbourhood of a pair ( x 0, µ 0) at which ƒ is in equilibrium (e.g. minima, maxima, saddle point). With these assumptions, the equilibrium surface is geometrically equivalent to the RiemannHugoniot or cusp catastrophe described by Thorn (1975). Figure 24 illustrates a possible topological interpretation of neural dynamics according to this model. The equilibrium surface represents stable modes of activity with postsynaptic potential kinetics and the membrane excitability as control parameters.
Time and the brain
48
Let us start at a point (a) on the equilibrium surface, corresponding to a high level of excitability and a relatively long decay time of the postsynaptic potentials, e.g. 12 ms. This may correspond to the tonic mode of firing described in the thalamo-cortical circuit (Steriade et al., 1990). With these “settings” an input pattern will evoke precisely timestructured activity. Different firing patterns may be evoked by the same input if the synaptic dynamics are modified with minor changes of the cellular excitability, as suggested for neuromodulatory mediators (Levitan, 1988). Furthermore, different input patterns of activity may subserve the same mode of activity, somewhat like attractors. The transitions between these states are represented graphically in Figure 24 by paths (ab-a), (a-e-a) and (a-g-a). When the excitability is decreased, several types of neurones in the cortex and in the thalamus tend to switch towards a rhythmic or bursty type of firing. This effect may be provoked by a hyperpolarization of the cell membrane, or by modifying the spike threshold level (Foote and Morrison, 1987). In the former case a smooth passage between time-structured activity and asynchronous firing is likely to occur, as suggested by path (b-c-b), especially if the synaptic decay is long. Conversely, if the synaptic decay is fast, and a modulatory input modifies the threshold potential, a sudden switch from temporal patterns of firing to desynchronized activity will occur, as indicated by paths
Figure 23 . Two-dimensional representation of dynamical systems that are topologically equivalent (see text for definition of equivalency).
Empirical evidence about temporal structure
49
Figure 24 . Topological interpretation of neural dynamics as a function of two control parameters, the cell excitability and the kinetics of the postsynaptic potentials. The equilibrium surface is represented by a cusp catastrophe, where transitions can occur either suddenly or continuously between temporally organized firing patterns and asynchronous activity. This equilibrium surface refers to the activity of a specific neural network, and it is possible that the same cell belongs to more than one cell assembly. If the cell assemblies are controlled only by one parameter in common, then temporal and rate code are not mutually exclusive.
(a-d) and (e-f). Complex spatiotemporal firing patterns may also occur with low levels of excitability, i.e. point (e) in the graph of Figure 24, as suggested by cholinergic switching within neocortical networks (Villa et al., 1996; Xiang et al., 1998). Such a point (e) on the equilibrium surface can be particularly unstable, because a further decrease in excitability [path (e-f)], but also an increase in synaptic decay [path (e-d)], may provoke a sudden change in the mode of activity, as observed in simulation studies (Villa, 1992; Hill and Villa, 1997). It is important to notice that if the excitability is low, e.g. during long lasting hyperpolarization, the kinetics of the postsynaptic potential is often irrelevant with regard to the input pattern, so that the output activity will always tend to be organized in rhythmic bursts. Conversely, if the excitability is increased from a starting point (f) and the time constant of the synaptic decay is fast (say 4–5 ms), the input patterns could switch on either stable [path (f-g)] or unstable temporally-organized modes
Time and the brain
50
of activity [path f-e] only through sudden transitions.
13. SUMMARY OF CHAOTIC DYNAMICS Point processes defined by spike trains can be analyzed from alternative conceptual frameworks. Although few reasons are generally considered for adopting these frameworks as complementary approaches to the same biological system, several observations, either from experimental or simulated data, and theoretical assumptions have lead us to raise serious questions about the relationships between Spatiotemporal firing patterns and chaotic processes in neural networks. The dual description of brain activity could be regarded as similar to the dual representation of the light, with corpuscular and wave equations representing two irreducible expressions of the same phenomenon. A careful investigation of the dual nature of highly converging/diverging neural networks could open unexpected new ways to answer the problems arising from observations at multiple scales (ranging from sets of single neurones to brain imaging) of brain activity. In mathematics, few years ago, the demonstration of the Taniyama-Shimura conjecture filled the gap between two worlds and opened the way to the demonstration of Fermat’s last theorem by Andrew Wiles. This conjecture says that every rational elliptic curve (e.g. characterized by doughnut shapes) is a modular form, and modular forms have special properties with their many symmetries. Likewise, it is appealing to suggest that the relationships between chaotic dynamics and complex patterns of firing may represent a dictionary where questions, intuitions and insights in the one world get translated to questions, intuitions and insights in the other world.
14. FUTURE PROGRAM The detection of precise Spatiotemporal firing patterns or attractors necessarily requires the stability of the generating processes over a relatively long period of time. Thus, precise spike patterns in single neurones, or across multiple neurones, may be mostly involved in long-term processes (e.g. memory traces, learned motor programs), whereas the ensemble coding based on systematic modulations of firing rate may be related to short-term operational processes (e.g. motor action, attentional or feature-binding). The research presented here is not asking the questions that most neurophysiologists usually ask: which is the most adequate stimulus for a given neurone?; how is the external world mapped in the cortex?; what precisely are the receptive fields of single units? What is being asked here is how to detect the participation of nerve cells in distributed brain information processing, e.g. sensory discrimination, preparatory activity or short-term memory. The quantitative measure of neuronal activity available to neurophysiologists is the time series of spike epochs—i.e. the spike train. Therefore, it is of utmost interest to study temporal patterns of firing and assess whether they might be carriers of information. It is true that in the nervous system the problem of learning is crucial and
Empirical evidence about temporal structure
51
can hardly be approached without taking into account synaptic modification. However, not all neural information needs to be processed through modifiable networks. Changes in strength of a particular link between two cells become meaningless, because, in a distributed system, equivalent changes at one synapse might correspond to different patterns of activity. Therefore, the information is not necessarily resident in the links among the units, but might be provided by activity organized in a highly precise temporal mode. Indeed a connectionist network with the same set of connection weights may transmit different signals (Mpitsos et al., 1988). Accurate application of complex pattern detection methods enables correlation between discrete brain states and measures of behavioural performance on a trial-by-trial basis. The fact that the timing of pattern occurrence could relate to reaction time indicates that the network phenomena underlying them reflect some state of the animal that is able to influence behavioural output. Changes in cortical network activity during the waiting period may therefore be related to the concepts of “attention” and “set”, with emphasis on processes related to motor output in the former (Wise and Kurata, 1989) and to sensory processing in the latter (Shinba et al., 1995). It is interesting to notice that in the behavioural experiments reported here we found relatively few significant patterns, in the order of few hundreds, for tens of hours of analyzed recording time. The recording of cell activity is done over short periods (10 s) per trial, but one behavioural experiment could last several hours. It is possible to postulate either that the single unit detection might be unstable over such a prolonged time, or that only those cells involved repeatedly in activity directly related to the animal behaviour can participate in the detected patterns, thus decreasing the chance of detection with a limited sample of 15 spike trains recorded simultaneously. The problem of finding several neurones from a synfire chain is analyzed by Abeles (example 7.6.2, p. 255 in Abeles, 1991). In this example, finding two or more neurones from the same chain, when recording 10 neurones simultaneously, where p 0 =0.15 is the estimated probability of finding one cell from a chain, can be performed with an approximate probability of 0.44. The probability of detecting the same number of neurones in simultaneous recordings of 15, 20 or 30 cells, increases significantly. If the assumptions of this example are correct, it becomes possible to detect 6 or more neurones belonging to the same synfire chain, when recording from 30 single units simultaneously, with a probability near to 0.30. These estimations take into account only one synfire chain, but a single neurone may be involved simultaneously in multiple synfire chains and a synfire chain can include the same neurone twice, or even a higher number of times (Abeles et al., 1994). Altogether the experimental data and theoretical assumptions presented here make it possible to outline a general framework of coding principles, in which time and rate coding do not represent mutually exclusive coding schemes. Temporal and/or noisy rate codes might operate simultaneously in a given network, and several tools of analysis should be used to test the general hypothesis that the activation of widespread cell assemblies is distributed through multiple patterns of neuronal activity. Synchronous or oscillatory activity might activate and start complex temporally-organized processes, e.g. encompassing distributed sensorimotor information over widespread brain areas or thalamo-cortical recurrent loops (Villa et al., 1996). State-of-the-art techniques have allowed several laboratories to record tens of cells simultaneously in behaving rats
Time and the brain
52
(Skaggs and McNaughton, 1996; Nicolelis et al., 1997). Simulation of large-scale neuronal networks formed by integrate-and-fire model neurones are also becoming popular to test hypotheses on the significance of temporal information processing in the brain (Amit and Brunel, 1997; Lumer et al., 1997). The possibility of inserting virtual electrodes in such networks and of recording from hundreds or thousands of virtual neurones will certainly become a routine analysis in the forthcoming years. Then, the possibility of detecting and estimating carefully the significance of high complexity spatiotemporal patterns of spikes appears to be a crucial step for the evaluation of coding schemes. The spatiotemporal firing pattern should not be considered as representing a Morse alphabet, but rather the co-representation of specific information and a certain state of “central arousal”. Future developments of temporal dynamics and topological interpretation of neuronal network activity could be an important issue in understanding the functional properties and information processing in the central nervous system. Whether this prediction is upheld or not, there is no doubt that these studies will open up new possibilities for research (new areas, new methods of analysis, new recording technique, computer-modeling assistance, etc.) and no possible outcome should be a dead end. After all, we are only 200 years from the intellectual legacy of Galvani and although we may say that the pre-history of Neuroscience is over we should not rule out the possibility that we are still in its proto-historical phase.
ACKNOWLEDGEMENTS This work was made possible particularly by the contribution of Igor V.Tetko and by the members of the Laboratory of Neuro-heuristics (www-lnh.unil.ch) and the staff personnel of the Institute of Physiology of the University of Lausanne. Partial support by the Swiss National Science Foundation (Bern) and the Istituto della Enciclopedia Italiana (Roma) is acknowledged.
APPENDIX I The following paragraphs illustrate a possible framework for system identification. Suppose that A is a stationary spike train. The overall mean rate of the A train is defined by Prob { A spike in ( t,t + h )}/ h. Suppose that B is another stationary spike train occurring simultaneously to spike train A and m B its mean rate. A first-order conditional rate function m AB (u) is defined as follows. For one cell B and close to any particular time t the AB cross-intensity function, m AB (u), measures the instantaneous rate of the B train—i.e. the probability whether there will be a B spike- u time units after an A spike, say
(1) The average change of the instantaneous rate at time u in a spike train B, when a single
Empirical evidence about temporal structure
53
spike occurs at time 0 in a spike train A is predicted at best by the kernel of first order a (u). If one defines m AA (u) as the auto-intensity function of A —i.e. the cross-intensity function of the A train with itself-then, where satisfies the following condition (Brillinger 1974),
(2)
Because of stationarity this function does not depend on t. If the spike trains are independent, then m AB (u)=m B for any u . If spike train A were Poisson noise, m AA (v) would be equal to m A for all v , and the solution of Equation (2) would be a(u)=m AB (u) −m B . In this case the kernel function would correspond to the cross-intensity function up to an additive constant and it could be estimated by the corresponding cross-correlation histogram. In order to construct a general solution a(u) of Equation (2) let us define two parameters, φ AA (λ) and φ AB (λ). The parameter
(3)
denotes the power spectrum of the A spike train (Brillinger 1974). This is proportional to the variance of the component of frequency λ of the A spike train, hence it may be interpreted as reflecting the power in each frequency component of that train. For pure Poisson noise, it is identically constant and equal to (2 p)−1 m A . The parameter (4)
denotes the cross-spectrum of the A and B spike trains. This is proportional to the covariance of the component of frequency λ of the A train with the corresponding component of the B train. Hence it may be interpreted as reflecting how a certain frequency component in the A train is associated with one in the B train. The Fourier transform of the solution a(u) is defined by
(5)
Taking the Fourier transform of Equation (2) and the definitions (3) and (4) leads to φ AB , the solution of Equation (λ)=A(λ) φAA (λ). It follows that, if (2) may be written as
Time and the brain
54
(6)
Finally, the measure of predictability of the kernel is provided by the coherence at frequency λ defined by It is never less than zero or greater than one. It is a measure of the association of the spike trains. A coherence of one would imply perfect linear prediction of the B train by the A train at frequency λ. A coherence of zero results if the two trains are statistically independent or if at frequency λ. Several studies have applied this approach to study the functional interactions between spike trains (Rosenberg et al., 1989; Brillinger and Villa, 1994, 1997).
REFERENCES Abbott, L.F., Varela, J.A., Sen, K., and Nelson, S.B. (1997) Synaptic depression and cortical gain control. Science, New York 275, 220–224. Abeles, M. (1982) Local Cortical Circuits: An Electrophysiological Study , Springer Verlag, Berlin. Abeles, M. (1983) The quantification and graphic display of correlations among three spike trains. IEEE Transactions on Biomedical Engineering 30, 235–239. Abeles, M. (1991) Corticonics: Neural Circuits of the Cerebral Cortex . Cambridge University Press, New York. Abeles, M., and Gerstein, G.L. (1988) Detecting Spatiotemporal Firing Patterns Among Simultaneously Recorded Single Neurones. Journal of Neurophysiology , 60, 909–924. Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993a) Spatiotemporal Firing Patterns in the Frontal Cortex of Behaving Monkeys. Journal of Neurophysiology , 70, 1629–1638. Abeles, M., Prut, Y., Bergman, H., Vaadia, E., and Aertsen, A. (1993b) Integration, synchronicity and periodicity. In: A.Aertsen, (ed.) Brain theory: spatio-temporal aspects of brain function , pp. 149–181. Elsevier, Amsterdam. Abeles, M., Vaadia, E., Bergman, H., Prut, Y., Haalman, I., and Slovin, H. (1993c) Dynamics of neuronal interactions in the frontal cortex of behaving monkeys. Concepts in Neuroscience , 4, 131–158. Abeles. M., Prut, Y., Bergman, H., and Vaadia, E., (1994) Synchronization in neuronal transmission and its importance for information processing, Progress in Brain Research , 102, 395–404. Addison, P.S. (1997) Fractals and Chaos: An illustrated Course . Institute of Physics Publishing, Bristol. Adrian, E.D. (1912) On the conduction of subnormal disturbances in normal nerve. Journal of Physiology (London) , 45, 389–412. Adrian, E.D. (1934) The basis of sensation: the action of the sense organs . Christophers, London. Aertsen, A.M.H.J., Gerstein, G.L., Habib, M.K., and Palm, G. (1989) Dynamics of neuronal firing correlation: modulation of effective connectivity. Journal of Neurophysiology , 61 , 900–917.
Empirical evidence about temporal structure
55
Amit, D.J. (1995) The Hebbian paradigm reintegrated: local reverberations as internal representations. Behavioral Brain Sciences , 18, 617–657. Amit, D.J., Brunel, N., and Tsodyks, M. (1994) Correlations of cortical Hebbian reverberations: theory versus experiment. Journal of Neuroscience , 14 , 6435–6445. Amit, D.J. and Brunel, N., (1997) Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex , 7, 237–252. Amit, D.J., Fusi, S., and Yakovlev, V. (1997) Paradigmatic working memory (attractor) cell in IT cortex. Neural Computation , 9, 1071–1092. Andronov, A., and Pontryagin, L. (1937) Systèmes grossiers. Doklady Akademii Nauk. SSSR , 14 , 247–251. Babloyantz, A., Salazar, J.M., and Nicolis, C. (1985) Evidence of chaotic dynamics of brain activity during the sleep cycle. Physics letters , 111A , 152–155. Bair, W., and Koch, C. (1996) Temporal precision of spike trains in extrastriate cortex of the behaving Macaque monkey. Neural Computation , 8, 1184–1202. Bialek, W., Rieke, F., de Ruyter van Steveninck, R.R., and Warland, D.S. (1991) Reading a neural code. Science, New York , 252, 1854–1857. Bialek, W., and Rieke, F. (1992) Reliability and information transmission in spiking neurons. Trends in Neuroscience , 15 , 428–434. Braitenberg, V., and Schüz, A. (1991) Anatomy of the cortex: statistics and geometry . Springer Verlag, Berlin. Brillinger, D.R. (1974) Cross-spectral analysis of processes with stationary increments including the stationary G/G/8 queue. Annals of Probability , 2, 815–827. Brillinger, D.R. (1975) The identification of point process systems. Annals of Probability , 3, 909–924. Brillinger, D.R., and Villa, A.E.P. (1994) Examples of the investigation of neural information processing by point process analysis. In: V.Z.Marmarelis (ed.) Advanced Methods of Physiological System Modeling, volume 3 , pp. 111–127, Plenum Press, New York. Brillinger, D.R., and Villa, A.E.P. (1997) Assessing connections in networks of biological neurons. In: D.R.Brillinger, L.T.Fernholz, S.Morgenthaler (eds) The Practice of Data Analysis , pp. 77–92, Princeton University Press, Princeton, NJ. Brillinger, D.R., Bryant, H.L. Jr., and Segundo, J.P. (1976) Identification of synaptic interactions. Biological Cybernetics , 22, 213–228. Bullock, T.H. (1961) The problem of recognition in an analyzer made of neurons. In: W.A. Rosemblith (ed.) Sensory communication , pp. 717–724, Wiley, New York. Castro, R., and Sauer, T. (1997) Correlation dimension of attractors through interspike intervals. Physics Reviews , A55, 287–290. Celebrini, S., and Newsome, W.T. (1994) Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. Journal of Neuroscience , 14 , 4109–4124. Celletti, A., and Villa, A.E.P. (1996) Low dimensional chaotic attractors in the rat brain. Biological Cybernetics , 74, 387–394. Celletti, A., Bajo Lorenzana, V.M., and Villa, A.E.P. (1997) Correlation dimension for paired discrete time series. Journal of Statistical Physics , 89, 877–884. Celletti, A., Tetko, I.V., and Villa, A.E.P. (1998) Analysis of the deterministic behaviour of experimental series. Journal de Physique IV 8, 209–214. Celletti, A., Froeschlé, C., Tetko, I.V., and Villa, A.E.P (1999) Deterministic behaviour of short time series. Meccanica . 34 . 145–152.
Time and the brain
56
Cutler, C.D., and Kaplan, D.T. (1997) Nonlinear dynamics and time series: building a bridge between the natural and statistical sciences. Fields Institute communications, vol. 11 , pp. 252, American Mathematical Society, Providence, R.I. Dayhoff, J., and Gerstein, G.L. (1983a) Favored patterns in spike trains. I. Detection. Journal of Neurophysiology , 49, 1334–1348. Dayhoff, J., and Gerstein, G.L. (1983b) Favored patterns in spike trains. II. Application. Journal of Neurophysiology , 49, 1349–1363. de Ruyter van Steveninck, R.R., Lewen, G.D., Strong, S.P., Koberle, R., and Bialek, W. (1997) Reproducibility and variability in neural spike trains. Science, New York , 275, 1805–1808. deCharms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds by the coordination of action-potential timing. Nature, London , 381, 610–613. Douglas, R., and Martin, K. (1991) A functional micro-circuit for cat visual cortex. Journal of Physiology (London) , 440, 735–769. Eckmann, J.P., and Ruelle, D. (1992) Fundamental limitations for estimating dimensions and Lyapunov exponents in dynamical systems. Physica D , 56, 185–187. Edmonds, B., Gibb, A., and Colquhon, D. (1995) Mechanisms of activation of glutamate receptors and the time course of excitatory synaptic currents. Annual Review of Physiology , 57, 495–519. Eckhorn, R., Grusser, O.J., Kroller, J., Pellnitz, K., and Popel, B. (1976) Efficiency of different neuronal codes: information transfer calculations for three different neuronal systems. Biological Cybernetics , 22, 49–60. Ferster, D., and Spruston, N. (1995) Cracking the neural code. Science, New York , 270, 756–757. Fetz, E.E. (1997) Neuroscience—temporal coding in neural populations. Science, New York , 278, 1901–1902. Foote, S.L., and Morrison, J.H. (1987) Extrathalamic modulation of cortical function. Annual Review of Neuroscience , 10, 67–96. Friston, K.J. (1995) Neuronal transients. Proceedings of the Royal Society of London— Series B , 261, 401–405. Frostig, R.D., Frostig, Z., and Harper, R.M. (1984) Information trains. The technique and its uses in spike train and network analysis with examples taken from the nucleus parabrachialis medialis during sleep-waking states. Brain Research , 322, 67–74. Frostig, R.D., Frostig, Z., and Harper, R.M. (1990) Recurring discharge patterns in multiple spike trains. I. Detection. Biological Cybernetics 62, 487–493. Fuster, J.M. (1990) Inferotemporal units in selective visual attention and short-term memory. Journal of Neurophysiology , 64, 681–697. Georgopoulos, A., Taira, M.L., and Lukashin, A. (1993) Cognitive neurophysiology of the motor cortex. Science, New York , 260, 47–52. Georgopoulos, A.P. (1995) Current issues in directional motor control. Trends in Neuroscience , 18, 506–510. Gerstein, G.L., and Mandelbrot, B. (1964) Random walk models for the spike activity of a single neuron. Biophysical Journal , 4, 41–68. Grassberger, P., and Procaccia, I. (1983) Characterization of strange attractors. Physics Review Letters , 50, 346–349. Hénon, M. (1976) A two-dimensional mapping with a strange attractor, Communications in Mathematical Physics , 50, 69–77. Herrmann, M., Ruppin, E., and Usher, M. (1993) A neural model of the dynamic activation of memory. Biological Cybernetics , 68, 455–463.
Empirical evidence about temporal structure
57
Hill, S.L., and Villa, A.E.P. (1997) Dynamic transitions in global network activity influenced by the balance of excitation and inhibition, Network , 8, 165–184. Klemm, W.R., and Sherry, C.J. (1981) Serial ordering in spike trains: what’s it trying to tell us? International Journal of Neuroscience , 14, 15–23. König, P., Engel, A.K., and Singer, W. (1996) Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neuroscience , 19, 130–137. Legéndy, C.R. (1975) Three principles of brain function and structure. International Journal of Neuroscience , 6, 237–254. Levitan, I.B. (1988) Modulation of ion channels in neurons and other cells. Annual Reviews of Neuroscience , 11, 119–136. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997) Neural dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cerebral Cortex , 7, 207–727. MacGregor, R.J. (1987) Neural and brain modeling . Academic Press, San Diego. Mainen, Z.F., and Sejnowski, T.J. (1995) Reliability of spike timig in neocortical neurons. Science, New York , 268, 1503–1506. Markram, H., Lubke, J., Frotscher, M., and Sakmann, B. (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APS and EPSPS. Science, New York , 275, 213–215. Martignon, L., Von Hasseln, H., Grim, S., Aertsen, A., and Palm, G. (1995) Detecting higher-order interactions among the spiking events in a group of neurons. Biological Cybernetics , 73, 69–81. McCormick, D.A., and Bal, T. (1994) Sensory gating mechanisms of the thalamus. Current Opinion in Neurobiology , 4, 550–556. McCulloch, W.S., and Pitts, W. (1943) A logical calculus of the ideas imminent in nervous activity. Bulletin of Mathematical Biophysics , 5, 115–133. Miller, R. (1996) Cortico-thalamic interplay and the security of operation of neural assemblies and temporal chains in the cerebral cortex. Biological Cybernetics , 75, 263–275. Mountcastle, V.B., Poggio, G.F., and Werner, G. (1963) The relation of thalamic cell response to peripheral stimuli varied over an intensive continuum. Journal of Neurophysiology , 26, 807–834. Mpitsos, G.J., Burton, R.M., and Creech, H.C. (1988) Connectionist networks learn to transmit chaos. Brain Research Bulletin , 21, 539–546. Mpitsos, G.J. (1989) Chaos in brain function and the problem of nonstationarity: a commentary. In: E. Başar, T.H.Bullock (eds) Dynamics of sensory and cognitive processing by the brain , pp. 521–535, Springer-Verlag, New York, N.Y. Mpitsos, G.J., and Burton, R.M.Jr. (1992) Convergence and divergence in neural networks: processing of chaos and biological analogy. Neural Networks , 5, 605–625, Mukhametov, L.M., Rizzolatti, G., and Tradardi, V. (1970) Spontaneous activity of neurones of nucleus reticularis thalami in freely moving cats. Journal of Physiology (London) , 210, 651–667. Nafe, J.P. (1968) Neural correlates of sensation. In: D.R.Kenshalo (ed.) The Skin Senses pp. 5–14, Clarence Thomas, Springfield. Nicolelis, M.A.L., Ghazanfar, A.A., Faggin, B.M., Votaw, S., and Oliveira, L.M.O. (1997) Reconstructing the engram—simultaneous, multisite, many single neuron recordings. Neuron , 18, 529–537. Nieman, R.E., Fisher, D.G., and Seborg, D.E. (1971) A review of process identification and parameter estimation techniques. International Journal of Control , 13 , 209–264.
Time and the brain
58
Payne, B.R., Lomber, S.G., Villa, A.E.P., and Bullier, J. (1996) Reversible deactivation of cerebral network components. Trends in Neuroscience , 19 , 535–542. Peixoto, M. (1962) Structural stability on 2-dimensional manifolds. Topology, 1, 101– 120. Perkel, D.H., and Bullock, T.H. (1965) Neural coding. Neurosciences Research Program Bulletin , 6 , 221–348. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967a) Neuronal spike trains and stochastic point processes. I. The single spike train. Biophysical Journal , 7, 391–418. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967b) Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophysical Journal , 7, 419– 440. Perkel, D.H., Gerstein, G.L., Smith, M.S., and Tatton, W.G. (1975) Nerve-impulse patterns: a quantitative display technique for three neurons. Brain Research , 100 , 271–296. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. (1994) Numerical Recipes in C. (2nd ed.) Cambridge University Press, New York. Prut. Y., Vaadia, E., Bergman, H., Haalman, I., Slovin, H., and Abeles, M. (1998) Spatiotemporal Structure Of Cortical Activity—Properties and Behavioral Relevance. Journal of Neurophysiology , 79, 2857–2874. Rapp, P.E., Zimmerman, I.D., Albano, A.M., Deguzman, G.C., and Greenbaum, N.N. (1985) Dynamics of spontaneous neural activity in the simian motor cortex: the dimension of chaotic neurons. Physics Letters , 110A , 335–338. Rapp, P.E., Zimmerman, I.D., Vining, E.P., Cohen, N., Albano, A.M., and JimenezMontano, M.A. (1994) The algorithmic complexity of neural spike trains increases during focal seizures. Journal of Neuroscience , 14 , 4731–4739. Rapp, P.E., Albano, A.M., Schmah, T.I., and Farwell, L.A. (1993) Filtered noise can mimic low-dimensional chaotic attractors. Physical Review , E47, 2289–2297. Riehle, A., and Requin, J. (1995) Neuronal correlates of the specification of movement direction and force in four cortical areas of the monkey. Behavioral Brain Research , 70, 1–13. Riehle, A., Grün, S., Diesmann, M., and Aertsen, A. (1997) Spike synchronization and rate modulation differentially involved in motor cortical function. Science, New York , 278, 1950–1953. Rieke, F., Bodnar, D.A., and Bialek, W. (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proceedings of the Royal Society of London—Series B , 262, 259–265. Rosenberg, J.R., Amjad, A.M., Breeze, P., Brillinger, D.R., and Halliday, D.M. (1989) The Fourier approach to the identification of functional coupling between neuronal spike trains. Progress in Biophysics and Molecular Biology , 53, 1–31. Segundo, J.P. (1986) What can neurons do to serve as integrating devices? Journal of Theoretical Neurobiology , 5, 1–59. Segundo, J.P., Moore, G.P., Stensaas, L., and Bullock, T.H. (1963) Sensitivity of neurons in Aplysia to temporal patterns of arriving impulses. Journal of Experimental Biology , 40, 643–667. Segundo, J.P., Perkel, D.H., and Moore, G.P. (1966) Spike probability in neurones: influence of temporal structure in the train of synaptic events. Kybernetik , 3, 67–82. Segundo, J.P., and Perkel, D.H. (1969) The nerve cell as an analyzer of spike trains. UCLA Forum in Medical Sciences , 11, 349–390. Shadlen, M.N., and Newsome, W.T. (1994) Noise, Neural Codes and Cortical
Empirical evidence about temporal structure
59
Organisation. Current Opinion in Neurobiology , 4, 569–579. Shadlen, M.N., and Newsome, W.T. (1998) The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. Journal of Neuro science , 18, 3870–3896. Shinba, T., Sumi, M., Iwanami, A., Ozawa, N., and Yamamoto, K. (1995) Increased neuronal firing in the rat auditory cortex associated with preparatory set. Brain Research Bulletin , 37, 199–204. Skaggs, W.E., and McNaughton, B.L. (1996) Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science, New York , 271, 1870–1873. Smale, S. (1967) Differential dynamical systems. Bulletin of the American Mathematical Society , 73, 747–817. Softky, W., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random epsps. Journal of Neuroscience , 13, 334–350. Steriade, M., Jones, E.G., and Llinás, R.R. (1990) Thalamic oscillations and signalling , Wiley & Sons, New York, NY. Stuart, G.J., and Sakmann, B. (1994) Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature, London , 367, 69–72. Tang, A.C., Bartels, A.M., and Sejnowski, T.J. (1997) Effects of cholinergic modulation on responses of neocortical neurons to fluctuating input. Cerebral Cortex , 7, 502–509. Tetko, I.V., and Villa, A.E.P. (1997a) Fast combinatorial methods to estimate the probability of complex temporal patterns of spikes. Biological Cybernetics , 76, 397– 407. Tetko, I.V., and Villa, A.E.P. (1997b) Efficient partition of learning datasets for neural network training. Neural Networks , 10, 1361–1374. Tetko, I.V., and Villa, A.E.P. (1997c) A comparative study of pattern detection algorithm and dynamical system approach using simulated spike trains. Lecture Notes in Computer Science , 1327, 37–42. Tetko, I.V., and Villa, A.E.P. (1999) Temporal interactions in multiple spike trains detected by pattern grouping algorithm. Journal of Neurophysiology (submitted). Theiler, J., and Rapp, P.E. (1996) Re-examination of the evidence for low-dimensional, nonlinear structure in the human electroencephalogram. Electroencephalography and Clinical Neurophysiology , 98, 213–222. Thorn, R. (1975) Structural stability and morphogenesis (English translation). W.A. Benjamin, Reading, MA. Thomson, A., and Deuchars, J. (1994) Temporal and spatial properties of local circuits in neocortex. Trends in Neuroscience , 17, 119–126. Thorpe, S., Fize, D., and Marlot, C. (1996) Speed of processing in the human visual system. Nature, London , 381, 520–522. Tsodyks, M.V., and Sejnowski, T. (1995) Rapid state switching in balanced cortical network models. Network , 6, 111–124. Tsodyks, M.V., and Markram, H. (1997) The neural code between neocortcal pyramid neurones depends on neurotransmitter release probability. Proceedings of the National Academy of Sciences, U.S.A. , 94, 719–723. Tsukada, M., Terasawa, M., and Hauske, G. (1982) Temporal pattern sensitive and nonsensitive responses in the cat’s retinal ganglion cells. Biological Cybernetics , 44, 197–203. Usher, M., and Stemmler, M. (1995) Dynamic pattern formation leads to 1/f noise in
Time and the brain
60
neural populations. Physical Review Letters 74, 326–329. Vaadia, E., Bergman, H., and Abeles, M. (1989) Neuronal activities related to higher brain functions—theoretical and experimental implications. IEEE Transaction in Biomedical Engineering , 36, 25–35. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London , 373, 515–518. Villa, A.E.P. (1988) Influence de I'écorce cérébrale sur I’activité spontanée et evoquée du thalamus auditifdu chat . Imprivite, Lausanne. Villa, A.E.P. (1990) Functional differentiation within the auditory part of the reticular nucleus of the cat. Brain Research Reviews . 15. 25–40. Villa, A.E.P. (1992) Les catastrophes cachées du cerveau. Le Nouveau Golem , 1, 33–63. Villa, A.E.P., and Abeles, M. (1990) Evidence for spatio-temporal firing patterns within the auditory thalamus of the cat. Brain Research , 509, 325–327. Villa, A.E.P., Rouiller, E.M., Simm, G.M., Zurita, P., de Ribaupierre, Y., and de Ribaupierre, F. (1991) Corticofugal modulation of information processing in the auditory thalamus of the cat. Experimental Brain Research , 86 , 506–517. Villa, A.E.P., and Fuster, J.M. (1992) Temporal correlates of information processing during visual short-term memory. NeuroReport , 3, 113–116. Villa, A.E.P., and Tetko, I.V. (1995) Spatio-temporal patterns of activity controlled by system parameters in a simulated thalamo-cortical neural network. In: H.Hermann, (ed.) Supercomputing in Brain Research: from tomography to neural networks , pp. 379–388, World Scientific Publishing, Singapore. Villa, A.E.P., Tetko, I.V., Riehle, A., and Requin, J. (1995) Selective spatio-temporal firing patterns in the motor cortex during a reaction-time task. Society for Neuroscience Abstracts , 21 , 516. Villa, A.E.P., Bajo, V.M., and Vantini, G. (1996) Nerve Growth Factor (NGF) modulates information processing in the auditory thalamus. Brain Research Bulletin , 39, 139– 147. Villa, A.E.P., Hyland, B., Tetko, I.V., and Najem, A. (1998a) Dynamical cell assemblies in the rat auditory cortex in a reaction-time task. Biosystems , 48, 269–278. Villa, A.E.P., Tetko, I.V., Celletti, A., and Riehle, A. (1998b) Chaotic dynamics in the primate motor cortex depend on motor preparation in a reaction-time task. Current Psychology of Cognition . 17, 763–780. Villa, A.E.P., Tetko, I.V., Hyland, B., and Najem, A. (1999a) Significance of spatiotemporal activity patterns among rat cortex neurons in performance of a conditioned task. Proceedings of the National Academy of Sciences, U.S.A. , 96, 1006– 1011. Villa, A.E.P., Tetko, I.V., Dutoit, P., de Ribaupierre, Y., and de Ribaupierre, F. (1999b) Corticofugal modulation of functional connectivity within the auditory thalamus of rat, guinea pig and cat revealed by cooling deactivation. Journal of Neuroscience Methods , 86, 161–178. Villa, A.E.P., Eriksson, J., Eriksson, C., Haeberli, C, Hyland, B., and Najem, A. (199c) Stimulus congruence affects perceptual processes in a novel Go/Nogo conflict paradigm in rats. Behavioural Processes , 48, 69–88. Villa, A.E.P., and Tetko, I.V. (1999) Spatiotemporal Activity Patterns Detected from Single Cell Measurements from Behaving Animals, Proceedings SPIE (International Society for Optical Engineering) , 3728, 20–34. Wilson, M.A., and McNaughton, B.L. (1993) Dynamics of the Hippocampal Ensemble
Empirical evidence about temporal structure
61
Code for Space. Science, New York , 261 , 1055–1058. Wilson, H.R., and Cowan, J.D. (1974) Excitatory and inhibitory interactions of localized populations of model neurons. Biophysical Journal , 12 , 1–24. Wise, S.P., and Kurata, K. (1989) Set-related activity in the premotor cortex of rhesus monkeys: effect of triggering cues and relatively long delay intervals. Somatosensory and Motor Research , 6, 455–476. Xiang, Z., Huguenard, J.R., and Prince, D.A. (1998) Cholinergic switching within neocortical inhibitory networks. Science, New York 281, 985–988. Xing, J., and Gerstein, G.L. (1996) Networks with lateral connectivity: I. Dynamic properties mediated by the balance of intrinsic excitation and inhibition. Journal of Neurophysiology , 75, 184–199. Zurita, P., Villa, A.E.P., de Ribaupierre, Y., de Ribaupierre, F., and Rouiller, E.M. (1994) Changes of single unit activities in the cat’s auditory thalamus and cortex associated to different anesthetic conditions. Neuroscience Research . 19 , 303–316.
2 Cross-Correlograms for Neuronal Spike Trains. Different Types of Temporal Correlation in Neocortex, their Origin and Significance Lionel G. Nowak 1 and Jean Bullier 2 1. Section of Neurobiology, Yale University School of Medicine, C303 Sterling Hall of Medicine, 333 Cedar Street, New Haven, CT06510, USA. e-mail: [email protected] 2. Jean Bullier: Centre de Recherche Cerveau et Cognition, CNRS-UPS UMR 5549, 133, route de Narbonne, 31062 Toulouse Cedex email: [email protected]
After a presentation of the method of computation of the cross-correlation histogram, the major interpretations of cortical synchronization are reviewed. These include indicators of different brain states (sleep, waking state), indication of cortical connectivity, participation in cell assemblies, and a different form of neural coding. It is then argued that there are several types of synchronization, reflected in the existence of CCH peaks of different widths. The function of these different forms of synchronization in cortical processing and their possible underlying generation mechanisms are discussed. KEYWORDS: Synchronization; Cell assembly; Binding; Cortex; Vision; Crosscorrelogram Synchronization of neuronal activity is a topic that has recently received a lot of interest. The major reason for this is certainly related to the publication of experimental results that supported the idea that the relative timing of action potentials between different neurones could play an essential role in perception. The enthusiasm raised by these experiments has not been shared by the whole community, and a number of criticisms have been made. Scientific controversies are often related to a misunderstanding of certain definitions. In our case, the essential question is: what does synchronization really mean and imply? This chapter is an attempt to examine synchronization in depth. First, we recall the basic principles of the cross-correlation technique, as it is the method used most frequently for tracking synchrony at the level of action potentials. Next, we present a historical perspective on synchronization, which leads to the realization that the meaning of synchronization has varied a lot at different periods. In a third section we show that one should not talk about synchronization as a
Time and the brain
64
single factor, but about several different types of synchronization. Having acknowledged this basic fact, we attempt to review the possible mechanisms that lead to the generation of these different forms of synchronization. Finally, we review recent results and the current status on the functional roles of synchronization.
1. THE CROSS-CORRELATION HISTOGRAM AS A TOOL The technique that has been used most commonly to reveal the temporal coherence in the firing of cortical neurones is to compute a cross-correlation histogram (abbreviated as CCH, also known as cross-correlogram) from their spike trains. Although the crosscorrelation technique had been used before, Perkel et al. (1967b) were the first to present formally its computation and interpretation: consider two cells that are recorded extracellularly (A and B in Figure 1). The CCH computed from the spikes emitted by neurones A and B is an estimate of the probability of firing of neurone B as a function of the time before or since a spike was fired by neurone A. This estimate is calculated by placing each spike of neurone B in a given time bin depending on its difference in time of emission from that of a spike in neurone A. It is important to note that in a CCH, a spike from neurone B usually appears in several bins, since the intervals between this spike and all the spikes of neurone A are taken into account. However, each interval between a spike in neurone A and a spike in neurone B is counted only once. Thus, a CCH is best described as a histogram of intervals between spikes-times in two neurones. Different patterns of connectivity can give rise to different forms of temporal coherence, and hence to different shapes in a CCH. We schematically illustrate these in Figure 1, considering the case of 2 neurones recorded during spontaneous activity.
Cross-correlograms for neuronal spike trains
65
Figure 1 . Typical shapes of cross-correlation histograms (CCH). Neurones A and B produce the spike trains 1 and 2. The CCH gives an estimate of the probability of a spike in neurone B at time t before or after a spike in neurone A. Uncorrelated neurones give a flat CCH. A single sharp displaced peak is the signature of a monosynaptic drive. A peak centered on the origin corresponds to synchronous firing that is usually interpreted as resulting from a common drive.
If they share no direct or indirect link, the CCH computed from the activities of the two cells (A and B in Figure 1) appears flat. This is interpreted as evidence that the cells fire independently (Figure 1a). A monosynaptic excitatory connection between 2 neurones appears in the CCH as a peak displaced from the origin of time by a latency corresponding to the sum of the conduction time in the axon and the synaptic delay (Figure 1b). In the vast majority of brain structures, not all spikes in cell A trigger a spike in cell B; conversely, not all spikes in B are triggered by a spike in cell A. As a consequence, the CCH peak emerges from a baseline probability level corresponding to those spikes in B that were not related to the spikes in A. In other words, those spikes that occurred independently in the two cells. The strength of the connection can then be estimated by different means, for example as the percentage of spikes in the presynaptic neurone that are related to spikes in the postsynaptic neurone (the “effectiveness”; Levick et al., 1972); as the percentage of spikes in the postsynaptic neurone that are related to spikes in the presynaptic neurone (the “contribution”; Levick et al., 1972); as the peak area normalized by the area under the peak (the “mean percent increase”, or “MPI”; Cope et al., 1987); as the peak height normalized by the CCH baseline (the “RMA”; Engel et al., 1990); as the number of
Time and the brain
66
spikes contained in the peak (the “asynchronous gain”; Abeles, 1991); etc. A monosynaptic inhibitory connection produces a dip in the CCH, provided the inhibitory postsynaptic potential (IPSP) is strong enough to reduce the firing probability in neurone B and that there is enough activity to be reduced by the IPSP (Aersten and Gerstein, 1985; Melssen and Epping, 1987). If the two cells are driven at the same time by a common presynaptic neurone (assuming homogeneous axonal conduction times and synaptic delays), they would tend to fire at the same time, which, in the CCH, results in a peak centered on the time origin (Figure 1b). The two cells in this case are said to synchronize their activities. Finally, a broad peak should appear in the CCH if the two neurones are simultaneously inhibited by a third neurone. This might appear counter-intuitive at first sight, but can be explained by the following reasoning: On average, the common inhibition tends to decrease their firing at the same time. When inhibition is released, the 2 cells increase their firing rate at the same time, hence the broad peak centered on the time origin. Sensory stimuli are often used to activate neurones, especially in regions of the cortex where spontaneous activity is too low to accumulate enough spikes to compute CCHs in a reasonable time. However, the use of a stimulus introduces a complication. In that case, the presence of a peak in the CCH can have two different origins: one corresponds to the stimulus-locked temporal coherence. It reflects the fact that the two neurones under study are activated with a constant temporal relationship by the sensory stimulation. This is exemplified in Figure 2 for the case of two neurones recorded, one in area V1 and the other in area V2 of the macaque visual cortex. The spike trains are shown schematically in Figure 2a. Figure 2b shows the peri-stimulus time histograms of the responses of the two cells to the onset and the offset of a flashing stimulus. The stimulus induced an increase in firing rate in both cells, at roughly the same time, and for a comparable duration. This coincident increase in firing rate contributes to the peak in the CCH (Figure 2c1, raw CCH). The delay and the size of the peak corresponding to the stimuluslocked temporal coherence depend on the
Cross-correlograms for neuronal spike trains
67
Figure 2 . Examples ofCCH and methods of the shift predictor. Recordings from units in areas V1 and V2 of the macaque monkey. A: examples of spike trains, as recorded (above) and after shifting the V2 spikes by one stimulus period (below). B: Post-stimulus histograms (PSTHS) of the responses of the two neurones to a small stimulus flashed in the receptive fields. The light is ON at time 0 and OFF at 3 s. C1: CCH computed from the normal spike trains (raw CCH) and after shifting the V2 spike train (Shift predictor), for the parts of the spikes trains corresponding to visual responses in the CCH. The deviation of the raw CCH from the Shift predictor indicates the presence of neural coupling. C2: Subtracted CCH and side peak. The subtracted CCH is the difference between the raw CCH and the shift predictor. The side peak is obtained by subtracting two shift predictors, one shifted by one and one shifted by two stimulus periods. It gives an indication of the amount of variability of the CCH trace. D1: same calculation as for C1 except that the spikes come from the period with no stimulus. D2: same as C2 for spontaneous activity (no stimulus). Note that a broad peak (H type) is observed in spontaneous activity whereas an intermediate peak (C type) is observed during visual stimulation.
latency difference in the activation of the neurones by the stimulus, as well as the duration of their responses. The other form of temporal coherence observed with sensory stimuli is the temporal
Time and the brain
68
coherence of neuronal origin. This form of coherence results from any of the neuronal interactions described above (Figure 1). To distinguish between the 2 forms of temporal coherence it is common to use a shift (or shuffle) predictor method. This method was first presented by Perkel et al. (1967b): a sensory stimulus, if repeated with a fixed periodicity, induces a periodic increase in firing rate (Figure 2a). The effect of the sensory stimulus, therefore, can be predicted, in isolation, by computing another CCH (called the shift predictor) after shifting one spike train with respect to the other by an amount of time corresponding to one (or several) periods of the stimulus (Figure 2a). Usually, the periodicity of the stimulus is several seconds. The temporal coherence of neuronal origin, on the other hand, exists only for a limited amount of time, most commonly below 10 msec in the case of fast synaptic transmission, and generally below 1 second in the case of slower (metabotropic) interactions. Therefore, by comparing the original CCH (the raw CCH) with its shift predictor it is possible to determine what is produced by the stimulation and what is due to the neuronal interactions (Figure 2c1. The subtracted CCH, corresponding to the subtraction of the shift predictor from the raw CCH, is used to characterize the temporal coherence of neuronal origin in isolation (Figure 2c2). Experimental studies were then designed to test the prediction from the theoretical study by Perkel et al. (1967b). Moore et al (1970); and Bryant et al. (1973) applied the cross-correlation technique to the study of the functional connectivity in Aplysia. The peaks observed in the CCH could be ascribed to different forms of synaptic interactions, which were directly inferred by virtue of dual intracellular recording for the same cell pairs. They showed that, indeed, a monosynaptic excitatory connection produces a displaced peak, and that a common input produces a peak centered on the origin of time. Modeling studies (Knox, 1974), and studies in the spinal cord (Kirkwood and Sears, 1978, 1982b; Fetz and Gustafsson, 1983; Gustafsson and McCrea, 1984; Cope et al., 1987) and recently in the cortex (Matsumura et al., 1996), attempted to establish a relationship between the shape of an excitatory postsynaptic potential (EPSP) and the shape of the CCH. The shape of the CCH does not reproduce the shape of the underlying EPSP. Instead, it is usually found that the shape of the peak in the CCH is similar to the temporal derivative of the EPSP; in other words, the width of the CCH peak is proportional to the rise time of the EPSP. Departure from this relationship is observed in cases of EPSPs of small size compared to the fluctuations of the membrane potential. In such cases the shape of the CCH peak can be reproduced by a linear combination of the EPSP shape and its derivative. Moore et al. (1970) and Bryant et al. (1973) showed further that the shape of the CCH peak is determined not only by the connectivity pattern, but also by the firing pattern of the neurones under study. This pattern is revealed by computing the autocorrelogram (ACH) of the neurone activity (cf. Perkel et al., 1967a). It is done in the same way as the CCH except that the ACH is a histogram of all the intervals (with or without intervening spikes) between the spikes recorded in one neurone. Moore et al. (1970) and Bryant et al. (1973) distinguished between the primary effects, which result from the synaptic interactions, and the secondary effects, which result from the firing pattern of the neurones under study. The ACH can show that some neurones fire action potentials that are not independent of each other. For example a neurone might be bursty, in which case
Cross-correlograms for neuronal spike trains
69
one action potential will usually be followed by the other action potentials of the burst. If a postsynaptic neurone shows a bursty firing, then the CCH will present a broader peak, because a spike in neurone A is followed on average by a burst in neurone B. Another case corresponds to an oscillatory common input neurone: the oscillatory firing will be replicated symmetrically in the CCH, leading to an oscillatory structure. Although the impact of the firing pattern of the cells was recognized in these early papers, many crosscorrelation studies have neglected to take it into account when interpreting the various shapes and widths shown by the peaks in the CCHs. We return to this question in Section 4.
2. SYNCHRONIZATION AND ITS MEANINGS: AN HISTORICAL PERSPECTIVE The cross-correlation technique has been used to characterize the neuronal interactions that take place within the central nervous system. In the cerebral cortex, it has shown that neurones tend to synchronize their activity in the majority of cases. However, this basic result has been open to multiple interpretations. To show how the interpretation of synchronization has evolved in parallel to the conceptual field within which it has been studied, we now trace the history of synchronization in the cortex. 2.1. Awakening and Desynchronization Synchronization of neuronal firing was first associated with sleep and anaesthesia, and was supposed to disappear when subjects or animals were awake. This notion can be traced back to the first studies on electroencephalographic activity, initiated by Berger, in which rest, sleep and anaesthesia were found to be associated with EEG traces showing large-amplitude slow waves, that appeared coherent over large distances. In contrast, awakening was associated with lower amplitude events occurring at a higher frequency, and usually independently at the different recording sites (e.g. Adrian and Matthews, 1934). This led to the hypothesis that the underlying neuronal activity should be synchronized in anaesthesia and sleep, while sensory stimulation and awakening would disrupt and suppress this synchronization. Adrian (1935), for example, stated in 1935 that “[…] if a large mass of nerve-cells is to pulsate with a co-ordinated beat it is clear that all of them must tend to pulsate at the same frequency. If afferent messages excite some to a higher frequency the co-ordinated beat must break down; and so these cortical rhythms are broken up by external stimulation, to return again when the animal is undisturbed”. Or, said in another way by the same author, “[…] groups of nerve-cells very often tend to act in unison when there is nothing to prevent them”. Hence, the first studies in which dual extracellular recording of action potentials was performed in cortex were designed to test the hypothesis that neurones synchronize their
Time and the brain
70
activity during anaesthesia and sleep, while sensory stimulation and awakening suppress this synchrony. Li (1959), recording from pairs of neurones in the somatosensory cortex of cats anaesthetized with a barbiturate, found that spontaneous activity is made up of simultaneous bursts of 5 to 10 action potentials when the animal is deeply anaesthetized. The tendency of neurones to discharge simultaneously tended to decrease as the anaesthesia level was reduced. Holmes and Houchin (1966) conducted a study in urethane anaesthetized rats, and constructed the equivalent of a CCH to detect synchronization of neuronal activity. They found it to be more prevalent in deeply anaesthetized animals, and observed it even for cells that were separated by several millimeters on the cortical surface. Synchronization was reduced, or suppressed, when the anaesthesia was lighter. They also used “strong peripheral stimuli” (which presumably were more like awakening stimuli than subtle sensory stimuli), and found that these also decreased the synchronization. Noda and Adey (1970) recorded the ongoing activity in the suprasylvian gyrus of unanaesthetized, behaving cats. They found that pairs of neurones strongly synchronize their activity during sleep, while synchronization, for the same pairs of neurones, was strongly reduced, if not completely absent, during arousal and rapid eye movement sleep. Noda and Adey (1973) completed their study by testing the action of the anaesthetic pentobarbital on neuronal activity and synchronization. The anaesthetic increased the synchrony and the burstiness of the neurones to a level comparable to that observed during sleep. With this anaesthetic, the bursts of action potentials occurred mostly in coincidence with the spindle waves appearing in the EEG. Burns and Webb (1979) added another argument in favour of a desynchronizing role for afferent sensory stimuli: They managed to deafferent pieces of cortex and to record from them. The neurones in these deafferented cortices synchronized their activities in all the cases, independently of the behavioural state of the cat, while the non-deafferented cortex only rarely showed signs of synchronization, especially when the animal was awake. As a conclusion for this section we quote Burns and Webb (1979): “[…] it is not unreasonable to state that when the brain is processing information, neighbouring cortical neurones are very unlikely to be found doing the same thing at the same time.” Hence experimental results supported the hypothesis of Adrian that neuronal activity is strongly synchronized during sleep and anaesthesia, while synchronization would be absent or at best strongly reduced in the awake brain. However, these studies can be criticized on one point, in that, in most cases, the neurones were studied during spontaneous activity; the effects of the sensory stimulation, if studied, were only poorly controlled. 2.2. Neuronal Connectivity and Common Inputs Within a different conceptual framework, inherited from the study by Perkel et al. (1967b), cross-correlation has been used as a method to provide data on the connectivity between neurones. In this context, what was studied was not the relationship between synchronization of neuronal firing and the behavioural state of the animal; instead, crosscorrelation was used as a tool to test for the presence of synaptic relationships between extracellularly recorded neurones.
Cross-correlograms for neuronal spike trains
71
Compared to other methods, calculating an index of cross-correlation between the firing of two neurones would appear as a better alternative to study their connectivity: the tracing of anatomical pathways says nothing about the physiological properties of the labelled neurones or their target cells. Electrical stimulation, as a method of identifying neurones that are targeted by a given structure, has a number of drawbacks, including the possible activation of axonal tracts issuing from other structures and passing near the stimulating electrode, or the problem of the activation of axon collaterals (e.g. Nowak and Bullier, 1998a,b). One powerful method consists of performing dual intracellular recordings, allowing the physiological and morphological identification of both the preand postsynaptic neurones involved in a synaptic connection. The use of dual intracellular recording in vitro has provided a number of important data (for recent reviews see Thomson and Deuchars, 1997; Markram, 1997), but its low yield and its requirement for long lasting, stable recordings, makes it impossible to use for in vivo mammalian studies. In the mammalian visual system, CCHs displaying narrow peaks offset from the time origin, indicating monosynaptic connections, have been observed in the subcortical visual pathways. Cleland et al. (1971a,b) and Levick et al. (1972) traced the functional connectivity between the retina and the lateral geniculate nucleus (LGN). Among other things, their results showed that a retinal ganglion cell projects on a thalamic neurone with a similar receptive field subtype. They also showed that a small number of retinal ganglion cells, sometimes only one, provides most of the excitatory drive to a thalamic neurone. After these investigations, the study of the thalamocortical pathway followed, with the studies by Tanaka (1983), who showed that ON-centre thalamic neurones provide the main drive to the ON subfield of cortical simple cells. The corresponding CCH peaks, a few msec wide, were displaced from the time origin by a latency 0.9 to 2.7 msec. Tanaka (1983) also showed that, on average, one spike in a thalamic neurone increases the discharge probability of a cortical simple cell by 10%. These results were recently confirmed by Reid and Alonso (1995), although the strength of the thalamic input appeared lower in their study, presumably due to the use of different visual stimuli (see also Swadlow, 1995, for the case of the somatosensory thalamus and cortex). The cross-correlation technique has also been used to study synaptic connectivity between cortical neurones recorded within the same area. After a pioneering work by Griffith and Horn (1963), the first cortical studies were those by Dickson and Gerstein (1974) in the auditory cortex, by Renaud and Kelly (1974) in the motor cortex, by Toyama et al. (1981a, b) and Michalski et al. (1983) in the visual cortex. The CCHs showed significant peaks, indicating temporal coherence of neuronal origin, in roughly half the cases when the neurones were close to each other (e.g. recorded from the same electrode). The incidence of significant CCHs decreased with increasing separation between the electrodes. In contrast to what was observed for the retinothalamic and thalamocortical pathways, the significant CCHs rarely displayed features indicating the presence of a monosynaptic connection: Displaced peaks and displaced troughs were observed in only 5–10% of the cases, and only when the neurones were located close to each other. Instead, most of the CCH peaks were centred on the time origin when neurones were close to each other, and only centred peaks were
Time and the brain
72
observed when they were distant. In other words, the activity of two simultaneously recorded cortical neurones appeared synchronized in the vast majority of the cases even when sensory stimuli were used to activate them. It was then inferred that such synchronized activity reflects the presence of common inputs (Figure 1), from other cortical areas or from subcortical structures. Gerstein and Dickson (1974) concluded that the connectivity between cortical neurones is made by an “arrangement that consists mostly of shared inputs.” 2.3. Functional Connectivity, Synchronization and Cell Assemblies Although cross-correlation was once thought to be a method for tracing connections in vivo, this hope had to be abandoned. The modelling study by Boven and Aertsen (1990) illustrates how a simple monosynaptic connection between two neurones can give rise to different CCHs, depending on the activity of the other neurones in the network. In their model, a neurone 1 projects to a neurone 2, with a relatively small synaptic strength α (Figure 3). For its firing to be influenced by that of neurone 1, it is necessary for neurone 2 to have its membrane potential close enough to the spike threshold, such that the relatively small synaptic events produced by neurone 1 would lead to an action potential. The membrane potential of neurone 2 is determined by the activity in the population of neurones (N) projecting onto it. If the activity in this population of neurones is low, then the membrane potential of neurone 2 would not be depolarized enough to allow the EPSPs produced by neurone 1 to trigger an action potential. As the activity in the population N increases, the probability (called effective connectivity—α’ in Figure 3) that an EPSP produced by neurone 1 leads to an action potential in neurone 2 increases, until this probability saturates. From this we can also extrapolate that if the activity in the population of neurones (N) is high, but out of phase with respect to that of neurone 1, then the signature of the monosynaptic connection will not be seen in the CCH. Furthermore, the size of the peak cannot be interpreted as an estimate of the synaptic strength α’, since it reflects the combined effects of the connection between neurones 1 and 2 and the activity of the network in which they are embedded. At this point, “one should distinguish between structural (or anatomical) connectivity on one hand and functional (or effective) connectivity on the other” (Aertsen and Preiβ1, 1990). The lack of a peak is not proof of a lack of connectivity. Conversely, the presence of a peak in a CCH computed between two neurones indicates not only the presence of a synaptic connection. It also means that the whole cortical network was in a state that allowed the expression of this connection… The appearance and disappearance of centred peaks can be envisaged simply in a situation where one common input neurone, C, projects to two neurones A and B (cf. Figure 1c). Neurones A and B may also receive a number of inputs they do not share, such that their activity with different stimuli might be of comparable strength. Neurone C could be activated by only one stimulus configuration, and only in this case
Cross-correlograms for neuronal spike trains
73
Figure 3 . Relationship between strength of effective connectivity α’ as a function of network activity Nl for different values of the synaptic weight β between the pool of neurones N and neurone 2. Neurones 1 and 2 are related by a monosynaptic connection of synaptic weight a. Nλ is the mean firing rate of the neurones in the pool N driven by the stimulus S. α’ is the ratio of the peak area in the CCH to the number of presynaptic spikes (equivalent to the “contribution” index of Levick et al., 1972). In all cases there is a strong dependence of the effective connectivity upon the activity of the network and a saturation for high activity levels. Note that the variation of the effective connectivity is much larger for small values of the network activity, which is likely to be similar to the case in the cortex, from Boveu and Aertoe (1990)
would a centred peak be visible in the CCH. The anatomical wiring is always present, but may not be expressed in all cases. Obviously the situation in the cortex is far more complex, where one neurone receives several thousands of synapses from neurones located in different cortical and subcortical structures, all with different stimulus selectivities. In support of these theoretical considerations, several experimental studies have shown that the temporal coherence of neuronal origin observed in the spike trains from the same neurone pair depends on the stimulus used to activate them, as well as on the behavioural
Time and the brain
74
context. In the auditory cortex, Frostig et al. (1983) found that 88% of the CCHs were modified by the stimulus presented: some CCHs, showing displaced peaks during spontaneous activity, became flat in the presence of a stimulus while other CCHs behaved in the opposite way. Later studies in auditory cortex confirmed this strong stimulus dependency (Vaadia and Abeles, 1987; Ahissar et al., 1992a; Eggermont, 1994; de Charms and Merzenich, 1996). Changes in correlated firing under different stimulus conditions have also been observed in subcortical structures (e.g. Epping and Eggermont, 1987; Sillito
Figure 4 . Oscillatory coupling between area 17 neurones of opposite cortical hemispheres. Decomposition of oscillatory correlation over peristimulus time. Right and left hemisphere multiunit signals are
Cross-correlograms for neuronal spike trains
75
correlated in response to a bar moving for 3 s, stationary for 5 s and moving back for 3 s. The receptive fields were partially overlapping and had optimal orientations differing by 45 degrees. A: The net peri-stimulus CCH (PSCCH) shows oscillations relative to time difference, with the position of the peak shifting slightly during the movement of the bar, and shifting in the other direction when the bar returns. Lowest panel shows PSTHs for right hemisphere (shaded) and left hemisphere (outlined). Note the high level of spontaneous activity between stimuli. Middle panel shows sections along the PST dimension for time difference windows of −3 to 3 msec (shaded) and 10 to 16 msec (outlined). Right hand panel shows CCHs obtained by summing over PST dimension. B: Array of equivalent Z-values indicating the statistical significance of the coupling. The middle panel shows profiles of the significance along the PST dimension, for time differences at the central peak (shaded) and at a negative lobe (outlined), along with the 2.5% one-sided significance level (dotted) for positive coupling (Z=1.96) and negative coupling (Z=−1.96). Coupling is highly significant during movement of the bar (Z=+/−5), while dropping below significance between movements. The panel to the right shows sections along the time-difference axis of mean Z value for PST in windows 10.3 to 11.3 seconds (shaded), and in the interstimulus period, for PST from 5 to 9 seconds (outlined) along with the one-sided 2.5% significance leves (z=+/−1.96; dotted line). The windows for the horizontal and vertical sections are indicated by brackets on their respective axes. Several lobes of the oscillatory profile during stimulation are significant, while the interstimulus profile is well below significance, in spite of high levels of spontaneous activity, from Nowak et al (1995)
et al, 1994). In frontal and prefrontal cortex, the correlated firing between pairs of neurones can also change in relation to behaviour, with a time course of a fraction of a second (Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995; Riehl et al., 1997). In the visual cortex, the first description of a change in synchronization related to a change of the stimulus-configuration can be found in the paper by Ts’o et al. (1986). They showed an example, where the CCH peak height decreased when the neurones under study were activated with 2 bars with optimal orientation, rather than with one single bar with an orientation in between the two optima. In the visual cortex, stimulus-dependent changes of synchronized activity have been investigated mostly for one form of synchronization, the synchronization of gamma range (between 20 and 80 Hz) oscillation initially described by Gray and Singer (1989) and Eckhorn et al. (1988). One example is shown in Figure 4, for the case of interhemispheric synchronization in cat. These synchronized oscillations are observed under stimulation by a moving stimulus, but not during spontaneous activity (Figure 4). Most importantly, they depend on the configuration of the stimuli in such a way that they are more prominent for one continuous stimulus activating groups of cells together and coherently, than for different stimuli activating the different neuronal populations separately in space and/or
Time and the brain
76
time (Gray et al., 1989; Engel etal., 1991c). In summary, these results show that the temporal coherence in the firing of two neurones is not invariant; instead it depends on the presence of one type of stimulus, and/or on one phase of the behaviour in which the animal is engaged. Therefore crosscorrelation is not simply an anatomical method. However, cross-correlation of spike trains can be used to show that, at a particular moment and in presence of a given stimulus, one cell can be linked to a larger group of cells, and that this link might disappear in other conditions. The acknowledgement of the specificity of this link brought the cross-correlation technique into another conceptual field, that of the cell assembly. The initial concept of the cell assembly was proposed by Hebb (1949) as a link between neuronal activity and behaviour (for other types of cell assemblies see also Gerstein et al., 1989). Hebb, as well as most contemporary neuroscientists, did not envisage a one-to-one relationship between the activity of one neurone and one perception or one behaviour. Instead perception and motor activity had to be based on the activity of pools of neurones (or “population coding”). The novelty of Hebb’s theory resided in the involvement of the connectivity between neurones, and most importantly in the functional expression of this connectivity, as the neuronal basis of perception and behaviour. He defined the cell assembly as: “a diffuse structure comprising cells in the cortex and diencephalon (and also, perhaps, in the basal ganglia of the cerebrum), capable of acting briefly as a closed system, delivering facilitation to other such systems and usually having a specific motor facilitation” (Hebb, 1949, p. XIX of introduction). Hebb (1949) also conceived that memory (in a broad sense) and perception are two aspects of the same phenomenon, supported by the same cell assemblies. Establishment of a memory trace would proceed in two steps: First, reverberation of activity within the cell assembly would provide a physiological basis for a transient memory of the stimulus. Second, this transient memory can become a long lasting one by means of structural changes. The way by which such structural changes could occur is Hebb’s famous rule, according to which the coincidence of pre- and postsynaptic activities leads to a strengthening of the synapse. Strengthening of synapses can indeed be obtained by pairing pre- and postsynaptic activities, as has been established in several studies, both in vivo and in vitro (Baranyi and Szente, 1987; Gustafsson et al., 1987; Frégnac et al., 1988; Bonhoeffer et al., 1989; Ahissar et al., 1992b; Frégnac et al., 1994b; Jester et al., 1995; Markram et al., 1997). Hebb conceived that some basic perceptual capacities are already present at birth. Some inherited neuronal mechanisms therefore would provide the basis for a primary form of segmentation and figure-ground segregation: “The primitive unity of the figure is defined here as referring to the unity and segregation from the background which seems to be a direct product of the pattern of sensory excitation and the inherited characteristics of the nervous system on which it acts” (Hebb, 1949, p. 19).
Cross-correlograms for neuronal spike trains
77
If enough repetitions of the same pattern occur, activations in these low levels would be combined and would lead to the generation of a cell assembly, supposedly taking place in association areas. Once established, the cell assemblies would provide support for the identity of a perceived figure and for the permanence of learning. One important feature of the cell assembly is the presence of convergence and divergence in the projection scheme. This provides one means by which a cell can be activated by different presynaptic elements, which in turn enables pattern completion: presentation of one part of a figure would activate the whole assembly, hence creating the internal representation of the whole. The associational properties of the cell assemblies would also enable the property of figure invariance (the fact that the recognition of an item is independent of the viewing distance, illuminating condition, viewing angle, etc.) and the property of generalization (the ability to recognize a whole class of items after familiarization with a subpopulation of it). The role of temporal coherence in the assembly was not unambiguous in Hebb’s words. On the one hand, temporal correlation of spike activities appears necessary for the strengthening of synapses. It is also clear that once the strengthening of synapses between two neurones has taken place, these will not act independently anymore: “A functional relationship of activity” should result. “The fundamental meaning of the assumption of growth at the synapse is in the effect this would have on the timing of action potentials by the efferent cells” (Hebb, 1949, p. 72). Using present day terminology and technology, this should appear as a displaced peak in a CCH. On the other hand, Hebb explicitly rejected the importance of a tight temporal coherence for the functioning of the assembly: “At each synapse there must be a considerable dispersion in the time of arrival of impulses, and in each individual fibre a constant variation of responsiveness; and one could never predicate a determinate pattern of action in any small segment of the system. In the larger system, however, a statistical constancy might be quite predictable” (Hebb, 1949, p. 76). There is one important limitation in Hebb’s theory. The paradox is that this limitation results exactly from what makes the cell assembly such an attractive theory, and can be traced back to the anatomical basis of the assembly: neighbouring neurones are strongly interconnected, and this presumably results from the application of Hebb’s rule, since neurones located close to each other are activated together by sensory stimuli. The presence of this net-like structure, with its convergence and divergence, creates a multipotentiality by virtue of which one neurone can belong to different assemblies and generalization and completion can be realized. However, when two, instead of one, figures are presented simultaneously and close to each other, it is likely that some neurones will be activated by both of them, such that the resulting increase in activity in these neurones cannot be traced back to one or the other figure: the cell assembly system is not able to encode more than one object without confounding them. This ambiguity as to what figure induced an increased activity in which assembly, was pointed out by von
Time and the brain
78
der Malsburg (1981) and is known as the “superposition catastrophe”. Hence there must be some mechanism that maintains different assemblies, segregated from one another. In terms of anatomo-functional connectivity, there must be a selection process, by which neurones will belong to one assembly or another, but not to two assemblies at the same time. Synchronization of neuronal firing appeared as one possible mechanism by which this selection procedure might be performed. A role for synchronization in this process of selection was first proposed by Milner (1974), who was concerned with the way that even elements as simple as lines can be perceived as different entities: “[…] we must attempt to explain how lines are immediately seen as units or ‘wholes’. One possibility worth considering is that primitive unity depends upon the contiguity of active cells in topographically organized parts of the projection pathway. […] If adjacent, or nearly adjacent, cells interact when excited, in such a way as to synchronize and perhaps intensify each other’s activity, this could provide the unifying characteristic that ties the elements of a figure together. At subsequent levels of the pathway, impulses from cells fired by one whole would arrive as synchronous volleys, whereas impulses from different figures would have a random temporal relationship to each other. The resulting difference in the effectiveness of temporal summation could serve to funnel the signals from different objects in the field into different paths, through the feature detection network, and thus minimize their mutual interference”. 1 A role for synchronization (or using a broader term, “correlation”) in figure-ground segregation was also proposed by von der Malsburg (1981) for whom “a figure is decomposed in parts, the strongest correlations signalling affiliation to one part of the figure, 1. Recent modeling studies by Carlos Brody (J. Neurophysiol. 80:3345–3351; Neural Comput. 11:1527–1536; Neural comput. 11:1537–1551) have shown how artefactual peaks can be obtained in cross correlation histograms, that are the consequences of 1) slow covariation of firing rates and 2) correlation of visual response latencies. Brody’s papers also provide indication as to how such correlations can be distinguished from those related to neuronal connectivity.
weaker ones affiliation to adjacent parts, and so on.” There is a “decomposition of the visual scene into a hierarchy of correlates”. Therefore, the use of a temporal coding, based on short term correlation, would solve the superposition problem—and by extension would provide a mechanism for segmentation and for figure-ground segregation. Neurones activated by one object would synchronize between each other, but not with the neurones activated by another object. Obviously the temporal segregation of neuronal activities can be more efficient if neurones have the possibility of engaging in oscillatory synchronization: then one figure would have its representation in the synchronous oscillation of one population of neurones, while the other population would have its own synchronized oscillation out of phase with those of the first. This provided the basis for a modelling study in which a fully connected set of neurones was nevertheless capable of segmentation by virtue
Cross-correlograms for neuronal spike trains
79
(among other things) of synchronized oscillatory activities (von der Malsburg and Schneider, 1986). Another important question also came into focus, which is how neuronal activity is propagated from one place to another. The basic fact is that one cortical neurone produces only small depolarizations in another cortical neurone. According to in vitro studies the average size of an EPSP elicited by one pyramidal neurone upon another is less than 1 mV in the majority of the cases (for reviews: Thomson and Deuchars, 1997; Markram, 1997). The amplitude of the depolarization required for triggering an action potential in a cortical neurone is of the order of 10–20 mV. Therefore it is required that a summation of EPSPs takes place. The number of EPSPs that have to be summated is much higher when these EPSPs occur independently from each other in time, than when they occur all together, that is, in synchrony. This led to the notion that neurones can behave as coincidence detectors, rather than as temporal summators. The role of synchronization for the propagation of activity is the starting point for the synfire chain concept proposed by Abeles (1982): “According to this hypothesis the activity of the neurons that transmit information is organized along a chain of sets of neurons. Each link in the chain is made of a set of neurons that fire in exact synchrony whenever the chain becomes active. Neurons of each set converge on neurons of the next, and therefore synchronized activity of one set elicits synchronized activity in the next set and so on. […] the relevant code is the combination of neurons that fire in synchrony.” Here synchronization serves a selection procedure, for only those neurones that are synchronized have a chance to transmit their information to the next stage. “This concept of dynamic formation and dissolution of functional chains is similar to the ‘cell assembly’ postulated by Hebb thirty years ago, except for the requirement that the transmission along the chain is secured by the synchronous firing of sets of cells.” One advantage of the synfire chain, is that it allows for the existence of “cross over” such that one neurone can belong to two different chains. Even so, the independence of the two chains is preserved, as required for the segregation of cell assemblies with common neurones, as long as they are not active in exact synchrony. Experimental supports were later given for the existence of synfire chains in the awake behaving monkey (Abeles et al., 1993). The stimulus-dependent changes in synchronization observed in visual cortex (Gray et al., 1989; Engel el al., 1991c; see also Engel et al., 1991b; Ghose et al., 1994; Sillito et al., 1995; Kreiter and Singer, 1996; Livingstone, 1996; Brosch et al., 1997) are consistent with a role for synchronization in the process of segmentation, and appeared as the experimental demonstration of the theoretical ideas developed by Milner, von der Malsburg and Abeles. Hence, this gave support to the notion that “synchrony might be the ‘glue’ that binds distributed neuronal activity into unique representations” (Engel et al., 1992).
Time and the brain
80
2.4. Synchronization and Attention We have just reviewed evidence indicating a role for synchronization in scene segmentation. There are, however, other experiments showing that, in awake behaving animals, synchronization does not always have a clear relationship with sensory stimulation or motor activity. Early studies demonstrated that high frequency oscillatory events are visible, if not prominent, in the EEG traces under behavioural conditions requiring attention and vigilance: Lopes da Silva et al. (1970) observed the presence of a “beta” rhythm (around 20 Hz) above the visual cortex of dogs that were trained to press a lever when the visual stimulation (whole field, sinusoidally modulated light) was ON. This rhythmic signal was absent if the animals were anaesthetized. Rougel et al. (1979) demonstrated the presence of a fast somatoparietal rhythm during combined focal attention and immobility in baboon and squirrel monkey. The frequencies were between 13 and 25 Hz, and appeared independently in two loci (S1 and area 5). Bouyer et al. (1981) reported the presence of 35–5 Hz oscillations, with their focus located in the frontal (area 4 or 6) and parietal cortex (area 5) in immobile cats during focused attention. In these last two studies, the oscillations appeared in cortical areas where no information had to be processed—sensory and motor activities were presumably shut down during this attentive immobile behaviour. Some of the changes in synchronization observed in relation to different phases of behaviour in the frontal cortex of awake monkeys (Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995; but see Riehle et al., 1997) and in cat visual and motor areas (Roelfsema et al., 1997) might be related to changes in attentiveness. In the somatosensory and motor cortex, Murthy and Fetz (1992) observed 25–35 Hz synchronized oscillatory activity, that did not depend on any particular movement or stimulus, but instead appeared to be more prominent during a difficult task presumably requiring attention. Sanes and Donoghue (1993), in a visuo-motor task, attributed the origin of these oscillations to preparation of movements, rather than to their execution, while Baker et al. (1997) suggested these oscillations could be associated with the motor cortex “resting” between demanding motor tasks. Finally, in monkey parietal cortex, Cardoso de Oliveira et al. (1997) showed that synchronization between pairs of neurones is stronger before the presentation of the visual stimulus that the monkey has to attend to, than during its presentation. Studies in humans also indicate several possible roles for gamma oscillations. The power of such oscillations is higher when real and illusory (Kanizsa) triangles are presented than for non-coherent stimuli (Kanizsa triangles with rotated inducers), thus suggesting a role in binding for object perception (Tallon-Baudry et al, 1996). However, using similar recording techniques, the same authors demonstrated that increased power of gamma oscillations can be recorded during a task of active visual search (TallonBaudry et al., 1997) and during the expectation period in a delayed match to sample task (Tallon-Baudry et al., 1998). These results suggest that gamma oscillations may be involved in the recollection of stored representations and in visual attention (see also Llinás et al., 1994).
Cross-correlograms for neuronal spike trains
81
2.5. Summary Four different interpretations have been advanced for the presence of synchronization in the cortex. First, it has been proposed that the synchronization of action potentials between pairs of neurones reflects the EEG waves that characterize the brain in an unconscious state—during sleep or when anaesthetized. Second, synchrony has been considered to be the manifestation of a connectivity pattern typical of the cortex, in which a pair of neurones receives common inputs from another group of cells. Third, it has been proposed that the synchronized firing of two cells indicates their participation in a cell assembly. From this it was proposed that synchronization constitutes another neuronal code, a new dimension for the neuronal representation of sensory inputs and motor outputs, to be added to the rate coding. Finally, it has been shown that synchronization of neuronal activity can occur with no relationship to sensory input or motor output, and reflects the state of the cortical network when the subject is in a state of expectancy or attention.
3. DIFFERENT TYPES OF SYNCHRONIZATION We have reviewed the accumulated evidence that synchronization is the form of temporal coherence that dominates when recording from pairs of neurones in cortex. So far we have shown that synchronization has received different and sometimes contradictory interpretations. However, different interpretations for the presence of synchronized firing may not be contradictory if there is not one, but several forms of synchronization. We will now show that, indeed, different forms of synchronization exist. 3.1. Non-oscillatory Synchronization within One Cortical Area As mentioned earlier, the large majority of CCH peaks obtained from neurones recorded in the same cortical area reflects synchronization of neuronal activity. Different types of synchronization can be distinguished on the basis of the width of the CCH peaks. One common finding among the earliest studies on cortical synchronization, when it was studied in relationship to sleep and anaesthesia, is the presence of rather large CCH peaks. The peaks presented by Holmes and Houchin (1966) were several tens or hundreds of msec wide; the peaks presented by Noda and Adey (1970, 1973) were similarly quite wide on average, although a large variability could be observed in their data, such that narrow peaks a few msec wide as well as very large peaks, several hundred msec wide, were reported. In contrast, in those reports in which neuronal interactions were studied within the context of neuronal connectivity, with sensory stimuli or motor responses, the widths of the peaks often appeared to be narrow. In cat visual cortex for example, Toyama et al. (1981a) reported the presence of very narrow centered peaks 1 msec wide, while
Time and the brain
82
Michalski et al. (1983) reported centered peaks with widths ranging from 10 to 120 msec. Several studies showed that two or three different types of peaks can be distinguished, when attempts have been made to classify them according to their widths (Murphy et al., 1985; Krüger and Aiple, 1988; Krüger and Mayer, 1990; Aertsen et al., 1991; Gochin et al., 1991; Hata et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992). Compilation of these results suggests a relatively clear segregation between three types of CCH: Some narrow peaks, with a width less than about 10 msec when measured at half width; a second class of peaks, (intermediate peaks), with widths between approximately 10 msec and a few hundred of msec, and broad peaks with widths broader than one hundred msec. These different types of peaks have been reported both in anaesthetized animals and in awake behaving monkeys. As a rule, the three different peak types are observed for neurones that are close to each other (for example, recorded from the same electrode), while mostly the intermediate and broad peaks are detected when neurones are separated by more than a few hundred microns and up to several mm (Murphy et al., 1985; Krüger and Aiple, 1988; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992; Cardoso de Oliveira et al., 1997). In primary visual cortex, orientation selectivity is another property that appears to be related to the incidence of different peak types: Narrow peaks are seen more often when neurones share a similar orientation preference, especially if the receptive fields do not overlap (Krüger and Aiple, 1988; Schwartz and Bolz, 1991; Hata et al., 1991; Livingstone, 1996; Tamura et al., 1996). Notice, however, that the probability of observing a peak in CCHs between cross-oriented or iso-oriented pairs of neurones might also depend on the type of stimulus configuration used to activate the cells (Sillito et al., 1995). In primate area V2, Tamura et al. (1996) did not observe any relationship between orientation selectivity and the probability of observing narrow peaks. The occurrence of intermediate peaks does not depend strongly on orientation selectivity (Krüger and Aiple, 1988). 3.2. Non-oscillatory Synchronization between Different Cortical Areas CCHs have been computed for neurones recorded in different visual areas. The incidence of significant peaks, indicating interactions of neuronal origin, seems to be lower than for within-area recording. The only study in which these incidences have been compared under the same conditions (Cardoso de Oliveira et al., 1997) shows a drop from 46 to 15%. In our study on inter-areal interactions in monkey visual cortex, significant peaks were observed in only 10% of the dual single unit recordings. In the other 90% of cases the subtracted CCH appeared flat, indicating the absence of neuronal interactions in the great majority of the cases (Nowak et al., 1999). The coupling strength also seems to be much weaker for between- compared to within-area CCHs (Ana Wang Roe, personal communication). Similar to reports for within-area recordings, the CCHs showed peaks centred on the time origin in the very large majority of the cases (Nelson et al., 1992; Nowak et al., 1995; Wang Roe and Ts’o, 1997; Cardoso de Oliveira et al,, 1997; Nowak et al., 1999).
Cross-correlograms for neuronal spike trains
83
Therefore, neurones belonging to different areas tend to synchronize their activity through mechanisms of neuronal origin. On the basis of the width of the peaks, synchronization between different areas can also be subdivided in three different groups. The tripartite distribution of peak width is illustrated in Figure 5 for three different studies in two animal species: Interactions between areas 17 and 18 in the cat (Nelson et al., 1992), interactions between areas 17 of opposite hemispheres in the cat (Nowak et al., 1995), and interactions between areas V1 and V2 of the same hemisphere in the monkey (Nowak et al., 1999). The tripartite distribution of peak widths becomes quite obvious when one makes histograms of the distribution of the logarithm of the peak width, as shown in Figure 5. From one study to another, there is some variability in the exact positions of the modes and of the gaps between the different groups of peaks. Nevertheless, it is quite clear that inter-areal neuronal interactions produce some narrow peaks, which we called T (“tower”) peaks, with width of a few msec up to approximately 15 msec; some intermediate peaks, which we called C (for “castle”) peaks, with widths between 10–20 msec and 100–200 msec (one example of a C peak is shown in Figure 2C2); and finally, some broad peaks (H peaks, “hills”), with width of several hundred msec up to 2 sec (one example is shown in Figure 2D2). Similar to what has been reported for intra-areal synchronization, the incidence of the different peak types observed in the inter-area studies depends on the separation of the neurones (in this case, not in terms of physical separation but in terms of receptive field separation) and on their orientation selectivities. T peaks are preferentially obtained when neurones have overlapping receptive fields and similar orientation preferences (Nelson et al., 1992; Nowak et al., 1995). C peaks can be observed even if the receptive fields do not overlap; however, the receptive field separation for which C peaks are observed appears bounded by an upper limit of 10 degrees in cat (Nelson et al., 1992; Nowak et al., 1995) and 5 degrees in macaque (Nowak et al., 1999), for the corresponding eccentricities of the recordings. In the cat, the incidence of C peaks is not related to the difference in orientation selectivity when the receptive fields are in overlap. However, C peaks show a weak dependence on orientation preference when the neurones have nonoverlapping receptive fields (Nelson et al., 1992). The broadest peaks (H peaks) occurred without any relationship with receptive field properties, except for a decreased incidence with increased receptive field separation in the monkey V1-V2 study (Nowak et al., 1999). Thus, the different types of peaks observed in the inter-area studies are not homogeneously distributed with respect to the properties of the neurones recorded. 3.3. Synchronized Oscillations Although it has received much of recent attention, gamma-range oscillatory synchronization (GROS, one example of which is shown in Figure 4) was the last one to be described in the cortex, although it was occasionally mentioned in early studies. In terms of incidence, this form of synchronization seems to be totally absent in some reports, but overwhelmingly present in others. One possible reason for this discrepancy might be
Time and the brain
84
Figure 5 . Distribution histograms of the logarithm of the width of CCH peaks. The width is measured at half height. CCHs are calculated between neurones in areas 17 and 18 of the cat (top), between areas 17/18 of both cortical hemispheres in the cat (middle) and between areas V1 and V2 in the macaque monkey (bottom). Note the tripartite distribution of peak width with narrow (T type), middle (C type) and
Cross-correlograms for neuronal spike trains
85
broad (H type) peaks for each case of cross-correlation between neurones located in different cortical areas.
the method used to reveal it: cross-correlation can reveal synchronized oscillations in the spike trains of neurones only if they are regular enough. Other methods are less strict in this regard. For instance, some methods based on signal frequency analysis, or synaptic or field potentials (like the power spectrum analysis) might reveal more or less broad “peaks”, centred close to 40 Hz, that correspond to signals which fluctuate within this frequency range (field potentials: Eckhorn et al., 1988, 1993; Munk et al., 1996; TallonBaudry et al., 1996, 1997; synaptic potentials: Jagadeesh et al., 1992; Gray and McCormick, 1996; Steriade et al., 1996a). Whether these fluctuations are large and regular enough to be termed “oscillations” or “aperiodic oscillations” (Freeman and Barrie, 1994) is not always clear. However, a membrane potential that contains a large amount of power in the gamma-range frequency does not necessarily lead to oscillatory auto- (or cross) correlation histograms (Nowak et al., 1997). The problem of the presence or absence of oscillations then becomes a methodological and terminological issue (see also Young et al., 1992; Frégnac et al., 1994a). When the presence of oscillations is tested in spike trains, another methodological problem appears, that is related to the use of averaging over long periods of recording: there is some variability in the frequency from one oscillatory event to another, if not within one period, such that averaging across multiple oscillatory episodes might result in a smoothing of the side peaks that characterise the oscillations in auto- and crosscorrelograms (Gray et al., 1990; Kreiter and Singer, 1992; Livingstone, 1996; Murthy and Fetz, 1996; Gray and Viana Di Prisco, 1997). Thus, the presence of one isolated peak in a CCH, like a T peak for instance, does not exclude the possibility that the neurones were, at one time or another, engaged in an oscillatory synchronous activity. We will concentrate on the cases where oscillations are clearly visible. Unfortunately, gamma-range oscillations do not constitute a homogeneous ensemble. Instead gammarange oscillations have been shown to occur in 3 different conditions: 1. Gamma-range oscillations have been reported in field potentials and intracellular recordings—but, to our knowledge, not in spike trains—during spontaneous activity in anaesthetized preparations. 40 Hz “brief waves” were observed by Adrian and Matthews (1934) and have been described anew with different anaesthetics, in various cortical areas and in different animal species (Steriade et al., 1996a,b; Brett and Barth, 1997). 2. GROS have also been described, that appear only during sensory stimulation, for neurones recorded within one visual cortical area in the cat (Gray and Singer, 1989; Gray et al., 1989; Eckhorn et al., 1988; Engel et al., 1990, 1991c; Roelfsema et al., 1994; König et al., 1995; Munk et al., 1996; Kruse and Eckhorn, 1996; Brosch et al., 1997; Gray and Viana Di Prisco, 1997) and the primate (area V1: Eckhorn et al., 1993; Livingstone, 1996; area V2: Frien et al., 1994; area MT: Kreiter and Singer, 1992). 3. Synchronized oscillation can also appear without any clear relationship to the presence of stimulation, but presumably in relation to the state of attentiveness (Lopes da Silva et al., 1970; Rougeul et al., 1979; Bouyer et al., 1981; Murthy and Fetz, 1992; Sanes and Donoghue, 1993; Baker et al., 1997).
Time and the brain
86
Gamma-range oscillations can be synchronized between two different cortical areas: Between areas 17 and 18 in the cat (Eckhorn et al., 1988); between areas 17 of the two hemispheres in the cat (Engel et al., 1991a, Nowak et al., 1995); between areas 17 and PMLS of the cat (Engel et al., 1991b). Oscillatory synchronization also occurs between the motor areas of the 2 hemispheres and between somatosensory and motor areas (Murthy and Fetz, 1996). The width of the central peak observed in the gamma oscillatory CCHs (3–5 msec in Engel et al., 1990) is similar to the width of the narrow peaks described above. Furthermore, there are instances in which a non-oscillatory CCH containing one narrow peak can be made to become oscillatory (Schwartz and Bolz, 1991; Nowak et al., 1995). This would suggest that non-oscillatory narrow peaks and gamma-range synchronized oscillation are somehow related. There is however one important difference: the gamma oscillations can be synchronized between neurones that are separated by large distances on the cortical surface (up to 7 mm in cat area 17; Eckhorn et al., 1988; Gray et al., 1989; Engel et al., 1990; König et al., 1995), while the non-oscillatory narrow peaks are restricted to neurones located close to each other, or with overlapping receptive fields (Murphy et al., 1985; Krüger and Aiple, 1988; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Eggermont, 1992; Nelson et al., 1992; König et al., 1995; Nowak et al., 1995). Similarly, the high frequency oscillations reported in the motor cortex of awake behaving monkeys appear coherent despite significant separation between the recording sites (Murthy and Fetz, 1996) while narrow peaks are observed only for small interneuronal separations (Murphy et al., 1985). In area 17, the stimulus-related GROS have been reported to occur between neurones displaying different orientation selectivities for overlapping receptive fields. Beyond overlap, the probability of observing this type of synchronization is higher when neurones display similar orientation preferences (Gray et al., 1989; Engel et al., 1990; Engel et al., 1991a; Livingstone, 1996). The motion of the stimuli also seems to be an important stimulus parameter to obtain GROS, as they do not appear with stationary stimuli (Engel et al., 1990; but see Ghose and Freeman, 1992). In fact, oscillations turned on by a moving stimulus can be disrupted by stationary stimuli (Kruse and Eckhorn, 1996). Summary Four different types of synchronization can be distinguished on the basis of the shape of the CCH peaks. One form corresponds to the gamma-range oscillatory synchronization. The three other forms correspond to non-oscillatory synchronization, that can be distinguished on the basis of different peak widths. These non-oscillatory forms of synchronization show different incidences depending on the properties of the recorded cells.
Cross-correlograms for neuronal spike trains
87
4. WHERE DOES SYNCHRONIZATION COME FROM? Before trying to determine the underlying mechanisms, we must first review the relationship between the different forms of synchronized activity, the waking state of the brain and the presence of a sensory stimulus. 4.1. Behavioural and Stimulus Dependency of the Different Types of Synchronization 4.1.1. Awakening and anaesthesia We have mentioned above that sleep and waking states, as well as the use of anaesthesia, affect the probability of observing synchronization between pairs of cortical neurones. Here we re-examine this issue with respect to the four different forms of synchronization defined above, in an attempt to determine their origins and functional roles. We have investigated the effects of different anaesthetic agents on the three types of non-oscillatory activities observed between areas V1 and V2 (Nowak et al., unpublished). Three anaesthetic agents have been used, which were isoflurane, halothane (both volatile anaesthetics), and the opiate fentanyl. The level of spontaneous activity was lower by a factor of 1.5 to 2 with isoflurane and halothane compared to fentanyl, but the strength of the visual responses was not significantly different between the three anaesthetics. In terms of incidence and strength of synchronized activity however, there was a major difference. The three non-oscillatory forms of synchronization were observed under all anaesthesia conditions. With isoflurane or halothane, the probability of observing a significant peak in a CCH computed from single unit activities was five time larger than with fentanyl. In parallel with this large difference in incidence, a strong difference was also observed in the proportion of action potentials contributing to the CCH peaks, which was more than twice as high with the volatile anaesthetics compared with fentanyl. This indicates a considerable reduction in the number of action potentials that were correlated by mechanisms of neuronal origin under fentanyl anesthesia. The origin of these differences is not clear, but it may be hypothesized that isoflurane and halothane have a reinforcing effect on the neuronal mechanisms generating sleepassociated rhythms, like the spindle wave and the “slow (<1 Hz) oscillation” (Adrian and Matthews, 1934; Steriade et al., 1993a). For example, the strength of GABAergic inhibition is increased by these anaesthetics (e.g., Franks and Lieb, 1994), which might lead to a strengthening of the spindling mechanisms in the thalamus (Keifer et al., 1994; for the cellular mechanisms of spindle wave generation see McCormick and Bal, 1997; Destexhe et al., 1998). Fentanyl, on the other hand, is likely to act on more specific targets related to nociception, and is less likely to influence directly the thalamocortical
Time and the brain
88
network (e.g., Pasternak, 1993). Therefore most of the C peaks observed under halothane or isoflurane might reflect the presence of spindling activity. In a similar line of evidence, barbiturate anaesthesia is known to induce spindles. At the cortical level this results in cross-correlation peaks with width very similar to that of C peaks (e.g., Contreras et al., 1997). Concerning H peaks, it is interesting to notice their similarity with the peaks observed in CCHs computed during the slow (<1 Hz) cortical oscillations associated with sleep (Amizca and Steriade, 1995). Concerning gamma-range oscillations, Livingstone (1996) found their occurrence not to be influenced by the halothane, nitrous oxide or sufentanyl. Gray and Viana di Prisco (1997) found that the incidence of gamma-range oscillations associated with visual stimulation is not different in awake behaving cats and in cats anaesthetized with halothane. This would suggest that gamma-range oscillations are neither disrupted nor reinforced by anaesthesia. On the other hand, the gamma-range oscillatory activities associated with attentive behaviour are unlikely to be preserved under anaesthesia. In keeping with this observation, Munk et al. (1996) found that stimulation of the reticular formation in anaesthetized cats, as a paradigm that would mimic awakening, increases the incidence and strength of oscillatory synchronization. Similarly, the gamma-range fluctuations that are observed transiently during the slow sleep oscillations become more sustained after stimulation of the mesopontine cholinergic nuclei (Steriade et al., 1996b). 4.1.2. Stimulus dependency The four different types of synchronized activity show different behaviour with respect to the presence of a sensory stimulus. As mentioned above, the GROS observed in the visual cortex occur only during the presentation of a visual stimulation, and are not found during periods of spontaneous activity (see Figure 4). Under fentanyl anaesthesia, we studied the stimulus dependency for two of the three classes of non-oscillatory peaks we observed between areas V1 and V2 of the monkey (Nowak et al., 1999). An example of this stimulus dependency is presented in Figure 2. It shows that the presentation of a stimulus led to the occurrence of a C peak, while during spontaneous activity an H peak was present instead (Figure 2C2 and 2D2). This behaviour summarizes the general tendency we observed. We could not establish any statistically significant trend for T peaks, due to the paucity of their occurrence. We found that C peaks were mostly associated with the presentation of a stimulus (Nowak et al., 1999). Hence, based on the effects of the presentation of a visual stimulus and on the effect of anaesthesia (see above), it might be hypothesized that there are not one, but two types of C peaks, one associated with the processing of sensory signals (see also Aertsen et al., 1991; Cochin et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995 for the occurrence of intermediate peaks in awake behaving animals), the other reflecting sleepand anaesthesia-induced rhythms. The H peaks on the other hand, showed a different behaviour with respect to the presentation of a visual stimulus, suggesting that most of them are associated with spontaneous activity rather than with visual responses. Notice that, contrary to what
Cross-correlograms for neuronal spike trains
89
Figure 2 may suggest, the C and H peaks do not necessarily replace each other depending on whether the neurones were visually driven, or were spontaneously active. For example, C peaks could be found for the visual response period while the CCH corresponding to the spontaneous activity period could be flat. This questions the relevance of H peaks as a neuronal correlate of sensory processing. However, a minority of H peaks do appear during visual stimulation only (Nowak et al., 1995, 1999), and peaks similar to H peaks have been shown to occur in awake, behaving monkeys (Aertsen et al., 1991; Gochin et al., 1991; Aertsen et al., 1992; Vaadia et al., 1995). Here again it might be that two different neuronal mechanisms generate CCH peaks with similar widths. Besides the difference between visually driven and spontaneous activity, the configuration of the visual stimuli can strongly influence the probability of observing synchronization. This has been well documented for the GROS, which tend to occur more often when the neurones are activated by one common stimulus rather than when two stimuli activate them separately or in a conflicting way (Gray et al., 1989; Engel et al., 1991b, c; Kreiter and Singer, 1996; Livingstone, 1996; Brosch et al., 1997). Whether non-oscillatory synchronization is modified when different types of stimulus configuration are used to activate the cells has not been studied as extensively as in the case of oscillatory synchronization. In cat area 17, Engel et al. (1991c) showed that nonoscillatory CCHs display a stimulus dependency similar to that shown for the oscillatory synchronization. In cat and monkey primary visual cortex, Sillito et al. (1995) showed examples of narrow CCHs that depended on the orientation context of the stimuli used. Some H peaks observed between the two hemispheres in cat appeared to be disrupted by the use of conflicting stimuli (Nowak et al., 1995). The stimulusdependent synchronization observed in frontal and auditory cortex of awake animals seems to apply for all the different types of peaks (Frostig et al., 1983; Vaadia and Abeles, 1987; Aertsen et al., 1991; Vaadia et al., 1991; Aertsen et al., 1992; Ahissar et al., 1992a; Vaadia et al., 1995). Altogether these data indicate that stimulus dependent changes are not restricted to gamma-range synchronous oscillations: they can also be observed for all the different types of non-oscillatory synchronization. 4.2. Neuronal Mechanisms Generating Different Types of Synchronization 4.2.1. Common inputs The peaks centred on the time origin in a CCH may result from the activation of the two recorded cells by a third cell, or by a pool of common presynaptic neurones. In the retina, two neighbouring ganglion cells can show synchronized firing within a short time scale, and Mastronade (1983a) concluded that this was due to a common input from amacrine cells. In the lateral geniculate nucleus of the cat, Sillito et al. (1994) described a form of synchronization, characterized by relatively broad peaks (whose width was in the range of tens of msec), that appears between neurones with non overlapping receptive fields,
Time and the brain
90
when they are simultaneously activated by light bars or gratings long enough to cover both receptive fields. This form of synchronization also depends on the orientation of the stimulus with respect to the alignment of the receptive fields. It is not observed, or is much smaller, when small spots of light covering only each receptive field are flashed simultaneously. This form of synchronization disappears when the visual cortex is lesioned. This suggests that this form of thalamic synchronization is due to common input fed back from cortical layer VI. As already stressed, synchronization of neuronal origin is the most often encountered type of neuronal interaction found in the cortex. Dickson and Gerstein (1974) calculated that the source of common inputs responsible for the synchronized activity they observed within the auditory cortex should be located more than 1.5 mm away from the recorded area. This obviously suggests the thalamus as one possible structure containing the neurones providing the common inputs. However, Engel et al. (1991c) and Nowak et al. (1995) showed that gamma-range oscillatory synchronization, as well as the three types of non-oscillatory synchronization, are present in the interactions between areas 17 of the two hemispheres. Since the projections of the thalamus onto the visual cortex are strictly ipsilateral, this shows that synchronized activity can be obtained without common thalamic inputs. Obviously this does not mean that the thalamic inputs do not contribute to the synchronization observed within the same hemisphere, although in this case, it is unlikely to be the source of shared inputs to neurones that are separated by a distance larger than the width of the terminal arbor of thalamic axons (Nelson et al., 1992). The corpus callosum contains only axons of cortical origin. Engel et al. (1991c) showed that lesion of the corpus callosum suppressed all instance of GROS between the two hemispheres. Munk et al. (1995) extended this result to the case of T and C peaks. This indicates that the corticocortical connections transiting through the corpus callosum are required for the intermediate and narrowest type of coupling, whether or not it is oscillatory. H peaks, on the other hand, remained quite numerous if only the posterior half of the corpus callosum was cut. They were reduced in incidence, although still not completely suppressed, only after 80% of the corpus callosum was severed (Munk et al., 1995). What types of connections sustain the different types of interhemispheric synchronization? Two types of corticocortical connections are present in the corpus callosum. The first corresponds to the fibres that link together the two areas 17. This group of connections is homologous to the horizontal connections within area 17. The other set of axons corresponds to feedback corticocortical connections: the areas 17 of both hemispheres receive common feedback inputs from several “higher” cortical areas (areas 19, 21a, PMLS and PLLS), by means of axons that bifurcate such that one branch innervates the hemisphere ipsilateral to its parent neurone; while the other innervates the hemisphere contralateral to it. Notice also that some form of “functional” bifurcation can occur if two adjacent neurones projecting one to one side, the other to the other hemisphere, are made to synchronize their activities through mechanisms of neuronal origin. Interestingly, C and H peaks are observed for separations that correspond to the known values of divergence for the feedback connections. The next step in the study of Munk et al. (1995) was therefore to establish the role of one or the other set of connections in the generation of synchronized activity. To this end, extensive lesions of
Cross-correlograms for neuronal spike trains
91
the extrastriate cortex were made, aimed at sparing only area 17 and the neighbouring strip of area 18 that represents the vertical meridian. In this way, the feedback connections were removed and the route for the polysynaptic propagation of activity through multiple cortical steps was interrupted as well. The corpus callosum was left intact in these instances (hence the “horizontal connections”). Despite these extensive lesions, the strength of the visual responses was not different from that in intact animals. The results showed that narrow peaks were not strongly affected. Similarly, gammarange oscillations were still observed, although their strength appeared to be reduced (Munk et al., unpublished). However, the incidence of C peaks was reduced, approximately by one half, in single unit paired recordings, and that of H peaks was reduced by two thirds. Together with this reduced incidence, a reduced coupling strength was observed for those H and C peaks that remained. H peaks remained unless all but the most anterior portion of the callosum was removed, but disappeared when the direct feedback inputs were removed. From this we can conclude that a majority of H peaks seem to reflect the transmission of a slow rhythm reflecting the anaesthetized or sleeping state of the animal, by way of multiple, polysynaptic pathways through the cortical mantle. Half of the C type synchronization seems to be associated with monosynaptic common inputs provided by feedback connections. It is not clear however, if higher cortical areas contain common input neurones that generate C peaks, or whether they are involved in modulating activity in a way that would make neurones more bursty, hence generating larger peaks (see below). T peaks and GROS do not seem to require a common input source from higher areas; they seem to be supported by the direct reciprocal connections between the two hemispheres. 4.2.2. The emergent properties of the cortical network At this stage we are faced with a paradox for the generation of the narrowest CCH peaks (T peaks and gamma-range oscillatory synchronization): They seem to be generated solely by the reciprocal connections linking the two synchronized neurones, but they are nonetheless centred on the time origin despite the large separation and large conduction delay between the recording sites. How is this possible? This issue has been addressed by modelling studies. A number of such studies have shown that networks composed of realistic neurones, interconnected with realistic synaptic strengths and conduction delays, show synchronized oscillatory activities when activated by an “input” that does not contain any temporal structure (some of the studies relevant to this question are: Sporns et al., 1989; Bush and Douglas, 1991; König and Schillen, 1991; Lytton and Sejnowski; 1991; Wilson and Bower, 1991; Ritz et al., 1994; Whittington et al., 1995; Bush and Sejnowski, 1996, Hansel and Sompolonski, 1996; Wang and Buzsáki, 1996; Lumer et al., 1997a,b). These studies obviously differ in their details but all suggest that synchronization can be obtained as an emergent property within the cortical network. These models provide a counter-intuitive result, namely, that the presence of a conduction delay between neurones does not prevent them from being
Time and the brain
92
engaged in synchronized activity. This is determined by the connectivity pattern, characterized by its divergence (one cell projects to hundreds of other cells), by the fact that excitatory neurones project to both excitatory and inhibitory neurones (and vice versa), by the small strength of the individual synapses, to some extent by the membrane properties of individual neurones, and, most important, to GABAergic inhibition that plays a key role in oscillatory synchronization. Model studies specifically designed to recreate the gamma-range oscillatory activity showed that neurones can be synchronized as long as the conduction delay is less than, approximately, one fourth of the oscillation period (see also Eckhorn, this volume). The presence of oscillatory forms of synchronization between the two cortical hemispheres for example, is not incompatible with the large distance between neurones and conduction delays up to about 5 msec—the conduction delay between the two hemispheres in cat cortex is below 5 msec in the majority of the cases (Harvey, 1980; Innocenti, 1980). Another issue related to the generation of gamma-range synchronous oscillation is the requirement for pacemaker neurones and their location. Some experimental results suggest that fast rhythms might be generated through synaptic interactions taking place within the cortical network, without pacemaker neurones, since oscillatory activity can be independent of the membrane potential of the recorded neurones (Bringuier et al., 1997). In some of the modelling studies mentioned above however, synchronized oscillations were obtained only after including excitatory neurones discharging in bursts of action potentials. Neurones displaying this property might be the “chattering cells”, recently described in cat cortex (Gray and McCormick, 1996; Steriade et aL, 1998), that have the property to discharge repetitive bursts of action potentials at frequencies up to 70 Hz. (However the majority of these neurones depart from “pure” pacemaker neurones in the sense that the burst frequency depends on the strength of their activation [Gray and McCormick, 1996].) In addition, some cells display subthreshold oscillations in their membrane potential when depolarized by DC injection (Llinás et al., 1991; Nuñez et al., 1992). Oscillatory activities in the gamma range do occur in subcortical structures such as the retina and the visual thalamus (Rodieck, 1967; Robson and Troy, 1987; Funke and Worgotter, 1995; Ghose and Freeman, 1992; Neuenschwander and Singer, 1996). It should be pointed out however, that the apparent “rhythmicity” observed at the level of some retinal ganglion cells during sustained activity basically reflects the regularity (low variability) of the discharges and that the frequency shown by these “rhythms” changes widely depending on the luminance conditions (Rodieck, 1967; Robson and Troy, 1987; Troy and Robson, 1992). According to Bouyer et al. (1981), the posterior thalamic group could be the pacemaker for the 40 Hz rhythms observed in the parietal cortex of attentive cats. Cells with rhythmic discharges within the gamma range have also been reported in the reticular and the intralaminar nuclei (Pinault and Deschênes, 1992a; Steriade et al., 1993c). Other modelling studies therefore, have suggested that neurones in the thalamus act as pacemakers for the production of cortical 40 Hz oscillations (Llinás et al., 1994; Funke and Worgotter, 1995; Ghose and Freeman, 1997). Nevertheless, these models cannot account for the presence of oscillations synchronized between the two hemispheres.
Cross-correlograms for neuronal spike trains
93
4.2.3. Generation of different peak widths Compared to the case of synchronized oscillations, the other forms of synchronization, despite being more commonly observed, have received less interest from modelling studies (but see: Erb and Aertsen, 1992; Koch and Schuster, 1992; Abeles et al., 1994; Arnoldi and Brauer, 1996; Hansel and Sompolinski, 1996). One aspect that has not received much attention is the variability of peak widths observed in the CCHs. Experimental data, however, can give a hint about how different peak widths (and therefore different peak classes) can be obtained. As far back as 1970, it was shown that the width of a CCH is strongly influenced by the presence of burstiness in one or both of the neurones (Moore et al., 1970). More recent studies have shown that variable amounts of burstiness generate different CCH peak widths. First, there is a similarity between the width of a CCH and the width of the ACHs of one or both of the neurones of the pair (Gochin et al., 1991; Nowak et al., 1995). Second, replacing the bursts by the first action potential of the burst, or using deconvolution methods, usually leads to a narrowing of the peak width (Mastronarde, 1983b; Eggermont et al., 1993; Eggermont and Smith, 1996). Recognizing that burstiness accounts for the width of a CCH peak just moves the problem a step farther: what neuronal mechanisms are responsible for the presence of bursty firing? Burstiness can be an intrinsic property of the neurones. Cortical neurones have been classified in different types, based on their spike response to current pulse injection in vivo and in vitro. Among these different types of neurones, two are characterized by their propensity to generate bursts of action potentials. One type corresponds to the “intrinsic bursting neurones” identified both in vitro (Connors et al., 1982; McCormick et al., 1985; Jones and Heinemann, 1988; Chagnac Amitai et al., 1990; Mason and Larkman, 1990) and in vivo (Pockberger, 1990; Baranyi et al., 1993; Gray and McCormick, 1996). The other category has been observed in vivo and corresponds to the “chattering cells” (Gray and McCormick, 1996; Steriade et al., 1998) that differ from the “intrinsic bursting” neurones by their thinner action potentials and their ability to discharge bursts in a sustained manner. Nevertheless, in both types of cells, each burst lasts for only a few msec. This would produce peaks in ACHs of a few msec width only. This would account only for the larger width shown by the narrowest class of CCHs. Intrinsic membrane properties of cortical neurones cannot therefore explain the width of the intermediate and broad CCH peaks. Neurones in the thalamus, provided they are sufficiently hyperpolarized, can discharge bursts of action potentials. The duration of a typical burst in the thalamus is of the order of ten to several tens of msec (Jahnsen and Llinás, 1984; Kim et al., 1995; Guido and Weyand, 1995). During spindle waves, these bursts are strongly synchronized such that the thalamus can be brought into a globally synchronized state (Kim et al., 1995; Contreras et al., 1997). In these conditions, the topographic organization of the thalamus, and the lack of direct interactions between relay cells, do not prevent neurones located in separate territories from behaving in unison. The corresponding ACH and CCH peaks display widths of several tens of msec (Kim et al., 1995). This synchronized activity,
Time and the brain
94
transmitted to cortex, would produce bursts of action potentials there, that would not be of intrinsic but of synaptic origin. This would generate CCH peaks with a width compatible with that of intermediate (C) peaks between spatially remote loci in cortex. Therefore, some intermediate peaks (those that are associated with sleep and anaesthesia) could be generated through the thalamocortical network, with a prominent role played by the membrane properties of thalamic neurons. For those C peaks that appear without a relationship to spindle activity (for example during sensory stimulation only, or in awake animals), it can be expected that the synaptic burstiness originates from within the cortex itself. If it is the same C peaks that appear to depend on feedback connections, then it could be that feedback connections are able to control the duration of cortical neuronal bursts. Since bursts are more likely to produce the temporal summation that is required to trigger action potentials in a post synaptic neurone, it could be hypothesized that feedback inputs from higher cortical areas would ultimately control the transmission of activity through the cortex. We have proposed that the majority of H peaks are correlates in the spike synchronization domain of the slow (<1 Hz) oscillations associated with sleep (Adrian and Matthews, 1934; Steriade et al., 1993a). Steriade et al. (1993b) showed that the slow oscillations survive extensive lesion of the thalamus, which indicates that it is generated within the cortex. The neuronal mechanisms that lead to the slow build up of the associated depolarizing synaptic potential, and to its resetting, remain to be elucidated. 4.3. A Global Picture As we entered into a more detailed examination of cortical synchronization, we found that there are not one, but four different forms of synchronization in cortex. Then, examining the stimulus and sleep/waking dependency of these different forms of synchronization, as well as the possible mechanisms that could account for their generation, we are forced to recognize that there might also be some subclasses within the different forms of synchronization. 1. The gamma-range synchronized oscillations might not be a single form of synchronization. One form appears in the membrane potential and in the field potential of neurones when animals are asleep or anaesthetized. Another form of gamma-range synchronized activity, that displays a strong stimulus dependency, depends apparently on the direct reciprocal connections between the recorded cells, although subcortical structures might contribute as well. There is a third form of gamma-range synchronized oscillation, that cannot be associated with the presentation of a stimulus, but which seems to be associated with attentiveness and expectancy. Whether these three forms of gamma-range synchronization share similar mechanisms for their generation, or whether their similarity is just coincidental, remains to be determined. 2. Non-oscillatory, narrow peaks. There are two types of narrow peaks. Those that are neatly separated from the time origin can be considered as representing a serial synaptic connection between the two recorded cells; this form of temporal coherence nonetheless appears extremely rare in the cortex. The majority of narrow peaks are centred on the time origin. They could be produced by common inputs from other
Cross-correlograms for neuronal spike trains
95
structures, especially when they are observed between neurones located close to each other. When the participation of common inputs can be dismissed, they appear to be generated by the direct reciprocal connections between the two recorded cells. 3. Intermediate and C peaks. Because they are observed between neurones that are spatially separated, it is necessary to postulate that they are generated through mechanisms that do not respect a strict topographic organization. In the visual cortex, feedback connections do not respect the retinotopic organization of the lower cortical areas, and it has been shown that these connections contribute to their generation. The feedback connections could further sustain the stimulus dependency exhibited by this type of synchrony. Another form of C peak could be related to spindle wave generation in the thalamus during sleep and anaesthesia. In that case their width could be related to the interaction taking place within the thalamocortical network. 4. Broad and H peaks. A majority of H peaks seem to reflect the transmission of a slow rhythm reflecting the anaesthetized or sleeping state of the animal, through multiple, eventually polysynaptic pathways. Since this form of coupling does not really show any relation to sensory processing, it is not surprising to observe it between neurones that have nothing in common in terms of receptive field properties. Nevertheless, there is another form of broad peak, that is observed in awake behaving animals. In these conditions too, they are observed between neurones that are spatially separated from each other. Since this type of synchrony can change with respect to the behavioural state of the animal, it is tempting to propose at it reflects slow, global changes, associated perhaps with the generation of sustained attention.
5. FUNCTIONAL ROLES FOR SYNCHRONIZATION We have shown that synchronization, in its different forms, can be explained as the consequence of the anatomical organization of corticocortical and thalamocortical connections and of their functional expression. The question to be solved now is whether synchrony is only the consequence of the network at work, or whether synchronization is also used for some purposes. In this attempt to examine the functional significance of synchronization, we will not examine the significance of the synchronization that occurs in the states of sleep and anaesthesia. Likewise, we will not discuss the importance of synchronization for plastic changes of neuronal connectivity. 5.1. Multiplexing of Information Synchronization of neuronal origin can be explained by the presence of common input to the two recorded cells. In two studies, one in retina (Meister et al., 1995) and the other in cortex (Ghose et al., 1994b), attempts have been made to map the receptive field of the synchronized action potentials. One result from the two studies is that the receptive field corresponding to the synchronized action potentials can differ, in size and properties,
Time and the brain
96
from those of the two recorded units. This means that the synchronized action potentials from two cells contain information not available from the response of each cell taken individually. By this way there would be a multiplexing of information, allowing for an increased amount of information transmission in the axons. However attractive this hypothesis is, it requires that neurones beyond the synchronized ones be able to demultiplex the inputs they receive. Whether such a subtle information processing can be performed in the cortex remains to be determined. 5.2. Binding by Synchronization 5.2.1. Recent results, challenges, and constraints Synchronization of neuronal activity has been proposed as a solution to the binding problem. In short, neurones that respond to the same object would synchronize their activity, while remaining desynchronized from neurones responding to different objects. Synchronization would provide a means of selecting specific neuronal populations, by coincident arrival of EPSPs. Electrophysiological evidence has been provided, showing that neurones synchronize their activities when activated by a common stimulus, but remain desynchronized when activated by conflicting stimuli (see above). Additional support for a role of synchronization in perception was given in two studies. In the first one, Roelfsema et al. (1994) studied amblyopia (poor vision in one eye) that was induced by strabismus in cats: cats are not able to perceive gratings of high spatial frequency through their amblyopic eye, but the strength of the neuronal responses to these high frequency gratings is similar for the two eyes. What differed instead between the two eyes was the ability for neurones to synchronize, which was lower for neurones activated by the amblyopic eye, compared to those activated by the normal eye. A deficit in perception appears correlated with a deficit in synchrony. In a second study, Fries et al. (1997) studied the effects of interocular rivalry in awake strabismic (but not amblyopic) cats. They found that presenting two competing stimuli to both eyes did not affect the strengths of visual responses. However, in the condition of rivalry, synchronization was stronger between neurones activated by the dominant eye, and lower in neurones activated by the non-dominant eye. This suggested that “at early stages of visual processing the degree of synchronicity rather than the amplitude of the responses determines which signals are perceived and control behavioural responses.” Additional support for a role of synchrony in binding comes from large scale modelling studies (Neven and Aertsen, 1992; Tononi et al., 1992). Based on Hebbian cell assembly organization, these artificial neural networks are able to learn and to recognize stimuli. In doing so they can solve the in variance and the occlusion problems. However, as already stressed, segmentation cannot easily be performed in a pure Hebbian cell assembly system. In these models therefore, it is synchronization that has been used to tag which parts belong to the same object. Hence these modelling studies demonstrate the plausibility of synchronization as a means of solving the binding problem. Nevertheless, there are a number of recent experimental studies that have challenged
Cross-correlograms for neuronal spike trains
97
the binding-by-synchrony hypothesis. First, Gochin et al. (1991), recording in inferotemporal cortex of awake behaving monkeys, found that different visual stimuli did not produce different CCH peaks, despite the differences of neuronal responses. Likewise, Gawne and Richmond (1993) in inferotemporal cortex, and Gawne et al. (1996) in primary visual cortex of awake behaving monkeys, found that correlation in the millisecond range does not correlate with the similarity of the responses of the neurones to a battery of different stimuli. This does not prove that synchrony does not play a role, but it suggests that a rate coding, eventually combined with a slow temporal coding, would do a better job in discriminating between the different stimuli used in these studies, than would a code based solely on synchronized activity in the millisecond range. Second, a group of studies in the motor cortex has failed to find a clear relationship between oscillatory synchronization and movement execution (Murthy and Fetz, 1992; Sanes and Donoghue, 1993; Baker et al., 1997). These results do not argue against a role for synchronization in binding in sensory cortex. However, the recent study by Cardoso de Oliveira et al. (1997), in parietal visual cortex of monkey, showed the intriguing result of a suppression of synchronization by visual stimulation, rather than an enhancement. In awake behaving animals therefore, synchronization might have a stronger relationship to attention than to segmentation. It has been suggested that 40 Hz oscillatory activities are instrumental for the generation of long range synchronization (König et al., 1995). Therefore perturbing 40 Hz oscillations should prevent synchronization, and ultimately prevent accurate perception. In two psychophysical studies however (Fahle and Koch, 1995; Kiper et al., 1996), it was found that perceptual performances are neither improved not impaired by the use of stimuli flickering in time near 40 Hz. If synchronization plays a role in binding, then it could be argued that either oscillations are not required for the generation of synchronization, or that non-oscillatory forms of synchronization play a role. Thorpe et al. (1996) demonstrated that it takes only 150 msec of processing for recognition of a visual image. The images used in this study were complex colour photographs, therefore requiring binding and figure-ground segregation to allow recognition. This does not argue against a role of synchronization in segmentation, but this imposes serious constraints on the onset of synchronous activity with respect to the onset of the neuronal responses in different cortical areas. Therefore, one important issue that remains to be investigated is that of the onset of synchronized activity in cortex.
Time and the brain
98
5.2.2. Coincidence detection The cornerstone of the binding-by-synchronization hypothesis is the requirement for neurones to behave as coincidence detectors: an action potential is triggered only after enough EPSPs have been summated (see above). The time it takes for this summation to occur corresponds to the integration time. The largest time window within which the summation of two EPSPs can take place is approximately equivalent to the time constant of the neurone. Beyond this time interval, the depolarization elicited by the first EPSP has vanished, since the membrane potential has returned to its resting level. Theoretical and modelling studies show that, for the same mean presynaptic firing rate, the number of randomly occurring EPSPs required to fire an action potential is much larger than the number of EPSPs required if they occur in synchrony (Abeles, 1982; Bernander et al., 1991; Reyes and Fetz, 1993b). This makes the neurone extremely sensitive to the occurrence of synchronized EPSPs. For this reason it has been proposed that cortical neurones may behave as coincidence detectors. All neurones perform temporal summation. The distinction between the temporal integrator and the coincidence detector resides in the time window during which this summation is accomplished. König et al. (1996) proposed a frontier between the two types, such that a neurone is considered as a coincidence detector if the integration time is smaller than the mean interspike interval, while the integration time of a temporal integrator would be equal or larger than the mean interspike interval. The critical value would be around 5 msec. Experimental measurements of the integration time of cortical neurones during sensory stimulation are still lacking. However, the variability of action potential timing has provided one entry to study the way neurones integrate synaptic activity: For a constant mean firing rate, the interspike intervals (ISI) can be highly variable, such that short and long ISIs can be observed adjacent to each other. The distribution of ISIs is then characterized by its skewed nature, giving a Poisson-like shape to the ISI histogram. The coefficient of variation (the standard deviation divided by the mean ISI) is often close to unity, indicating apparent randomness of spike timing (e.g. Softky and Koch, 1993). The variability of ISIs has been noticed since the earliest electrophysiological investigations of the central nervous system. Softky and Koch (1993) attempted to determine its origin by confronting experimental data obtained from awake monkey visual cortex with different computer models of synaptic integration. One important conclusion they reached is that the variability observed in vivo is incompatible with the integration of random EPSPs that are simply summated: if EPSPs of small size have to be summated in order to yield a constant mean firing rate, then the total synaptic current is almost constant in time, and action potential timing would not vary. One way, however, to obtain the high variability in the membrane potential and therefore in spike timing, is to have the neurone depolarized not by small random EPSPs, but by large compound EPSPs. In other words, it is possible to explain the presence of a high variability of action potential timing if the action potentials are triggered by synchronous EPSPs. This provided evidence favouring the role of synchronization in determining action potential triggering. In the light of these results, it was proposed that, indeed, cortical neurones do
Cross-correlograms for neuronal spike trains
99
behave as coincidence detectors. Unfortunately, later studies have indicated that variability of action potential timing might be explained by other mechanisms. Shadlen and Newsome (1994) showed that variability of action potential timing can be obtained if neurones integrate not only random EPSPs, but also random IPSPs. For this neuronal model to work in accordance with the in vivo data, the EPSPs and the IPSPs have to be balanced (that is to say, the total inhibitory current should be equivalent or close to the total excitatory current a neurone receives). The role of inhibition in generating spike timing variability has been confirmed in other modelling studies (Hansel and Sompolinski, 1996; van Vreeswijk and Sompolinski, 1996; Schindler et al., 1997; Troyer and Miller, 1997). Large scale modelling studies might provide an indication about the usefulness of synchronization in eliciting activity in cortical neurones. In their model, Lumer et al. (1997b) could modify the amount of synchrony between neurones without changing other parameters (a procedure impossible to achieve experimentally). They found that reducing synchronization decreases the firing rate of the neurones in the network. However, in the worst case, the firing rate was not reduced to zero, but to a value approximately half the original one. This suggests that synchronization can help the transmission of signal and maintenance of activity, but also that synchronization is not absolutely necessary for spike generation. Altogether it could be concluded that neurones might behave as coincidence detector, in which case synchronization would help transmitting information. It does not mean, however, that neurones cannot work as temporal summators. The experimental demonstration that cortical neurones are activated by synchronized EPSPs remains to be done. 5.2.5. Alternatives to the binding-by-synchronization The central issue in the question of segmentation within a Hebbian cell assembly is that of the selection, among those anatomical connections that are present, of those that have any relevance to the stimulus. Synchronization has been proposed as one possible mechanism, but it is important to ask whether there are other neuronal mechanisms of segmentation. Another possible solution to the segmentation problem, which at the same time provides the correct selection of neuronal pathway, is to suppress the possibilities of cross talk: this is falling back into the labelled line and cardinal neurone hypotheses, that have been largely criticized (for example for the lack of flexibility they imply), and will not be discussed further here. Another possibility to be considered is that the selection among a set of anatomical connections does rely on the inhibition of the activity of neurones that are not relevant to the situation. Centre-surround interactions, that have been well documented in various visual cortical areas (Nelson and Frost, 1975; Allman et al., 1985; Schein and Desimone, 1990; Knierim and Van Essen, 1992; DeAngelis et al., 1994; Lamme, 1995; Sillito et al., 1995), might play this role. In short, centre-surround interactions are such that, as long as no discontinuity is present in a stimulus pattern, the activity of neurones is actively
Time and the brain
100
maintained at a relatively low level. It is only when there are discontinuities that neuronal activities are higher, and therefore are more likely to transmit information to the next processing stage. In this case the transmission of information does not have to depend on the presence of synchronization, whether or not it is oscillatory, and the postsynaptic neurones do not have to behave as coincidence detectors. In the future it will be important to determine, for the same neurone-pairs, which of the two mechanisms (selection by synchronization, selection by inhibition) have the largest impact on neuronal processing. In conclusion, we have shown that there are multiple types of synchronization in cortex. These different forms of synchronization can be related, to some extent, to different states of the cortical network related to sleep and anaesthesia, to information processing, or to attentiveness and expectancies. Synchronization associated with information processing has been proposed as a solution to the binding problem. However, not all experimental results seem to agree with this notion, and alternative mechanisms besides synchronization exist, that could also potentially play a role in segmentation and figure-ground segregation. Whatever the outcome of this question, synchronization of neuronal activity will remain an important phenomenon, for it reflects, in one way or another, the functioning of the cortex.
ACKNOWLEDGEMENTS Thanks to Joshua Brumberg for comments and help with the English. Thanks to Naura Chounlamountri for help with the Figures.
REFERENCES Abeles, M. (1982) Local cortical circuits: An electrophysiological study . Springer Verlag. Abeles, M. (1991) Corticonics: Neuronal circuits of the cerebral cortex . Cambridge University Press. Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993) Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology , 70, 1629–1638. Abeles, M., Prut, Y., Bergman, H., and Vaadia, E. (1994) Synchronization in neuronal transmission and its importance for information processing. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal Coding in the Brain , Springer Verlag, Heidelberg, pp. 39–50. Adrian, E.D. (1935) The electrical activity of the brain. Proceedings of the Royal Society of Medicine , 29 , 197–200. Adrian, E.D., and Matthews, B.H.C. (1934) The interpretation of potential waves in the cortex. Journal of Physiology (London) , 81 , 440–471. Aertsen, A., and Gerstein, G.B. (1985) Evaluation of neuronal connectivity: sensitivity of cross-correlation. Brain Research , 340 , 341–354. Aertsen, A., and Preißl, H. (1990) Dynamics of activity and connectivity in physiological neuronal networks. In: H.G.Schuster (ed.) Nonlinear Dynamics and Neuronal Network , VCH, pp. 281–301.
Cross-correlograms for neuronal spike trains
101
Aertsen, A., Vaadia, E., Abeles, M., Ahissar, E., Bergman, H., Karmon, B., Lavner, Y., Margalit, E., Nelken, I., and Rotter, S. (1991) Neural interactions in the frontal cortex of a behaving monkey: signs of dependence on stimulus context and behavioral state. Journal für Hirnforschung , 32, 735–743. Ahissar, M., Ahissar, E., Bergman, H., and Vaadia, E. (1992a) Encoding of sound-source location and movement: Activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. Journal of Physiology (London), 67, 203–215 Ahissar, E., Vaadia, E., Ahissar, M., Bergman, H., Arieli, A., and Abeles, M. (1992b) Dependence of cortical plasticity on correlated activity of single neurons and on behavioral context. Science, New York , 257, 1412–1415. Allman, J., Miezin, F., and McGuinness, E. (1985) Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception , 14 , 105–126. Amzica, F., and Steriade, M. (1995) Short- and long-range neuronal synchronization of the slow (<1 Hz) cortical oscillation. Journal of Neurophysiology , 73, 20–38. Arnoldi, H.M., and Brauer, W. (1996) Synchronization without oscillatory neurons. Biological Cybernetics , 74, 209–223. Baker, S.N., Olivier, E., and Lemon, R.N. (1997) Coherent oscillations in monkey motor cortex and hand muscle EMG show task-dependent modulation. Journal of Physiology (London) , 501 , 225–241. Baranyi, A., and Szente, M.B. (1987) Long lasting potentiation of synaptic transmission requires postsynaptic modification in the cortex. Brain Research , 423, 378–384. Baranyi, A., Szente, M.B., and Woody, C.D. (1993) Electrophysiological characterization of different types of neurons recorded in vivo in the motor cortex of the cat. I. Patterns of firing activity and synaptic responses. Journal of Neurophysiology , 69, 1850–1864. Bernander, O, Douglas, R.J., Martin, K.A.C., and Koch, C. (1991) Synaptic background activity influences spatiotemporal integration in single pyramidal cells. Proceedings of the National Academy of Sciences, U.S.A. , 88, 11569–11573. Bonhoeffer, T., Staiger, V., and Aertsen, A. (1989) Synaptic plasticity in rat hippocampal slice cultures: Local “hebbian” conjunction of pre- and postsynaptic stimulation leads to distributed synaptic enhancement. Proceedings of the National Academy of Sciences, U.S.A. , 86, 8113–8117. Bouyer, J.J., Montaron, M.F., and Rougeul, A. (1981) Fast fronto-parietal rhythms during combined focused attentive behavior and immobility in cat: cortical and thalamic localizations. Electroencephalograhy and Clinical Neurophysiology , 51 , 244–252. Boven, K.-H., and Aertsen, A. (1990) Dynamics of activity in neuronal networks give rise to fast modulations of functional connectivity. In: R.Eckmiller, G.Hartmann and G.Hauske (eds) Parallel Processing in Neural System and Computers , Elsevier Science Publishers B.V. (North Holland), pp. 53–56. Bringuier, V., Frégnac, Y., Baranyi, A., Debanne, D., and Shulz, D. (1997) Synaptic origin and stimulus dependency of neuronal oscillatory activity in the primary visual cortex of the cat. Journal of Physiology (London) , 500 , 751–774. Brett, B., and Barth, D.S. (1997) Subcortical modulation of high-frequency (gamma range) oscillating potentials in auditory cortex. Journal of Neurophysiology , 78, 573– 581. Brosch, M., Bauer, R., and Eckhorn, R. (1997) Stimulus-dependent modulations of correlated high-frequency oscillations in cat visual cortex. Cerebral Cortex , 7, 70–76. Bryant, H.L., Ruiz Marcos, A., and Segundo, J.P. (1973) Correlation of neuronal spike discharges produced by monosynaptic connections and by common inputs. Journal of
Time and the brain
102
Neurophysiology , 36, 205–225. Burns, B., and Webb, A.C. (1979) The correlation between discharge times of neighbouring neurons in isolated cerebral cortex. Proceedings of the Royal Society of London, series B. , 203, 347–360. Bush, P.C., and Douglas, R.J. (1991) Synchronisation of bursting action potentials discharge in a model network of neocortical neurons. Neural Computation , 3, 19–30. Bush, P., and Sejnowski, T. (1996) Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network model. Journal of Computational Neuroscience . 3 , 91–110. Cardoso de Oliveira, S., Thiele, A., and Hoffman, K.-P. (1997) Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. Journal of Neuro science , 17, 9248–9260. Chagnac-Amitai, Y., Luhmann, H.J., and Prince, D.A. (1990) Burst generating and regular spiking layer 5 pyramidal neurons of rat neocortex have different morphological features. Journal of Comparative Neurology , 296, 598–613. Cleland, B.C., Dubin, M.W., and Levick, W.R. (1971a) Simultaneous recording of input and output of lateral geniculate neurones. Nature (New Biology) , 231, 191–192. Cleland, B.G., Dubin, M.W., and Levick, W.R. (1971b) Sustained and transient neurones in the cat’s retina and lateral geniculate nucleus. Journal of Physiology (London) , 217, 473–496. Connors, B.W., Gutnick, M.J., and Prince, D.A. (1982) Electrophysiological properties of neocortical neurons in vitro. Journal of Neurophysiology, 48, 1302–1320. Contreras, D., Destexhe, A., Sejnowski, T.J., and Steriade, M. (1997) Spatiotemporal patterns of spindle oscillations in cortex and thalamus. Journal of Neuroscience , 17, 1179–1196. Cope, T.C., Fetz, E.E., and Matsumura, M. (1987) Cross-correlation assessment of synaptic strength of single Ia fibre connections with triceps surae motoneurones in cats. Journal of Physiology (London) , 390, 161–188. DeAngelis, G.C., Freeman, R.D., and Ohzawa, I. (1994) Length and width tuning of neurons in the cat’s primary visual cortex. Journal of Neurophysiology , 71, 347–374. de Charms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds by the coordination of action potential timing. Nature, London. , 381, 610–613. Destexhe, A., Contreras, D., and Steriade, M. (1998) Mechanisms underlying the synchronizing action of corticothalamic feedback through inhibition of thalamic relay cells. Journal of Neurophysiology , 79, 999–1016. Dickson, J.W., and Gerstein, G.L. (1974) Interactions between neurons in auditory cortex of the cat. Journal of Neurophysiology , 37, 1239–1261. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., and Reitboek, H.J. (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Biological Cybernetics , 60, 121–130. Eckhorn, R., Frien, A., Bauer, R., Woelbern, T., and Kehr, H. (1993) High frequency (60–90 Hz) oscillations in primary visual cortex of awake monkey. NeuroReport , 5, 2273–2277. Eggermont, J.J. (1992) Neural interaction in cat primary auditory cortex. Dependence on recording depth, electrode separation, and age. Journal of Neurophysiology , 68, 1216– 1228. Eggermont, J.J., Smith, G.M., and Bowman, D. (1993) Spontaneous burst firing in cat primary auditory cortex: age and depth dependence and its effect on neural interaction measures. Journal of Neurophysiology , 69, 1292–1313.
Cross-correlograms for neuronal spike trains
103
Eggermont, J.J. (1994) Neural interaction in cat primary auditory cortex. II. Effects of sound stimulation. Journal of Neurophysiology , 71, 246–270. Eggermont, J.J., and Smith, G.M. (1996) Neuronal connectivity only accounts for a small part of neural correlation in auditory cortex. Experimental Brain Research , 110, 379– 391. Engel, A.K., König, P., Gray, C.M., and Singer W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: inter-columnar interaction as determined by crosscorrelation analysis. European Journal of Neuroscience , 2, 588–606. Engel, A.K., König, P., Kreiter, A.K., and Singer, W. (1991a) Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex. Science, New York , 252, 1177–1179. Engel, A.K., Kreiter, A.K., König, P., and Singer, W. (1991b) Synchronization of oscillatory neuronal reponses between striate and extrastriate visual areas of the cat. Proceedings of the National Academy of Science, U.S.A., 88, 6048–6052. Engel, A.K., König, P., and Singer, W. (1991c) Direct physiological evidence for scene segmentation by temporal coding. Proceedings of the National Academy of Science, U.S.A. , 88, 9136–9140. Engel, A.K., König, P., Kreiter, A.K., Schillen, T.B., and Singer, W. (1992) Temporal coding in the visual cortex: new vistas on integration in the nervous system. Trends in Neuroscience , 15, 218–226. Epping, W.J.M., and Eggermont, J.J. (1987) Coherent activity in the auditory midbrain of the grass-frog. Journal of Neurophysiology , 57, 1464–1483. Erb, M., and Aertsen, A. (1992) Dynamics of activity in biology-oriented neural network models: Stability at low firing rates. In: A.Aertsen and V.Braitenberg (eds.) Information processing in the cortex: Experiments and theory , Springer, Heidelberg, pp. 201–223. Fahle, M., and Koch, C. (1995) Spatial displacement, but not temporal asynchrony, destroys figural binding. Vision Research , 35, 491–494. Fetz, E.E., and Gustafsson, B. (1983) Relation between shapes of post-synaptic potentials and changes in firing probability of cat motoneurones. Journal of Physiology (London) , 341, 387–410. Franks, N.P., and Lieb, W.R. (1994) Molecular and cellular mechanisms of general anesthesia. Nature, London , 367, 607–614. Freeman, W.J., and Barrie, J.M. (1994) Chaotic oscillations and the genesis of meaning in cerebral cortex. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 13–37. Frégnac, Y., Shulz, D., Thorpe, S., and Bienenstock, E. (1988) A cellular analog of visual cortical plasticity. Nature, London , 333, 367–370. Frégnac, Y., Bringuier, V., and Baranyi, A. (1994a) Oscillatory neuronal activity in visual cortex: a critical re-evaluation. In: G.Buzsáki, R.Llinás, W.Singer, A.Berthoz and Y.Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 81–102. Frégnac, Y., Burke, J.P., Smith, D., and Friedlander, M.J. (1994b) Temporal covariance of pre- and postsynaptic activity regulates functional connectivity in the visual cortex. Journal of Neurophysiology , 71, 1403–1421. Frien, A., Eckhorn, R., Bauer, R., Woelbren, T., and Kehr, H. (1994) Stimulus-specific fast oscillations at zero phase between areas V1 and V2 of awake monkey. NeuroReport , 5, 2273–2277. Fries, P., Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997)
Time and the brain
104
Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proceedings of the National Academy of Science, U.S.A. , 94, 12699–12704. Frostig, R.D., Gottlieb, Y., Vaadia, E., and Abeles, M. (1983) The effects of stimuli on the activity and functional connectivity of local neuronal groups in the cat auditory cortex. Brain Research , 272, 211–221. Funke, K., and Wörgötter, F. (1995) Temporal structure in the light response of relay cells in the dorsal lateral geniculate nucleus of the cat. Journal of Physiology (London) , 485, 715–737. Gawne, T.J., and Richmond, B.J. (1993) How independent are the messages carried by adjacent inferior temporal cortical neurons? Journal of Neuro science , 13, 2758–2771. Gawne, T.J., Kjaer, T.W., Hertz, J.A., and Richmon, B.J. (1996) Adjacent visual cortical complex cells share about 20% of their stimulus-related information. Cerebral Cortex , 6, 482–489. Gerstein, G.L., Bedenbaugh, P., and Aertsen, A.M.H.J. (1989) Neuronal assemblies. IEEE Transactions on Biomedical Engineering , 36, 4–14. Ghose, G.M., and Freeman, R.D. (1992) Oscillatory discharge in the visual system: Does it have a functional role? Journal of Neurophysiology , 68 , 1558–1574. Ghose, G.M., Ohzawa, I., and Freeman, R.D. (1994) Receptive-field maps of correlated discharge between pairs of neurons in the cat’s visual cortex. Journal of Neurophysiology , 71, 330–346. Ghose, G., and Freeman, R.D. (1997) Intracortical connections are not required for oscillatory activity in the visual cortex. Visual Neuroscience , 14, 963–979. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of awake macaque. Experimental Brain Research , 84 , 505–516. Gray, C.M., and Singer, W. (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 86, 1698–1702. Gray, C.M., König, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-colomnar synchronisation which reflects global stimulus properties. Nature, London . 338, 334–337. Gray, C.M., Engel, A.K., König, P., and Singer, W. (1990) Stimulus-dependent neuronal oscillations in cat visual cortex: receptive field properties and feature dependence. European Journal of Neuroscience , 2, 607–619. Gray, C.M., and McCormick, D.A. (1996) Chattering cells: Superficial pyramidal neurons contributing to the generation of synchronous oscillations in the visual cortex. Science, New York , 274, 109–113. Gray, C.M., and Viana Di Prisco, G. (1997) Stimulus dependent neuronal oscillations and local synchronization in striate cortex of the alert cat. Journal of Neuroscience , 17, 3239–3253. Griffith, J.S., and Horn, G. (1963) Functional coupling between cells in the visual cortex of the unrestrained cat. Nature, London , 199, p. 876 and 893–895. Guido, W., and Weyand, T. (1995) Burst responses in thalamic relay cells of the awake behaving cat. Journal of Neurophysiology , 74, 1782–1786. Gustafsson, B., and McCrea, D. (1984) Influence of stretch-evoked synaptic potentials on firing probability of cat spinal motoneurones. Journal of Physiology (London) , 347, 431–451. Gustafsson, B., Wigström, H., Abraham, W.C., and Huang, Y.-Y. (1987) Long-term
Cross-correlograms for neuronal spike trains
105
potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials. Journal of Neuroscience , 7, 774–780. Hansel, D., and Sompolinsky, H. (1996) Chaos and synchrony in a model of a hypercolumn in visual cortex. Journal of Computational Neuroscience , 3, 7–37. Harvey, A.R. (1980) A physiological analysis of subcortical and commissural projections of area 17 and 18 of the cat. Journal of Physiology (London) , 302, 507–535. Hata, Y., Tsumoto, T., Sato, H., and Tamura, H. (1991) Horizontal interactions between cortical neurones studied by cross-correlation analysis in the cat. Journal of Physiology (London) , 441, 593–614. Hebb, D.O. (1949) The organization of behavior: A neuropsychological theory . John Wiley & Sons, New York. Holmes, O., and Houchin, J. (1966) Units in the cerebral cortex of the anesthetized rat and the correlations between their discharges. Journal of Physiology (London) , 187, 651–671. Innocenti, G.M. (1980) The primary visual pathway through the corpus callosum: Morphological and functional aspects in the cat. Archives Italiennes de Biologie , 118, 124–188. Jagadeesh, B., Gray, C.M., and Ferster, D. (1992) Visually evoked oscillations of membrane potential in cells of cat visual cortex. Science, New York , 257, 552–554. Jahnsen, H., and Llinás, R. (1984) Electrophysiological properties of guinea-pig thalamic neurons, an in vitro study. Journal of Physiology (London) , 349, 205–226. Jester, J.M., Campbell, L.W., and Sejnowski, T.J. (1995) Associative EPSP-spike potentiation induced by pairing orthodromic and antidromic stimulation in rat hippocampal slices. Journal of Physiology (London) , 484, 689–705. Jones, R.S.G., and Heinemann, U. (1988) Synaptic and intrinsic responses of medial enthorinal cortical cells in normal and magnesium-free medium in vitro. Journal of Neurophysiology , 59 , 1476–1496. Keifer, J.C., Baghdoyan, H.A., Becker, L., and Lydic, R. (1994) Halothane decreases pontine acetylcholine release and increases EEG spindles. NeuroReport , 5, 577–580. Kim, U., Bal, T., and McCormick, D.A. (1995) Spindle waves are propagating synchronized oscillations in the ferret LGNd in vitro. Journal of Neurophysiology , 74, 1301–1323. Kiper, D.C., Gegenfurtner, K.R., and Movshon, J.A. (1996) Cortical oscillatory responses do not affect visual segmentation. Vision Research , 36, 539–544. Kirkwood, P.A., and Sears, T.A. (1978) The synaptic connexions to intercostal motoneurones as revealed by the average common excitation potential. Journal of Physiology (London) , 275, 103–134. Kirkwood, P.A., and Sears, T.A. (1982) The effect of single afferent impulses on the probability of firing of external intercostal motoneurones in the cat. Journal of Physiology (London) , 322, 315–336. Knierim, J.J., and Van Essen, D.C. (1992) Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. Journal of Neurophysiology , 67, 961–980. Knox, C.K. (1974) Cross-correlation functions for a neuronal model. Biophysical Journal , 14 , 567–582. Koch, C., and Schuster, H. (1992) A simple network showing burst synchronization without frequency locking. Neural Computation , 4, 211–223. König, P., and Schillen, T.B. (1991) Stimulus-dependent assembly formation of oscillatory responses: I.Synchronization. Neural Computation , 3, 155–166. König, P., Engel, A.K., and Singer, W. (1995) Relationship between oscillatory activity
Time and the brain
106
and long-range synchronization in cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 92, 290–294. König, P., Engel, A.K., and Singer, W. (1996) Integrator or coincidence detector? the role of cortical neurons revisited. Trends in Neuroscience , 19 , 130–137. Kreiter, A.K., and Singer, W. (1992) Oscillatory neuronal responses in the visual cortex of the awake monkey. European Journal of Neuroscience , 4, 369–375. Kreiter, A.K., and Singer, W. (1996) Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. Journal of Neuroscience , 16 , 2381–2396. Krüger, J., and Aiple, F. (1988) Multimicroelectrode investiguation of monkey striate cortex: Spike train correlations in the infragranular layers. Journal of Neurophysiology , 60 , 798–828. Krüger, J., and Mayer, M. (1990) Two types of neuronal synchrony in monkey striate cortex. Biological Cybernetics , 64 , 135–140. Kruse, W., and Eckhorn, R. (1996) Inhibition of sustained gamma oscillations (35–80 Hz) by fast transient responses in cat visual cortex. Proceedings of the National Academy of Science, U.S.A. , 93, 6112–6117. Lamme, V.A.F. (1995) The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience , 15, 1605–1615. Levick, W.R., Cleland, E.G., and Dubin, M.W. (1972) Lateral geniculate neurons of cats: Retinal inputs and physiology. Investigative Ophthalmology , 11 , 302–311. Li, C.-L. (1959) Synchronization of unit activity in the cerebral cortex. Science, New York , 129, 783–784. Livingstone, M.S. (1996) Oscillatory firing and interneuronal correlations in squirrel monkey striate cortex. Journal of Neurophysiology , 75, 2467–2485. Llinás, R.R., Grace, A.A., and Yarom, Y. (1991) In vitro neurons in mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10- to 50-Hz frequency range. Proceedings of the National Academy of Science, U.S.A. , 88, 897–901. Llinás, R., Ribary, U., Joliot, M., and X.-J. Wang (1994) Content and context in temporal thalamocortical binding. In: G. Buzsáki, R. Llinás, W. Singer, A. Berthoz and Y. Christen (eds.) Temporal coding in the brain , Springer Verlag, Heidelberg, pp. 251– 272. Lopes da Silva, F.H., van Rotterdam, A., Storm van Leeuwen, W., and Tielen, A.M. (1970) Dynamic characteristics of visual evoked potentials in the dog. I. Beta frequency selectivity in evoked potentials and background activity. Electroencephalography and Clinical Neurophysiology , 29 , 260–268. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997a) Neuronal dynamics in a model of the thalamocortical system. I. Layers, loops and the emergence of fast synchronous rhythms. Cerebral Cortex , 7, 207–227. Lumer, E.D., Edelman, G.M., and Tononi, G. (1997b) Neuronal dynamics in a model of the thalamocortical system. II. The role of neuronal synchrony tested through perturbations of spike timing. Cerebral Cortex , 7, 228–236. Lytton, W.W., and Sejnowski, T.J. (1991) Simulation of cortical pyramidal neurons synchronized by inhibitory interneurons. Journal of Neurophysiology , 66 , 1059–1079. Markram, H. (1997) A network of tufted layer 5 pyramidal neurons. Cerebral Cortex , 7, 523–533. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, New York , 275, 213–215.
Cross-correlograms for neuronal spike trains
107
Mason, A., and Larkman, A. (1990) Correlations between morphology and electrophysiology of pyramidal neurons in slices of rat visual cortex. II. Electrophysiology. Journal of Neuroscience , 10, 1415–1428. Mastronarde, D.N. (1983a) Correlated firing of cat retinal ganglion cells. I. Spontaneously active inputs to X- and Y-cells. Journal of Neurophysiology , 49 , 303– 324. Mastronarde, D.N. (1983b) Correlated firing of cat retinal ganglion cells. II. Responses of X-and Y-cells to single quantal events. Journal of Neurophysiology , 49, 325–349. Matsumura, M., Chen, D.-F., Sawaguchi, T., Kubota, K., and Fetz, E.E. (1996) Synaptic interactions between primate precentral cortex neurons revealed by spike-triggered averaging of intracellular membrane potentials in vivo. Journal of Neuroscience , 16 , 7757–7767. McCormick, D.A., Connors, B.W., Lighthall, J.W., and Prince, D.A. (1985) Comparative electro-physiology of pyramidal and sparsely spiny stelate neurons of the neocortex. Journal of Neurophysiology , 54, 782–806. McCormick, D.A., and Bal, T. (1997) Sleep and arousal: thalamocortical mechanisms. Annual Review of Neuroscience , 20 , 185–215. Meister, M., Lagnado, L., and Baylor, D.A. (1995) Concerted signaling by retinal ganglion cells. Science, New York , 270, 1207–1210. Melssen, W.J., and Epping, W.J.M. (1987) Detection and estimation of neuronal connectivity based on crosscorrelation analysis. Biological Cybernetics , 57, 403–414. Michalski, A., Gerstein, G.L., Czarkowska, J., and Tarnecki, R. (1983) Interactions between cat striate cortex neurons. Experimental Brain Research , 51 , 97–107. Milner, P.M. (1974) A model for visual shape recognition. Psychological Review , 81 , 521–535. Moore, G.P., Segundo, J.P., Perkel, D.H., and Levitan, H. (1970) Statistical signs of synaptic interaction in neurons. Biophysical Journal , 10, 876–900. Munk, M.H.J., Nowak, L.G., Nelson, J.I., and Bullier, J. (1995) The structural basis of cortical synchronization. II. Localization of the common input. Journal of Neurophysiology , 74, 2401–2414. Munk, M.H.J., Roelfsema, P.R., König, P., Engel, A.K., and Singer, W. (1996) Role of reticular activation in the modulation of intracortical synchronization. Science, New York , 272, 271–274. Murphy, J.T., Kwan, H.C., and Wong, Y.C. (1985) cross-correlation studies in primate motor cortex: synaptic interaction and shared input. Canadian Journal of Neurological Science , 12 , 11–23. Murthy, V., and Fetz, E.E. (1992) Coherent 25- to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys. Proceedings of the National Academy of Science, U.S.A. , 89, 5670–5674. Murthy, V.N., and Fetz, E.E. (1996) Synchronization of neurons during local field potential oscillations in sensorimotor cortex of awake monkeys. Journal of Neurophysiology , 76, 3968–3982. Nelson, J.I., and Frost, B.J. (1978) Orientation-selective inhibition from beyond the classical receptive field. Brain Research , 139 , 359–365. Nelson, J.I., Salin, P.A., Munk, M.H.-J., Arzi, M., and Bullier, J. (1992) Spatial and temporal coherence in cortico-cortical connections: A cross-correlation study in area 17 and 18 in the cat. Visual Neuroscience , 9, 21–37. Neuenschwander, S., and Singer, W. (1996) Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature, London , 379 ,
Time and the brain
108
728–733. Neven, H., and Aertsen, A. (1992) Rate coherence and event coherence in the visual cortex: a neuronal model of object recognition. Biological Cybernetics , 67 , 309–322. Noda, H., and Adey, W.R. (1970) Firing of neuron pairs in cat association cortex during sleep and wakefulness. Journal of Neurophysiology , 33, 672–684. Noda, H., and Adey, W.R. (1973) Neuronal activity in the association cortex of the cat during slepp, wakefulness and anesthesia. Brain Research , 54, 243–259. Nowak, L.G., Munk, M.H.J., James, A.C., Nelson, J.I., and Bullier, J. (1995) The structural basis of cortical synchronization. I. 3 types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–2400. Nowak, L.G., Sanchez-Vives, M.V., and McCormick, D.A. (1997) Influence of low and high frequency fluctuations on spike timing in visual cortical neurons. Cerebral Cortex , 7, 487–501. Nowak, L.G., and Bullier, J. (1998a) Axons, but not cell bodies, are activated by electrical stimulation in the cortical gray matter. I. Evidence from chronaxie measurements. Experimental Brain Research , 118, 477–488. Nowak, L.G., and Bullier, J. (1998b) Axons, but not cell bodies, are activated by electrical stimulation in the cortical gray matter. II. Evidence from selective inactivation of cell bodies and axon initial segment.Experimental Brain Research , 118, 489–500. Nowak, L.G., Munk, M.H.J., James, A.C., Girard, P., and Bullier, J (1999) Crosscorrelation study of the temporal interactions between areas V1 and V2 of the macaque monkey. Journal of Neurophysiology . 81:1057–1074. Nuñez, A., Amzica, F., and Steriade, M. (1992) Voltage-dependent fast (20–40 Hz) oscillations in long-axoned neocortical neurons. Neuroscience , 51, 7–10. Pasternak, G. (1993) Pharmacological mechanisms of opioid analgesics. Clinical Neuropharmacology , 16 , 1–18. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967a) Neuronal spike trains and stochastic point processes. I. The single spike train. Biophysical Journal , 7, 391–418. Perkel, D.H., Gerstein, G.L., and Moore, G.P. (1967b) Neuronal spike trains and stochastic point processes. II. simultaneous spike trains. Biophysical Journal , 7, 419– 440. Pinault, D., and Deschênes, M (1992) Voltage-dependent 40 Hz oscillations in rat reticular thalamic neurons in vivo. Neuroscience , 51, 245–258. Pockberger H. (1991) Electrophysiological and morphological properties of rat motor cortex neurons in vivo. Brain Research. 539, 181–190. Reid, R.C., and Alonso, J.-M. (1995) Specificity of monosynaptic connections from thalamus to visual cortex. Nature, London , 378, 281–284. Renaud, L.P., and Kelly, J.S. (1974) Identification of possible inhibitory neurons in the pericruciate cortex of the cat. Brain Research , 79, 9–28. Reyes, A.D., and Fetz, E.E. (1993) Effects of transient depolarizing potentials on the firing rate of cat neocortical neurons. Journal of Neurophysiology , 69 , 1673–1683. Riehle, A., Grün, S., Diesmann, M., and Aertsen, A. (1997) Spike synchronization and rate modulation differentially involved in motor cortical function. Science, New York , 278, 1950–1953. Ritz, R., Gerstner, W., Fuentes, U., and van Hemmen, J. (1994) A biologically motivated and analytically soluble model of collective oscillations in the cortex. II. Application to binding and pattern segmentation. Biological Cybernetics , 71, 349–358. Robson, J.G., and Troy, J.B. (1987) Nature of the maintained discharge of Q, X, and Y
Cross-correlograms for neuronal spike trains
109
retinal ganglion cells of the cat. Journal of the Optic Society of America , Series A, 4, 2301–2307. Rodieck, R.W. (1967) Maintained activity of cat retinal ganglion cells. Journal of Neurophysiology , 30, 1043–1071. Roelfsema, P.R., König, P., Engel, A.K., Sireteanu, R., and Singer, W. (1994) Reduced synchronization in the visual cortex of cats with strabismic amblyopia. European Journal of Neuroscience , 6, 1645–1655. Roelfsema, P.R., Engel, A.K., König, P., and Singer, W. (1997) Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature, London , 385, 157–161. Rougeul, A., Bouyer, J.J., Dedet, L., and Debray, O. (1979) Fast somato-parietal rhythms during combined focal attention and immobility in baboon and squirrel monkey. Electroencephalography and Clinical Neurophysiology , 46, 310–319. Sanes, J.N., and Donoghue, J.P. (1993) Oscillations in local field potentials of the primate motor cortex during voluntary movement. Proceedings of the National Academy of Science, U.S.A. , 90 , 4470–4474. Schein, S.J., and Desimone, R. (1990) Spectral properties of V4 neurons in the macaque. Journal of Neuroscience , 10, 3369–3389. Schindler, K.A., Bernasconi, C.A., Stoop, R., Goodman, P.H., and Douglas, R.J. (1997) Chaotic spike patterns evoked by periodic inhibition in rat cortical neurons. Zeitschrift für Naturforschung . 52a, 509–512. Schwarz, C, and Bolz, J. (1991) Functional specificity of a long-range horizontal connection in cat visual cortex: A cross-correlation study. Journal of Neuroscience , 11, 2995–3007. Shadlen, M.N., and Newsome, W.T. (1994) Noise, Neural codes and cortical organization. Current Opinion in Neurobiology , 4, 569–579. Sillito, A.M., Jones, H.E., Gerstein, G.L., and West, D.C. (1994) Feature linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London , 369, 479–82. Sillito, A.M., Grieve, K.L., Jones, H.E., Cudeiro, J., and Davies, J. (1995) Visual cortical mechanisms detecting focal orientation discontinuities. Nature, London , 378, 492– 496. Softky, W., and Koch, C. (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. Journal of Neuroscience , 13, 334–350. Sporns, O., Gally, J.A., Reeke, G.N., and Edelman G.M. (1989) Reentrant signaling among simulated neuronal groups leads to coherency in their oscillatory activity. Proceedings of the National Academy of Science, U.S.A. , 86, 7265–7269. Steriade, M., Nuñez, A., and Amzica, F. (1993a) A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. Journal of Neuroscience , 13, 3252–3265. Steriade, M., Contreras, D., Curro Dossi, R., and Nuñez, A. (1993b) The slow (<1 Hz) oscillation in reticular thalamic and thalamocortical neurons: scenario of sleep rhythm generation in interacting thalamic and neocortical networks. Journal of Neuroscience , 13, 3284–3299. Steriade, M., Curró Dossi, R., and Contreras, D. (1993c) Electrophysiological properties of intralaminar thalamocortical cells discharging rhythmic (~ 40 Hz) spike-bursts at ~ 1000 Hz during waking and rapid eye movement sleep. Neuroscience , 56, 1–9. Steriade, M., Nuñez, A., and Amzica, F. (1996a) Synchroniszation of fast (30–40 Hz)
Time and the brain
110
spontaneous cortical rhythms during brain activation. Journal of Neuroscience , 16, 392– 417. Steriade, A., Contreras, D., Amzica, F., and Timofeev, I. (1996b) Synchronization of fast (30–40 Hz) spontaneous oscillations in intrathalamic and thalamocortical networks. Journal of Neuroscience , 16, 2788–2808. Steriade, M., Timofeev, I., Dürmüller, N., and Grenier, F. (1998) Dynamic properties of corticothalamic neurons and local cortical interneurons generating fast rhythms (30–40 Hz) spike bursts. Journal of Neurophysiology , 79, 483–90. Swadlow, H.A. (1995) Influence of VPM afferents on putative interneurons in S1 of the awake rabbit: evidence from cross-correlation, microstimulation, and latencies to peripheral sensory stimulation. Journal of Neurophysiology , 73, 1584–1599. Tallon-Baudry, C., Bertrand O., Delpuech C., and Pernier, J. (1996) Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. Journal of Neuroscience , 16, 4240–4249. Tallon-Baudry, C., Bertrand, O., Delpuech, C., and Pernier, J. (1997) Oscillatory gammaband (30–70 Hz) activity induced by a visual search task in human. Journal of Neuroscience , 17, 722–734. Tallon-Baudry, C., Bertrand, O., Peronnet, F., and Pernier, J. (1998) Induced gammaband activity during the delay of a visual short-term memory task in humans. Journal of Neuroscience , 18, 4244–4254. Tamura, H., Sato, H., Katsuyama, N., Hata, Y., and Tsumoto, T. (1996) Less segregated processing of visual information in V2 than in V1 of the monkey visual cortex. European Journal of Neuroscience , 8, 300–309. Tanaka, K. (1983) Cross-correlation analysis of geniculostriate neuronal relationship in cats. Journal of Neurophysiology , 49 , 1303–1318. Thomson, A.M., and Deuchars, J. (1997) Synaptic interactions in neocortical local circuits: Dual intracellular recordings in vitro. Cerebral Cortex , 7, 510–522. Thorpe, S., Fize, D., and Marlot, C. (1996) Speed of processing in the human visual system. Nature, London , 381, 520–522. Tononi, G., Sporns, O., and Edelman, G.M. (1992) Reentry and the problem of integrating multiple cortical areas: simulation of dynamic integration in the visual system. Cerebral Cortex , 2, 310–335. Toyama, K., Kimura, M., and Tanaka, K. (1981a) Cross-correlation analysis of interneuronal connectivity in cat visual cortex. Journal ofNeurophysiology , 46, 191– 201. Toyama, K., Kimura, M., and Tanaka, K. (1981b) Organization of cat visual cortex as investigated by cross-correlation technique. Journal of Neurophysiology , 46, 202–214. Troy, J.B., and Robson, J.G. (1992) Steady discharges of X and Y retinal ganglion cells of cat under photopic illuminance. Visual Neuro science , 9, 535–553. Troyer, T.W., and Miller, K.D. (1997) Physiological gain leads to high ISI variability in a simple model of a cortical regular spiking cell. Neural Computation , 9, 971–983. Ts’o, D.Y., Gilbert, C.D., and Wiesel, T.N. (1986) Relationship between horizontal interactions and functinal architecture in cat striate cortex as revealed by crosscorrelation analysis. Journal of Neuroscience , 6, 1160–1170. Vaadia, E., and Abeles, M. (1987) Temporal firing patterns of single units, pairs and triplets of units in the auditory cortex. Israel Journal of Medical Science , 23, 75–83. Vaadia, E., Ahissar, E., Bergman, H., and Lavner, Y. (1991) Correlated activity of neurons: A neural code for higher brain functions? In: Krüger, J. (ed) Neuronal Cooperativity , Springer-Verlag Berlin Heidelberg, 1991, 205–279.
Cross-correlograms for neuronal spike trains
111
Vaadia, E., Haalman, I, Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, London , 373, 515–518. van Vreeswijk, C., and Sompolinski, H. (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, New York , 274, 1724–1726. von der Malsburg, C. (1981) The correlation theory of brain function. Internal Report 81–2, Dept. of Neurobiology, Max-Plank-Institute for Biophysical Chemistry, 3400 Göttingen, W-Germany. von der Malsburg, C., and Schneider, W. (1986) A neural cocktail-party processor. Biological Cybernetics , 54, 29–40. Wang, X.-J., and Buzsáki, G. (1996) Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. Journal of Neuroscience , 16, 6402–6413. Wang Roe, A., and Ts’o, D.Y. (1997) The functional architecture of area V2 in the macaque monkey. In: K.E.Rockland, J.H.Kaas and A.Peters (eds), Cerebral Cortex, Volume 12, Extrastriate Cortex in primate , Plenum Press, New York and London, pp. 295–257. Wilson, M.A., and Bower, J.M. (1991) A computer simulation of oscillatory behavior in primary visual cortex. Neural Computation , 3, 498–509. Whittington, M.A., Traub, R.D., and Jeffreys, J.G.R. (1995) Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, London , 373, 612–615. Young, M.P., Tanaka, K., and Yamane, S. (1992) On oscillating neuronal responses in the visual cortex of the monkey. Journal of Neurophysiology , 67, 1464–1474.
3 The Space-Time Continuum in Mammalian Sensory Pathways Asif A.Ghazanfar and Miguel A.L. Nicolelis Department of Neurobiology, Box 3209 Duke University Medical Center, Durham, NC 27710 correspondence to: Miguel A.L.Nicolelis, M.D., Ph.D, Department Neurobiology, Box 3209, Duke University Medical Center, 101 Research Drive, Bryan Research Building, Durham, NC, 27710 USA Phone: 919–684–4580; FAX: 919–684–5435; e-mail: [email protected]
Animals in their natural environments actively process spatiotemporally complex sensory signals in order to guide adaptive behaviour. It therefore seems likely that the properties of both single neurones and neural ensembles should reflect the dynamic nature of such interactions. We review several recent studies that demonstrate the existence of timedependent receptive fields and distributed coding of sensory signals by neurones in the auditory, visual and somatosensory thalamocortical pathways—features that seem to be constrained by the need to interpret actively time-varying stimuli. Based on these results, the central nervous system does not appear to operate as a static decoder, composed of highly-specialized feature-detector neurones, which sits idly until inputs from the environment are conveyed through it. On the contrary, the evidence indicates that perception emerges as the outcome of highly distributed interactions between populations of broadly-tuned neurones throughout the multiple and reciprocally-connected subcortical and cortical areas that define a sensory system. KEYWORDS: Neural ensembles; Coding; Spatiotemporal; Thalamocortical; Feedback “The trace of any activity is not an isolated connexion between sensory and motor elements. It is tied in with the whole complex of spatial and temporal axes of nervous activity which forms a constant substratum of behaviour”—Lashley, 1950.
1. INTRODUCTION It is no longer tenable to ignore the fact that the temporal structure of both natural signals
The space-time continuum in mammalian sensory pathways
113
and exploratory behaviours used by animals in their ecological niche constrain the mechanisms employed by the central nervous system to acquire, process, and decode sensory information. Until recently, however, very few neurophysiological theories of perception incorporated these dynamic aspects of sensory information processing. Instead, the emphasis of mainstream neural coding theories was placed on the existence of static and topographic mapping functions, which had as their central elements highly specialized “feature extracting” single neurones (Barlow, 1972; Hubel and Wiesel, 1962). The fact that sensory cues in the natural environment typically have intricate timevarying properties, and that animals can perceive such sensory stimuli very rapidly, suggests that the mammalian central nervous system is capable of processing complex time-dependent signals very efficiently. For example, speech and other vocalization signals have unique spectro-temporal signatures in which the timing and order of basic components (e.g. syllables, phonemes, motifs, etc.) play a critical role in distinguishing the meaning of different sounds. Thus, neural circuits have to decode the temporal structure of these signals in order to allow the subject to perceive and respond adaptively to them. The necessity of having time-dependent neural processes underlying perception is also indicated by the fact that, in natural settings, animals gather information about the surrounding environment by actively moving their sensory receptor arrays as they engage in exploratory behaviours. For example, during active exploration of their haptic environments, primates employ stereotypical movements of their hands to perceive minute tactile differences in object size, texture, and shape. When primates voluntarily move their fingers across an object, their ability to discriminate a variety of tactile features is improved considerably when compared to the passive reception of the same features (Darian-Smith, 1984). Analogous dependencies between active exploration and optimal sensory discrimination have been observed in other sensory modalities, such as saccadic eye movements during visual search and object recognition or head orientation movements for sound localization and recognition. Thus, most of the sensory information gleaned from the environment is sampled through actions, and leads to the genesis of percepts that seem to unfold continuously over time rather than in a series of discrete events. In this review, we address one of the central questions in modern neuroscience: how does the central nervous system rely on time-dependent computations to process behaviourally relevant sensory signals? In the last decade or so, many laboratories have dealt with a variety of questions that revolve around this central theme. Here, an attempt is made to summarize some of these new results. In the first half of this chapter, we describe how the discovery of spatiotemporal receptive fields across the thalamocortical loop has revealed a new dimension of sensory information-coding, which emphasizes the ability of the CNS to process time-varying signals during exploratory behaviours. In the second half of this chapter, we discuss how time-dependent neural coding schemes can be exploited by populations of neurones in the visual, auditory, and somatosensory thalamocortical pathways. The data reviewed here suggest that the presence of stimulus-dependent changes in the temporal structure of neuronal spike trains can be used to encode different types of sensory information. The temporal precision required in most of these schemes is of the
Time and the brain
114
order of 5–50 milliseconds, a time-scale very similar to those used to describe the spatiotemporal organization of thalamic and cortical neuronal receptive fields.
2. DYNAMIC RECEPTIVE FIELDS 2.1. Beyond the Classical Receptive Field Since the early work in the visual system (Hartline, 1950), the analysis of the receptive field (RF) properties of single neurones has occupied centre stage in sensory neurophysiology and, as a consequence, has guided the definition of a variety of cardinal principles in the investigation of sensory systems. Classically, the RF of a neurone is defined in terms of spatial co-ordinates (for the visual and somatosensory system), spectral co-ordinates (in the case of the auditory system), or by an arbitrary parameter space that has behavioural relevance (such as a set of odours or tastes for the olfactory and gustatory systems). Simply put, the RF of a neurone has been defined as the area of the particular “parameter space” within which a stimulus can elicit a neuronal response (measured in terms of the number of action potential) which is significantly above the spontaneous firing rate. Although the response properties of neurones are inherently both spatial and temporal, most studies have focused only on the spatial aspects of the RF of single neurones. This is evident from the voluminous amounts of work that have focused on “mapping” the organization of topographic representations of sensory space found in cortical and subcortical areas of the brain. As a result of the limited scope of these studies, neurones with particular spatial or spectral RF properties have been characterized simply as static filters of sensory inputs—filters that describe a neurone’s selectivity for features such as orientation or frequency tuning. The development of new RF mapping techniques has facilitated the characterization of RFs in both the spatial and temporal domains. Application of these methods in different sensory systems has demonstrated the existence of time-dependent RFs in multiple processing levels of the neuraxis. The central results of all these studies was the demonstration that it is inadequate to describe stimulus selectivity or representation in terms of the maximal or average spike counts averaged over a large time epoch, because the structure of RFs varies as a function of time. 2.2. Dynamic Frequency Tuning of Auditory Cortical Neurones: Adaptations to Complex Sound Processing In no other system has the role of time-dependent variables been emphasized more than in the auditory system. Over the years, information embedded in the form of time duration, interval and temporal order of stimulus features has been demonstrated to convey important information-bearing parameters for auditory signal processing in different species. Elegant studies (Hauser et al., 1998; Knudsen and Konishi, 1979; Margoliash and Fortune, 1992; Suga et al., 1983) have established that different nervous
The space-time continuum in mammalian sensory pathways
115
systems use a variety of temporal cues (i.e. interaural time difference, echolocation, sequences of sounds, etc.) to interpret auditory signals which are used to guide adaptive behaviours. In this context, even a small change in the temporal structure of an auditory signal can have a dramatic effect on its behavioural significance. Despite this wealth of information, very few neurophysiological studies have addressed how the spectrotemporal information embedded in complex sounds is reflected in the receptive fields of single neurones in the mammalian auditory system. In general, studies of the auditory cortex have focused on neuronal responses to simple stimuli, such as pure tunes and broad-band noise at different intensities. Only a handful of studies have been carried out to examine the time-dependence of frequency tuning in the mammalian auditory cortex. These studies have revealed that auditory cortical neurones can exhibit “spectrotemporal” RF, i.e. their frequency tuning may vary as a function of post-stimulus time. In other words, these neurones have “multi-peaked” tuning curves, such that they respond best to a particular frequency interval at one post-stimulus time, and to another frequency range at a later post-stimulus epoch. Thus, whereas most neurones in the peripheral auditory pathways have RFs that respond to a clearly definable frequency range, there is a small proportion of cortical neurones whose RFs contain two or more frequencies that can drive the neurones at the same intensity level (Abeles and Goldstein, 1972; Sutter and Schreiner, 1991). In a systematic study of these neurones, Sutter and Schreiner (1991) presented 675 different frequency-level combinations to each primary auditory cortical neurone in a pseudo-random order. The approach was very similar to the “response plane” technique used in some visual RF mapping studies (see below). These authors found that the two frequencies that elicit robust responses in multi-peaked neurones are often separated by frequencies to which the neurones do not respond at the same intensity levels. Moreover, the bandwidth of each peak in a multi-peaked RF was not significantly different from that of single-peaked neurones. The authors found that there were latency differences between peaks, and that these differences were highly variable; most often, however, the lowest frequency stimulus induced the shortest latency neuronal response, while the highest frequency stimulus led to the longest latency responses. Thus, these auditory cortical neurones seem to exhibit what one would consider a spectrotemporal RF. This means that the frequency tuning of the neurone changes as a function of post-stimulus time, or that the frequency response and time are coupled. Interestingly, the spectral and temporal ranges of these multi-peaked RFs corresponded to the ranges of behaviourally-relevant auditory signals for the species used for these studies. This raised the question of how these spectrotemporal RFs relate to how auditory neurones respond to natural auditory signals. Studies of the auditory responses to vocalizations in squirrel monkeys have demonstrated that neurones in the auditory cortex are sensitive to species-specific calls and have unique firing patterns for different call types (Winter and Funkenstein, 1973; Wollberg and Newman, 1972). Auditory cortical neurones responded with temporally-complex patterns of discharges, and for a given call type, a variety of temporal firing patterns were observed among different cortical neurones. Likewise, a given cortical neurone could respond to different call types. Editing out parts of a particular vocalization pattern indicated that single cortical neurones could dynamically integrate information from different call components into
Time and the brain
116
their firing patterns (Wollberg and Newman, 1972). Of the cortical neurones that responded to vocalizations, the majority of them responded to steady pure tones as well. Winter and Funkenstein (1973) reported that the responses to vocalizations could be predicted qualitatively by the neurone’s response to simpler stimuli, such as clicks and pure tones. More recent experiments on call-selective neurones in the auditory cortex of anaesthetized rhesus (Rauschecker et al., 1995) and marmoset monkeys (Callithrix jacchus) (Wang et al., 1995) have largely supported the latter results from squirrel monkeys. In the marmoset, primary auditory neurones phaselocked to the envelope portion of the vocalization, which was centred around the neurone’s frequency tuning (Wang et al., 1995). In the rhesus monkey, when different segments of a particular vocalization were used as stimuli for a call-selective neurone, the response could also be predicted by the frequency tuning curve of the neurone (Rauschecker et al., 1995). Thus, the spectral and temporal nature of a particular auditory neurone’s RF can be used, at least to a first approximation, to predict the firing patterns of a neurone to behaviourally-relevant vocal signals. Further testing of this assumption was performed by mapping the RF of primate auditory cortical neurones with reverse-correlation techniques, and by generating corresponding synthetic sounds (deCharms et al., 1998; Tramo et al., 1996). In these cases, the auditory stimuli used for the reverse-correlation consisted of rapid sequences of tones or chords. As mentioned above, for cats, in the primate primary auditory cortex, neurones have RFs whose best-frequency varies as a function of time. In addition, these spectro-temporal RFs can be used to a certain extent to predict the responses of these neurones to synthetic versions of species-specific vocalizations (Tramo et al., 1996). In the primary auditory cortex of the owl monkey, deCharms et al. (1998) showed that neurones have complex patterns of spectrotemporal RFs that indicate preferences for stimulus edges, stimulus transitions in frequency or intensity, and conjunctions of different stimulus features. To test the integrative properties of these neurones, different classes of auditory stimuli were generated based on the structure of their spectro-temporal RFs. Once these synthetic stimuli were presented they were able to drive the corresponding auditory cortical neurone at much higher rates than simple stimuli, the when the optimal stimulus was expanded or contracted in either the spectral or temporal domain (deCharms et al., 1998). These results support the notion that spectrotemporal RF properties reflect the solution found by the central nervous system for processing and interpreting natural, time-varying, and behaviourally-relevant auditory signals. In this sense there is a reciprocal relationship between the dynamic structure of single neurone RFs and the time-varying signature of natural sounds. Indeed, the responses to natural vocal signals could be used to predict, to some extent, the spectrotemporal structure of a neurone’s RF, and, conversely, the spectrotemporal RF could be used to predict a given neurone’s response to complex synthetic sounds. 2.3. Receptive Field Dynamics in the Lateral Geniculate Nucleus Traditionally, the RFs of lateral geniculate nucleus (LGN) neurones have been described
The space-time continuum in mammalian sensory pathways
117
in purely spatial terms as circular with an “on-centre”, which responds to the onset of a bright stimulus, and an “off-surround”, which responds to the onset of a dark stimulus (or the off-set of a bright stimulus). LGN neurones of the opposite polarity exist as well. Perhaps, the earliest study suggesting that time was an important dimension in the structure of LGN RFs was conducted by Stevens and Gerstein (1976). To characterize the RF of LGN neurones, these authors used the “response plane” technique, a method in which the visually-responsive area of a neurone is divided into a matrix and the spiking of the neurone elicited by a brief flash of light (a spot, or any other stimulus feature) is measured at each position of the matrix. A number of post-stimulus time histograms (equal to the size of the stimulus matrix) are generated and then used to compile the temporal sequences of responses. Using this approach, Stevens and Gerstein (1976) found that the simple, spatial description of cat LGN RFs, such as “on-off” and “centresurround”, was insufficient to describe completely even the simplest of LGN RFs. Overall, these authors reported two different types of LGN RFs. One type displayed spatially homogeneous distributions of excitatory and inhibitory domains, and another type exhibited spatially heterogeneous excitatory and inhibitory domains, which shifted as a function of post-stimulus time. These different dynamic behaviours were hypothesized to originate from two different functional streams, X and Y, within the cat visual system. Thus, these authors state that, “time is at least as important an analytical variable as space to visual neurones” (Stevens and Gerstein, 1976) emphasizing that the coupling between space and time is an integral component of LGN RFs. More recent experiments have supported the original results of Stevens and Gerstein by demonstrating the existence of spatiotemporally coupled visual RFs in both the cat (Dan et al., 1996) and primate LGN (Golomb et al., 1994). These studies employed “reverse-correlation” techniques to map the spatiotemporal response function of a given neurone. In this approach, the structure of the receptive field of a single neurone can be described fully by the measured response to any complete set of stimuli presented in rapid, pseudo-random sequences. One complete set of stimuli is defined by localized flashes of light presented in several random sequences. The average spike count of the neurone is then correlated with the past location and time of each flash, i.e. “reversecorrelated”. In the primate LGN, Golomb et al. (1994) found that both parvocellular and magnocellular neurones had spatiotemporal RFs. Thus, although these neurones still exhibited centre-surround organization, the response to the surround was delayed relative to the response of the centre. Thus, the RFs of these neurones could only be described as space-time inseparable, since at a given moment in time the overall spatial distribution of the RF was different from those observed in a previous or a future time epoch. These RFs could also be used to predict the neurone’s response to complex pattern stimuli (Golomb et al., 1994; see below). Do the spatiotemporal response properties of LGN neurones elicited by simple stimuli relate to their function in coding and representing natural visual signals? This is an important issue, because the demonstration that the spatiotemporal structure of a single neurone RF could be used to predict the neurone’s temporal coding of different visual stimuli would support the notion that the coupling of spatial and temporal dimensions are relevant for sensory information processing. This question was addressed by experiments done in the cat LGN, where the RF structure of single neurones was used to predict the
Time and the brain
118
responses to natural, time-vary ing visual stimuli (Dan et al., 1996). Natural scenes are highly redundant, having both spatial and temporal correlations. Because coding efficiency could be advantageous, both evolutionarily and computationally, it was hypothesized that LGN neurones would decorrelate natural scene stimuli in order to transform this information into a less redundant signal (Atick and Redlich, 1990). This was tested by recording the activity of LGN neurones in response to natural, time-varying scenes and measuring the power spectra of the neural signals (Dan et al., 1996). Indeed, the responses of LGN neurones to movies of natural scenes revealed that their individual spike trains were temporally decorrelated, that is their power spectra were essentially flat from 3Hz to 15Hz in response to natural scenes. Thus, although the natural scenes have intrinsic correlations in their structure, the LGN neuronal responses do not reflect these redundancies and, hence, code the information more efficiently. Furthermore, these authors reported that this decorrelation of LGN responses to natural scenes could only be predicted by the filtering properties of their spatiotemporal RFs (Dan et al., 1996). Thus, in both cat and primate the time-dependent RF structures of LGN neurones have been used to predict the responses and temporal coding of stimuli, for both artificial (Golomb et al., 1994) and natural stimuli (Dan et al., 1996). In both of these studies, feedforward circuitry was hypothesized to account for the dynamic behaviour LGN RFs. However, no experimental test of this “feedforward” hypothesis has been carried out thus far. 2.4. Dynamic Organization of the Primary Visual Cortex Traditionally, visual cortical neurones are classified according to their response selectivity and the shapes of their RFs. “Simple” cells of the primary visual cortex receive most of the LGN inputs, and have RFs that are classically described as being oriented spatially. These RFs exhibit subregions that respond primarily to bright or dark elongated stimuli. “Complex” cells respond to both bright and dark stimuli anywhere within their RFs. Recently, approaches similar to those used for mapping the spatial and temporal properties of LGN neurones have been used to characterize the RFs of visual cortical neurones, in both the cat and the monkey. Both response-plane and reverse-correlation approaches have demonstrated that the RF structure of visual cortical neurones is not static, but varies as function of post-stimulus time. These studies revealed that primary visual cortical neurones also have spatiotemporal RFs whose time-dependent structure can be measured by using a variety of stimulus parameters (overall RF size, location of on-off regions, orientation tuning, directional tuning, and length tuning). Receptive field mapping, in both space and time, has shown that V1 neurones respond to flashes of light in different subregion of their RFs at different times. Using the response plane technique, Dinse et al. (1991) reported that each subfield within the RF of visual cortical neurones, for both on- and off-responses, exhibits its own temporal characteristics. When each response-plane is plotted in sequence (from 20 to 200 milliseconds), a wave of excitation seems to travel and spread across the entire RF. Using the reverse-correlation analysis, where the stimuli consisted of rapidly-flashed patterns of
The space-time continuum in mammalian sensory pathways
119
spots or bars, DeAngelis et al. (1993) found that simple cells of the cat visual cortex have bright and dark excitatory elongated subregions that move as a function of time. The elongated on-off subregions of these spatiotemporal RFs could be used to predict the responses of these simple cells’ responses to bars of light (DeAngelis et al., 1995). In addition to the time-dependent organization of on-off responses, other properties of visual cortical neurones, such as orientation tuning, end-stopping, and directional selectivity, also show a robust time-dependence on a millisecond scale. The first evidence for this time-dependence was provided by Dinse et al. (1991), who employed flickering bars of light of different lengths to investigate the time course of orientation tuning and end-stopping in cat visual cortical neurones. Classically, orientation tuning is defined simply as the selectivity of neuronal responses to bars of a particular orientation, while end-stopping is the selectivity to bars of a particular length. Compiling the sequences of tuning curves for each of these stimulus features in post-stimulus time-steps of 20 milliseconds, these authors demonstrated that the selectivity of single neurones to these features evolved over post-stimulus time. For instance, in the case of orientation tuning, some neurones first exhibited a state of excitation that did not show selectivity for any particular orientation, followed by the gradual sharpening of the selectivity over time. Other neurones showed selectivity for a particular orientation that remained stable over time, while for yet another group of neurones, the selectivity changed over time (i.e. responding best to one orientation at one time epoch, and another orientation at a later time epoch). Nearly identical results were found for length-tuning (Dinse et al., 1991). A laminar comparison of dynamic orientation tuning in the macaque primary visual cortex was performed by applying the reverse-correlation technique in the orientation domain. Ringach et al. (1997) found that orientation selectivity developed after approximately 30–45 milliseconds, and persisted for up to 85 milliseconds over poststimulus time. Comparing the nature of orientation selectivity across different layers of macaque V1, these authors observed that the degree of dynamic orientation tuning depended on the laminar position of the neurones. The cortical layers which received direct input from the LGN (layers 4Ca and 4Cb) contained neurones which showed a single, broadly-tuned orientation preference, that did not change over time, while neurones in the output layers of V1 (layers 2, 3, 4B, 5 and 6) showed a range of dynamic behaviours. In these “output” layers, orientation tuning was narrower than neurones of the input layers, and their preferred orientations changed over post-stimulus time. Furthermore, as reported by Dinse et al. (1991) for cat visual cortex, a subset of neurones in V1 showed orientation tuning that increased in sharpness over time (Ringach et al., 1997). Taken together, the results from these experiments indicate that the neurones in the primary visual cortex are not simply a collection of static spatial filters or feature detectors; instead, their RF size and feature selectivity vary as function of post-stimulus time. 2.4.1. Potential circuit mechanisms underlying visual cortical dynamics An important and unresolved question concerns the neural circuitry underlying the
Time and the brain
120
genesis of visual spatiotemporal receptive fields. Both feedforward and feedback mechanisms have been proposed to account for the type of cortical and subcortical dynamics described above. Feedforward mechanisms could include the convergence of multiple cortical neurones, which do not exhibit spatiotemporal RFs, upon neurones which do. Alternatively, feed-forward inputs from two different classes of LGN neurones, each with different temporal dynamics (lagged versus non-lagged LGN neurones [Saul and Humphrey, 1992]) could account for some of the time-dependency of RFs in V1. As proposed by DeAngelis et al. (1995), simultaneous, multi-site recordings of LGN and V1 neurones (Reid and Alonso, 1995) are needed to test these hypotheses. Feedback cortical circuitry, on the other hand, could account for the dynamic feature tuning and the increases in selectivity of V1 neurones observed over poststimulus time (Dinse et al., 1991; Ringach et al., 1997). For orientation tuning and end-stopping, many neurones displayed orientation tuning which became narrower over time. It is conceivable that this increase in selectivity results from a “sculpturing” process, carried out by the asynchronous convergence of multiple feedback projections to the same set of V1 neurones (Dinse et al., 1991). According to this hypothesis, the initial state of widespread excitation of the cortex caused by the onset of the stimulus would be followed by reverberating activity from nearby local circuitry that could increase the selectivity of cortical neurones. In addition to local circuitry, other cortical areas could contribute, via feedback projections, to the definition of the RF structure of V1 neurones at longer latencies (Mignard and Malpeli, 1991). Overall, these results from both the LGN and visual cortex indicate that spatiotemporal RFs properties are likely to exist at most, if not all, processing levels of the visual system, and that this time-dependency in single neurone firing likely results from the asynchronous convergence of many inputs from different feedforward and feedback projections to individual neurones. To test such a hypothesis, one would need to carry out simultaneous multi-site recordings of neuronal responses (i.e. from thalamus and cortex) that are obtained before, during, and after selective inactivation of feedforward and/or feedback pathways. Experiments of this type are more feasible using the rat somatosensory pathway as a model system. 2.5. Spatiotemporal Receptive Fields Across the Somatosensory Thalamocortical Loop 2.5.1. Spatiotemporal receptive fields in the somatosensory thalamus Ensembles of well-isolated single neurones from the ventral posterior medial (VPM) nucleus of the thalamus, and the posteromedial region of the primary somatosensory (SI) cortex, of awake and anaesthetized rats have been simultaneously recorded using a technique based upon chronically-implanted arrays of microwires (Nicolelis and Chapin, 1994; Nicolelis et al., 1997a). These brain regions are known to process tactile information coming from the large facial whiskers on the rat snout. As in traditional studies, the RFs of thalamic and cortical neurones were reconstructed in these studies,
The space-time continuum in mammalian sensory pathways
121
following repetitive mechanical deflections of individual whiskers. For this analysis, a single neurone’s response to the stimulation of ~5x5 matrix of whiskers (deflected one at a time) was divided into 5 millisecond post-stimulus epochs. For each epoch, the whisker that elicited the greatest response (in terms of spike counts) was defined as the “RF centre” for that time epoch. As the “centre” whisker often varied as a function of poststimulus time, we were able to map spatiotemporal RFs for most neurones in both the VPM and SI cortex (Nicolelis and Chapin, 1994; Ghazanfar et al., 1995). This approach is identical to the response plane techniques used to map spatiotemporal RFs in the visual system (Dinse et al., 1991; Stevens and Gerstein, 1976). The chronic and simultaneous neural ensemble recording approach is uniquely suited to carrying out a quantitative analysis of the response properties of neurones across the thalamocortical pathway. First, the stable recordings obtained using the chronic placement of microwires allows one to sample from populations of well-isolated single neurones for several hours at a time, permitting the use of a large stimulus set (5 whiskers) and a large number of trials (360) per site. Second, since many neurones are recorded in parallel, several non-stationary effects (e.g. anaesthesia, placement of stimulus, etc.) can be reduced or eliminated. Finally, this approach permits one to record from the exact same set of neurones before, during and after experimental manipulations, such as deafferentation and/or pharmacological inactivation of different pathways. Traditionally, the RF of a somatosensory neurone is defined as the area of the body surface that, when stimulated, triggers a significant increase in this neurone’s firing rate. As in other species, most classical electrophysiological studies of the “whisker region” of rodent somatosensory system have focused on the characterization of the spatial attributes of the RF of neurones located in the subcortical and cortical processing centres of the trigeminal pathway. Recently, Nicolelis, Chapin and their colleagues (Nicolelis et al., 1993a; Nicolelis and Chapin, 1994) employed simultaneous neural ensemble recordings to sample the sensory responses of populations of neurones distributed across the whisker representation area of the main thalamic nucleus of the trigeminal somatosensory system, the VPM. These authors carried out a detailed analysis of the RFs of individual VPM neurones in both awake and anaesthetized animals, and revealed that single VPM neurones have large, multiwhisker RFs whose centre is defined by one whisker whose stimulation elicits the strongest sensory response at the shortest latency. This whisker is classically known as the principal whisker (PW) of the RF, and is commonly used to identify the location of a given neurone in the topographic maps of the whisker pad observed across the trigeminal system. Further analysis, however, revealed that the location of the PW (or RF centre) of a given VPM neurone could vary as a function of poststimulus time. Indeed, VPM neurones could be divided into two functional classes, according to the time-dependency of their RF centres. In about 41% of the recorded VPM neurones, the spatial position of the RF centre shifted over poststimulus time (Figure 1, bottom panel). The remaining neurones exhibited RF centres that remained in the same whisker over time (Nicolelis and Chapin, 1994).
Time and the brain
122
Figure 1 . Top panel. Spatiotemporal receptive field of a cortical neurone located in layer V of the rat primary somatosensory cortex. Bottom panel Spatiotemporal receptive field of neurone located in the ventral posterior medial nucleus of the rat thalamus. Notice that the spatial domain of these receptive fields varies as a function of post-stimulus time.
The first class of neurones exhibited a spatiotemporal coupling of their RFs, because the position of the RF centre, and its surround, varied as a function of post-stimulus time. In other words, the PW in the short latency component (PWSL) was different from the PW in the long latency component (PWLL) of the response. Interestingly, all spatiotemporal VPM RFs exhibited time-dependent RF shifts of centres in a caudal-to-rostral direction, and, consequently, were termed C->R cells. For example, one particular C->R neurone responded best to its PWSL, whisker C1, at 5–10 milliseconds post-stimulus time, while still responding significantly to many other caudal and rostral whiskers which defined its RF surround. By 15–20 milliseconds, the strongest response for this neurone was no longer elicited by whisker C1, but by whisker B3, which became the PW of the RF at long-latency (or PWLL). Thus, the RF shift observed in the VPM was clearly defined by a smooth transition in the spatial distribution of sensory responses observed in the first 30 ms of post-stimulus time. In general, C->R neurones had the largest RFs (average of 17.2 whiskers). The time-dependent shifting of the centres of these cell’s RFs seemed to evolve gradually, and appears to involve two factors: 1) an early component composed of short-latency responses from the caudal whiskers; and 2) a time-dependent
The space-time continuum in mammalian sensory pathways
123
enhancement of responses to more rostral whiskers. Conversely, a second class of VPM neurone was observed, which did not exhibit any evident time-dependent shift in their RFs. Invariably, this group of neurones had their PWs located in the rostral part of the whisker pad, which contains small and immobile vibrissae. These neurones were labelled “rostral position (RP) cells”. The RFs of these neurones was characterized by the existence of a single whisker (the PW) which elicited the largest response at all post-stimulus times, and by the presence of surrounding whiskers which constituted the RF surround. 2.5.2. Dynamic receptive fields in the primary somatosensory cortex Classically, neurones located within a given cortical barrel column are believed to respond preferentially to the deflection of one “principal” whisker (PW), and to fire less vigorously to stimulation of several adjacent whiskers that define the surround region of the neurone receptive field (SRF) (Armstrong-James and Fox, 1987). As in the VPM, responses to the PW occur at shorter latencies than responses to SRFs; the distribution of response latencies to centre and surround whiskers is clearly bimodal. In a series of experiments, Ghazanfar and Nicolelis, (1999) investigated whether the differences in latency observed between a stimulus applied to a PW or the SRFs could define spatiotemporal RF in the rat SI cortex. This was an interesting question to ask, because until recently most studies of rat SI cortex had focused primarily on the spatial aspects of cortical RF organization. Although multi-whisker RFs had been described in the rat SI before (Chapin, 1986; Simons, 1978), and the existence of a time-dependent spread of cortical recruitment had been proposed (Armstrong-James et al., 1992), the spatiotemporal organization of SI cortical neurone RFs was never characterized. Ghazanfar and Nicolelis, (1999) analyzed in detail the time-dependent RF dynamics of 197 cortical neurones located in layer V of the rat SI cortex. Overall, these authors observed that, for the vast majority of these neurones, the spatial structure of their RFs changed as a function of post-stimulus time (Figure 1, top panel). However, the patterns of these SI cortical spatiotemporal RFs varied widely, unlike those reported for the VPM, which shifted primarily in a caudal to rostral direction, or not at all (Nicolelis and Chapin, 1994). We identified four directions of SI cortical RF shifts: 1) rostral-to-caudal (RC); 2) caudal-to-rostral (CR); 3) dorsal-to-ventral (DV); and 4) ventral-to-dorsal (VD). Neurones whose RFs traversed an equal distance in two or more directions were defined as unclassifiable (UC). Out of the 197 neurones in our sample, 175 (88.8%) exhibited time-dependent shifts in their RF centres, while 22 (11.2%) did not. Of those with spatiotemporal RFs, 17.7% were RC shifters, 16.0% CR shifters, 23.4% DV shifters, and 6.3% VD shifters. The remaining 36.6% (64 out of 175 neurones) could not be classified as having a directional bias. In this study, Ghazanfar and Nicolelis, (1999) also demonstrated that the direction of the spatiotemporal RF shift was dependent upon the location of the PWSL. If a neurone’s PWSL was located in the dorsal part of the whisker pad, then the neurone was more likely to exhibit a DV spatiotemporal RF. If a neurone’s PWSL was in the more rostral part of the whisker pad, then it was more likely to exhibit a RC spatiotemporal RF. In summary,
Time and the brain
124
the authors observed that the large, asymmetrical RFs of layer V SI cortical neurones, and the variations in minimal latencies for surround whiskers, define complex spatiotemporal RFs in which the centre (or the PW) varies as function of post-stimulus time. Four classes of spatiotemporal RF types could be characterized based on directional biases of the RF movement, and these biases were, in part, dependent upon the location of the PWSL. Therefore, the existence of such time-dependent changes in RF structure suggests that populations of SI neurones located in layer V provide rather dynamic representations of the whisker pad, which incorporates both spatial and temporal domains of tactile sensory responses. 2.5.3. Hypothesis concerning the origins of spatiotemporal RFs in the somatosensory thalamocortical pathway The organization of VPM and SI cortical layer V RFs indicated that both thalamic and cortical neurones exhibit a variety of directional biases and complex spatiotemporal coupling. In their original studies (Nicolelis et al., 1993a; Nicolelis and Chapin, 1994), Nicolelis and Chapin postulated that the dynamic nature of these RFs resulted from the asynchronous convergence of feedforward and feedback pathways on to VPM neurones. The same hypothesis was put forth later by Ghazanfar and Nicolelis, (1999) to account for the genesis of spatiotemporal RFs in the SI cortex. For both VPM and SI cortical neurones, asynchronous inputs could arise from parallel, feedforward inputs from the ascending lemniscal and paralemniscal pathways. These pathways arise from the trigeminal brain-stem nuclei, and are known to have different temporal lags. Such a mechanism would be similar to that proposed for the genesis of spatiotemporal RFs in the visual system (DeAngelis et al., 1995). Moreover, feedback connections from a variety of cortical somatosensory areas also converge upon VPM and SI neurones (Fabri and Burton, 1991; Beck et al., 1998). These cortical feedback projections could also contribute to the time-dependent changes in RF centre, observed in both the VPM and the SI cortex. For instance, examination of the VPM C->R RF shifts suggested that these RFs could be broken down into early and late temporal components. These different temporal components of the RFs could emerge as the result of the temporally-asynchronous convergence of multiple excitatory and inhibitory afferents to VPM neurones. In this model, the early component of VPM RFs would result from inputs from the fast ascending somatosensory lemniscal pathways that include afferents from the principal and spinal trigeminal complex of the brainstem (which terminate on the same neurones in the VPM). The late component is a little more difficult to account for, but potential sources of input would include slower-conducting trigeminal pathways, and the very dense excitatory corticothalamic inputs that converge on the distal dendrites of VPM neurones. Although the spatioternporal properties of cortical neurones are likely to vary according to their laminar location, the dynamic RFs of SI cortical neurones described above are also likely to arise from the asynchronous convergence of multiple afferents on the same population of cortical neurones. Such inputs could arise from parallel, feedforward inputs from the lemniscal and paralemniscal pathways, which converge onto
The space-time continuum in mammalian sensory pathways
125
layer V neurones at different timelags, from intracortical feedback connections within SI (Gottlieb and Keller, 1997; Hoeflinger et al., 1995), or from other sensory and motor cortical areas (Fabri and Burton, 1991; Koralek et al., 1990). 2.5.3.1. Testing the asynchronous hypothesis for generation of thalamic spatiotemporal RFs Two types of experiments were employed to test whether an “asynchronous” convergence of inputs could account for the generation of spatiotemporal RFs in the rat VPM nucleus. First, the contribution of feedforward somatosensory pathways to the genesis of their RFs was tested by inducing reversible deafferentation of the whisker pad, through subcutaneous infusions of a local anaesthetic, lidocaine. This local anaesthetic allowed measurements of changes in RF organization which emerge as a consequence of altering the flow of tactile information through ascending feedforward trigeminal somatosensory pathways to the VPM and SI cortex (Nicolelis et al., 1993b; Faggin et al., 1997; Nicolelis, 1997). As predicted, this reversible peripheral deafferentation induced immediate reorganization in both the spatial and temporal properties of VPM and SI RFs. The two major effects were 1) a temporal shift resulting in the elimination or reduction of early, short-latency component of RFs, located within the “anaesthetized zone”, with the enhancement of the later components, and 2) a spatial shift where RFs that were originally centred in the “anaesthetized region” of the whisker pad shifted, immediately following the induction of the deafferentation, to neighbouring territories of the whisker pad. These experiments suggest that peripheral deafferentation largely effects the early, “feedforward” components of the VPM and SI cortical RFs. In a second series of experiments, the potential role of corticofugal projections in the genesis of spatiotemporal structure of VPM RFs was investigated (Ghazanfar et al., 1997; Krupa et al., 1999). The results obtained in these studies were complimentary to those reported for peripheral deafferentation (Nicolelis et al., 1993b; Faggin et al., 1997). Using a combination of multi-site chronic microwire arrays and a stereotaxically-placed cannula, Ghazanfar et al. (1997) and Krupa et al. (1999) were able to inactivate the SI cortex pharmacologically, using the GABA agonist, muscimol. Since the cannula was placed adjacent to a chronically implanted microwire array in the SI cortex, these authors were able to monitor the onset, duration, and tangential spread of muscimol. These experiments revealed that corticofugal projections contribute very importantly to the genesis of longer latency neuronal responses in the VPM nucleus, since, in most cases, blockade of cortical activity attenuates these later response components. As mentioned before, these long-latency components are critical to the definition of spatiotemporal RFs in VPM (Nicolelis and Chapin, 1994), and for the C->R RF shifts observed in this nucleus, since the PWLL (principal whisker or RF centre at long-latency) differs from the one defined at short-latency (the PWSL). Furthermore, it was also observed that cortical feedback projections can account for up to 50% of the immediate plastic effects observed in the VPM following peripheral deafferentation (Krupa et al., 1999). As described above, the results obtained with both peripheral deafferentation and inactivation of corticofugal projections support the asynchronicity hypothesis, and
Time and the brain
126
demonstrate that spatiotemporal RFs, at least in the rat VPM thalamus, are defined by large-scale interactions between multiple brain structures. 2.5.4. Active tactile exploration is required for the genesis of spatiotemporal RFs How do the spatiotemporal structures of VPM and SI RFs emerge in the first place? Upon the discovery of directional RF shifting in the rat VPM, a series of studies tested the possibility that experience-dependent mechanisms, via active whisker movements occurring from the time of early postnatal life, could account for the “sculpturing” of directional bias into the spatiotemporal structure of VPM and cortical RFs. Rats use rhythmic protractions of their long caudal whiskers to explore objects that are placed in front of them (Welker, 1964), while the small, immobile rostral whiskers are used primarily to establish and maintain contact with objects. Because active tactile exploration plays a critical role in the normal development of haptic perception in mammals (Bushnell and Boudreau, 1991), Nicolelis et al. (1996) and Oliveira et al. (1997) postulated that the spatiotemporal RFs of layer V SI cortical and VPM thalamic neurones in the rat probably reflect the pattern of spatiotemporal whisker inputs that occurs during the active “whisking” of object surfaces during natural behaviours. Patterned tactile stimulation generated by active exploration during postnatal development may be required for the normal functional organization of the somatosensory system. To test this hypothesis, the effects of neonatal disruption of active whisker movements on the development of thalamic and cortical spatiotemporal RFs was investigated (Nicolelis et al., 1996; Oliveira et al., 1997). In rats, the facial muscles used during whisker movements are innervated solely by branches of the facial nerve, which is clearly separated from the infraorbital nerve, a branch of the trigeminal nerve, which carries low-threshold tactile information from the facial whiskers. Because of this anatomical peculiarity, it was possible to eliminate the production of active whisker movements in neonatal rats by sectioning the facial nerve before these animals experienced any sort of active whisker movements. Complete peripheral nerve section prevented rats from making active whisker movements on one side of the face throughout their development and adulthood, without affecting the sensory innervation—via the infraorbital nerve—of their facial vibrissae. After the rats reached adulthood, the spatiotemporal structure of somatosensory RFs was examined and it was observed that preventing the expression of active tactile exploration during development dramatically alters the spatiotemporal RF organization of VPM and SI neurones. First, the size of neuronal RFs in both structures was significantly reduced. In the VPM, for instance, the reduction was on average from 13 whiskers (normal animals) to 6 whiskers (in animals with facial nerve section). The spatiotemporal organization of RFs was also disrupted in the VPM and SI cortex. For example, in normal adult rats 100% of VPM neurones whose PWSL were located in the caudal part of the whisker pad exhibited a caudal-to-rostral shift in their RF centres over post-stimulus time. In rats with neonatal facial nerve sections, on the other hand, only 19% of VPM neurones with PWSL in the caudal whisker pad exhibited time-dependent shifts in their RF centres (Nicolelis et al., 1996). In other words, the dynamic properties of VPM neurones were dramatically reduced by
The space-time continuum in mammalian sensory pathways
127
preventing these animals from actively exploring their environment during development. Therefore, these results underscore the importance of active movements for the functional maturation of normal spatiotemporal RFs, in both cortical and subcortical relays of the rat somatosensory system, and implicate experience-dependent mechanisms for the establishment of spatiotemporal RFs in the somatosensory system.
3. SPATIOTEMPORAL CODING BY SINGLE NEURONES AND NEURAL ENSEMBLES 3.1. Rate Versus Temporal Coding? Although the analysis of single neurone RFs continues to play an important role in classical neurophysiology, more contemporary views recognize that the RF of a neurone offers only a probabilistic description of the “potential” responses that can be exhibited by that neurone within the parameter space of a particular type of sensory stimulus (e.g. all the facial whiskers, or all the orientations of a bar of light). Thus, the investigation of how neurones encode information derived from a subset of the parameter space of a sensory stimulus does not rely exclusively on the analysis of single neurone RFs. The investigations of neural codes revolves around identifying a measurement or metric of neuronal activity to describe a particular stimulus parameter, behaviour, or cognitive skill. Neural codes are also studied in the context of how different areas of the brain exchange information and relate to each other. Many neural metrics have been proposed to operate as candidate neural codes in different neural structures and in different species. In general, most of these algorithms have been dichotomized as variations of two main themes: “rate” or “temporal” codes (Shadlen and Newsome, 1994). The prevailing view that firing rate is the universal code was first advocated by Adrian (1928), who suggested that a neural code is simply the mean firing rate of the action potentials evoked from a neurone, and rate variability is “noise”. Because this noise can be filtered out, or reduced, by averaging across time or by recording the activity of large populations of neurones, a rate code performs very well in the presence of noise, but has limited information-carrying capacity. In contrast to the classical rate code hypothesis, multiple schemes have been proposed in which temporal aspects of neuronal firing play a significant role in conveying information. Temporal coding schemes include measurements based on: 1) the precise timing of individual spikes (in a 1–3 millisecond range); 2) the precise temporal sequence of firing across a population of neurones (in a range of <5 millisecond); 3) the occurrence of widely distributed synchronous neuronal firing activity at frequency ranges varying from 1–100 Hz (periods between >10 ms and <1000 ms); 4) the time-dependent modulation of firing rate exhibited by single neurones; and, 5) the occurrence of stimulus or behaviour-related spatiotemporal patterns of firing across populations of neurones (ranging from 10 to 100 ms). Although the jury is still out and not likely to return an early verdict, it seems fair to say that the discussion between rate versus temporal coding is an artificial one. First,
Time and the brain
128
most of the schemes of temporal coding actually represent a particular case of a rate code, which is limited to a particular time epoch. On the other hand, as discussed above, the high level of stimulus selectivity of neurones seems to vary over time, and thus, the identity of a stimulus cannot be ascertained simply by time-averaging each neurone’s response function. This suggests that a strict rate code does not seem to do justice to the richness of neuronal dynamics observed throughout the central nervous system. Instead, the coding of stimuli seems to be best represented when the temporal modulation of the sensory response is taken into account. 3.2. Temporal Coding of Space by Single Neurones 3.2.1. The presence of stimulus-related information in the response waveform ofLGN neurones For many decades, the dominant assumption in visual physiology has been that the strength of a visual neurone’s response is the only variable which contains relevant information about the stimulus (Barlow, 1972). Consequently, the description of temporal structures in the sensory responses of neurones was used simply to classify different LGN cell types (Marrocco, 1976) or to investigate how these neurones integrate their inputs (Dawis et al., 1984). More recently, however, studies using complex patterns of visual stimulation have demonstrated that neurones in the visual pathway respond to a variety of different stimulus attributes. These studies have also revealed that, by including the temporal structure of their sensory responses as an additional dimension of coding, one can increase the information carrying capacity of these neurones. Using behaving primates performing a fixation task, McClurkin et al. (1991a,b) studied the visual responses of single neurones in the parvocellular layers of LGN to a set of two-dimensional black-and-white patterns, called Walsh patterns, that could be used for the linear reconstruction of any picture of the same resolution (in this case, 8 x 8 pixels). The neuronal responses to these different patterns were characterized by examining the strength and temporal pattern of the responses using principle component analysis (PCA). These authors found that parvocellular LGN neurones displayed a variety of temporal firing patterns, and that these firing patterns varied in a reproducible way as a function of the patterns presented. The differences in firing patterns included changes in the magnitude, width, and number of peaks seen in the initial response evoked by the stimulus, as well as changes in the magnitude of the sustained, longer-latency component of the responses. These results were interpreted as an indication that the time-dependent modulation of neuronal firing, i.e. the temporal distribution of LGN spikes, might also encode information in addition to the total number of spikes fired in a given epoch of time (McClurkin et al., 1991a). McClurkin et al. (1991b) then employed PC A to quantify the differences in temporal response patterns elicited by the set of Walsh patterns. By definition, PCs are linearly weighted polynomia which are orthogonal to each other. By correlating the derived PCs with different attributes of neuronal responses, these authors showed that the first
The space-time continuum in mammalian sensory pathways
129
principal component was highly correlated with the overall strength of the response. Higher principal components either were related to the shape of the response (i.e. the temporal distribution of spikes) or identified the particular stimulus eliciting the response. To assess how much stimulus-related information was carried simply in the firing rate versus both the magnitude and shape of the response, information theory analysis was applied to the response strength alone versus the first three principal components, which carry both response strength and temporal patterning information. This analysis revealed that the inclusion of both the response strength and temporal distribution of spikes elicited by a given stimulus increased the information carrying capacity of LGN neurones 1.5 times when compared to the response strength alone. Based on these results, the authors hypothesized that this form of temporal coding cannot be explained by traditional models of LGN neurone properties, and proposed that these neurones and other neurones along the visual pathway should be considered as spatiotemporal filters. To test this hypothesis directly, Golomb et al. (1994) analyzed the spatiotemporal RF properties of primate LGN neurones and used this information to predict their responses to the Walsh patterns used by McClurkin et al. (1991a,b). They found that the spatiotemporal RF properties could indeed be used to predict the temporal response patterns seen by McClurkin et al. (1991a,b). Similarly, information theory analysis revealed that these spatiotemporal firing patterns could be used to distinguish between different Walsh stimuli on the basis of a single response. Therefore, these two studies (McClurkin et al., 1991a,b; Golomb et al., 1994) provided a direct link between the spatiotemporal RF properties and temporal coding capacity of LGN neurones. 3.2.2. Stimulus-dependent nature of temporal coding in the visual cortex Using an experimental approach identical to that used to study LGN neurones McClurkin et al. (1991a,b), Richmond et al. (1990) and Richmond and Optican (1990) studied the potential coding dimensions of primate primary visual cortical neurones. Overall, the results were very similar to those found for LGN neurones. Responses of both simple and complex V1 neurones showed robust differences in their response strength, and temporal modulation to different Walsh patterns (Richmond et al., 1990). Once again, principal component and information theory analyses revealed that the nature of the stimulus could best be predicted by taking into account both the response magnitude and the temporal pattern of spiking (Richmond and Optican, 1990). These authors also reported that V1 complex neurones exhibited substantially different temporal response profiles when Walsh patterns (or oriented bars) and their contrast-reversed mates were employed as stimuli. Along similar lines, McClurkin and Optican (1996) reported that the number of spikes elicited by V1, V2 and V4 primate visual cortical neurones varied only for one or the other of the stimulus types (colour vs. pattern). However, the temporal distribution of spikes was affected by both stimulus types, suggesting that neurones in multiple visual cortical areas can encode multiple stimulus features simultaneously, both through changes in number of spikes and by time-dependent modulation of firing rates (McClurkin and Optican, 1996; Richmond et al., 1990). Put in another way, these results indicate that different features of a stimulus do not need to be encoded by distinct
Time and the brain
130
populations of neurones, each devoted to a particular stimulus attribute. Instead, the same population of neurones could encode multiple stimulus attributes simultaneously. Using an entirely different analytical approach, Victor and Purpura (1996) have proposed that temporal coding of stimuli in V1 and V2 visual cortical areas of primates performing a fixation task is dependent upon the nature of the stimulus. These authors developed a metric to compare the similarities between two spike trains, based on three basic parameters: the total number of spikes, the intervals between spikes, and the absolute spike time. Each of these parameters was then used to calculate the distance between all pairs of spikes within each data-set. By varying the width of their analysis window, these authors were able to characterize parametrically the precision of spike timing across different stimulus conditions. Their analyses revealed that five different stimulus attributes (contrast, check size, orientation, spatial frequency, and texture) could elicit responses from V1 and V2 neurones that varied systematically in both the number of spikes and their temporal patterns (Victor and Purpura, 1996). Interestingly, there was evidence for temporal coding for all the stimulus attributes tested, but the precision of the temporal code varied. For instance, contrast stimuli elicited spike trains with a ~ 10–30 millisecond precision, while texture stimuli elicited responses with a ~ 100 millisecond precision. Like the results of Richmond et al. (1990), these findings suggest that multiple stimulus attributes could be represented simultaneously in the spike trains of visual cortical neurones. For example, at a high temporal resolution, stimulus contrast could be encoded, while at a lower resolution, the same spike trains could encode texture (Victor and Purpura, 1996). 3.2.3. Coding of sound location by the temporal discharge patterns of single auditory cortical neurones In addition to identifying different types of sounds such as vocalization, the auditory system must also determine the source of sounds. In the visual and somatosensory systems, it is presumed that stimulus location, at least at early stages of these pathways (i.e. up to the level of the primary sensory cortices) is somehow represented by the orderly retinotopic or somatotopic maps found in these brain regions. Although neuropsychological studies have demonstrated that lesions of the auditory cortex result in sound localization deficits (Wortis and Pfeiffer, 1948), no analogous topographic maps, formed by neurones sharply-tuned to sound-source location, have been found in the auditory cortex (Middlebrooks et al., 1998). For instance, auditory cortical neurones in the anterior ectosylvian sulcus of the cat respond to noise bursts, and are broadly-tuned for sound source location (Middlebrooks et al., 1994, 1998). Thus, by using only their average firing rates, one cannot get a very precise estimation of the location of a sound. Middlebrooks et al. (1994, 1998) tested whether the patterns of spiking of single neurones could more reliably encode the location of a sound. An artificial neural network (ANN) was trained to classify the temporal spike patterns of single auditory cortical neurones to sounds coming from different locations in azimuth. Surprisingly, it was found that the spiking pattern of single neurones could encode the location of a sound across 360 degrees of azimuth at more than twice the level of chance. In other words, the
The space-time continuum in mammalian sensory pathways
131
neurones were not tuned to a single “best area” of sound localization. When the resolution of the spike trains was decreased beyond 1 millisecond, the ability of the ANN to discriminate different patterns degraded. Based on these results, Middlebrooks et al. (1998) argued that the organization of the auditory system is uniquely suited to preserving the temporal fidelity of sounds. Nevertheless, as they pointed out, since the performance of single neurones was still below the psychophysical ability of the animal to locate sounds (Jenkins and Masterton, 1982), it is likely that a distributed population of neurones is necessary to identify the precise location of a sound in a real world scenario. 3.3. Dynamic and Distributed Representations of Sensory Inputs by Neural Ensembles The presence of neurones with large receptive fields, whose tuning changes as a function of time, across different sensory pathways, makes it apparent that even the simplest of stimuli is likely to elicit temporally-complex responses from large populations of cortical and sub-cortical neurones. The distinction between temporal and rate coding, in this context, can be seen only as a heuristic dichotomy, since both measures supply one with important information about different stimuli. It, therefore, seemed reasonable to focus our experimental efforts to measure the dynamics of populations of neurones distributed across a given brain region, or multiple brain regions simultaneously, in order to address the question of how time is used by the central nervous system to encode information. With the recent advent of electrophysiological and analytical techniques for multielectrode recordings (Nicolelis, 1998 in press), multi-site neural ensemble recordings have sparked renewed interest in the properties of neural assemblies and their potential roles in brain function. Much of this has focused on pair-wise analysis of neurones, but in some instances larger-scale approaches have been successful in recording and analyzing small ensembles across different brain structures (Ghazanfar and Nicolelis, 1997; Nicolelis et al., 1997a). Therefore, it is not surprising that many laboratories have begun applying neural ensemble recordings to investigate how neuronal populations encode sensory and motor information. In the next sections, some of this recent work carried out in the mammalian visual, auditory, and somatosensory systems is reviewed. 3.3.1. Ensemble coding of visual features Due to the extensive interconnections between neurones both within and across multiple brain structures, even the simplest stimuli will elicit a response from a large population of neurones. This gives rise to the problem of how multiple stimuli can be represented by overlapping populations of neurones. To address this problem, a relational neural code based on synchronous neuronal firing has been suggested as a solution. According to this hypothesis, the joint activity of neurones could allow the many different features of a particular stimulus to be “bound” together; i.e. “feature extractor” neurones should fire synchronously in order to represent the multitude of stimulus features that define a visual scene (Singer and Gray, 1995). In support of this contention, multi-electrode experiments
Time and the brain
132
have been carried out in a variety of visual structures, and these studies have shown that, under certain conditions, neurones can synchronize their firing in response to global stimulus features. In the visual cortex of the anaesthetized cat, Gray and Singer (1989) showed that neurones in a cortical column fire synchronously during the presentation of a moving bar of light. From this result, they hypothesized that synchronous firing may occur between neurones in two spatially distinct cortical columns. To test this idea, they recorded multiunit responses to oriented bars simultaneously from several spatially separate sites in cat primary visual cortex (Gray et al., 1989). Using cross-correlations, they demonstrated that neurones with non-overlapping RFs from spatially distinct locations (up to 7 mm apart), but with similar orientation preferences, could robustly synchronize their firings if the stimulus extended across both neurones’ RFs. When two separate, smaller stimuli were moved together, at the same time and in the same direction, to drive each neurone, weak correlations occurred. However, moving the two stimuli in opposite directions completely abolished synchronization. These results suggest that this neuronal synchronization depended on global features of the stimuli, such as coherent motion (movement in the same direction vs. opposing directions), and continuity (one long bar vs. two short bars) (Gray et al., 1989). Similar results have been reported for neurones located in different cortical areas and between hemispheres (Nelson et al., 1992; Nowak et al., 1995). Nonetheless, it is important to emphasize that, implicitly, this theory still supports the notion that single neurones behave as feature extractors and that explicit mechanisms for “binding information” are required. The presence and potential relevance of neuronal synchronization in the visual system has also been investigated by studying the correlated activity of LGN neurones in the cat. Sillito et al. (1994) have shown that neurones in the LGN of the cat may show significant correlated activity under the influence of stimulus continuity and coherent motion. These authors reported that LGN neurones with non-overlapping receptive fields did not show any significant temporal correlations when activated by a pair of stationary spots. However, when a single bar or moving gratings that covered both neurones’ receptive fields were presented, the same cells would synchronize their firing. By inactivating the overlying visual cortex, these authors also demonstrated that cortical feedback to the LGN was required for the occurrence of this synchronization. Similarly, Alonso et al. (1996) recorded from pairs of cat LGN neurones and found that neurones that had overlapping RFs and monosynaptic connections with primary visual cortical neurones would correlate their firing during visual stimulation. These authors reported that this correlated firing of LGN neurones enhances the efficacy of their inputs upon visual cortical neurones. Whether synchronization under these circumstances was due to feedforward or feedback influences remains to be investigated. In a departure from pairwise cross-correlation studies, Radons et al. (1994) investigated visual neuronal population responses to moving bars, both light and dark. The data-set, containing both multiple- and single-units, was obtained from a 30microelectrode array implanted in the primary visual cortex of an anaesthetized monkey (Kruger and Aiple, 1988). These authors used hidden Markov models, a patternrecognition technique, to determine how well the spatiotemporal firing patterns of neural ensembles could be used to recognize the different visual stimuli. Using this approach,
The space-time continuum in mammalian sensory pathways
133
they found that the identity of the stimulus types could be classified with 90% accuracy, several times above chance levels, when the time window of the analyses was between 25 and 100 milliseconds; increasing or decreasing the time window resulted in a drop in performance. Because of this relatively coarse time scale, the recognition of stimulus types was based simply on the spatiotemporal patterns of mean firing rates and not on the covariance of firing among neurones within the ensemble. Thus, these results argue that the representation of a stimulus can be achieved using the graded responses of a population of neurones without the need for high levels of neuronal synchrony. Another solution to “feature binding” is the convergence of many neurones coding for particular features upon a small numbers of neurones which code for the conjunction of those features (Hubel and Wiesel, 1962). Higher order visual areas are the places where one would expect to find such “grandmother” neurones. Gochin et al. (1994) tested whether neural ensemble coding was necessary for the complex feature detection in a higher order visual area. This was accomplished by focusing on the inferior temporal (IT) cortex, a region located along the ventral pathway of the macaque visual cortex (the “what” pathway, encoding object identities). Because of its position in the visual pathway, this cortical region is ideally-suited for investigating the existence of local versus distributed coding schemes. Although some of these IT neurones are highly selective, and can respond selectively to complex visual patterns such as faces and hands (Desimone et al., 1984), most of them do not respond only to one particular visual stimulus. It follows, therefore, that the stimulus discrimination capability of the IT area would best be assessed by studying groups of neurones. Thus, Gochin et al. (1994) tested the ability of ensembles of IT neurones, in a paradigm in which each neurone was recorded sequentially from a monkey performing a paired-association task, to encode different complex visual patterns. Data were analyzed by constructing firing rate vectors (time-scale =100 milliseconds) from serially recorded neurones, and applying linear discriminant analysis techniques to assess the ability of these reconstructed ensembles to distinguish between five different visual images. The main finding was that ensembles of IT neurones were able to discriminate between different stimuli much better than single IT neurones. Furthermore, the discrimination capacity varied as a function of ensemble size: the bigger the ensemble, the better the performance. Systematically dropping neurones from the ensemble resulted in “graceful” degradation of discrimination capacity. Consistent with this finding, the application of information theory analysis revealed that the distribution of information across the IT neural ensemble was unimodal, and thus, not segregated to particular neurones within the ensemble. Interestingly, the amount of information carried by the ensemble was less than the arithmetic sum of the information carried by individual neurones, suggesting that there is considerable redundancy across the population. In an important control, data from simultaneously recorded IT neurones were analyzed in the same way, and there was no significant increase in the amount of information contained in the ensembles. Based on this result, the authors suggest that, at least in IT cortex, little information is contained in the covariance of neural activity. Nevertheless, in a previous study Gochin et al. (1991) demonstrated, by recording small ensembles of IT neurones simultaneously from a 3-microwire bundle, that nearby neurones can synchronize their firing patterns in response to a visual stimulus, while neurones that
Time and the brain
134
were far apart are weakly correlated. The role of such synchronous firing, if any, in information processing of complex visual images remains to be investigated. 3.3.2. Encoding of sounds by neural ensembles in the auditory cortex In the auditory cortex of the rat, Maldonado and Gerstein (1996) were interested in determining the changes in neuronal ensemble dynamics that follow sensory reorganization induced by intracortical microstimulation (ICMS). ICMS is known to produce a broadening of the receptive fields of cells located at the stimulation site. In addition, neurones recorded from adjacent electrodes have been shown to increase their responsiveness to the best frequency of the cells recorded from the stimulating electrode. The functional relationships between neurones distributed within the primary auditory cortex were assessed by using the gravity analysis (Gerstein et al., 1985), a method in which the temporal relationships between neuronal spike trains are represented as a series of clusters in a multidimensional space. In this multidimensional space, neurones attract or repel each other depending on the coincidence of their neuronal firing. In this study, the responses of up to eight neurones were recorded simultaneously following auditory stimuli and ICMS. The experiments demonstrated that the functional clustering of a subset of the simultaneously recorded neurones, obtained during the delivery of the auditory stimuli, could be strengthened following ICMS. Interestingly, the formation of these functional clusters of neurones did not necessarily relate to the anatomical distance between the cells. In other words, neurones that were anatomically close did not necessarily have a stronger interaction, and neurones that were far apart did not necessarily have weaker interactions. Maldonado and Gerstein’s results indicate that neural ensembles can be established transiently in time, and that they are not necessarily composed of neurones within a highly circumscribed location (e.g. a cortical column). Moreover, their data suggest that the membership of a neural ensemble is mutable as a function of reorganization induced by ICMS. In a direct test of Middlebrooks et al.’s (1994, 1998) distributed coding of sound location hypthesis, Ahissar et al. (1992) demonstrated that broadly-tuned auditory neurones could encode sound location and movement by their pair-wise interactions. By recording the simultaneous extracellular activity of up to nine auditory cortex neurones in awake monkeys trained to sit still, Ahissar et al., studied the sensitivity of primary auditory cortical activity to azimuth and angular movements of acoustic stimuli. They found that two neurones, each broadly-tuned for spatial location, could synchronize their firing to predict spatial location more accurately. Analysis of cross-correlation histogram shapes revealed that different stimulus conditions could modulate the “effective connectivity” of the two neurones; that is, the magnitude and time course of synchronization varied according to stimulus parameters. These authors also noted that correlation of firing between neurones could occur in the absence of changes in the firing rates of each individual neurone. This suggests that temporal interactions between pairs of broadly-tuned auditory neurones, whose firing rates alone were unable to predict spatial location, could be used to discriminate between different spatial locations and angular movements of sound.
The space-time continuum in mammalian sensory pathways
135
The results of Ahissar et al. (1992), suggesting a role for relational coding in auditory processing, are supported by another data set obtained in the primary auditory cortex of anaesthetized marmosets. In this study, cross-correlation analyses revealed that pairs of neurones, or multiunit clusters, may also synchronize their firing according to another stimulus parameter, sound duration (deCharms and Merzenich, 1996). Using pure tone stimuli appropriate to the tuning of the recorded neurones, deCharms and Merzenich showed that short duration stimuli could change the firing rates of auditory cortical neurones. However, the same stimuli delivered for a longer duration could coordinate the firing patterns of neurones recorded simultaneously from two nearby locations. Importantly, this synchronization occurred without changes in the overall firing rates of each of the neurones. The coordination of activity between neurones could last for the duration of the stimulus, while the firing rate remained unchanged. Thus, only pair-wise corticocortical interactions were able to represent duration of a sound. Similar to the results reported by Ahissar et al., (1992), the degree of synchronization varied according to stimulus parameters. The correlation of firing between neurones was strongest when the stimuli were adjusted to fit their frequency tuning profiles, and tended to weaken when the distance between neurones increased. As mentioned in the introduction to this chapter, both the behavioural relevance and the past experience of an animal are believed to play an important role in the function of sensory systems. The recognition of species-specific vocalizations is particularly relevant for many animals since an important part of their everyday life depends on these sensory experiences. In the anaesthetized marmoset, Wang et al. (1995) have shown that subpopulations of neurones in the primary auditory cortex respond selectivity to a speciesspecific vocalization, the “twitter” call. Although most of the neurones were recorded serially, the time-varying and spatially distributed population responses were reconstructed according to their frequency tuning and post-stimulus time. This study revealed that the twitter call could elicit activity from a population of neurones that were spatially distributed across the primary auditory cortex, and that the pattern of firing was correlated with the spectro-temporal characteristics of the vocalization, but on a coarser time scale. In other words, the firing of these neurones phase-locked to large portions of the stimulus, and not to the finer-grained changes in frequency modulation. Two different vocalizations elicited two different cortical population response profiles, with considerable overlap in terms of the members of the population. Temporally-manipulated synthetic versions of the call, which retained the appropriate spectral content, resulted in a degradation of neuronal responses and decreased the amount of synchronization across the population of neurones. This suggests that the temporal characteristics of the individual neurones’ RFs play a role as important as the spectral features. 3.3.3. Encoding of tactile stimuli by thalamocortical somatosensory ensembles Temporal interactions between neuronal populations located in different brain regions played a fundamental role in the concept of cell assemblies first put forward by Donald Hebb (1949). Testing Hebb’s assumptions requires one to record from populations of neurones distributed across multiple cortical areas and subcortical nuclei. Using a chronic
Time and the brain
136
multiple-electrode recording preparation, Nicolelis et al. (1995, 1997a) were able to record from populations of neurones distributed throughout the brainstem, thalamic and cortical processing levels of the trigeminal somatosensory pathway in both anaesthetized and awake rats. The results of these studies have emphasized that the co-ordinated activity of large ensembles of neurones, distributed across multiple cortical and subcortical structures, may provide the basis for the encoding of tactile information in the mammalian somatosensory system. Ensemble recordings of neurones in both VPM and SI cortex have revealed that, even in structures classically thought to employ a labelled line coding scheme, punctate tactile stimuli can elicit distributed responses across each of these brain regions (Figure 2). During normal exploratory behaviours, rats make many simultaneous whisker contacts with objects in the environment. It is important, therefore, to investigate how the somatosensory system integrates information that are more complex than punctate stimuli such as
The space-time continuum in mammalian sensory pathways
137
Figure 2 . Population histograms reveal the distributed nature of responses to punctate tactile stimuli in both the SI cortex (top panel) and VPM thalamus (bottom panel). Notice that individual neurones, all recorded simultaneously, within each population vary in their response magnitudes and latencies.
single whiskers. Studies in other sensory systems have uncovered non-linear neuronal properties when naturalistic stimuli are used to elicit sensory responses (e.g., Rauschecker et al., 1995; Wang et al., 1995). Towards this goal, Ghazanfar and Nicolelis
Time and the brain
138
(1997) recorded concurrently the responses of several tens of well-isolated single neurones from the VPM and primary somatosensory (SI) cortex following single- and multi-whisker deflections. These multi-whisker deflections consisted of three coincidentally-displaced whiskers oriented in rows or columns within the matrix of facial whiskers. Examination of ensemble responses revealed that both single and multiwhisker stimuli could elicit widespread and concurrent activation of neurones across the thalamocortical loop for several milliseconds. Multi-whisker stimuli oriented along columns of whiskers elicited non-linear cortical ensemble responses by activating significantly SI cortex more vigorously, and faster than the arithmetic sum of singlewhisker ensemble responses. This was not the case for multi-whisker stimuli oriented along rows of whiskers. Interestingly, this bias towards whisker columns was not apparent in the single neurone data for SI cortex. Therefore, these results suggest that ensembles of cortical neurones can generate complex spatial transformations that are not seen at the single neurone level. 3.3.4. Large-scale neural ensemble interactions across the somatosensory system In another set of studies, simultaneous multi-site neural ensemble recordings were used to investigate the large-scale interactions between populations of neurones distributed across multiple processing levels of the rat trigeminal somatosensory system (Nicolelis et al., 1995). These recordings included neurones located in the trigeminal ganglion, the principal and spinal subnuclei of the trigeminal brainstem complex, the VPM thalamus, and the infragranular layers of the SI cortex in freely behaving animals. The first important observation derived from this study was the discovery that cortical and subcortical somatosensory neurones can exhibit widespread 7–12 Hz synchronous oscillations (peak frequency at 10Hz), which tend to precede the initiation of tactile exploratory behaviours (whisker twitching movements). By correlating the electrophysiological data with behavioural measurements, obtained through quantitative analysis of facial electromyograms and frame-by-frame video records, Nicolelis et al. (1995) observed that these oscillations tended to appear during periods in which rats assume an immobile and alert posture (Figure 3). Interestingly, these oscillations preceded, by an average of 581 ms, the occurrence of small amplitude rhythmic whisker movements (whisker twitches) in the same frequency range (7–12 Hz). In about 80% of these episodes, the oscillations were first observed in the SI cortex where, on average, 92% of the neurones exhibit oscillatory activity as measured by autocorrelation functions. About 20 ms after the oscillations appeared in cortex, 98% of the VPM neurones, located in both hemispheres, were entrained into these oscillations (Figure 3). Just prior to the initiation of whisker twitches, about 49% of the neurones located in the spinal nucleus of the trigeminal brainstem complex began to exhibit the same oscillations. Neurones located in the principal nucleus of the trigeminal complex or in the trigeminal ganglion did not exhibit these oscillations. By stimulating individual whiskers during the occurrence of these oscillations, we observed that only stimuli delivered during the peak of the oscillations were capable of eliciting sensory-evoked responses. Stimuli delivered outside this small time window
The space-time continuum in mammalian sensory pathways
139
would either induce a phase shift in the underlying oscillatory activity or have no effect. Interestingly, stimuli delivered in a particular phase of these oscillations could block the oscillatory activity for several seconds. In some instances, tactile stimuli could also trigger the occurrence of such oscillations. Altogether, these results raised the hypothesis that top-down influences, generated primarily in the neocortex, are capable of entraining most of the somatosensory system into a pattern of firing (oscillatory mode), which may be ideal for processing sensory information obtained during rhythmic active exploration (through whisker movements) of the surrounding environment.
Figure 3 . Local field potentials recorded simultaneously from VPM and SI cortex before and during whisker twitching behaviour. Oscillations begin first in SI cortex, then in VPM, just prior to whisker twitching movements. Arrows denote the beginning of SI cortical oscillations. First six channels were recorded from electrodes in SI cortex, and last two were recorded in VPM (figure courtesy of Erika E.Fanselow).
3.3.5. Effects of chronic peripheral deafferentation on somatosensory ensembles Many studies have investigated the effects of peripheral sensory deafferentation on the properties of topographic maps in different brain regions (Kaas, 1991; Nicolelis, 1997). In the somatosensory system, peripheral deafferentation is known to result in plastic changes across multiple cortical and subcortical relays of the pathway (Florence and Kaas, 1995). For example, neonatal rats that underwent a complete unilateral removal of their facial whiskers had an extensive reorganization of their ventral posterior medial (VPM) nucleus (Nicolelis et al., 1991). Specifically, this reorganization resulted in the
Time and the brain
140
expansion of RFs to include abnormal regions of the body, such as the shoulder and forepaw, an increase in the response latencies, and an increase in the reverberatory bursting activity of VPM neurones. The effects of this reorganization of the dynamic properties of VPM neurones on the ability of VPM neural ensembles to represent the location of a tactile stimulus were investigated by Nicolelis et al. (1997b). Until recently, no study had examined how such peripheral manipulations affect the ability of the nervous system to encode various stimulus attributes. Ensembles of 13–20 single neurones were recorded simultaneously, and their spatiotemporal firing patterns were analyzed using discriminant analysis (DA). DA was used to quantify the effects of neonatal sensory deprivation on the ability of VPM neural ensembles to “discriminate”, on a trial-by-trial basis, the location of a tactile stimulus. Using this approach, it was determined that when the spatiotemporal patterns of VPM neural ensemble responses in normal rats were used, the neural ensembles were able to classify correctly the location of a tactile stimulus, among three different sites, on 88.2% of the trials. In contrast, ensembles of VPM neurones from whisker-deprived animals performed significantly worse, classifying correctly the location of the stimulus on 73.5% of the trials, even though the three different stimulus locations chosen were much farther apart (making it easier to discriminate) than in the normal animals. Furthermore, removing, one-by-one, the neurones that contributed the most weight in the population response, revealed a graceful degradation of ensemble performance, demonstrating the distributed nature of normal VPM function. Overall, these results suggest that peripheral deafferentation, by altering both the spatial and temporal aspects of single VPM neuronal properties, can decrease the ability of somatosensory neural ensembles to discriminate the location of a tactile stimulus and provides an unique experimental/neurophysiological model for studying chronic peripheral sensory deprivation that is similar to clinical disorders such as phantom limbs. 3.3.6. Simultaneous coding of tactile stimulus location across multiple primate somatosensory cortical areas One of the most basic functions of the nervous system is to identify the location of sensory stimuli. Information about stimulus location is transduced first by activation of the peripheral receptor array. From there, this information may travel along several parallel pathways in the central nervous system until reaching the neocortex. In the complex somatosensory system of primates, which includes many reciprocally-connected cortical areas, a fundamental question is whether information about tactile stimulus localization is preserved throughout these many cortical areas, or whether such information is conserved within distinct cortical areas that then serve as a general reference for other cortical areas. It has been suggested that the degradation of topographic maps of the body surface, due to increases in receptive field sizes as one proceeds from early to later stages of cortical processing, should result in a corresponding degradation in the ability of higher order areas to localize a tactile stimulus. For example, the secondary somatosensory cortex and the different fields of the posterior parietal cortex have long been considered to be uninvolved in stimulus location coding, because
The space-time continuum in mammalian sensory pathways
141
the large receptive fields of neurones in these regions would seem to preclude the representation of this information (Kaas and Pons, 1988). It is seemingly easier to locate a stimulus using a population of neurones with very small RFs, and such a population would be, functionally, a labelled line system. Thus, a fundamental question is whether information about tactile stimulus location is lost in cortical areas where neurones have large receptive fields. It is possible that information about stimulus location could be transformed from a spatial code at the periphery, easily recognized at a single neurone level, to a population code, best recognized in the simultaneous activities of large populations of cortical neurones. Nicolelis et al. (1998) tested these two potential coding schemes by chronically and simultaneously recording populations of neurones from the hand, arm and face regions of three different somatosensory cortical areas in owl monkeys (Aotus trivirgatus). Quantitative analyses of single neurone RF sizes in the infragranular layers of primary somatosensory (area 3b), secondary somatosensory (SII), and parietal area 2, revealed that most neurones had broadly-tuned RFs. As such, single neurones were not reliable indicators of stimulus location on a single-trial basis. As reported previously for the rat somatosensory system (Ghazanfar et al., 1995; Ghazanfar and Nicolelis, 1997; Nicolelis and Chapin, 1994), punctate stimuli elicited unique spatiotemporal patterns of population responses from each of these primate somatosensory cortical areas. In order to test whether these distributed population responses could identify the location of a stimulus on a single-trial basis, neural ensemble analyses were carried out using multivariate statistical pattern recognition approaches (Nicolelis et al., 1998 in press). These analyses, which included an artificial neural network based on learning vector quantization (LVQ), revealed that small ensembles of neurones (~20–40 neurones) from area 3b, SII, or area 2 could encode the location of a stimulus reliably, on a single trial basis (Figure 4a). For example, an ensemble of 24 neurones from area 3b could correctly identify the stimulus location among three different skin sites on 72.5% of the trials (chance is 33.3%). Similarly sized neural ensembles in SII and area 2 also demonstrated above-chance discrimination on a single trial basis. Decreasing ensemble sizes, by removing sequentially the “best predictor” neurone, revealed a graceful degradation of ensemble performance in all three cortical areas, a hallmark of distributed coding (Figure 4a). In order to investigate which parameters of the neural ensemble response were important for localizing the tactile stimulus, two data manipulations were carried out: 1) decreasing the temporal resolution of the response by increasing bin sizes; and 2) shuffling the trials, to eliminate the covariance structure between elements of the population. Increasing the bin size, from 3 milliseconds to 45 milliseconds, significantly reduced the performance of SII ensembles to discriminate the location of a stimulus, suggesting that the temporal modulation of ensemble firing plays an important role for encoding stimulus location in this cortical area (Figure 4b). Surprisingly, ensembles in areas 3b and 2 did not show a deficit in performance, demonstrating that simple firing rate changes were sufficient (Figure 4b). This finding, which is reminiscent of the results in the visual system (Richmond and Optican, 1990; Victor and Purpura, 1996), suggests that different somatonsensory cortical areas may employ different encoding strategies to represent the same type of tactile information (i.e. stimulus location).
Time and the brain
142
Finally, independent shuffling of trial order for each single neurone had little effect on the ability of neural ensembles in any of the three areas to correctly classify stimulus location. If anything, ensemble performance sometimes improved following trial shuffling. These results indicate that, for the set of analytical tools used in this study, neither intra-trial precise timing of ensemble spike patterns, neural synchronization, nor linear covariance across the population of neurones were necessary for effective single trial discrimination of tactile stimulus location. Instead, the location of a stimulus could be predicted by the unique spatiotemporal patterns of sensory responses across the neural population. In consideration of these results, it is important to realize that the use of multi-site neural ensemble recordings from areas 3b, SII, and 2 revealed that neural responses occur in these regions concurrently. As such, it does not appear likely that information is passed only sequentially from one cortical area to another; rather, the different encoding strategies used by each area may occur almost simultaneously. Indeed, analysis of the
The space-time continuum in mammalian sensory pathways
143
Figure 4 . a. For two cortical areas analyzed (SII and 3b) the single trial discrimination capability varied as a function of the ensemble size. This figure illustrates the effects of sequential removal of the best predictor neurons from the population on single trail discrimination of the stimulus location. These curves also revealed a graceful decay in discrimination performance as a function of ensemble size. b. Effect of bin size on the discrimination capability of two different cortical areas. While a significant reduction in discrimination capability was observed when bin clumping (i.e. increasing the bin describing neuronal firing from 3 to 45 ms) was applied to SII neural ensembles, no effect was observed in area 3b neural ensembles. These results suggest that the temporal structure of population responses could play a more fundamental role in area SII than in the other two somatosensory regions for encoding stimulus location.
time epoch in which the maximum amount of information is contained revealed that there was a 15 millisecond overlap between area 2 and areas 3b and SII sensory responses.
Time and the brain
144
4. CONCLUSION The findings reviewed in this chapter underscore the fundamental relevance of time in sensory information processing by the auditory, visual, and somatosensory systems of mammals. Altogether, these data reveal that the existence of spatiotemporal coupling in both single neurone responses and neural ensemble firing is pervasive throughout these sensory pathways. Furthermore, dynamic RFs and the spatiotemporal coding of stimulus attributes, which have been now identified in all three sensory systems, seem to emerge and to be constrained by both the need to interpret time-varying stimuli and the fact that active exploratory behaviours are used by animals to scan their surrounding environment. Regarding the mechanisms used to generate the spatiotemporal properties of single neural RFs and neural ensemble coding, the evidence suggests that the asynchronous convergence of multiple afferent pathways gives rise to the dynamic neural properties observed in all three sensory systems. In this context, descending corticofugal projections play as crucial a role in sensory information processing as the feedforward pathways, since they provide the anatomical substrate for propagating top-down signals from the neocortex to subcortical relays and, consequently, act upon the ascending flow of sensory information. It is also likely that the recurrent connectivity of sensory systems contributes to the finding that slight alterations at either peripheral or central levels can trigger system-wide alterations in the functional organization of a sensory pathway. Therefore, embedded in this model is the notion that adaptive neural phenomena (i.e. immediate and long-term sensory plasticity, learning, etc.), occurring at different time scales (i.e. seconds, minutes, hours, days, etc.) both influence and are constrained by the dynamics of sensory information processing at a millisecond scale. Thus, in addition to contributing to an extra dimension for information coding, the dynamic properties of neural ensembles provide the substrates for triggering immediate system-wide reorganization that follow changes in sensory experience over the course of the organism’s life (Nicolelis, 1997). Fundamentally, mammalian sensory systems can no longer be considered as a collection of passive filters or feature extractors that are specialized in detecting the presence of particular stimulus attributes (i.e. orientation, colour, tones, etc.). The experimental evidence reviewed in this chapter indicates that perception emerges as the outcome of highly distributed interactions between populations of broadly tuned neurones dispersed throughout the multiple and reciprocally-connected subcortical and cortical areas that define a sensory system. Put in another way, the central nervous system does not operate as a static decoder, which sits idle until inputs from the environment are conveyed through it, by feedforward neural pathways that connect the sensory periphery to the cerebral cortex. Instead, a more parsimonious hypothesis is that during adulthood, the spatiotemporal organization of sensory systems reflect their active attempt, during exploratory behaviours, to compare the meaning encoded into a time-varying stimulus with the accumulated representation of the sensory experiences acquired during a lifetime.
The space-time continuum in mammalian sensory pathways
145
REFERENCES Abeles, M., and Goldstein, M.H. (1972) Responses of single units in the primary auditory cortex of the cat to tones and to tone pairs. Brain Research , 42, 337–352. Adrian, E.D. (1928) The basis of sensation . Norton, New York. Ahissar, M., Ahissar, E., Bergman, H., and Vaadia, E. (1992) Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. Journal of Neurophysiology , 67, 203–215. Alonso, J.-M., Usrey, W.M., and Reid, R.C. (1996) Precisely correlated firing in cells of the lateral geniculate nucleus. Nature, London , 383, 815–819. Armstrong-James, M., and Fox, K. (1987) Spatiotemporal convergence and divergence in the rat SI “barrel” cortex. Journal of Comparative Neurology , 263, 265–281. Armstrong-James, M., Fox, K., and Das-Gupta, A. (1992) Flow of excitation within rat barrel cortex on striking a single vibrissa. Journal of Neurophysiology , 68, 1345– 1358. Atick, J.J., and Redlich, A.N. (1990) Towards a theory of early visual processing. Neural Computation , 2, 308–320. Barlow, H.B. (1972) Single units and sensation: a neuron doctrine for perceptual psychology? Perception , 1, 371–394. Beck, P.D., Fitzpatrick, L., and Nicolelis, M.A.L. (1998) Corticothalamic projections of somatosensory and motor cortical areas in the rat. Society for Neuroscience Abstracts , p. 131. Bushnell, E.W., and Boudreau, J.P. (1991) The development of haptic perception during infancy. In: M.A.Heller and W.Schiff (eds.) The psychology of touch . Erlbaum, Hillsdale, NJ, pp. 139–161. Carvell, G.E., and Simons, D.J. (1990) Biometric analyses of vibrissal tactile discrimination in the rat. Journal of Neuroscience , 10, 2638–2648. Chapin, J.K. (1986) Laminar differences in sizes, shapes, and response profiles of cutaneous receptive fields in the rat SI cortex. Experimental Brain Research , 62, 549– 559. Dan, Y., Atick, J.J., and Reid, R.C. (1996) Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. Journal of Neuroscience , 16, 3351–3362. Darian-Smith, I. (1984) The sense of touch: performance and peripheral neural processes. In: I. Darian-Smith (ed.) Handbook of physiology. I. the nervous system. Sensory processes. Vol. 3 , American Physiological Society, pp. 739–787. Dawis, S., Shapley, R., Kaplan, E., and Tranchina, D. (1984) The receptive field organization of X-cells in the cat: spatiotemporal coupling and asymmetry. Vision Research , 24, 549–564. DeAngelis, G.C., Ohzawa, I., and Freeman, R.D. (1993) Spatiotemporal organization of simple-cell receptive fields in the cat’s striate cortex. I. General characteristics and postnatal development. Journal of Neurophysiology , 69, 1118–1135. DeAngelis, G.C., Ohzawa, I., and Freeman, R.D. (1995) Receptive field dynamics in the central visual pathways. Trends in Neurosciences , 18, 451–458. deCharms, R.C., Blake, D.T., and Merzenich, M.M. (1998) Optimizing sound features for cortical neurons. Science, New York , 280, 1439–1443. deCharms, R.C., and Merzenich, M.M. (1996) Primary cortical representation of sounds
Time and the brain
146
by the coordination of action-potential timing. Nature, London , 381, 610–613. Desimone, R., Albright, T.D., Gross, C.G., and Bruce, C.J. (1984) Stimulus selective properties of inferior temporal neurons in the macaque. Journal of Neuroscience , 4, 2051–2062. Dinse, H.R., Kruger, K., Mallot, H.A., and Best, J. (1991) Temporal structure of cortical information processing: cortical architecture, oscillations, and non-separability of spatio-temporal receptive field organization. In: J.Kruger (ed.) Neuronal Cooperativity Springer-Verlag, Berlin, pp. 68–104. Fabri, M., and Burton, H. (1991) Ipsilateral cortical connections of primary somatic sensory cortex in rats. Journal of Comparative Neurology , 311, 405–424. Faggin, B.M., Nguyen, K.T., and Nicolelis, M.A. (1997) Immediate and simultaneous sensory reorganization at cortical and subcortical levels of the somatosensory system. Proceedings of the National Academy of Sciences, U.S.A. , 94, 9428–33. Florence, S.L., and Kaas, J.H. (1995) Large-scale reorganization at multiple levels of the somato-sensory pathway follows therapeutic amputation of the hand in monkeys. Journal of Neuroscience , 15, 8083–8095. Gerstein, G.L., Perkel, D.H., and Dayhoff, J.E. (1985) Cooperative firing activity in simultaneously recorded populations of neurons: detection and measurement. Journal of Neuroscience , 5, 881–9. Ghazanfar, A.A., and Nicolelis, M.A.L. (1999) Spatiotemporal properties of layer V neurons of the rat primary somatosensory cortex. Cerebral Cortex , 9, 348–361. Ghazanfar, A.A., Krupa, D.J., and Nicolelis, M.A.L. (1997) Tactile processing by thalamic neural ensembles: the role of cortical feedback. Society for Neuroscience Abstracts . Ghazanfar, A.A., and Nicolelis, M.A.L. (1997) Nonlinear processing of tactile information in the thalamocortical loop. Journal of Neurophysiology , 78, 506–510. Gochin, P.M., Colombo, M., Dorfman, G.A., Gerstein, G.L., and Gross, C.G. (1994) Neural ensemble coding in inferior temporal cortex. Journal of Neurophysiology , 71 , 2325–2337. Gochin, P.M., Miller, E.K., Gross, C.G., and Gerstein, G.L. (1991) Functional interactions among neurons in inferior temporal cortex of the awake monkey. Experimental Brain Research , 84, 505–516. Golomb, D., Kleinfeld, D., Reid, R.C., Shapley, R.M., and Shraiman, B.I. (1994) On temporal codes and the spatiotemporal response of neurons in the lateral geniculate nucleus. Journal of Neurophysiology , 72 , 2990–3003. Gottlieb, J.P., and Keller, A. (1997) Intrinsic circuitry and physiological properties of pyramidal neurons in rat barrel cortex. Experimental Brain Research , 115 , 47–60. Gray, C.M., Konig, P., Engel, A.K., and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, London , 338 , 334–337. Gray, C.M., and Singer, W. (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Sciences, U.S.A. , 86 , 1698–1702. Hartline, H.K. (1950) The receptive fields of optic nerve fibres. American Journal of Physiology , 130 , 690–699. Mauser, M.D., Agnetta, B., and Perez, C. (1998) Orienting asymmetries in rhesus monkeys: effect of time-domain changes on acoustic perception. Animal Behaviour , 56, 41–47. Hebb, D.O. (1949) The organization of behavior. A neuropsychological theory . John Wiley and Sons.
The space-time continuum in mammalian sensory pathways
147
Hoeflinger, B.F., Bennett-Clarke, C.A., Chiaia, N.L., Killackey, H.P., and Rhoades, R.W. (1995) Patterning of local intracortical projections within the vibrissae representation of rat primary somatosensory cortex. Journal of Comparative Neurology , 354, 551– 563. Hubel, D.H., and Wiesel, T.N. (1962) Receptive fields, binocular interactions, functional architecture in the cat’s visual cortex. Journal of Physiology (London) , 195 , 215–243. Jenkins, W.M., and Masterton, R.B. (1982) Sound localization: effects of unilateral lesions in central auditory system. Journal of Neurophysiology , 47 , 987–1016. Kaas, J.H. (1991) Plasticity of sensory and motor maps in adult mammals. Annual Review of Neuroscience , 14 , 137–168. Kaas, J.H., and Pons, T.P. (1988) The somatosensory system of primates. In: H.D. Steklis and J. Erwin (eds) Comparative Primate Biology, Vol. 4 , Alan R.Liss, New York, pp. 421–468. Knudsen, E.I., and Konishi, M. (1979) Mechanisms of sound location in the barn owl (Tyto alba). Journal of Comparative Physiology , 133 , 13–21. Koralek, K.-A., Olavarria, J., and Killackey, H.P. (1990) Areal and laminar organization of corticocortical projections in the rat somatosensory cortex. Journal of Comparative Neurology , 299 , 133–150. Kruger, J., and Aiple, F. (1988) Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. Journal of Neurophysiology , 60, 798–828. Krupa, D.J., Ghazanfar, A.A., and Nicolelis, M.A.L. (1999) Immediate thalamic sensory plasticity depends on corticothalamic feedback. Proc. Natl. Acad. Sci . USA, 96, 8200– 8205. Lashley, K.S. (1950) In search of the engram. Symposium of the Society of Experimental Biology , 4, 454–482. Maldonado, P.E., and Gerstein, G.L. (1996) Neuronal assembly dynamics in the rat auditory cortex during reorganization induced by intracortical microstimulation. Experimental Brain Research , 112 , 431–41. Margoliash, D., and Fortune, E.S. (1992) Temporal and harmonic combination-sensitive neurons in the zebra finch’s HVc. Journal of Neuroscience , 12 , 4309–4326. Marrocco, R.T. (1976) Sustained and transient cells in monkey lateral geniculate nucleus: conduction velocities and response properties. Journal of Neurophysiology , 39 , 340– 353. McClurkin, J.W., Gawne, T.J., Optican, L.M., and Richmond, B.J. (1991 a) Lateral geniculate neurons in behaving primates. II. Encoding of visual information in the temporal shape of the response. Journal of Neurophysiology , 66 , 794–808. McClurkin, J.W., Gawne, T.J., Richmond, B.J., Optican, L.M., and Robinson, D.L. (1991b) Lateral geniculate neurons in behaving primates. I. Responses to twodimensional stimuli. Journal of Neurophysiology , 66 , 777–93. McClurkin, J.W., and Optican, L.M. (1996) Primate striate and prestriate cortical neurons during discrimination. I. simultaneous temporal encoding of information about color and pattern. Journal of Neurophysiology , 75, 481–495. Middlebrooks, J.C., Clock, A.E., Xu, L., and Green, D.M. (1994) A panoramic code for sound location by cortical neurons. Science, New York , 264 , 842–844. Middlebrooks, J.C., Xu, L., Eddins, A.C., and Green, D.M. (1998) Codes for soundsource location in nontonotopic auditory cortex. Journal of Neurophysiology , 80 , 863–881. Mignard, M., and Malpeli, J.G. (1991) Paths of information flow through visual cortex.
Time and the brain
148
Science, New York , 251, 1249–1251. Nelson, J.I., Salin, P.A., Munk, M.H., Arzi, M., and Bullier, J. (1992) Spatial and temporal coherence in cortico-cortical connections: a cross-correlation study in areas 17 and 18 in the cat. Visual Neuroscience , 9, 21–37. Nicolelis, M.A.L. (1997) Dynamic and distributed somatosensory representations as the substrate for cortical and subcortical plasticity. Seminars in Neuroscience , 9, 24–33. Nicolelis, M.A.L. (1998, in press) Methods in Neural Ensemble Recordings . CRC Press, New York. Nicolelis, M.A., Chapin, J.K., and Lin, R.C. (1991) Thalamic plasticity induced by early whisker removal in rats. Brain Research , 561, 344–9. Nicolelis, M.A., Lin, R.C., Woodward, D.J., and Chapin, J.K. (1993a) Dynamic and distributed properties of many-neuron ensembles in the ventral posterior medial thalamus of awake rats. Proceedings of the National Academy of Sciences, U.S.A. , 90, 2212–6. Nicolelis, M.A.L., Lin, R.C.S., Woodward, D.J., and Chapin, J.K. (1993b) Induction of immediate spatio-temporal changes in thalamic networks by peripheral block of ascending cutaneous information. Nature, London , 361, 533–536. Nicolelis, M.A.L., and Chapin, J.K. (1994) Spatiotemporal structure of somatosensory responses of many-neuron ensembles in the rat ventral posterior medial nucleus of the thalamus. Journal of Neuroscience , 14, 3511–3532. Nicolelis, M.A., Baccala, L.A., Lin, R.C., and Chapin, J.K. (1995) Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system. Science, New York , 268, 1353–8. Nicolelis, M.A.L., Oliveira, L.M.O., Lin, R.C.S., and Chapin, J.K. (1996) Active tactile exploration influences the functional maturation of the somatosensory system. Journal of Neurophysiology , 75, 2192–2196. Nicolelis, M.A.L., Ghazanfar, A.A., Faggin, B.M., Votaw, S., and Oliveira, L.M.O. (1997a) Reconstructing the engram: simultaneous, multisite, many single neuron recordings. Neuron , 18, 529–537. Nicolelis, M.A.L., Lin, R.C.S., and Chapin, J.K. (1997b) Neonatal whisker removal reduces the discrimination of tactile stimuli by thalamic ensembles in adult rats. Journal of Neurophysiology , 78, 1691–1706. Nicolelis, M.A.L., Ghazanfar, A.A., Stambaugh, C.R., Oliveira, L.M.O., Laubach, M., Chapin, J.K., Nelson, R.J., and Kaas, J.H. (1998) Simultaneous encoding of tactile information by three primate cortical areas. Nature, Neuroscience , 1 (in press). Nicolelis, M.A.L., Stambaugh, C.R., Brisben, A., and Laubach, M. (1998 in press) Methods for simultaneous multisite neural ensemble recordings in behaving primates. In: M.A.L.Nicolelis (ed.) Methods in Neural Ensemble Recordings . CRC Press, New York. Nowak, L.G., Munk, M.H., Nelson, J.I., James, A.C., and Bullier, J. (1995) Structural basis of cortical synchronization. I. Three types of interhemispheric coupling. Journal of Neurophysiology , 74, 2379–400. Oliveira, L.M.O., Shuler, M.G., Casal, S.G., and Nicolelis, M.A.L. (1997) Effects of neonatal unilateral facial nerve cuts on tactile responses in adult rat SI cortex. Society for Neuroscienc Abstracts. , p. 1798. Radons, G., Becker, J.D., Dulfer, B., and Kruger, J. (1994) Analysis, classification, and coding of multi-electrode spike trains with hidden Markov models. Biological Cybernetics , 71, 359–373. Rauschecker, J.P., Tian, B., and Hauser, M. (1995) Processing of complex sounds in the
The space-time continuum in mammalian sensory pathways
149
macaque nonprimary auditory cortex. Science, New York , 268, 111–114. Reid, R.C., and Alonso, J.-M. (1995) Specificity of monosynaptic connections from thalamus to visual cortex. Nature, London , 378, 281–284. Richmond, B.J., and Optican, L.M. (1990) Temporal encoding of two-dimensional patterns by single units in primate primary visual cortex. II. Information transmission. Journal of Neurophysiology , 64, 370–80. Richmond, B.J., Optican, L.M., and Spitzer, H. (1990) Temporal encoding of twodimensional patterns by single units in primate primary visual cortex. I. Stimulusresponse relations. Journal of Neurophysiology , 64, 351–69. Ringach, D.L., Hawken, M.J., and Shapley, R.M. (1997) Dynamics of orientation tuning in macaque primary visual cortex. Nature, London , 387, 281–284. Saul, A.B., and Humphrey, A.L. (1992) Evidence of input from lagged cells in the lateral geniculate nucleus to simple cells in cortical area 17 of the cat. Journal of Neurophysiology , 68, 1190–1207. Shadlen, M.N., and Newsome, W.T. (1994) Noise, neural codes and cortical organization. Current Opinion in Neurobiology , 4, 569–579. Sillito, A.M., Jones, H.E., Gerstein, G.L., and West, D.C. (1994) Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature, London , 369, 479–482. Simons, D.J. (1978) Response properties of vibrissa units in rat SI somatosensory neocortex. Journal of Neurophysiology , 41, 798–820. Singer, W., and Gray, C.M. (1995) Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience , 18, 555–586. Stevens, J.K., and Gerstein, G.L. (1976) Spatiotemporal organization of cat lateral geniculate receptive fields. Journal of Neurophysiology , 39 , 213–238. Suga, N., O’Neill, W.E., Kujirai, K., and Manabe, T. (1983) Specificity of combinationsensitive neurons for processing biosonar signals in auditory cortex of the mustached bat. Journal of Neurophysiology , 49 , 1573–1626. Sutter, M.L., and Schreiner, C.E. (1991) Physiology and topography of neurons with multipeaked tuning curves in cat primary auditory cortex. Journal of Neurophysiology , 65 , 1207–1226. Tramo, M.J., Bellew, B.F., and Hauser, M.D. (1996) Discharge patterns of auditory cortical neurons evoked by species-specific vocalizations and synthetic complex signals in alert Macaca mulatta. Society for Neuroscience Abstracts , p. 1623. Victor, J.D., and Purpura, K.P. (1996) Nature and precision of temporal coding in visual cortex: a metric-space analysis. Journal of Neurophysiology , 76 , 1310–1326. Wang, X., Merzenich, M.M., Beitel, R., and Schreiner, C.E. (1995) Representation of a species-specific vocalization in the primary auditory cortex of the common marmoset: temporal and spectral characteristics. Journal of Neurophysiology , 74, 2685–2706. Welker, W.I. (1964) Analysis of sniffing of the white albino rat. Behaviour , 12, 223– 244. Winter, P., and Funkenstein, H.H. (1973) The effect of species-specific vocalizations on the discharge of auditory cortical cells in the awake squirrel monkey (Saimiri sciureus) Experimental Brain Research , 18, 489–504. Wollberg, Z., and Newman, J.D. (1972) Auditory cortex of squirrel monkeys: response properties of single cells to species-specific vocalizations. Science, New York , 175, 212–214. Wortis, S.B., and Pfeiffer, A.Z. (1948) Unilateral auditory-spatial agnosia. Journal of Nervous and Mental Disease , 108, 181–186.
4 Information Flow along Neocortical Axons Harvey A.Swadlow Department of Psychology (U-20), The University of Connecticut, Storrs, Connecticut 06269 Tel: (860) 486–2252; FAX: (860) 486–2760; e-mail: [email protected]
Neocortical axonal systems convey the results of cortical processing to cortical target structures. This chapter is concerned with the timing of this information flow and with delays introduced by different types of cortical axon. Neocortical white matter is composed of axons that range from <0.1 µm to >10 µm in diameter. Such axons generate a similarly wide range of conduction velocities, and conduction times between soma and axon terminals. Work from the author’s laboratory is reviewed in which conduction properties of corticocortical and descending corticofugal efferent systems of rabbit neocortex were examined in each of four sensory and motor cortical regions. These data are compared with corresponding axonal systems of primates, cats and rodents. Whereas some cortical efferent systems are composed of exclusively fast-conducting axons, others have a large number of very slowly conducting elements, and impulse conduction times along these latter elements may take several tens of milliseconds. Properties of specific thalamocortical axonal systems are also reviewed, and the role of axonal conduction time in the generation of sharply and broadly synchronous cortical activity is explored. KEYWORDS: Axonal conduction; Cerebral cortex; Cortico-cortical axons; Conduction velocity; Conduction delay; Synchrony
1. INTRODUCTION Neocortical axonal systems convey the results of cortical processing to a host of destinations. This chapter is concerned with the timing of this information flow, and with delays introduced by different types of cortical axons. Considerable information is available concerning the laminar and regional origins of neocortical efferent pathways and the locus of terminal synapses. Less attention has been paid to the axon lying between soma and synaptic terminals. General surveys of cortical white matter (Bishop and Smith, 1964; Waxman and Swadlow, 1977) have shown that neocortical axons exhibit a wide spectrum of diameters. Unmyelinated axons range from <0.1–0.6 µm, and often have a median diameter of ~0.2 µm (e.g. Waxman and Swadlow, 1976b; Swadlow et al., 1980). Most such axons cannot be visualized without the aid of electron
Information flow along neocortical axons
151
microscopy. Myelinated axons may be as fine as 0.2 µm in diameter (e.g. Harding and Towe, 1985; Waxman and Bennett, 1972). The diameter of the largest myelinated axons found in a given cortical system is species dependent (see below), and may be >10 µm (e.g. Biedenbach et al., 1986). Thus, neocortical axons span a range of >1:10,000 in cross-sectional area, and a range in expected conduction velocities of >1:100 (Waxman and Bennett, 1972, see below). This chapter reviews work from the author’s laboratory in which conduction properties of corticocortical and descending corticofugal efferent systems of rabbit neocortex were examined in each of four sensory and motor cortical regions. These data are compared with results from corresponding axonal systems of primates, cats, and rodents. This is not intended to be an all-inclusive review of neocortical axonal systems, and many axonal systems are not included. The cortical regions examined in rabbit were primary visual cortex (V1), primary and secondary somatosensory cortices (S1, S2) and motor cortex (Swadlow, 1988, 1990, 1991, 1994). Efferent systems studied in each of these regions include: (1) the contralateral callosal projection (CC axons); (2) ipsilateral corticocortical projections (C-IC axons) linking primary and secondary sensory cortical areas, or motor and sensory cortical areas; (3) descending corticofugal projections of layer 5 (CF-5 axons); and (4) descending corticofugal projections of layer 6 (CF-6 axons). In addition, sharply synchronous (±1 ms), non-oscillatory activity between putative GABAergic neocortical interneurones is described (Swadlow et al., 1998), and the relationship of such activity to properties of thalamocortical axons is explored.
2. METHODOLOGICAL CONSIDERATIONS Information about axonal conduction delays can be derived from several sources, and each has characteristic limitations. 2.1 Conduction Delays Inferred from Fibre-diameter Spectra The theoretical basis for the relationship between axonal diameter and conduction velocity has been well-explored, and conduction time can be inferred from a knowledge of axon diameter and conduction distance. The conduction velocity of myelinated axons is expected to be linearly related to axon diameter (Rushton, 1951; Goldman and Albus, 1968) and that of non-myelinated axons should be proportional to the square root of diameter (Bennett and Waxman, 1972; Goldman and Albus, 1968; Matsumoto and Tasaki, 1977; Rushton, 1951). These relations assume that axons are “dimensionally similar”, and are dependent upon on factors such as specific membrane properties, internodal distances and myelin thickness. The scaling factor for myelinated axons has been derived largely from peripheral nerve, and considerable variability among axonal systems has been observed (cf., Boyd and Kalu, 1979). Assuming a scaling factor of 5.5 (5.5 m/s for a one micron fibre, including the myelin) and a minimal diameter of 0.2 µm (Waxman and Bennett, 1972),
Time and the brain
152
the smallest myelinated central axons should conduct at velocities of 1.1 m/s, and a 10 µm axon should conduct at about 55 m/s. Central unmyelinated axons are generally from 0.08–0.6 µm in diameter (e.g. Swadlow et al., 1980; Waxman and Swadlow, 1976b). In the periphery they are considerably larger, and axons of 1.1 µm in diameter have conduction velocities of ~2.1 m/s (e.g. Rushton, 1951). However, no systematic studies have examined the scaling factor of central unmyelinated axons, which often conduct at velocities as low as 0.3 m/s (e.g. Bester et al., 1995; Kaba et al, 1986; Manis, 1989; Swadlow, 1994). Assuming that this velocity value represents the finest unmyeli-nated axons, the largest unmyelinated central axons should conduct at a velocity of ~0.8 m/s (given the square root relationship described above). Assuming that axon diameter-conduction velocity relationships are known, two practical considerations seriously limit inferences of conduction time based on axonal diameter and conduction distance: (1) Only a few cortical efferent systems (e.g. the corpus callosum, pyramidal tract) are clearly packaged as fibre bundles accessible to the electron microscope. The axons of most cortical efferent systems mingle in both grey and white matter, and specific systems cannot be readily disentangled (however, see Woodward and Coull, 1984). In classical studies of axonal degeneration following cortical lesions, efferent axons seen coursing towards their targets were often characterized as “course” or “fine” (e.g. Guillery, 1966, 1967). However, such methods, using light microscopy, are generally biased towards the detection of myelinated axons because most unmyelinated axons fall below the level of resolution of the light microscope. The same limitation applies to most studies of cortical axons using retrograde or orthograde tracers (e.g. Boyapati and Henry, 1984; Rockland, 1992). (2) Electron microscopic procedures generally result in each axon being measured at a single point along it’s trajectory. Since axons may vary considerably in diameter and myelination along their length, inferences regarding conduction delays are thereby limited. 2.2. Conduction Delays Inferred from Antidromic Latencies Electrical stimulation of the axonal pathway will elicit antidromic responses in the soma, which can be differentiated from synaptic responses using collision and other procedures (e.g. Bishop et al, 1962; Fuller and Schlag, 1976). A major advantage of this procedure, when compared with inferring conduction time from single measurements of axon diameter, is that the antidromic latency reflects the conduction time along the entire length of axon between stimulation site and soma. However, this method is subject to numerous sampling biases, most of which converge to select against the detection of neurones with slowly conducting axons (see Swadlow, 1998, for a recent review). Factors that contribute to this bias include: (1) Neurones with fine-diameter axons often have small somata which are more difficult to sample than are neurones with larger somata (Humphrey and Weller, 1988; Stone, 1973; Towe and Harding, 1970).
Information flow along neocortical axons
153
(2) Cortical neurones with very fine diameter axons often show no spontaneous impulse activity, and lack sensory receptive fields (Swadlow, 1988, 1990, 1991, 1994). Such neurones are difficult to detect, and are often passed or killed by the microelectrode before they can be studied. (3) Fine diameter axons have higher thresholds to electrical stimulation than do larger diameter axons (Nowak and Bullier, 1996; Ranck, 1975). Thus, whereas a stimulating electrode in close proximity (e.g. <0.5 mm) to a small group of fibres may activate both large and small diameter axons, a somewhat more distant stimulating electrode will selectively activate the larger diameter axons. Increasing stimulus intensity will activate larger axons yet further from the stimulation site, but few unmyelinated axons will be activated at distances of >0.5 mm, even when stimulus intensity is very high (e.g. Swadlow, 1998 [Figure 3]). This results in a strong bias against fine-diameter axons unless stimulating electrodes are in very close proximity to the axons under study. This bias will be heightened when it is deemed necessary to keep stimulus intensity to very low levels in order to limit stimulus spread to a small cortical region. (4) The conduction time along fine preterminal branches may easily be excluded from the measured antidromic conduction time. Although great care may be taken in placing stimulating electrodes near the expected site of axon termination, the impulse may be first initiated some distance away, in a myelinated segment of the axon (where threshold is much lower). Because unmyelinated axons may conduct at velocities of <0.3 m/s (below), one or two millimeters of unmyelinated terminals can add several milliseconds to axonal conduction. (5) Fine-diameter axons often exhibit significant variations in axonal conduction velocity, that result from prior impulse activity (e.g. Ferreyra Moyano and Molina, 1980; Segal et al., 1983; review by Swadlow et al., 1980; see below). Antidromic latency may, therefore, vary considerably from trial-to-trial (in some cases by >15%), depending on the timing of stimulus pulses to antecedent “spontaneous” impulse activity (Swadlow et al., 1978a, b). Although the antidromic nature of the response in such cells can be verified by collision (e.g. Bishop et al., 1962; Fuller and Schlag, 1976) and other procedures (Swadlow et al., 1978b), many experimenters employ a “constant latency” (often defined as <0.1 or 0.2 ms variability) as a defining requirement for antidromic identification; cells showing greater latency variability are thereby rejected from analysis. This creates a strong bias towards the fast-conducting axons, which generally do have a relatively constant antidromic latency. 2.3. Conduction Delays Inferred from Synaptic Latencies Stimulation of one region of cortex will elicit synaptic effects in other regions, and the latencies of these effects depend on conduction times along intervening axons. There are a number of limitations to such procedures: (1) The number of intervening synapses mediating a specific effect is often unknown; (2) In reciprocal pathways, orthodromic synaptic effects cannot be readily differentiated from synaptic effects that are mediated by antidromically-activated recurrent collaterals; and
Time and the brain
154
(3) Synaptic effects mediated by large diameter axons with short conduction times are usually easily observed (because impulses from many such axons arrive nearly simultaneously, and result in considerable temporal summation of EPSPs). However, long latency effects are often temporally dispersed and below threshold for generating spikes or evoked potentials. These problems place limits on the relevance of measured synaptic latencies to understanding the normal conduction delays occurring between cortical areas (cf., Swadlow et al., 1979). However, such procedures may be very useful in determining the fastest possible conduction times between cortical regions.
3. RESULTS FROM THE RABBIT Extracellular recordings were obtained from antidromically identified efferent neurones in rabbit V1, S1, S2 and motor cortex (Swadlow, 1988, 1990, 1991 and 1994). These studies, conducted under virtually identical experimental conditions, were aimed at understanding the common features of parallel processing within diverse neocortical regions, and axonal conduction times were one of the variables under study. Estimated axonal conduction velocities and antidromic latencies of the efferent populations studied are presented in Figures 1 and 2. Also shown are conduction velocities and antidromic latencies of specific thalamocortical axons projecting to V1 and S1 (Swadlow, 1985, 1995). Conduction velocities presented in Figure 1 reflect mean axonal conduction velocities between stimulation sites and recording sites near the soma. 3.1. Callosal Axons Figure 1a shows the estimated axonal conduction velocities for rabbit callosal axons originating in V1, S1, S2 and motor cortex. Note that the axonal stimulation sites in these studies were just contralateral of the midline, so slowing of impulses as they invade preterminal branches in not reflected here. In each of the cortical systems of the rabbit, the mean conduction velocities ranged from 1.47–3.5 m/s, and the slowest callosal axons conducted at velocities of 0.29–0.6 m/s. In each system, a significant population was seen that conducted at velocities of <1 m/s, and a small number of axons conducted at velocities of >5 m/sec. Thus, these axon conduction velocities are indicative of a mixed population of unmyelinated and finediameter myelinated axons. In fact, ultrastructural studies (Waxman and Swadlow, 1976b) found 45% of rabbit splenial axons to be unmyelinated (diameters of 0.08–0.6 µm, mean=0.2 µm) and the remaining to be myelinated with diameters of 0.3–1.85 µm (mean=0.74 µm). Figure 2a shows the antidromic conduction times for the same callosal axons shown in Figure 1a. Note that these conduction times represent somewhat less than 1/2 of the interhemispheric conduction time (this is because the terminal 1/2 of the axonal length is not included, and the excluded portion contains the highly branched and more slowly
Information flow along neocortical axons
155
conducting terminal region). In each of the cortical regions studied in rabbit, a few fast conducting callosal axons result in a small number of very short conduction times (1–2 ms). However, in each system, the conduction times along most fibres are quite long. In S2, for example, conduction times of up to 29.9 ms were seen, implying an interhemispheric conduction time of >60 ms for this neurone. In the above studies of V1, S1, S2 and motor cortex, some rabbit callosal axons projected a collateral to some ipsilateral cortical areas (to V2, S2, S1 and S1 for the studies of V1, S1, S2 and motor cortex, respectively). The proportion of identified callosal axons that projected such branches was smaller in V1 (8%) and S1 (12%) than in S2 or motor cortex (24% in each: Swadlow, 1988, 1990, 1991, 1994). In nearly all cases the site of branching was shown to be temporally near the soma (<1 ms), and conduction velocities were considerably higher in the callosal branch than in the ipsilateral
Time and the brain
156
Figure 1 . Histograms of axonal conduction velocities of various axonal populations. From above downwards: CC, C-IC, CF-5 and CF-6 axons of V1, S1, S2 and motor cortex. In lowest histogram, conduction velocities of thalamocortical axons projecting from LGNd and VB thalamus to V1 and SI, respectively, are shown. These data, and those presented in Figure 2 are derived from Swadlow, 1988–
Information flow along neocortical axons
157
1995. Asterisks shown above uppermost histogram show comparable results from callosal axons of monkey prelunate gyrus (Swadlow et al., 1978a)
Figure 2 . Antidromic latencies of the same populations shown in Figure 1
corticocortical branch (Swadlow, 1991, 1994). A similar ipsilateral projection of callosal
Time and the brain
158
axons has been found in primate (Schwartz and Goldman-Rakic, 1982). 3.2. Ipsilateral Corticocortical Axons Figure 1b presents results from ipsilateral cortical systems studied in each of the cortical regions of the rabbit. In V1 and SI, the projections to V2 and S2, respectively, were studied. In S2 the “feedback” projection to SI was studied, and in motor cortex the projection to SI was studied. In each of these systems, stimulating electrodes were placed, under physiological guidance, in the topographically aligned region of the target structure under study. The diverse nature of these corticocortical projections might lead one to expect considerable differences in their axonal conduction properties. However, this was not found to be the case. In each of the cortical systems, the mean conduction velocities ranged from 0.86–1.4 m/s. In each system, a very significant population was seen to conduct at velocities of <1m/s (lowest velocities of 0.21–0.4 m/s in the four cortical regions), and a small number of axons conducted at velocities of >4 m/sec. Thus, these axon conduction velocities are appropriate for a mixed population of unmyelinated and fine-diameter myelinated axons. Figure 2b shows the antidromic conduction time for the C-IC axons shown in Figure 1b. In each of these systems studied in rabbit, the few fastest conducting axons resulted in a small number of very short conduction times (1–2 ms). However, in each system, the conduction times along most fibres were quite long. In the motor cortical projection to S1, for example, conduction times of up to 32.5 ms are seen. 3.3. Descending Corticofugal Axons of Layer-5 Figure 1c presents results for CF-5 axons studied in the four cortical regions of the rabbit. In V1, results are limited to corticotectal neurones that were antidromically activated via stimulating electrodes placed in topographically aligned regions of the superior colliculus. In SI and in S2, stimulating electrodes were placed within and above the topographically aligned region of ventrobasal thalamus, and in the motor cortex stimulating electrodes were placed within and above ventrolateral thalamus. In the studies of S1, S2 and motor cortex, the thalamic stimulating electrodes were in close proximity to the internal capsule, and high-intensity stimulation was used to activate axons terminating within the thalamus as well as those descending to regions beyond the thalamus. Thus, the CF-5 axons studied in each of these cortical regions undoubtedly represent a heterogeneous mixture of descending corticofugal neurones originating in layer 5. It is, therefore, somewhat surprising that their conduction properties were so homogeneous, and distinct from those of the callosal and ipsilateral corticocortical axons described above, and also distinct from the CF-6 axons described below. Figure 1c shows that mean axonal conduction velocities of each of the CF-5 axonal systems studied ranged from 10.9–12.9 m/s. In none of the systems were axons observed which conducted at velocities of <2 m/s (lowest velocities for the four regions were 3.7–5.7 m/s). Thus, these axon conduction velocities are suggestive of a population of myelinated axons, and no
Information flow along neocortical axons
159
evidence was seen for the presence of any unmyelinated axons. Figure 2C shows the antidromic conduction time for the CF-5 axons. In each of the systems studied, many axons had very short conduction times (1–2 ms), and very few axons with conduction times of >3 ms were seen. 3.4. Descending Corticofugal Axons of Layer-6 Figures 1d and 2d present results for CF-6 axons. In V1, stimulating electrodes were placed within and above the topographically aligned region of the dorsal lateral geniculate nucleus. In SI and in S2, stimulating electrodes were placed within and above the topographically aligned region of ventrobasal thalamus, and in motor cortex stimulating electrodes were placed within and above ventrolateral thalamus. Highintensity stimulation was used to maximize the probability of activating the axons of all of the layer-6 neurones projecting to the thalamus. The mean conduction velocities of CF-6 axons ranged from 1.73–2.4 m/s in the four cortical regions studied. In each region, a large population of CF-6 axons conducted at velocities of <1m/s (lowest velocities of 0.26–0.52 in the four cortical regions), and a small number of axons conducted at velocities of >6 m/sec. Thus, as was the case for the CC and C-IC populations, these axon conduction velocities are consistent with a mixed population of unmyelinated and finediameter myelinated axons. Figure 2d shows the antidromic conduction time for the above CF-6 axons. In each of the regions studied in rabbit, the few fastest conducting axons result in a small number of very short antidromic latencies (1–2 ms). However, in each region, the conduction times along most CF-6 axons were quite long, and antidromic latencies of >25 ms were not uncommon. 3.5. Specific Thalamocortical Axons Figures le and 2e show the high axonal conduction velocities and short conduction times, respectively, of “specific” thalamocortical axons projecting to V1 (from LGNd; Swadlow and Weyand, 1985) and S1 (from VB; Swadlow, 1995). 3.6. Axonal Conduction Velocities and Neuronal Response Properties In each of the cortical regions studied in rabbit, CF-5 axons were more rapidly conducting than CC, C-IC or CF-6 axons. CF-5 neurones also had much higher rates of spontaneous impulse activity, and were much more responsive to peripheral sensory stimulation than were CC, C-IC or CF-6 neurones. Many CC, C-IC and CF-6 neurones lack both spontaneous impulse activity and a demonstrable suprathreshold peripheral receptive field, and in each of the cortical regions studied these characteristics were observed most frequently in neurones with axonal conduction velocities of <1 m/s (Swadlow, 1988, 1990, 1991,1994). It has been shown that many such “silent”, slowly conducting cortical elements do have subthreshold receptive fields (Swadlow and Hicks,
Time and the brain
160
1996, 1997), and the conditions under which such fields reach threshold is currently under study. These observations indicate that cortical neurones with very slowly conducting axons form functionally distinct subclasses within the CC, C-IC and CF-6 populations.
4. RESULTS FROM CATS, MONKEYS AND RODENTS 4.1. Callosal Axons 4.1.1. Anatomy The corpus callosum is one of the most thoroughly studied axonal systems of the neocortex. A large body of information is available concerning the regional and laminar distribution of both the cells of origin and the sites of synaptic contact in the contralateral hemisphere (e.g. Innocenti, 1986 for review). Because callosal axons remain segregated from other white matter systems for considerable distances on either side of the midline they are readily studied using anatomical procedures. Callosal axons have been examined in mouse (Tomasch, 1957), rat (Seggie and Berry, 1972), rabbit (Waxman and Swadlow, 1976), cat (Fleischhauer and Wartenburg, 1967), monkey (LaMantia and Rakic, 1990; Swadlow et al., 1978) and man (Aboitiz et al., 1989, 1992; Tomasch, 1954). In each of these species, a considerable proportion of unmyelinated axons (19–45%) is reported, and the mean diameter of myelinated axons is ~1 µm. Although the mean axonal diameters of all species studies are very similar, a very few large-diameter axons (5–9 µm) axons are present in species with larger brains (i.e., cat, monkey and man). In monkey, regional differences in axonal composition have been observed in the anterior-posterior axis of the corpus callosum (LaMantia and Rakic, 1990). The most rostral sector, for example, contains ~28% unmyelinated axons, and the mean diameter is 0.7 µm (not including the myelin sheath). The most posterior section (splenium), in contrast, was found to contain only 3.5% unmyelinated axons and the mean axonal diameter was 0.9 µm. (However, Swadlow et al., 1980, reported 31% of splenium axons of a single macaque monkey to be unmyelinated). 4.1.2. Conduction velocities and conduction delays Conduction velocities similar to those of rabbit have been reported in sensorimotor cortex of rat (mean axonal velocity=2.9 m/s, range=0.4–8.3 m/s, Catsman-Berrevoets et al., 1980) and visual cortex of the mouse (mean conduction velocities of <1 m/s Simmons and Pearlman, 1983). However, conduction velocities in carnivores and primates may be considerably higher than those seen in rodents and rabbit. Stimulation and recording methods virtually identical to those used in rabbit were applied to callosal axons of the
Information flow along neocortical axons
161
macaque prelunate gyrus (Swadlow et al., 1978a). Figures 1a and 2a (asterisks) show, respectively, the conduction velocities and conduction times (following stimulation near the midline) for these axons. Conduction velocities were considerably higher than those seen in rabbit (median=9 m/s, range=5–27 m/s), and no conduction velocities consistent with unmyelinated fibres were seen. In cat visual cortex, several authors have reported mean or median conduction velocities of 10–15 m/sec (Harvey, 1980; Innocenti, 1980; McCourt et al., 1990; Toyama et al., 1974), although a few neurones with very slowly conducting axons were reported (up to 15 ms; Innocenti, 1980). A higher proportion of very slowly conducting axons has been found in sensorimotor cortex of the cat. Miller (1975) reported antidromic latencies of <2 ms to >30 ms to stimulation near the callosal midline, and estimated axonal conduction velocities to be from <1 m/s to 10 m/s. Zarzecki (1989) reported a mean antidromic latency of 6.7 ms (range=1.9–15.2 ms) between motor cortices of the cat. Assuming a conduction distance of 30 mm, this implies a mean conduction velocity of 4–5 m/s, considerably lower than that reported in visual cortex. The question of whether interhemispheric conduction times are invariant over a large range of species and brain sizes has been recently addressed by Ringo et al. (1994). These authors argue, both from measurements of callosal axon diameter in a broad range of species and from allometric considerations, that conduction times cannot be held constant across this axonal system by compensatory variations in mean axonal diameter. They point out that to equalize the interhemispheric delay in mouse and man would require human callosal axons to be 15 times the diameter of those of the mouse, but that a diameter difference of only 26% is actually observed. They note, however, that extremely rare “giant” axons are seen in the corpus callosum of large brains. For example, Lamantia and Rakic (1990 [Figure 6]) presented an electron micrograph of one such axon in the monkey corpus callosum, with a diameter of ~9 µm. This axon was believed to originate in somatosensory cortex. Given an interhemispheric conduction distance of about 54 mm in this species (Swadlow et al., 1978a), the conduction time along this axon would be ~1 ms (Waxman and Bennett, 1972). Thus, although the mean axonal conduction time of the callosal system of different species is not scale-invariant, the high-velocity end of the frequency distribution may well be. 4.2. Ipsilateral Corticocortical Efferent Systems 4.2.1. Anatomy Each region of the neocortex projects to numerous cortical areas within the ipsilateral hemisphere, and hundreds of retrograde and anterograde tracing studies have characterized the laminar distribution and morphology of the cells of origin of these systems, their terminal arborization patterns and their sites of synaptic contact. There is considerably less information available regarding the axons of specific ipsilateral corticocortical systems. Some degeneration and/or labelling studies have characterized the apparent diameter of axons coursing towards a particular target (e.g. Rockland, 1989,
Time and the brain
162
1992, 1997; Zeki, 1978). In a recent review, Rockland (1997) concluded that most feedback as well as feedforward corticocortical axons of