Brain, Mind and the Signifying Body

Brain, Mind, and the Signifying Body

Open Linguistics Series Series Editor Robin Fawcett, University of Wales, Cardiff This series is 'open' in two senses. First, it provides a forum for works associated with any school of linguistics or with none. Most practising linguists have long since outgrown the unhealthy assumption that theorizing about language should be left to those working in the generativistformalist paradigm. Today large and increasing numbers of scholars are seeking to understand the nature oflanguage by exploring one or other of various cognitive models oflanguage, or in terms of the communicative use of language, or both. This series is playing a valuable part in reestablishing the traditional 'openness' of the study of language. The series includes many studies that are in, or on the borders of, various functional theories oflanguage, and especially (because it has been the most widely used of these) Systemic Functional Linguistics. The general trend of the series has been towards a functional view of language, but this simply reflects the works that have been offered to date. The series continues to be open to all approaches, including works in the generativist-formalist tradition. The second way in which the series is 'open' is that it encourages studies that open out 'core' linguistics in various ways: to encompass discourse and the description of natural texts; to explore the relationships between linguistics and its neighbouring disciplines - psychology, sociology, philosophy, cultural and literary studies - and to apply it in fields such as education, language pathology and law. Recent titles in this series

Analysing Academic Writing, Paul]. Thibault Classroom Discourse Analysis, Frances Christie ConstruingExperience through Meaning: A Language-based Approach to Cognition, M. A. K. Halliday and Christian M. I. M. Matthiessen CulturallySpeaking: Managing Rapportthrough Talk across Cultures, Helen Spencer-Oatey (ed.) Development of Language, Geoff Williams and Annabelie Lukin (eds) Educating Eve: The 'Language Instinct' Debate, Geoffrey Sampson EmpiricalLinguistics, Geoffrey Sampson Genre and Institutions: Social Processes in the Workplace and School; Frances Christie and J. R. Martin (eds) The Intonation Systems of English, Paul Tench Language, Education and Discourse, Joseph A. Foley (ed.) Language Policy in Britain and France: The Processes of Policy, Dermis Ager Language Relations across Bering Strait: Reappraising the Archaeological and Linguistic Evidence, Michael Fortescue Learning throughLanguage in Early Childhood, Clare Painter Multimodal Discourse Analysis, Kay L. O'Halloran (ed.) Pedagogy and the Shaping of Consciousness: Linguistic and Social Processes, Frances Christie (ed.) Register Analysis: Theory and Practice, Mohsen Ghadessy (ed.) Relations and Functions within and around Language, Peter H. Fries, Michael Cummings, David Lockwood and William Spruiell (eds) Researching Language in Schools and Communities: Functional Linguistic Persepctives, Len Unsworth (ed.) SummaryJustice: JudgesAddressJuries, Paul Robertshaw Syntactic Analysis and Description: A Constructional Approach, David G. Lockwood ThematicDevelopments in English Texts, Mohsen Ghadessy (ed.) Ways of Saying: Ways of Meaning. Selected Papers of Ruqaiya Hasan, Carmen Cloran, David Butt and Geoffrey Williams (eds) Words, Meaning and Vocabulary: An Introduction to Modern English Lexicology, HowardJackson and Etienne Ze Arnvela Workingwith Discourse: Meaning beyond the Clause, J. R. Martin and David Rose

Brain, Mind, and the Signifying Body An Ecosocial Semiotic Theory

Paul J. Thibault with a foreword by

M. A. K. Halliday

Continuum The Tower Building, 11 York Road, London SEl 7NX 15 East 26th Street, New York NY 10010 www.continuumbooks.com © Paul J. ThibauIt 2004 Foreword © M. A. K Halliday 2004

All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage or retrieval system, without prior permission in writing from the publishers. First published 2004 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. ISBN 0-8264-6965-5 (hardback) Library of Congress Cataloging-in-Publication Data A catalogue record for this book is available from the British Library.

Typeset by Kenneth Burnley in Wirral, Cheshire Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wilts

Contents

List of Figures List of Tables Foreword by M. A. K. Halliday Preface

IX X XI

xiii

PART I

Chapter 1: Introduction

2 3 4 5 6 7 8 9 10 11 12

The Body-brain System, Meaning-making Activity, and Ecosocial Environment: Building a New, Unified Discourse The Conceptual Framework of the Ecosocial Semiotic Perspective Gibson's Ecological Theory of Perception and the Three-level Scalar Hierarchy View of Organism-plus-Environment Transactions The Brain as Regulator of Sensori-motor Activity: Implications for Social Semiosis Topological and Typological Modes of Semiotic-material Interdependence Contextualization and Meta-redundancy A Critique of the Causal View of Brain-mind Relations Defining and Extending the Notion of Meaning in Terms of the Three-level Hierarchy and the Specification Hierarchy The Signifying Body: Rethinking the Stratified view of Semiosis in Terms of the Three-level Scalar Hierarchy Systemic-functional Linguistic Theory: Bringing Together the Intraorganism and Inter-organism Perspectives on Meaning-making Reconnecting the Semiotic Concept of Value to the Body-brain System and to Meaning-making Activity A Thumbnail Sketch of the Book

3 8 11 18 23 26 30 34 39 46 49 54

PART 11 Chapter 2: Sensori-motor Activity, Movement, and Social Meaning-making: Rethinking the Expression Plane of Semiosis 1 2

Some Early Signposts from Saussure and Hjelmslev: The Expression Plane as Embodied Articulatory Movement Energy Exchange and the Complementarity of Interacting Body-brains

59 68

vi 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17

CONTENTS The Stratified Model of Semiosis: The Problem of Conceptual Abstractness and Scalar Homogeneity Blackboxing the Sensori-motor Dimension: Language Seen as Modality-independent Centralized Processing Mechanism The Intentional Character of 'Inner' and 'Outer' Sensori-motor Activity: Towards a Unified Account The Symbolic Possibilities of Bodily Movement Articulatory Movement Seen as Actional Semiotic, Not Physical Behaviour Inner and Outer Body States and Social Semiosis The Semiotic Mediation and Entraining of Embodied Bio-kinematic Potential Metafunctional Diversity on the Expression Plane The Expression Plane is the Interface between Body and Ecosocial Environment The Metafunctional Basis of Vocal-tract Articulatory Activity Subjectivity, Agency, and the Prosodic Realization of Interpersonal Meaning Vocal-tract Gestures and Grammar: Symptom and/or Supervenience? The Intentional Character of Articulatory Activity Embodying the Metafunctions: The Example of Vocal-tract Articulatory Activity The Metafunctional Basis of Space and of Bodily Movement in Ecosocial Space-time

69

72 75 77 78 81 83 86 87 90 94 98 100 103 105

Chapter 3: Body Dynamics, Meaning-making, and Scale Heterogeneity: Expression and Content as Cross-scalar Semiotic Processes Embedding the Body-brain in its Ecosocial Environment

2 3 4 5 6 7 8 9 10 11

The Dynamical Character of Expression and Content and the Crosscoupling of Diverse Scalar Levels of Semiotic Organization The Expression Stratum and the Principle of Alternation Stratification in Relation to Expression and Content The Integration of Iconic, Indexical and Symbolic Modes of Meaning in Phonology Rhythm and the Foot The Metafunctional Organization of Phonology as seen from the Perspective of Rhythm and the Foot The Integration of Iconic, Indexical and Symbolic Modes of Meaning in Lexicogrammar Language as Particle, Wave, and Field The Brain as Selective Recognition System: Language and Edelrnan's Theory of Recategorization The Integration oflndividuals to their Semiotic Trajectories The Intrinsically Time-bound Nature of Semiosis: The Integration of Temporal Dynamics Across Scales of Expression and Content

108 109 116 118 120 122 126 134 139 146 148

CONTENTS

12 13

Scalar Heterogeneity and the Phonological and Lexicogrammatical Rank Scales The Emergence of Linguistic Categories from the Child's Primary Forceful Interactions with its Environment

vii

154 162

Part III Chapter 4: The Semiotic Basis of Consciousness

1 2 3 4 5 6 7 8 9 10 11

12 13 14 15 16 17 18 19

First-person and Third-person Accounts of Consciousness The Representation of Subjective Experience in Consciousness in Relation to the Higher-scalar Environment of the Individual Locating the Seat of Consciousness The Meaning-making Capacity of the Body-brain Complex through the Discrimination of Difference Language Functions and the Cortical Organization of the Brain: Implications for Higher-order Consciousness Experiential Meaning and the Assimilation of the Phenomena of Experience to Knowable Categories Interpersonal Meaning as Exploratory and Orienting Activity in Relation to the Ground The Textual Metafunction as Semiotic Means for Giving Unity and Wholeness to Meaning-making Experiential and Interpersonal Meaning in Gaze Proto-interpersonal Meaning and the Child's Exploration of its Environment Bogdan's Theory of Mental Sharing and Topical Predication Interpersonal Meaning, Value, and Action Procedural Knowledge, Declarative Knowledge and the Semiotic Spiral towards Symbolic Consciousness Interpersonal Meaning, Goal-seeking Activity, and the Goal Hierarchy Consciousness and Semiotic Stratification An Alternative Reading of Descartes in the Internalist Perspective of Interpretive Activity The Entropic Character of Meaning Consciousness as the Contextualization of Experience in the Perspective of the Self The Embedding of Consciousness in a Higher-scalar System of Interpretance

171 173 176 184 189 195 198 200 201 202 205 209 212 215 217 224 226 227 231

Chapter 5: The Metafunctional Character of Consciousness: Some Correlations Between the Neurobiological and Semiotic Dimensions

1 2 3

The Contextual Character of Consciousness The Metafunctions and the Shape of Consciousness Vague Contours of the Metafunctions in the Infant's Early Perceptual-motor Engagements with the Environment

236 238 241

VIII

CONTENTS

4 5 6

The Structure of (Self)Consciousness in Perceptual Awareness Damasio's Neurobiological Theory of Consciousness The Proto-metafunctional Structure of Core and Extended Consciousness Integrating the Interaction System and the Meaning System Perspectives on (Self) Consciousness Minding the Gap between Minds: Mirror Neurons and Interpersonal Meaning Inner Speech as Linguistically Realized Higher-order Thinking The Metafunctional Character of Inner Speech as Linguistically Constituted Thought The Re-grounding of the Perspectives of Self and Other in Symbolic Consciousness Text, Social Meaning-making Practices and Higher-order Consciousness

7 8 9 10 11

12

246 248 257 264 267 271 272 276 277

Chapter 6: Brain, Meaning, and Consciousness

1

2 3 4 5 6 7

8 9

Biological and Socio-cultural Factors Form a Single System of Complexly Related and Interacting Factors: Putting Time and Activity Back into the Picture Re-interpreting Flohr's Brain-based Theory of Phenomenal Awareness: A Three-level Hierarchy View of the Emergence of Proto-meaning in the Brain Brain, Meaning, and Symbolic Consciousness The Embodiment of the Material and the Conscious Modes in Expression and Content Periodicity and the Intrinsic Temporal Organization of the Expression Stratum of Speech on Diverse Scalar Levels The Creation of Symbolic Objects of Consciousness in Semantic Neural Space Contextualizing Relations, the Principle of Meta-redundancy, and the Brain as Contextualizing Tool A Definition of Consciousness in Terms of the Semiosis that Occurs Within the Brain The Three-level Hierarchy, Neuronal Activity, and the Emergence of the Self-perspective

Epilogue References Name Index Subject Index

281 283 289 290

295 296

300 302 310 314 318 331 334

List of Figures

Example of gestural scores for /p<en/ ('pan') and /b<en/ ('ban'); borrowed from Browman and Goldstein (1995: 189) 3.1 Syllable structure of sail, showing moraic and non-moraic elements 3.2 Trajectory of wave of stressed and unstressed syllables in clause complex, showing alternation of stressed and unstressed syllables as syntagm unfolds in time 3.3 Trajectory of tonicity wave in clause complex 3.4 Rhythmic field of the word sailed, showing Ictus and Remiss phases of its temporal trajectory 3.5 Dependency relations between Ictus and Remiss phases in never 3.6 Thematicity wave in the English clause complex, showing Theme and Rheme as two peaks of informational quanta 3.7 Multiple modalizing fields acting on the syntagmatic domain of a proposition, showing the scope over which each field's modalized influence extends 3.8 Simplified determiner system, showing the categorial distinction between the values 'specific' and 'non-specific', as symbolized by English the and a 3.9 Expression stratum as combinatorial hierarchy, showing reorganization of higher levels as recouplings of elements on lower levels thereby allowing for the emergence of new systemic possibilities and, hence, new system-environment couplings 3.10 The integration of initial at-oneness with the world to the expansion of information-meaning at more specified levels of semiosis along an individuating trajectory 3.11 Linguistic categorization and its emergence from pre-linguistic forceful interactions between body and environment 4.1 Integration-cum-presupposition hierarchy of iconic, indexical, and symbolic modes of grounding 4.2 Classification of input and output impulses involved in the symbolic transduction of stimulus information and sensori-motor activity in semiosis (adapted and modified from Gibson 1983 [1966]: 46) 6.1 Reentrant loop of conscious experience, showing relationship between self and object of experience 6.2 Content and expression and their relations to the central and peripheral nervous systems; adapted from Peng (1994) 6.3 Open-ended hierarchy of semiotic triplicates of levels showing the integration of perceptual, conceptual, and symbolic phenomena of conscious experience 2.1

91 113

114 114 123 124 129 131

133

143 149 168 211

219 288 294

307

List of Tables

2.1 2.2 2.3 3.1 3.2 3.3

3.4 3.5 4.1 4.2 4.3 4.4 4.5 4.6 6.1 6.2

The robustness of the agent and its interfacing with the expression and content strata of language Stratified model of language, showing vertical hierarchy of different levels of abstraction Metafunctional analysis of exchange unit: phonological, lexicogrammatical and discourse semantic strata; tonic segments in upper case The expression stratum of spoken language in relation to the three-level hierarchy Semiotic properties of icon, index, and symbol The metafunctional organization of both phonology and lexicogrammar on the expression and content strata illustrating the operation of mixed-mode semiosis, combining both discrete typological-categorial distinctions and continuous topological variation on both strata The content stratum of language in relation to the three-level hierarchy Emergence of embodied category formation and its differentiation into 'learning about' and 'acting on' through forceful interaction with environment Lateralization of language functions in the left and right hemispheres according to Deacon A comparison of four accounts of language form and function in context, showing an emerging consensus concerning the multifunctional nature of language and its contextual motivation Proto-experiential meaning in gaze vector Proto-interpersonal meaning in gaze vector, showing ground functions Multimodal eo-deployment of perceptual-semiotic resources in the exchange between the child and his mother in Halliday's example Types of knowledge and stages of semiotic development in early infant semiosis Metafunctional analysis of the imperative clause (you) look at the mess Scalar hierarchy of anatomical structures of perception and relative functions

64 69 96 110 135

152 158 166 191 194 201 202 208 213 298 304

Foreword

It is a privilege to be invited to introduce a work of this range and importance.

Paul Thibault's book is appearing at a time when the disciplinary borders inherited from the previous century - no longer felt as enabling, but rather as constraining, as boundaries rather than borders - are tending to fuzz out and disappear; and new strategies of thought, new dimensions of knowledge are emerging. This book makes a significant contribution to the ongoing dialogue around these critical themes. One feature that contributes to the strength and effectiveness of Paul Thibault's treatment of his topic is the way he engages with the work of leading scholars of the past hundred years whose ideas can be seen as having in some sense anticipated the directions of change. In this respect he is following up his own earlier (1997) study of Ferdinand de Saussure, the scholar usually claimed as 'the father of modern linguistics' and certainly the linguist most frequently revisited and commented on during the entire twentieth-century period. In the present book, Paul Thibault discusses the theoretical contributions of Saussure's most important successor, Louis Hjelmslev, as well as those of other scholars such as Karl Biihler and Gregory Bateson who helped to shape the pattern of knowledge for their own and the succeeding time. By picking up on their work Paul Thibault provides an intellectual context for interpreting the theoretical advances made by contemporary scholars who, in their turn, are redefining the parameters of our own thinking - people such as Gerald Edelman, Jay Lemke, Antonio Damasio and Terrence Deacon. The central organizing concept running through the book is that of meaning, or semiosis - the realm of our existence that is distinct from, and complementary to, the material realm. Paul Thibault theorizes meaning as 'trajectories' taking place in time, in the form of activities on a number of different scalar levels from neuronal activity in the individual brain through to movement across human populations; but manifested critically in the social meaning-making practices of the members of human groups. Here too there is an echo of a previous motif in his work, that of 'Social Semiotics as Praxis' (1991) developed against the background of theory of narrative. All such activities are grounded in biological processes: 'meaning-making is to be explained in terms which are consistent with what we understand about the biological basis of semiosis' (present volume p. 24) - but this is not to say that the whole of semiosis can be explained by reduction to biology.

Xll

FOREWORD

The most powerful manifestation of semiosis is human language; and in this book Paul Thibault's central concern is the interpretation of language itself as a self-organizing complex system. His concept of the 'signifying body' encapsulates the notion of the human body as the locus where meaning is made, through the operation of the brain at the critical interfacing of the semiotic with the material - the content plane with the ecosocial environment, the expression plane with the physiological resources of the production and reception of speech. Thus all linguistic processes are grounded in processes of the body. But language cannot be reduced to bodily processes, and much of the discussion centres on the organization of language as it appears in depth with this dual perspective. Paul Thibault defines his starting point in these terms: ' ... the intrinsic organization oflanguage has evolved in the species (phylogenesis) and develops in the individual (ontogenesis) so that it cross-eouples both with the biological makeup of the body-brain and with the socio-cultural organization of our ecosocial environment in ways that closely relate to the kinds of social activities that human beings perform' (present volume p. 48). To me as a linguist perhaps the most significant feature of Paul Thibault's many-faceted approach is the way he builds up the intellectual context both for language and for linguistics, language in its relation to the human condition, linguistics in its relation to human knowledge. In a sense these are two aspects of the same contextualizing process, since language figures as the centrepiece of semiosis, or meaning-making activity, and all construction of knowledge is the making of meaning. Paul Thibault's project is not so much interdisciplinary as transdisciplinary, coaligning the resources for thinking about his topic along a new thematic discussion. This is the kind of thinking that is needed so that new questions can be asked wherever there are problems to be solved. M. A. K.

HALLIDAY

Hong Kong November 2003

Preface

The writing of this book has arisen out of the conviction that there is an urgent need for a materialist ecosocial semiotics which is able to reconnect body-brain processes and interactions both to the social and cultural practices which directly act upon and affect human bodies, as well as to the ways in which bodily and brain processes directly participate in and are a constitutively inseparable part of our meaning-making activity. This book is concerned with the role of the body-brain complex in our social meaning-making practices. In recent years, the role of the body and the brain in our social meaning-making practices has been a source of considerable interest and discussion. However, it is my contention that these discussions do not move our understanding of the body's central role in meaning-making beyond the discourse- and language-centred models of textual 'representation' which continue to inform most accounts of the constitutive role of the body in social semiosis. A different orientation is called for. This orientation is what I refer to, following Jay Lemke, as an ecosocial semiotics of human meaning-making activity. Like human meaning-making activity itself, this book is very much a hybrid phenomenon. This reflects my conviction that the science of human meaningmaking activity is necessarily a transdisciplinary theory and praxis. While working on the lectures of the course 'Saussure and Beyond' that I wrote for the Cyber Semiotics Institute between 1996 and 1998 (WWW site: http:// www.chass.utoronto.ca/epc/srb). it became increasingly clear to me that many of the fundamental questions concerning ecosocial semiotic theory can only be answered by engaging in a dialogue with the foundational concepts and questions of both the social and the life sciences. In some ways, my web course 'Saussure and Beyond' is a kind of Prolegomenon to the present study. In the past two decades or so, the dialogue between the human and life sciences has been given new direction and impetus by the development of the theory of complex adaptive dynamic open systems. Such systems are characterized by the fact that their component parts give rise to newly emergent levels of organization that are not reducible to the sum of their lower-level components. Instead, the higher, global levels take on dynamical characteristics that have constraining effects on the lower, local levels. Furthermore, dynamic open systems bring time and history to the centre of the theoretical enquiry. Time, irreversibility, the embedding of systems in higher-scalar environments, and individuation, are all important features of this new way of understanding the world, including the part we play in it. Thus, the prediction and control of phenomena

PREFACE

xiv

and the universalizing principles and modes of explanation in the natural and social sciences that have been the hallmarks of the' classical' approach to science in the past three or four centuries no longer hold sway in this new view of the world and our place in it. Meaning, the interpretation of meaning across many different scalar levels, and our being necessarily inside such systems of interpretance, will be seen to play a key role in this perspective. The new perspectives that the theory of complex dynamic systems makes available to the science of human meaning-making activity allow us to ask new questions. What are the foundational concepts of this approach? What is the relationship between the physical-biological and the social-cultural dimensions of our being? What does this approach entail for the relationship between the expression and content strata of semiotic systems? Does meaning originate in 'mind' or 'brain', or is it the result of complex, time-bound semiotic trajectories that loop between the individual's intrinsic dynamics and the ecosocial environment in which the individual is embedded? What is an ecosocial environment? How does the body-brain complex relate to meaning-making activity? How does consciousness relate to meaning-making activity? What is the status of the discourse- and language-centred models of textual 'representation' in relation to the centrality of activity? How do the material and the semiotic dimensions of meaning-making activity combine with each other? What is the political significance of an ecosocial semiotic theory? It is only through the attempt to understand the constitutive inseparability of the semiotic-discursive and physical-material cross-couplings and dynamics that we can adequately theorize our and others' embodiments, our subjective experience of our ecosocial environments, our perceptions of our inner states and sensations and the meanings we attribute to these both in our internal dialogues in 'inner' speech activity, as well as in our interactions with others, and the materiality of the body as playing a central, not marginal, role in social meaning-making activity. In this volume, I have sought to engage with the relevant perspectives and theoretical insights of a range of disciplines - e.g. linguistics, semiotics, biology, and psychology - in order to understand their relevance to the ecosocial semiotic framework of this book and to reconstitute their perspectives and insights within that framework. In doing so, I have tried to organize the diverse insights and theoretical perspectives that I engage with in this book into a coherent set of principles and analytical tools that will serve to extend the ecosocial semiotic framework into new areas of enquiry and practice. A second book, which I have completed and which will appear after the present one, will focus more specifically on the cultural, social, interpersonal and developmental aspects of the ways in which body-brain systems are cross-coupled to their ecosocial semiotic environments. In many ways, the arguments of the two books are complementary in terms of the overall ecosocial semiotic perspective. I am grateful to Professor Gugliemo Cinque, former Dean of the Facolta di Lingue e Letterature Straniere of the University of Venice, and to Professor Paolo Balboni, then Director of the Dipartimento di Scienze del Linguaggio of the same University. Both Gugliemo and Paolo generously supported my application for study leave from the University of Venice during 2002, as well as facilitating its passage through the University bureaucracy. This leave was crucial in allowing me to complete this project.

PREFACE

xv

This book was completed in August 2002 during my stay as a Visiting Professor in the Department of English in Lingnan University in Hong Kong. I wish to express my appreciation and gratitude to my colleagues at Lingnan University and especially to the Head of the Department of English, Professor Barry Asker, for providing so much support and assistance during the period of my stay at Lingnan University and also for making my stay there such an enjoyable and rewarding one. Wendy Wong, the departmental secretary, was a constant source of invaluable secretarial and practical assistance. I have been privileged to be able to present some of the arguments developed in this book to colleagues and students in universities and research institutes in the following countries: Australia, Belgium, Canada, China, Hong Kong, Italy, Japan, Norway, San Marino, Singapore, Spain, the United Kingdom and the United States. It is impossible to thank all of you by name, but I am most grateful to all of you for providing a forum for my ideas, as well as for your critical discussion and input. I also wish to express my gratitude to the following individuals with whom I have had discussions, or whose advice, support and friendship have been critically important for the completion of this project: John Alexander, Felipe Alcantara Iglesias, Anthony Baldry, Marco Battacchi, Olga Battacchi,Jim Benson, Paul Bouissac, David Butt, Ray Cattell, Magda Cortelli, Stephen Cowley, Michael Cummings, Fan Dai, Kristin Davidse, Ersu Ding, Paolo Fabbri, Robin Fawcett, Gillian Francis, Peter Fries, Nancy Fries, Andrew Goatly, Bill Greaves, Michael Gregory, Morag Harris, Mike Ingham, Marcel Kinsbourne, Lisa Leung, Marc Lorrimar, Bob Lumsden, Eva Maagere, Bill McGregor, Blair McKenzie, Ng Lai Ping, Kay O'Halloran, Carlo Prevignano, Helen Price, Duane Savage-Rumbaugh, Susan Savage-Rumbaugh, Stuart Shanker,Jared Tagliatela, Godfey Tanner, Carol Taylor Torsello, Gordon Tucker, Eija Ventola, and David Wallace. I owe a special debt of gratitude - both intellectual and personal- to Michael Halliday, Ruqaiya Hasan and Jay Lemke for the critically important dialogues that I have had with them on the subject matter of this book over many years. Without their constant encouragement and support, the development of the ideas in this book would have been considerably poorer. I also wish to thank my mother, Sylvia, my father, Philip, my brother, Mark, and my sister, Linda, for believing in me and helping to make it all happen. To my daughter, Ilaria, my profound appreciation and gratitude for her understanding and love. I have dedicated this book to the memory of Morag Harris. My deep sense of loss at her passing cannot adequately be expressed in words. Finally, to Ordy, for filling my life with her radiance and love. PAUL

J.

THIBAULT

Bologna October 2003

In memory of

Morag Anne Harris 17 November 1954-11 December 2000

Part I

This page intentionally left blank

1

Introduction

1. The Body-brain System, Meaning-making Activity, and Ecosocial Environment: Building a New, Unified Discourse This book is concerned with the relations between the body-brain system, meaning-making activity, and the ecosocial environment in which the former are embedded. An increasing number of studies in recent years show the inadequacies of the view that the brain is the seat of disembodied mental processes, or that cognitive and meaning-making activities stop at the skin of the individual organism. Moreover, the idea that the 'inner' mental life of the individual can be disjoined from our meaning-making (semiotic) activity itself needs to be reexamined. Traditionally, Western cultural traditions have evolved two radically distinct discourse traditions for talking about notions such as mind, cognition, and the mental, on the one hand, and society, communication, and the semiotic, on the other (see Gee 1992; Lemke 1995a: 81-82; Manghi 1988; Walkerdine 1997). Furthermore, the material basis of both 'inner' cognitive processes and 'outer' meaning-making, or semiotic, activity has been disjoined from our discourses about the mental (e.g. thinking) and meaning (e.g. discourse, language). Another problem stems from the way in which semiosis is narrowly defined in terms of linguistically informed notions of text. One negative consequence of this restriction is the failure to see not only the continuities that link human semiosis with the semiosis of other species, but also the continuities that exist between perception, non-linguistic conceptual thinking, mental imaging, consciousness, and semiosis. A second negative consequence lies in the way in which linguistic models of text have been used as models of embodied meaning-making activity - activity which is embedded in its ecosocial environment. Textual products and records are produced and/or used in different kinds of social activities (see Lemke 1984a: 78-80; Thibault 1991a: chapters 2-4,1994, 2003a: 58). The more fundamental notion in the theoretical framework of the present study is that of activity. Texts are integral parts of activities in particular ecosocial contexts, yet they have a secondary and derived status with respect to the activities in which they are made and in which they participate. Activity is primary in the present framework (see also Goodwin 1996a, 1996b; Lave 1997). Activity extends beyond the individual into the ecosocial environment. Meaning-making activity is a trajectory-in-time. In saying this, I am adapting the notion of an ontogenetic trajectory in the work of Salthe (1993: 181-5). The locus

4

BRAIN, MIND, AND THE SIGNIFYING BODY

of meaning is not the organism. It cannot be reduced to the semiotic forms that are eo-deployed in a given meaning-making activity or their physical-material substrate. Nor is it the object text that may result from this activity. Rather, the locus of meaning is the trajectory. It is useful to consider meaning-making activity as a semiogenetic trajectory that reaches back in time as well as forwards into the future. The relevant viewpoint here is that of the selves who jointly engage in such activities and the perspectives that they implicate. Furthermore, semiogenetic trajectories can be seen as embedded in much larger-scale historical-biographical (ontogenetic) and ecosocial-eultural trajectories at the same time as they integrate the much smaller scales of the neural and sensormotor dynamics of the body-brain to their own larger-scale dynamics and timescales. A trajectory is a persistence-in-time that arises through the organization of processes. A meaning-making, or semiogenetic, trajectory is, therefore, an organization of semiotic processes that persist on some timescale. The existence of the trajectory on its level is dependent on both higher-scalar contextual constraints and lower-scalar affordances, or enabling conditions, as well as on meaning which is stored within the system (e.g. the biological organism which supports the trajectory) as biological initiating conditions (Salthe 1993: 107). A semiogenetic, or meaning-making, trajectory does not require preestablished plans or rules locked within the central nervous system of the organism that govern, cause, or otherwise specify from the outset what meanings can be made and how. All that is required to get things started is a motivation or intention which provides the initial impetus. The self-organizing processes that enable a semiogenetic trajectory to emerge in time during the course of the activity will do the rest. That is, the reentrantly correlated neural groups that are built up from diverse sources of information both within the brain, in the environment of the agent, the agent's past history of interactions, and so on, constitute attractors. These attractors interact with the motivation or intention (seeJuarrero 1999: 199) in ways which open up possibilities in the individual's neurally organized semantic state space. Attractors, by definition, enact constrained pathways or trajectories through this state space. Given the recursive nature of cerebral organization as a network of network of networks ... etc., as discussed by Kinsbourne (2001: 244-5; see below), more specific networks are nested within more schematic ones. A vague motivation or intention to do something, as Juarrero (1999: 199) shows, is embodied in a very broad, shallow attractor. A broad or shallow attractor allows for very many lower-level possibilities for its realization in motor behaviour, depending on which sub-networks of neural patterns are activated. If, for example, a linguistic strategy for realizing the intention is activated - e.g. an imperative utterance directing someone to bring about the desired state of affairs - rather than, say, a strategy based on direct physical action such as reaching for the desired object and grasping it in one's hand, then some neural pathways and motor processes rather than others will be activated. These patterns of activation do not require conscious planning or deliberation on the part of the agent. Being an agent does not entail this. Rather, being an agent depends on the overall context of the activity: the activation of

INTRODUCTION

5

one pathway rather than some other sets up particular patterns of constraints on various levels of organization in the overall context as it unfolds in time. The agent's initiating conditions include a body-brain habitus of dispositions and ways of orienting to and being 'in' particular kinds of situations that the agent has accumulated over time, how the agent feels about them, the agent's perceptualmotor abilities at a given point in time, and the categories - perceptual-motor, conceptual, and semiotic - that the agent has established in his or her neural dynamics on the basis of previous engagements with the environment. Once a particular trajectory has been activated through the agent's neural dynamics such that a given motor process - the neuromuscular activity of vocaltract articulation - is entrained to the patterns of activation of the agent's neural dynamics, then higher-scalar contextual constraints come into play. Contextsensitive semiotic constraints provide global principles of organization whereby the action-trajectory is given a determinate organization and is modulated by both higher-scalar ecosocial constraints and lower-scalar (e.g. biological) enabling conditions as it unfolds in time towards its completion. The discourse genres and social activity-structure types which are characteristic of a particular social network are context-sensitive constraints in this sense. They are not fixed recipes for action, but systems of higher-order contextual constraints which cross-couple and entrain semiotic-discursive and physical-material processes to their principles of organization (see Andersen 2000). In so doing, they skew the probability distributions of the semiotic-discursive selections and their crosscoupling to physical-material, including bodily, processes, such that some outcomes are preferred and hence are more likely than others. It is in this way that meaning-making activity is given a determinate shape and vectoriality as an instance of an act of a given type. Rather than relying on an intuitive and common-sense appeal to intentions, wants, desires, and so on, as internal mental states that are prior to selforganizing meaning-making activity, intentions are emergent properties of this activity. Agents, in their meaning-making activity, attribute intentions to themselves and to others as a way of regulating and interpreting the contributions of each other to the eo-construction of the activity itself. In this view, intentions, rather than belonging to an epistemically private inner mental realm, are emergent discourse constraints and meanings that agents deploy in order, dialogically, to co-ordinate their respective contributions to the evolving discourse as it unfolds in real-time. The semiogenetic trajectory, therefore, affords the integration of the body-brain's material interactivity with its here-now environment with past events and occasions of interacting with others, as well as with anticipated possible future outcomes. For example, a near-six-month-old child's high-pitched squeak is contextually integrated with the perceptual pickup of information about an environmental event (the scattering of some pigeons), the child's turning towards and attending to this event, as well as the parent's response to the child's vocalization (Halliday 1993: 95-6; see chapter 4, section 10, pp. 202-5, for discussion). The child's vocalization is itself an environmental event, which the mother both hears and interprets. She interprets the vocalization as a meaningful act that requires a response from her. At the same time, the child's high-pitched squeak is contextually integrated with previous

6

BRAIN, MIND, AND THE SIGNIFYING BODY

experiences of the child's vocalizations and the parents' history of interpreting them in context-specific ways. The notion of the meaning-making (semiogenetic) trajectory thus provides a way of conceptualizing how body-brains contextually integrate information deriving from different perceptual modalities (e.g. seeing, hearing, moving) in the here-now of their own and others' bodily activities to events, activities, and so on, on other space-time scales, both in the actual past and in the anticipated future, involving either the same or other participants (see Thibault 2000a: 303-6; 2003b). It is in this way that infants are integrated, through their active participation in such events, to the higher-scalar ecosocial processes and dynamics where semiotic entities such as, for example, 'wants', 'beliefs', 'intentions' and 'desires' are recognized and used as the basis both for adjusting to and accounting for the behaviour of others qua selves to whom we attribute such things as wants, beliefs, intentions, and desires (see Matthiessen 1993; Thibault 1993; see also Zelazo 1999). On this view, intentions are not, as I pointed out above, epistemically private mental states that belong to an ontologically distinct domain of 'mind' with independent causal powers. Instead, they are meaning-making resources that belong to the higher-scalar system of in terpretance of some ecosocial semiotic system. As such, agents use them for the purposes of interpreting and orienting to each other in discourse, in the process adjusting their own activity on the basis of the belief in our culture in the notion that persons are in possession of intentions qua epistemically private mental states that can be read off behaviour as evidence for the existence of a private mental realm that language simply serves to reveal. This account is essentially a realist one: intentions have a separate existence in the mind with respect to the signs that refer to them. In my account, intentions, desires, wants, and so on are semiotic categories that agents appropriate from the interpersonal moral orders in which their actions are recognized, understood, and accounted for (Harre 1983). As such, they are not independent of the semiotic resources that we use in discourse to construe and enact them. They function in meaning-making activity to interpret and orient our own and others' activities. Semiosis is also a physical-material phenomenon. That is, semiosis is enacted and materially supported by the biologically based physical interactions that occur between individual organisms and their physical-material environments. From this point of view, semiosis is integrated with the material processes of our bodies and brain in interaction with their external environments. Semiotic activity both originates in physical-material (e.g. biological) interactions and, at the same time, has material effects on the environment, including other individuals and their bodies. In particular, meaning-making activity, or semiosis, is cross-coupled to physical-material processes which constitute the substrate of semiosis. By the same token, the activity is guided and modulated along its trajectory by higherscalar semiotic constraints that arise in the symbolic (e.g. semantic) neural space of the individual until the activity's completion. Semiotically mediated activity (meaning-making activity) always has both a semiotic-discursive and a physical-material dimension (see also Hasan 1999: 245, who points out that

INTRODUCTION

7

context can 'never be dissociated from the material and institutional aspects of a culture'). Meaning-making is a semiotic-discursive phenomenon in the sense that its dynamical processes enact meaningful patterns and relations that integrate the particular occasion or situation to the systems of semantic and other semiotic categories that the members of a given community or social network recognize and interpret as meaningful. Furthermore, these categories serve to link the particular event and the particular individuals who participate in that event to other activities, other practices, other individuals and social institutions on diverse space-time scales that go beyond the particular here-now event. These linkages are possible because the relevant systems of categories have evolved in historical time and have been adapted to the diverse needs and purposes of the individuals who constitute a particular community or social network. Those who argue in favour of formalist models of language maintain that language is autonomous with respect to both its physical-material basis in the body-brain complex of the individual organism and the wider social and cultural practices and meaning systems that characterize and constitute a given human community. In this book, I shall emphasize the continuity between semiotic processes and our other embodied resources for integrating organisms to their environments. Thus, perception, non-linguistic conceptual thinking, reasoning, sensori-motor exploration, memory and many forms of pre- or non-linguistic consciousness can be seen to be components of a much larger, phylogenetically older, complex of resources whereby organisms are integrated with and engage in meaningful transactions with their inner and outer environments on diverse space-time scales. Language and other semiotic resources (e.g. depiction, movement, and gesture) are not discontinuous with, or even necessarily qualitatively distinct from, these other systems in all respects. On the other hand, this does not mean that there are no differences between these older cognitive resources and social semiotic ones. Instead, our symbolic resources for making meanings with others in social contexts have evolved as part of this larger, older complex. The symbolic categories of language and other semiotic systems can be seen to be the further specification of these prior, noncultural systems. This position is not dissimilar to the recent arguments advanced by Hauser, Chomsky, and Fitch (2002) that the 'language faculty in the broad sense (FLB) , is shared with other species, whereas the 'language faculty in the narrow sense (FLN) , 'may be unique to humans' (Hauser et al.: 1578). Aside from the question of recursion, which these authors argue belongs to the FLN, their position argues for considerable continuity between animal communication and human language with respect to the FLB. I shall discuss in more detail later in this book the reasons why the mentalist discourse for talking about notions such as 'cognition', 'mental representation', and 'internal symbol manipulation' (Schank and Abelson 1977), cannot serve as a useful or adequate conceptual framework for understanding the ways in which individuals are integrated to their material and semiotic environments on many different scalar levels of organization. The mentalist discourse starts with the assumption of an individual mind which must solve problems and so on, in an external environment 'out there'. Early cognitive science focused on the internal

8

BRAIN, MIND, AND THE SIGNIFYING BODY

mechanisms of cognition, especially on the role of the information-processing strategies, the mental representations, and the computations performed on these representations. There was little or no interest in the neural processes that formed the physical substrate of these cognitions. The prevailing view was that neural processes merely provided the physical medium through which cognition was implemented. Cognition was seen as independent of the physical medium and its workings. This view was based on the models and metaphors of human cognition that the digital computer both inspired and made available. However, the body-brain complex, unlike the digital computer, is a dynamic open system which is both sensitive to and constrained by its contexts. In this book, I take the view that the biology of the body-brain complex cannot be separated from the structure and function of meaning-making activity. Furthermore, the discourse of mind has remained silent about the larger-scale ecosocial systems in which individuals and their activities are embedded and which mediate and link the activities of individuals together as component subsystems which function in larger-scale, systemically organized wholes (see Thibault 1986a). Quite aside from the individual-centred, even ideologically charged, assumption of a mind which is separated from a body and an individual actor who is separate from society, the causal and ontological priority assigned to that reified entity that we usually refer to as 'mind' leads us into ways of thinking and acting that are inconsistent with theoretical assumptions of this book. In the first place, no 'entity' exists independently of some context. Individual organisms are integrated to and are constituent and functioning parts of larger-scale systems and processes at the same time as they themselves are the integrating contexts for entities and processes on still smaller scales. Rather than a constitutive separation of mind, body, and environment, the focus of the present study will be on the ways in which individuals and their interactions with both their inner and outer environments are mediated by higher-scalar systems of interpretance and the social practices in and through which these systems of interpretance are deployed in particular contexts. Individuals, on their particular scale, are constitutive parts or subsystems of these higher-scalar environments. They do not simply exist 'in' an environment which is external to them (Salthe 1993: 20). Rather, the semiotically mediated activities in which they engage actively constitute their environments by virtue of the continuous exchanges of matter, energy, and information-meaning between organism and environment (see also Bateson 1973a: 285-91; Gibson 1986 [1979]: 8).

2. The Conceptual Framework of the Ecosocial Semiotic Perspective The conceptual framework of the present study is an ecosocial semiotic one. This term derives from and is a further development of the concept of ecosocial dynamics proposed by Lemke (1995b). Human socio-cultural systems, Lemke points out, are systems of social practices (actions, activity formations) linked to socially and historically specific meaning formations (thematic and axiological) in which the former are construed as meaningful (Lemke 1983, 1985; Thibault 1986b, 1989, 1990, 1991b, 2003a). Furthermore, every social practice 'is simulta-

INTRODUCTION

9

neously also an instance of some material process' (Lemke 1995b: 106). Social practices of all kinds are simultaneously material processes in a complex, hierarchically organized, developing and evolving ecosocial system. Socio-cultural systems are not, then, comprised of meaningful human practices alone. They are also material-physical processes. From the perspective of an ecosocial dynamics, there is a dialectically dual relationship of complementarity between the semiotic and the material dimensions of the couplings of the two that are enacted in social meaning-making activity. These couplings form the basis of the dynamics of an ecosocial system. Lemke has formulated the fundamental unity of these material and semiotic couplings in the constitution of a unitary ecosocial system as follows: Such systems are hierarchically organized at many different scales through complex couplings of processes which feed back to one another to produce entirely surprising, emergent phenomena (self-organization). In the dynamics of complex, tightly coupled systems with strong multiple feedback loops, even small local changes can produce surprising global effects. Semiotic formations, which slightly bias the linkages of material processes according to their semiotic meanings for a human culture, are essential elements in the material dynamics of human communities. There cannot be two systems here, one material and the other cultural, each changing according to separate laws, relatively independent of one another. There can only be one unitary ecosocial system, material and semiotic, with a single unified dynamics. (Lemke 1995b: 107; italics in original) In other words, there are the dynamics of the meaning-making (semioticdiscursive) practices of the community; there are also (and simultaneously) the material-physical dynamics of the material resources and the matter and energy processes and flows which embody, constrain, and, in part, organize the former. Ecosocial semiotic formations are actions-formations-cum-material-processes that are organized in terms of their semiotic relations and salience as social practices in a human community. Only some of the many possible crosscouplings between the semiotic-discursive and the physical-material are regularly and typically made and recognized in a given community. The dialectically dual character of these cross-eoupling relations also means that neither component is privileged or viewed as dominant. Nor is the one reducible to the other. Social meaning-making practices are not, therefore, uniquely or one-sidedly contextualized in relation to the paradigmatically organized sets of potentially meaningful differences that are typically recognized as being semiotically salient in a given community. Equally important, socially meaningful acts are contextualized in and through their relations to what Lemke (1984b: 113-21; 1995c: 166-75) has called the Interaction System of the community. That is, the material, eco-biological and physical relations to other system processes in a given ecosocial system. It is only through the attempt to understand the constitutive inseparability of these semiotic-discursive and physical-material crosscouplings and dynamics that we can adequately theorize our own and others' embodiments, our subjective experience of our ecosocial environments, our

10

BRAIN, MIND, AND THE SIGNIFYING BODY

perceptions of our inner states and sensations and the meanings we attribute to these both in our internal dialogues in 'inner' speech activity, as well as in our interactions with others, and the materiality of the body as playing a central, not marginal, role in social meaning-making. In the perspective that I have outlined above, one of the central questions is the following: how does patterned meaning-making activity, or semiosis, emerge and evolve in time? The fundamental importance of time and process is now becoming clearer after decades of theorizing and research which privileged plans and/or causal mechanism that were assumed to be in the organism from the beginning and which could be postulated as the driving forces of cognitive, developmental, and semiotic processes (see Thelen and Smith 1994). Meaningmaking activity involves very many heterogeneous elements, on many different space-time scales, ranging from the neurological processes in the human brain to the persistence in historical time of an entire culture's ways of making meaning and the resources for doing so. As we saw above, meaning-making arises from the cross-coupling of both physical-material and semiotic-discursive processes. The time-bound nature of these cross-couplings is fundamental. As Thelen and Smith show, the time-bound cross-couplings of elements from very many different levels of organization are development and cognition. The possibilities of combination of the two domains are seemingly endless: there are very many degrees of freedom. However, the cross-coupling of the two gives rise to pattern and organization because the many heterogeneous elements that are involved self-organize around fewer, simpler, more global principles of organization that give the resulting whole coherence along its temporal trajectory. As we shall see, the reduction of the many degrees of freedom to produce the pattern and organization that we recognize and interpret as meaningful according to our particular observer perspectives is the result of very complex, nonlinear interactions among the processes and relations on the many different scalar levels of organization that are involved. Ecosocial systems are stable, far-from-equilibrium dynamic open systems. Contrary to our expectations, based on the behaviour of closed systems, stable, far-from-equilibrium systems increase their organization and complexity in time. The far-from-equilibrium conditions that are necessary for this increase in organization and complexity are maintained through continuous transactions of energy, matter, and information between the system and its environment. Such systems were called dissipative structures by Prigogine and Stengers (1985 [1984]: 12-14, 143-4; Salthe 1993: 106-11) because they maintain (and increase) their own order and complexity by obtaining needed energy, matter, and information from the environment and then dissipating back to the environment some of this waste energy in the form of higher-entropy outputs that export entropy or disorder to the environment. Ecosocial semiotic systems and the human individuals who participate in these are, on their respective scalar levels, dynamic open systems. Such systems - physical, chemical, biological, social - are comprised of very many interacting components on different scalar levels that interact in nonlinear and heterogeneous ways. Moreover, they are dissipative structures: their existence-in-time and their structural integrity are maintained by their constant transactions of matter,

INTRODUCTION

11

energy, and information between the system and its environment. Dynamic open systems are, therefore, dependent on their contextualized relations with their 'external' environment. Their persistence-in-time cannot be explained in terms of more fundamental causal principles on lower levels. Instead, their persistencein-time is the organization of processes which depend on their transactions with the system's environment. Dynamic open systems are also self-organizing systems (Varela and Maturana 1980). The nonlinear interactions among the heterogeneous array of components that comprise the system enable patterned arrangements that did not previously exist to emerge spontaneously and to behave in ways that were not predictable on the basis of the very many elements that comprise the system. Instead, the system as a whole is attracted to a smaller number of global principles that come to constrain the behaviour of the system as a whole such that the individual components of the system are attracted to one preferred stable configuration rather than to other possible ones. The enormous potential variability of the system gives rise to points of localized instability such that concentrations of matter, energy, and information attract other elements to their emerging patterns. In turn, these local asymmetries may be amplified and hence bifurcate such that a new mode of organization is selected by the system as its preferred state. Meaning is not 'caused' by mind or brain states and processes that arise from within the individual. Instead, the body-brain complex provides initial (biological) conditions qua material affordances that interact with processes on other levels such as the ecosocial one. A heterogeneous diversity of processes and relations on many different space-time scales cooperate to give rise to the meanings that we recognize and interpret in our interactions with others.

3. Gibson's Ecological Theory of Perception and the Three-level Scalar Hierarchy View of Organism-plus-Environment Transactions Ecosocial semiotic theory also implicates a very different idea of perception, as compared to the traditional account of perception in psychological theory. Traditional theories of perception, as Grene (1990:18) points out in her discussion of J.J. Gibson's (1986 [1979]) ecological theory of perception, are based on the notion of association: organisms of all types 'build up associations from many instances' prior to perception. This notion is akin to the nativist claim that individual stimulus-receptor cells are endowed with an intrinsic, latent and natively provided content, which is 'triggered' by appropriate, externally derived 'primitive sensations' (see Maze 1991: 171-2 for a useful critique). Cibson's ecological theory of perception provides an alternative, which is entirely compatible with the conceptual framework of ecosocial semiotics. Gibsori's account is not a representational theory of perception. For Gibson, perception occurs through the organism's active exploration of its Umwelt. Organisms do not receive sense data from the outside and then elaborate a representation of this, which, in turn, forms the basis of a perception. Instead, they 'can discriminate information that is species-specifically relevant to their survival

12

BRAIN, MIND, AND THE SIGNIFYING BODY

in their environment' (Grene 1990:18). Gibson places the emphasis on the exploratory activities of embodied, active participants in their material environments. For instance, the participants in human social dyads are dialogically oriented both to significant others and the environment (the non-self), as constituted in and through the dyad. The active and embodied nature of perception, as defined by Gibson, accords with this view. In dialogically orienting to the other and/or to the environment, the individual learns to orient him- or herself, to localize, and to converge upon relevant distinctions, or semiotically construable differentiations, in its Umwelt. In the process, the individual does not so much give meaning to these as receive meaning from them in and through the exchange processes which constitute both. Thus, the 'intentional' and active orientation to this principle of alterity represents a predisposition to receive relevant meanings from the outside. At the same time, the autopoetic (Le. self-organizing) nature of these processes mean that in the making of more and more relevant differentiations, the members of the dyad both develop to higher levels of organizational complexity and at the same time expand their Umwelt (Harre 1990: 300). Gibson has rejected the notion of perception as a passive neural representation in the form of a data structure which is stored wholly in the brain. Instead, he has written of the 'mutuality' of animal and environment (Gibson 1986 [1979]: 8)~ The relation between the two terms - animal and environment - is one of complementarity: the one implies and could not exist without the other. The environment of an animal, Gibson points out, cannot be reduced to or explained in terms of the physical environment. The environment of living things is also informationally constrained, although for reasons that are explained in section 8, pp. 34-9 below, it is more appropriate to rethink the concept 'information' in terms of the 'meaning' or the 'significance' of the environment of a given animal. The reductionist view that the environment consists of physical constraints per se 'neglects the fact that an environment is ambient for a living object in a different way from the way that a set of objects is ambient for a physical object' (Gibson 1986 [1979]: 8). As Gibson also points out, the animal is both a perceiver and a behaver in its environment. Gibson's theory of affordances is built on this premise. The affordances of a given animal's environment are what it offers or affords the animal. Again, the complementarity, or mutuality, of animal and environment is crucial here. The environment is not the same for different species of animal. Animal and environment are mutually adapted to each other so that different species orient to different environmental affordances. According to Gibson, the perceptual pick-up of information about environmental events provides the animal with information about that event and what it affords the animal for action or behaviour (Gibson 1986 [1979]: 127). In this way, the perceptual information which is picked up provides the organism with information about environmental events and how the given environmental event can enable the organism to act or behave in particular ways. The importance of Gibson's theory for our present purposes is twofold. First, it shows that organisms perceive the environment in ways that closely and internally relate to the ways they act or behave in the environment. Secondly,

INTRODUCTION

13

the particular affordances to which the animal has eo-adapted provide the animal with information which is already an intrinsic part of its environment. The perceptual pick-up of environmental stimulus information is already primed by the ways in which perception mediates action. There is no requirement here that all of the relevant information be stored in the animal in the form of passive and behaviour-neutral databases that the animal consults in order to infer or reason about the appropriate course of action to undertake. Instead, the environmental stimulus information which is ambient to the organism provides information that is directly constitutive of the animal's action repertoire. That is, the organism does not process information or data about an objective environment 'out there' and then infer an appropriate course of action. Rather, the affordances of the environment directly specify possible courses of action for the organism. In specifying what an environmental event is, they also specify what the animal can do in relation to that event. We need not worry about all the details of Gibson's theory to understand the main point for our present purposes. Perception and action are closely linked to each other at the same time as they implicate, on-line, and constantly respond to, neural, motor, and contextual (environmental) factors. The traditional conception of the brain as a central processing unit that 'represents' or 'models' an external world has no place here. Rather, the brain is primarily concerned with the regulation of sensori-motor activity. That is, the brain regulates relations between the body and the environment at the same time as these bodyenvironment transactions act on and shape body-brain functions. The brain is then a contextual organ which regulates the organism's engagements with its environment. Gibson's theory helps us to see that brain, body, and environment are interlinked as functioning components in a larger-scale system that cannot be explained in terms of or reduced to anyone component part of the larger whole. Gibson's notion of affordances is important here for two reasons. First, affordances are environmental phenomena (objects, events, and so on) that are ambient to or surround the animal in some way that is potentially significant to the animal. Secondly, the affordances of an animal's environment are a functional component of the trajectory that extends from the organism's central nervous system through its bodily (sensori-motor) activity and into its environment and then loops back again to the organism. The point is that affordances are functional components of a trajectory that extends beyond the biological organism and which selectively imports features of the organism's environment back into the organism's internal dynamics. The notion of the trajectory is a fundamental one in this book (see Salthe 1993: 181-8). Trajectories, as we shall see later, occur on many different space-time scales. In the present context, both cognition and meaning-making activity can be thought of as time-bound trajectories that extend from the body-brain complex into the environment only to loop back again such that the information- or meaning-potential of the environment is selectively re-contextualized in the internal dynamics of the individual (see also Bateson 1973a: 285-91;Juarrero 1999: 212-13). Now, the traditional model of the brain as the source of reason and rationality and hence as the central executive or governor which oversees and controls behaviour does, of course, require that brains make contact with the external

14

BRAIN, MIND, AND THE SIGNIFYING BODY

world. The difference between the approach I am proposing here and the traditional one can be explained as follows. First, in the approach I am arguing in favour of here, the brain perse is not the sole locus of cognitive activity, as in the traditional approach. Instead, cognition and/or semiosis are a distributed activity implicating a trajectory which links brain, body, the available semiotic resource systems, and ecosocial environment. The seat of cognition and semiosis is the trajectory; it is not the brain per se. Meaning-making activity is not unilaterally caused by a central executive inside the brain. Rather, it is the emergent consequence of the ways in which both the intrinsic dynamics of the organism e.g. patterns of neural activation, neuromuscular activity, cognitive capacities such as memory, attention, and so on - interact with contextual factors, relevant intertexts, the available material and semiotic resources such that both intrinsic and extrinsic factors are jointly recruited and eo-deployed in the real-time assembling of the meaning-making trajectory. Secondly, Gibson's theory of affordances is one theory which shows that the environment of an animal is meaningful for the animal in non-trivial ways. That is, meanings are located and stored at the higher-scalar level of the environment which surrounds the animal, rather than being located in or caused by lower-scalar biological mechanisms that are entirely within the organism. In this way, we can begin to glimpse the need for a theory which can discuss different scalar levels that are implicated in the organism's transactions with the affordances in its environment. Salthe points out that, for most purposes, a basic three-level system is 'minimally needed in order to preserve the complexity' (1993: 46). The three-level system therefore avoids the temptation of reducing all levels to a single, privileged level at the same time as it preserves the recognition that there are processes taking place on different scalar levels. Higher-scalar levels provide the integrating contexts for lower-scalar ones. They act as contextual constraints or boundary conditions on the lower levels. Lower-scalar levels constitute enabling conditions of higher-scalar ones. Generally speaking, the intermediate level is selected as the focal level. The interactions between the organism and some affordance can be taken as the focal level in the present account. Below this level, there are the biological properties of organisms and the physical-material and possibly biological properties of the affordances that organisms interact with and which predispose them to some kinds of organism-affordance engagements rather than others. Above the focal level of these organism-affordance transactions, there is the environment of the organism. Importantly, it is in the environment of the organism that the meanings of environmental events are located and interpreted. This is so because meanings, in the extended sense I am using the term here, are not solely located in the individual organism, but are distributed across an entire population and its environmental niche. It is the environment as repository of meanings that mediates and brings into relation an individual organism qua member of that species and a particular environmental affordance in a given time and place. But it is the environment of the animal, as defined above, and, therefore, the possibilities of perception and action, in the first instance, that it affords the animal which mediates this relation and which makes possible its interpretation for the organism. To paraphrase Bateson (1973b: 392), there is

INTRODUCTION

15

contextual redundancy (see section 6, pp. 26-30) in the organism-plusenvironment system such that the morphology and behaviour of the organism provide information concerning the environment that the organism inhabits. In the processes of eo-adaptation and eo-evolution that create the organism-plusenvironment system, the environment is imported into the organism's internal dynamics such that it entrains these to its own dynamics (see above). The relevant system of relations is hierarchically organized in terms of three levels. The three-level hierarchical system which I have derived can be formalized as follows: L+1: Environment of animal qua system of interpretance which brings into relation (mediates) animal and its affordances and provides the higherscalar principles whereby these affordances and their relation to the animal can be interpreted in ways that afford perception and action; L:

The focal level of the animal's engagements with the affordances that it encounters in its environment;

L-l: The biological and other physical-material properties of organisms and environmental affordances that enable them and predispose them to engage in transactions with one another and selected aspects of their environment. The system of relations that I have outlined here provides a synoptic description of a cognitive and/ or semiotic process that extends beyond the individual organism and into its environment. The three-level scalar hierarchy view shows that the organism's transactions with the affordances it encounters and interprets are embedded in and mediated by a higher-scalar environment which cannot be defined in terms of any given individual perse. Furthermore, the lowerscalar neural processes in the brain and central nervous system, along with the body's sensori-motor activity, are neither the source nor the 'cause' of meanings and their interpretation. Instead, the physical properties of body and brain and the dynamical processes which these enable on their timescale are integrated to and contextualized by processes occurring on the still larger timescales of the progressively higher-scalar levels Land L+1. This also means that lower-scalar neural and bodily dynamics are entrained to the dynamics of the higher scales and their processes. It is only when such entrainment to a higher-scalar semantic or other meaningful property takes place that we can say that bodily activities and the physical properties of objects qua artefacts are involved in the instantiation of operations that are meaningful on higher levels. In other words, it is only when such activities and properties are functional in the processes of enacting, storing, and transmitting information which the organism can construe as meaningful in the service of its particular goals that we can say they are performing cognitive and/ or semiotic functions. The focal level L is the level on which the individual's agency is manifested and enacted in specific encounters with environmental affordances and the actions they enable the agent to carry out. The agent is not the biological

16

BRAIN, MIND, AND THE SIGNIFYING BODY

organism. Rather, the agent represents the integration of the lower-scalar neural and bodily (sensori-motor) resources of its organism on level L-l to the perspective of the self and the agency which manifests its selfness on level L. The individual is an agent in its own processes of individuation because it can deploy and change its knowledge, experience, and perspectives through its own activity. The agent can use its own neurological and bodily processes, as well as those of others, as a tool or an affordance in its own activities. The agent is not defined in terms of its neurological processes or its sensori-motor activities. These are lower-scalar (L-l) phenomena whereas the agent is a higher-scalar phenomenon on the focal level L. The lower-scalar phenomena are sub-personal with respect to the agent. Agents contextually integrate these lower-scalar affordances to their own level, in the process completing them by integrating them to the personal perspective ofa determinate self (Salthe 1993: 49; see also Prodi 1989). At the same time, the agent and its activities on level L are constrained and mediated by the higher-order system of interpretance on level L+ 1. The agency which is manifested on the focal level L is an emergent phenomenon which results from the complex, non-linear relations between (l) the 'upwardly' emergent affordances of the body-brain on level L-l and how these are contextually integrated to the next higher-scalar level (L) and (2) the 'downwardly' acting system of contextual constraints on the higher level L+l of the system of interpretance which is in operation. Thus, the agent and its agency on the focal level are constituted by the ways in which higher-order constraints and lower-order enabling conditions (affordances) give rise to an emergent intermediate level of organization which is a consequence of the ways in which the semiotic and material processes in the observed system of relations are organized on the intermediate (focal) level. Thus, bodily activities such as gaze orientation, pointing, gesturing, speaking, facial expressions, body movement, neural dynamics, and so on provide resources which afford joint social meaning-making with others. The extension of these possibilities to extra-somatic artefacts such as texts qua material objects, tools, toys, calculating machines, and so on, means that typological semiotic resources not only spread throughout the ecosocial environment and colonize it, as Lemke (2000a: 203) has put it, but they also adapt to and colonize the brain itself (Deacon 1998 [1997]: 327). The brain is not the 'cause' of these processes; rather, it is a functional participant in a semiotic-material trajectory that extends across potentially many diverse space-time scales. It is by virtue of his or her being embedded in a higher-scalar ecosocial environment and the meaningmaking trajectories which this enables that we can say that the individual, in the sense of the person who participates in social meaning-making with others, is not the biological organism, but the trajectory-in-time. This trajectory extends from the neural dynamics of the individual in ways that regulate, entrain and modulate motor dynamics so that the meaningful content of the neural dynamics is transduced into an instance of a given act-token. Writing on how intentions organize and channel action trajectories that originate in semantic neural space,]uarrero has formulated this question as follows:

INTRODUCTION

17

Thinking of prior intentions as self-organized dynamics within conscious semantic space, of proximate intentions as control knobs governing that space, and of act-types as attractors through that space allows us to conceptualize act-tokens as individual behavioural trajectories within that semantic space. These trajectories originate in intentional dynamics that both activate and then channel and regulate motor processes top-down such that the latter preserve and execute the invariant relations that embody the intention's meaningful content. (Juarrero 1999: 212) Semantic neural space is itself the result of the ways in which the higher-scalar systems of interpretance emanating from the ecosociallevel have entrained the brain's neurological processes to their own dynamics. In this way, the body-brain is contextually in tegrated to the dynamical (ecosocial semiotic) processes of the higher-scale. Kinsbourne (2001: 246) argues that meaning and experience in the brain are not assembled piecemeal fashion from smaller bits that are aggregated into larger wholes, bottom-up fashion. Rather, the global network of neurons, of which the newborn has a full complement, progressively differentiates out of a less differentiated whole into an increasingly more differentiated network of networks of networks ... (Kinsbourne 2001: 246) by virtue of the ways in which the brain is functionally embedded in and entrained by ecosocial dynamics, as well as by its own internal dynamics. The brain is then sculpted by the dynamics of the topological and typological functions of the body and its extra-somatic, artefactual extensions and the ways these operate across different space-time scales in the ecosocial environment. This approach suggests that we need to develop a new discourse for talking and thinking about the ways in which brain, body, and ecosocial semiotic environment are embedded in and are functioning participants in higher-scalar systems that link all three components in complex, hierarchically organized and non-linear interactions across the many levels of relations and space-time scales that are involved (Togeby 2000). Such an approach needs to be combined with an ecosocial theory of meaning-making activity. This also suggests that the shape - the intrinsic patterned organization - of our semiotic resource systems constitute principles of environmental organization that can be seen as fundamental components of brain organization itself. Neural dynamics and neural architecture will thus be seen to be in tune with the dynamics of the body and the ecosocial environment. Consider in this respect the following observations made by Kinsbourne on the left-right hemispherical organization of the brain: We learn that most people's left cerebral hemisphere is concerned with language, their right with spatial orientation. More broadly still, the left caters to sequential analysis and the generating of action sequences, the right to setting such activities into a spatial framework. Most general of all, the left hemisphere controls motivated approach sequences (handling, eating, and so on), progressively focusing and acting upon the target; the right hemisphere is more involved with the person's movement through the intervening space and the spatial background of the target. The left hemisphere's activities can be context free, whereas the activities of the right are context bound. The

18

BRAIN, MIND, AND THE SIGNIFYING BODY hemispheres are complementary in their functioning. (Kinsbourne 2001: 236)

These observations by Kinsbourne show the diverse, yet complementary, roles that the two hemispheres play in the regulation of sensori-motor activity. The left hemisphere is 'analytical' and 'focal'; the right hemisphere is 'general' and 'global'. It is not difficult here to see how the metafunctional organization of language that is postulated by systemic-functional theory can be related to the properties of the global hemispherical network, encompassing both left and right hemispheres. According to systemic-functional theory, language form is organized in terms of a number of diverse functional regions which are simultaneously configured in the forms of language on various levels of organization (see section 10, pp. 46-9, below). Given that social semiosis is an extension and further elaboration of the body-brain's neural and sensori-motor activity, we can postulate that the brain is attuned to the metafunctional organization of language and other semiotic modalities (see Chapter 5). For example, the 'analytical' and 'focal' properties of the left hemisphere that Kinsbourne identifies in the above quotation can be related to the typologicalcategorial bias of the experiential and logical metafunctions. On the other hand, the 'general' and 'global' properties of the right hemisphere can be related to the topological-eontinuous bias of the interpersonal and textual metafunctions. This claim has nothing whatsoever to do with the notion that language is coded in the brain in the form of a neural code, seen as the ultimate neural substrate of contextualized, linguistically mediated activity. However, it does claim that brain dynamics, on their particular scale, are entrained to the dynamics of semiotically mediated activity and in ways which enable brains to regulate the body's meaning-making activity so that it will be both integrated with and appropriate to its contexts. The biological architecture of the brain is consistent with the organization of ecosocial semiotic processes, though this does not mean that the latter can be explanatorily or causally reduced to the former.

4. The Brain as Regulator of Sensori-motor Activity: Implications for Social Semiosis The brain is not a symbolic processor, but a regulator of sensori-motor activity. As we shall see later, sensori-motor activity can be seen as proto-metafunctional in its organization. Social semiosis emerges through the entrainment of embodied sensori-motor activity to the higher-scalar meaning-making practices of the ecosocial system. It is through brain-regulated sensori-motor activity that individuals extend their selves and, hence, their trajectories into their ecosocial environment at the same time that that environment is selectively incorporated into the individual's body-brain dynamics in the form of higher-scalar or topdown contextual constraints. The cross-modal basis of the brain's regulation of sensori-motor activity is, in this way, extended into semiotically mediated forms of activity (meaning-making). That is, the original cross-modal basis of the bodybrain's sensori-motor engagements with its environment gets extended into

INTRODUCTION

19

increasingly specified and socially constrained multimodal forms of meaningmaking involving both somatic and extra-somatic resources and their combinations. By the same token, the body-brain's lower-scalar sensori-motor activity constitutes bottom-up affordances which are entrained to the higher-scalar, top-down constraints of the social meaning-making practices and conventions in which the lower-scalar body-brain dynamics of individuals are embedded. It is the interaction and combination of these two levels which enables newly emergent intermediate levels such as, for example, the dyads that characterize the 'primary intersubjectivity' of mother-newborn interaction, to emerge (Thibault 2000a). Moreover, it is the embedding of body-brain dynamics in and their entraining to the higher-scalar dynamics of the socia-cultural processes that are characteristic of some community or social network that allows us to recognize and interpret the emergent properties of brain-regulated sensori-motor activity as being both semiotically salient and socially functional. In this way, the metafunctional organization of linguistically mediated and other forms of social semiosis can be seen as an emergent consequence of higher-scalar patterns on the ecosocial scale. The logic of the three-level hierarchy view is evident here. Thus, the emergence of social semiosis in the individual is not the result of a biological or mental programme on level L-l. On this level, biological initiating conditions in the form of the values (Edelman 1992; Thelen and Smith 1994: 185-6) that nudge the individual organism along some developmental pathways rather than others (e.g. orienting to faces as a source of meaning and affect; attending to human voices, and so on) are entrained by higher scalar factors on the ecosocial level. At the focal level of the (mother-infant) dyads in which infant semiosis is observable and at which level the infant engages with our perspectives, the infant moves from dynamic attractor to attractor as development unfolds in time. Lower-scalar sensori-motor activity (L-l) constitutes affordances which stand in no single or fixed relation to higher scalar meanings. For example, the neurological processes which regulate sensori-motor activity and emergent social meanings stand in no fixed relation to these meanings. Neural firings in the brain do not generate meanings. Nor do they represent symbolic tokens that are stored in the brain. Rather, lower-scalar neural activity self-organizes to form higher, more global patterns of brain activity that embody the brain states that we recognize and experience, as well as attribute to others, as thoughts, intentions, feelings, meanings, and so on. These higher-scalar mental states do not stand in any fixed relationship to the neurological processes that realize them. It does not therefore matter which neurological patterns on level L-l realize higher-scalar mental states (Juarrero 1999: 190). In this sense, the lowerscalar neurological processes are affordances which are entrained to the dynamics of higher-scalar meanings. The lower-scalar neurological processes afford their selective reorganization as more global patterns which are meaningful on the higher-scalar perspective of the self. Juarrero argues that behaviour is meaningful action depending on 'whether or not the neural processes transmit information as mental' (1999: 191). In this way, arm movement induced by an involuntary spasm and arm movement that is construable as an instance of an act-type such as greeting

20

BRAIN, MIND, AND THE SIGNIFYING BODY

someone are distinguished. Only the latter's trajectory, Juarrero (1999: 191) points out, originates in and takes place in semantic space. The main point here is the role that meaning plays in regulating and bringing about the given motor behaviour qua act-token. Neurological processes therefore do not cause meaningful action. Rather, they are affordances in the sense that they constitute an information potential which is selectively reorganized on higher levels as 'meaning', in the broadest sense of the term, in the perspective of a self. As we shall see in more detail later in this study, the information potential of the lower-scalar neurological and sensor-motor dynamics (level L-l) are reorganized by the next-higher level (level L) as semiotically salient differences or relations - typological or topological - that are recognizable and interpretable for the relevant system of interpretance on level L+1. Strictly speaking, it is not pace Juarrero (see above) a matter of whether the neurological processes 'transmit' a meaningful content or not. Rather, it is a question of whether lower-scalar neurological processes constitute information which is able to be reorganized in ways that are meaningful to higher-scalar levels of organization. The principles which I have outlined here have been formulated by Lemke (1999) as the Principle of Alternation. We shall see later that Lernke's principle is a powerful and elegant theoretical tool that plays a key role in building the theoretical bridges between brain, body, meaning, and ecosocial environment. Furthermore, it fits perfectly with Edelman's theory of neuronal group selection. There is no homunculus or other ultimate causal or executive factor in the brain which directs the process of meaning-making. Rather, neural dynamics, on their timescale, interact with and provide support to events on other timescales. Meaning is the emergent outcome of the interaction across these processes on diverse levels. The crucial factor here is the operation of different timescales on the different levels that constitute the given system. Salthe points out: 'Processes at different scales change direction, cycle, or go to completion at different rates (have different rate constants), with lower-scale ones behaving faster and having smaller relaxation times than higher-scale ones (which, however, can travel at greater absolute speeds)' (1993: 46). Salthe's observations pertain to the organization of scalar hierarchical systems (see section 3, pp. 14-16). With respect to the physical brain processes that realize it, a mental content is a higher-scalar process; it has a slower, larger timescale with respect to the much faster scale of the neurological processes that are its substrate. The latter occur beyond the level of our awareness, whereas mental contents on the higher-scalar level constitute the phenomena of awareness. This suggests that faster, lower-scale neurological processes are reorganized across different scalar levels as perceptual and mental contents that are salient to and congruent with the timescale on which the self experiences and interprets meanings of all kinds (see also Finnemann 2000: 300). Again, we see here the impossibility of insulating one timescale from the other, for the very fast timescale of the neurological processes, say, is not isolated from the slower, higher-scalar level of phenomenal awareness and experience, but is integrated to it such that no insulation of the dynamical relations and processes on each level can be postulated. This integration across levels suggests that the emergence of

INTRODUCTION

21

meaningful contents in the brain is based on the principle of the heterarchical, rather than strictly, hierarchical organization of the different scalar levels that are involved. Kinsbourne (2001: 240) argues on the basis of extensive neuropsychological observation and research that cerebral organization can be understood as 'an architecture of nested functional units'. To quote Kinsbourne: We emerge with an understanding of cerebral organization as an architecture of nested functional units. Within an overall sequential analytic mode, the left hemisphere generates word sequences in one of its parts, action sequences in another, and sequential identification (for instance, of letters) in yet another. Within subareas of the respective areas, it enables both recognition and expression. In subareas of those subareas, it enables material-specific processing, for example, of action words, color names, names of animals, or the connotations of 'right' and 'left.' Within its overall spatial relational mode, the right hemisphere enables orientation in ambient space in one of its parts and pattern perception in another. In subareas of these subareas, it permits subskills like map reading, face recognition, or identifying sketched shapes ('perceptual closure'). A recursive organization emerges; superordinate processing modes differentiate into distinctive domains of functioning, and these again into different specific applications. I envisage a relatively undifferentiated whole that progressively differentiates (in child development? in evolution?) into the rich set of specifically human potential skills and alternative strategies. (Kinsbourne 2001: 240) The brain is organized as a global network of networks of networks of networks . . . (Kinsbourne 2001: 241). Furthermore, the higher mental functions constitute a further development of the brain's original adaptive function as a regulator of sensori-motor activity. Abstract, symbolic forms of thinking, reasoning, and meaning-making are not localized in special modules that developed later, but are emergent from the requirements of sensori-motor regulation. Moreover, in emerging from sensori-motor regulation, abstract semantic categories are able to reorganize and to direct the flow of sensorimotor behaviour along semantically guided trajectories. This is so both from the point of view of behaviour and from the point of view of perception: we can semantically entrain our own behaviour so that it instantiates a token of a recognizable act-type at the same time that we can construe observed behaviour as an instance of this or that act-type. This is clearly evident in the ways in which the semantically organized lexicogrammatical categories of, for example, [AgentAction-Affected] and their more specific instantiations (e.g. [Actor-ActionGoal], [Sayer-Verbal Process-Verbiage], and so on) can be seen as deriving from sensori-motor categorizations at the same time as constituting their more abstract reorganization as the semantic categories that we use in higher-order mental functions such as verbal reasoning, planning, and symbolic construal of our inner and outer environments. The recursive organization of cerebral organization is reflected in the recursive organization of language itself (see above quotation). This does not

22

BRAIN, MIND, AND THE SIGNIFYING BODY

mean that cerebral architecture is hard-wired for universal principles of linguistic organization. Rather, it reflects the way in which neurological processes are entrained to higher-scalar attractor states such as the meanings that derive from the ecosocial level. Specifically, it reflects the way in which the brain is entrained to principles of linguistic organization that are shared by very large numbers of people on the much slower, larger timescale of the historical evolution of a language community of many generations. Boundary conditions are established by higher-order, more schematic categories which set the parameters within which more specific categories and their instantiations in behaviour are meaningful. Thus, Patients with focal brain lesions do not violate the rules that governed their premorbid performances. They remain guided by preexisting parameters but become less specific, slower, and less stable in their responses. (Kinsbourne 2001: 243) Patients with focal brain lesions can be said to recede further 'up' the scale of instantiation such that the broader, shallow systemic properties are retained in ways which constrain behaviour in some more schematic domain whereas the more delicate, more context-specific properties are impaired, compromised, or lost in ways which adversely affect the patient's ability to make very fine-tuned contextual adjustments to more specific domains. This suggests that the neural networks for language and those for other domains are organized according to similar principles of recursive organization. In each case that Kinsbourne describes, the patient regresses to more schematic, hence less specific, categories that regulate behaviour and specify its meaningfulness in increasingly broader, shallower ways. These observations of Kinsbourne suggest that the information that is potentially afforded by lowerscalar neurological processes and the reorganization across progressively higherscalar levels of organization is what enables linguistic forms to 'differentiate out ofless-specific preconscious precursor states, with the word order implicit in the precursor state. The brain models and remodels until the utterance is perfected in its analytical detail' (Kinsbourne 2001: 244). In terms of the Principle of Alternation (Lemke 1999), we can say that the brain organizes and reorganizes topological pre-semantic states on lower levels of neural organization until the typological-categorial linguistic form emerges. Rather than a bottom-up assemblage of parts into whole, language in the brain is a result of the crossscalar reorganization across levels of the information potential that is afforded by lower levels as meaningful patterns on higher levels. Kinsbourne expresses the matter in the following terms: While no truly apt metaphor for how the brain works comes to mind, 'crystallizing out' seems more fitting than 'assembling together.' Interestingly, brain development proceeds according to similar principles. The newborn has a full complement of neurons; further development proceeds by selective cell death and elimination of synaptic connections. The biological chisel prefigures the microgenesis of brain states. (Kinsbourne 2001: 246)

INTRODUCTION

23

5. Topological and Typological Modes of Semiotic-material Interdependence Mental states, experiences, feelings, and so on, are generally assumed to have properties that are intrinsic to them. But it is more useful to think of such phenomena as complexly constructed meanings that we define in terms of patterned contextual relations. The term 'meaning' is normally reserved for the kinds of patterned relations that we recognize and use in, for example, language. Terms such as mental states and experience refer to particular patterns of meaningful relations that our community's habitual ways of thinking and acting have taught us to regard as phenomenal givens that are the personal property or possession of a particular individual. However, our ability to construe something as a mental state or an experience depends on our making sense of patterns of relations whereby we connect some things to other things and not to others (Gendlin 1962). There is a widespread tendency for people to say, for example, that they are conscious of something, that they are having or have had a certain experience, or that a certain mental state occurred in their mind. In the present study, I shall assume that the given experience, mental state, and so on, is made meaningful for us by the ways in which we construe it in relation to other experiences, mental states, material phenomena, and so on. That is, we construct contexts for something through the patterned relations that we construe between some events, things, inner body states, and so on, and others. Experience of all kinds is based on and depends on patterned contextualizing relations which select or foreground some patterns rather than others, in the process drawing attention to only some aspects of all the possible ways in which our inner and outer environments are meaningful to us. Following Lemke (2000a: 194), we can say that selves relate to and interact with both their inner and their outer environments on the basis of two kinds of semiotic-material interdependence. The first kind is topological and is based on continuous variation. Lemke (2000a: 194) points out that this topological mode of semiosis is 'primary and characteristic of thermodynamic and biological systems'. Topological semiosis is based on quantitative or analogue differentiation or variation that makes a difference in relation to some system of relations. Topological semiosis is not based on differences or contrasts between discrete (digital) categories. The second kind of semiotic interdependence is typologicaL It functions on the basis of discrete categories, e.g. the lexicogrammar and semantics of natural language (Halliday 1988). Typological semiosis creates meaning through the contrasting of discrete categories and types. A given sign is contrasted with other signs in ways which are, in turn, construed by still higher-order contextualizing relations that specify how a given combination of signs is relevant and meaningful in relation to that context, rather than other possible contexts in which it might have occurred and in which its meaning might have been different. The distinction between topological and typological semiotic interdependencies is a further development of Bateson's (1973c: 342-4) distinction between analogic and digital coding (see also Wilden 1980 [1972]: chapter 7, 1981:

24

BRAIN, MIND, AND THE SIGNIFYING BODY

19-21). The former, Bateson argues, is concerned with 'patterns and contingencies of relationship' (1973c: 337, 1973b: 388). The latter solves the problem as to 'how to be specific about something other than relationship' (1973c: 337). Bateson (1973b: 387) also argues that human verbal language is not derived 'in any simple way' from the iconic-indexical semiotic modalities of, say, 'kinesics' and 'paralanguage'. In my view, this means that human language, with its typological-categorial grammatical and semantic categories, did not simply emerge, bottom-up fashion, from these evolutionarily prior modes. Nor does it mean that language is a unique, one-off result of a language gene that distinguishes us from all other animal species. Both of these accounts, in their different ways, are reductionist. The first account is reductionist because it assumes that lower-level enabling conditions such as the motor processes of the body are building blocks which can be assembled into human language in bottom-up fashion. The second account is reductionist because it invokes explanations based on innate genetic factors existing on the lower-scalar level of the organism (not the person). In this account, privileged sub-personal causal agents that originate from within the biological organism are primary whereas environmental factors play no more than a subsidiary or 'triggering' role. Language is an emergent intermediate level of organization that is the result of the interaction between both higherscalar ecosocial constraints and lower-scalar biological (neurophysiological, genetic) affordances. The fact that the (phylogenetically and ontogenetically) prior topological semiotic modalities that are concerned with relationship are not transcended by language, but are integrated to and reorganized by it, requires explanation. In the Darwinist view, lower-level processes bring about effects on higher levels through a process of bottom-up emergence. In this view, the whole is an aggregate of its component parts. But this view has nothing to say about how higher-level processes act on and bring about constraining effects on lower-scalar levels. How can our systems of meaning-making (L+1) and the agents (L) who deploy them act upon and bring about effects on lower-level (L-l) biological processes? (see also Finnemann 2000: 281). The old dichotomies, though still persistent, between 'biology' and 'culture', or between 'nature' and 'nurture', are both useless and irrelevant. As the previous discussion of Gibson's ecological theory of perception emphasized, both the information afforded by the environment and the neurophysiological characteristics of the biological organism are component levels of a single system comprising many different levels and their dynamic interactions in time. What is important is not the reduction of one level to the other, but an account in which social semiosis, or meaning-making, is explained in terms that are consistent with what we understand about the biological basis of semiosis without, however, reducing all of semiosis to biology per se. Rather than explanations based on one-way efficient causality and the models and metaphors derived from the digital computer, the emphasis will be on a conceptual framework of complex, dynamic, open, adaptive and goal-seeking systems which share certain fundamental thermodynamic properties with all living systems, as well as with some abiotic systems such as ponds, eddies, and

INTRODUCTION

25

deep-sea vents whose macroscopic configurations qua dissipative structures provide both the thermodynamic potentials and the kinetic substrate for living systems to emerge (Salthe 1993: 277-8). Human agents qua trajectories-in-time, as well as the ecosocial semiotic systems they are embedded in, though on different scalar levels, are systems which evolve or individuate (Salthe 1993: 147-51). The same can be said of particular semiogenetic trajectories. Evolving systems are neither designed nor rule-based. Their organization, their structural integrity at any given moment of their existence, is a result of the ways in which the relational dynamics of the system as a whole constantly adjust to and selforganize in response to changing contextual parameters on diverse timescales. Topological semiosis is characteristic of our interactivity with our immediate material environment, including the external world, our own bodies, and felt inner states of the body. We can then ask the question as to how our semiotic differentiations - both topological and typological - function in the ways in which we construe and construct our relations between our inner and outer states, our bodies, and the material world with which we interact. For example, the ways in which we construct meaningful patterned relations between perceived or felt inner states of the body and our (always constructed) sense of a self with an individual identity, or the ways in which we use our bodies to engage with and to interact with our external material environments, entail the use of the body itself as a material artefact whose topological interactivity with its immediate environment provides the basis for the emergence of typological modalities of semiosis that greatly extend the possibilities for meaningful interaction between our body-brains and the wider, or higher-scalar, ecosocial system in which organisms are embedded and to whose dynamics their body-brains are entrained. In this view, topological semiosis, which forms the ground for all of our interactions with our environment, enables typological semiosis to emerge and, at the same time, the former is never transcended by the latter. In this way, the body-brain itself, along with its extra-somatic artefactual projections in the form of texts, tools, toys, electronic gadgets, and so on, are vectors (Lemke 2000a: 202) for the spread of typological semiosis and its categories through a particular ecosocial system. Both continuous variation (topological) and discrete contrast (typological) can be picked up by the organism and construed as signs of inner and/or outer environmental events and experiences. In this way, organisms build up and elaborate in the Innenwelten that are stored in their central nervous systems models, or systems of interpretance, for responding to and acting on the world in ways that are mediated by topological or typological modes of semiosis (see section Il, pp. 49-54). It is not difficult to see here that the 'signs' which most immediately engage the organism are topological ones, based on the ways in which continuous change or variation in material phenomena are picked up and interpreted as signs of environmental events. For example, continuous variation in a given haptic or olfactory stimulus is picked up by the receptor organs and transduced into neural impulses that stand in no necessary relation to the environmental event. In contrast, the stimulus information about some environmental that was picked up by the receptors stands in an indexical, therefore, a necessary relation to that event. This

26

BRAIN, MIND, AND THE SIGNIFYING BODY

transduction into neural impulses means that the resulting neural impulses stand in a symbolic relation to the environmental event, i.e. they are translated into information stored in neural impulses that constitute a higher-scalar system of symbolic possibilities, based on informational-semiotic constraints, for interpreting and responding to specific events that are (indexically) picked up by the organism.

6. Contextualization and Meta-redundaney Typically, we say that we tasted, smelt, felt, and so on, such and such, as if it were a thing, a phenomenal given that we experience as such. However, this way of talking overlooks the essentially relational and contextual nature of the experience. The relational and contextual nature of the patterns of experience can be described in terms of the principle of meta-redundancy that Lemke (l984c: esp. 35-9) has developed on the basis of the earlier proposals of Gregory Bateson concerning meta-communication and meta-learning (e.g. 1973d: 102-11). Redundancy refers to the likelihood or the probability with which two items, say, connect or combine to form a larger patterned whole or relation. Consider, for example, the eo-occurrence of language and gesture. A redundancy relation exists between some combination of linguistic utterance and gesture when not all possible combinations of words and gestures are likely or possible when we speak. When such a redundancy relation exists, this means that the word and the gesture mutually predict each other's eo-occurrence. There is a better than random chance of their combining or occurring together. Thus, linguistic expression and gesture are redundant with (redound with) each other in the sense that the occurrence of one predicts the probability of its being combined with the other. Now, language and gesture are very different semiotic modalities, organized according to different principles. It is not the case that each modality is simply a different material means for expressing the same kinds of meanings. Language is predominantly typological-eategorial; it is based on discrete categorial contrast or difference. Gesture, on the other hand, is topological-continuous; it is based on continuous variation of visual and spatial relations. The two semiotic modalities do not simply express the same meanings by alternative means of expression. Instead, they make different meanings on the basis of their different principles of organization. Furthermore, there are no fixed, context-free meanings which language and gesture have independently of the contexts in which they are combined. This means that the two modalities can combine in many different, situation-specific ways such that their combinations produce new possibilities for meaning in different contexts. By the same token, there are also limits on the number and kinds of combinations that actually do occur. Not all possible combinations are equally probable and some combinations are more likely or typical than others even though the range of possible combinations of the two modalities is obviously very large indeed. It is this flexibility of combination, within limits, which allows for the creation of meanings that would not be possible on the basis of words or gestures taken separately.

INTRODUCTION

27

The combination of words and gesture depends on the context. The combination of word and gesture forms a patterned relationship in which the words and the gesture are redundant with each other. The words and the gesture constitute a first-order redundancy relation. However, the patterned combination of words and gesture on this particular level depends, in part, on the expectations people have about the contexts in which this particular combination is typically used. The context is redundant with the combination of words and gesture, not with the words or the gesture, taken separately. That is, the context is redundant with the redundancy between the words and the gesture on the first level. Situations are of a higher logical type than are the combinations of words and gestures. For this reason, a situation or context is said to stand in a secondorder or meta-redundancy relation to the first-order redundancy relation between words and gestures. Words and gestures combine to form patterned relationship on the first level. However, this level in itself does not specify the context which the patterned relationship on the first level indexes or in some way creates or defines. For this reason, it is necessary to specify the second-order or meta-redundancy relation of the context in which a particular combination of words and gestures might have the meaning that it does. Importantly, there is usually no fixed relationship between the context and the combination of words and gestures: the meaning of the latter can vary according to the different contextual relations that they participate in or are redundant with. Following the me ta-redundancy formalism proposed by Bateson (l973d: 102-7) and developed further by Lemke (l984c), the different orders of relations that are involved in a given contextualizing relation are represented by the use of the slash, i.e. 'I'. A single slash indicates a first-order redundancy relation, a double slash a second-order or me ta-redundancy relation, and so on. In the following example, the two orders of relations are therefore represented as follows: word I gesture I I situational context Let us now return to the previous discussion of examples involving our experience of taste, smell, and so on. It is not difficult to show how the same basic logic of contextualization applies here as well. For example, the perceptual pick-up of stimulus information about some environmental event is a first-order redundancy relation along the lines discussed above. In this case, the stimulus information that is picked up by the receptor organs in the form of energy that is produced by the given environmental event is redundant with that event. Thus, stimulus information and environmental event are redundant with each other. Importantly, redundancy relations are always symmetrical. Thus, stimulus information is redundant with some environmental event and, at the same time, the environmental event is redundant with the stimulus information that is picked up by the perceptual systems of the organism. Likewise, words are redundant with the gestures that eo-pattern with them just as the gestures are redundant with the words. The same logic also applies to the meta-redundancy relations between the redundancy relations on the first-level and the contextualizing relations on the second level of the given hierarchy of contextualizing relations. Thus, context, C, is meta-redundant with the redundancy of words and

28

BRAIN, MIND, AND THE SIGNIFYING BODY

gestures, just as the redundancy of the words and gestures are meta-redundant with context C. To return to our example involving the perceptual pick-up of stimulus information, we can postulate the existence of symbolic information in neural impulses for interpreting the significance of the first-order redundancy relation between stimulus information and environmental event. Once again, higherorder contextualizing relations define the significance of the first-order relations and, at the same time, the latter create patterned relations that constitute the former. This is so because contexts are enacted, created, and defined by the patterned combinations of items on the level of first focus at the same time as the higher-order context operates as a frame of reference in which the relations on the first level are interpreted, or given meaning for the observer. In the present example, the orders of relations involved may be represented as follows: stimulus information / environmental event / / interpretation: response / / / SI in central nervous system ... More concretely, we can imagine a scenario such as the following by way of illustration. A monkey encounters a species of venomous snake in the tropical rain forest. In so doing, the monkey, in detecting the movement of the reptile through the debris of the jungle floor, picks up stimulus information (visual, auditory, and so on) about the given environmental event, i.e. the presence of the snake, its movement, its location relative to the monkey, and so on. The stimulus information which is picked up provides the monkey with information about a material event in its environment. Stimulus information and environmental event constitute a first-order redundancy relation. The monkey's reaction is one of fear and he withdraws from the scene so as to avoid the danger posed by the snake. The monkey possesses in his central nervous system a model as to how to behave with respect to snakes. That is, his central nervous system constitutes a higher-scalar system of interpretance which enables the monkey to interpret the lower-order redundancy relationships in the appropriate way danger! - as well as to adopt an appropriate course of action (flight). In this case, the levels of contextualizing relations involved may be approximated as follows: stimulus information / presence of snake / / danger: flight / / / SI in central nervous system ... Thus, the monkey's physiological reaction ('fear') and its behavioural response ('flight') are second-order redundancy relations which are redundant with the redundancy between the stimulus information and the environmental event on the first level. In this example, a third-order relationship shows the presence in the monkey's central nervous system of a system of interpretance which enables it to interpret the first-order relation in the appropriate way and therefore to adopt an appropriate course of action. Now, the symmetrical or two-waynature of (meta)-redundancy relations does not assume a determinate or fixed relationship between levels in the metaredundancy hierarchy. Such an assumption would mean that, for example,

29

INTRODUCTION

second-order or meta-redundancy relations are reducible to and, therefore, indistinguishable from the first-order ones which constitute them. In other words, this would mean that the description of the second-order relation is exhausted by the description of the first-order one. This would also mean that there is no distinction between context on the second level and the patterned combination of relations on the first level. However, we have seen that a given combination of, for example, words and gestures on the first level can be specified by different contexts. The meaning of the first-level combination eovaries according to the higher-order context in relation to which it is interpreted. On the other hand, the assumption of a fixed or determinate relation between the two levels would mean that no such variation between lower-level (linguistic and gestural) forms and higher-level contexts could exist. The principle of redundancy, on the other hand, demonstrates that the relations between higher and lower levels allow for constrained variability in the relations between levels (Thibault 1991a: 92; Finnemann 2000: 280). Thus, the same lower-level redundancy relations may participate in a higher-order metaredundancy relation with different contextualizing relations. The same formal relation can be contextualized in different ways. Secondly, the same contextual relation, the same meaning, may be realized in different lower-level patterned combinations oflexicogrammatical, gestural, pictorial, or other formal relations. There is no fixed or wholly determinate relationship across levels in the metaredundancy hierarchy. The various examples discussed above were all hypothetical. In sense, they constitute thought-experiments, though this has been for a purpose. My purpose has been to show that both the interpretative activities of different species (human and monkey) and what we commonly think of as semiosis and perception can be seen to share a similar underlying logic. This does not mean that the differences between perception and activity are being disregarded. Nor does it mean that the me ta-redundancy formalism as presented here can reveal the detailed neurophysiological and other architecture of either perception or semiosis. What I am interested in here are some general principles which they both share. In particular, we see that it is the recognition of patterned relations as occurring in certain contexts rather than others which is the salient point in both cases. In relating patterned relations to some contexts and not others, both humans and monkeys are able to attribute significance to these patterns and hence to act accordingly. Moreover, the patterned relations to which a given individual human or monkey - attributes significance are constrained by higher-order boundary conditions. In the monkey case, the boundary conditions are the tropical rainforest and the existence of life-threatening species of snakes, in relation to which the monkey adopts - partly through learning, partly through genes - a particular behavioural stance. In the human case, the boundary conditions are the cultural practices which constrain and regulate human activity in a particular community or social network. The principle of meta-redundancy refers to a system of contextualizing relations among different classes of acts, e.g. words and gestures. Firstorder redundancy relations are constrained by second- and higher-order

a

30

BRAIN, MIND, AND THE SIGNIFYING BODY

meta-redundancy relations. The principle of meta-redundancy thus shows how meanings are made on the basis of relations of relations of relations ..., of systems of systems of systems . . ., and so on. Higher-order relations act as contextual constraints on lower levels by synchronizing the component parts of the lower levels in ways which permit order and patterned arrangements or meaningful combinations to emerge. That is, the probability of combination of the items on a given level is contextually dependent on the higher-order relations which constrain its possible patterns of combination. These constraints impose limits on the possible patterns of combination on lower levels. In the case of language, the syntagmatic and paradigmatic patterns of its lexicogrammar result from the correlating of the previously discrete functional regions that are typical of protolanguage into a more complexly differentiated and interrelated whole. This shows how different functional regions in the grammar of a language, or in different semiotic modalities, depend on the ways in which the other components of the whole behave in relation to each other. The first-order redundancy relation that we considered above between word and gesture means that the occurrence of the particular words depends on and predicts the likelihood of its eo-occurrence with a given gesture, and vice versa. The establishment of such a two-way or symmetrical redundancy relation between the two items means that they are systematically and internally related to each as parts of some larger whole, e.g. an occasion of talk or some part of this. To the extent that word and gesture are now internally (contextually) related to each other, or, in other words, to the extent that they are now mutual contexts for each other's interpretation, we can say that words, say, have been integrated to the internal dynamics of gesture, and vice versa. Meta-redundancy relations are not static and timeless. Instead, they are dynamic, time-bound, and historical relations and processes. A given contextualizing relation at a given moment in time depends on its history on diverse timescales - the history of the unfolding discourse occasion, its intertexts (what was said, done, and so on, yesterday, last week, last year, and so on), the individual histories of the participants, the history of the community and its meaning-making practices.

7. A Critique of the Causal View of Brain-mind Relations Semiotic relations and practices are always materially embodied in physicalmaterial processes and interactions, though not reducible to them. As I pointed out in section 2 (pp. 8-11) we can talk about social semiotic practices in two ways: (I) in terms of their meaning relations to other practices in a given social formation; and (2) in the language of physics and biology, in terms of the physical processes and relations which embody them (Lemke 1995b: 106--7). Physical and biological processes can only have the meanings they do in a given social and historical formation through the social meaning-making practices of that formation. In this section, I shall consider some of the problems that arise when social semiosis is reduced to or causally explained in terms of physical brain processes.

INTRODUCTION

31

The two domains - the semiotic and the physical - are distinct, though complementary. This complementarity means that the social semiotic can be seen from the dialectically dual perspective of these two conceptual domains, which are not reducible the one to the other. The trouble starts, as Bhaskar (1979: 126) points out, when the social and psychological sciences seek to overcome their own conceptual inadequacies by reducing the domain of their inquiry to the conceptually more highly developed domains of the biological, chemical and physical sciences. Such a move, as Bhaskar points out, can only be explanatorily adequate when the 'to-be-reduced science' is itself conceptually adequate. Given the current conceptual and epistemological inadequacies and confusions which are rife in the social sciences, this means that the resulting reduction explains very little. Social meaning making (viz. semiosis) is only possible through the enactment of material processes which acquire meaning as social semiotic practices. The conflation of the two domains misses a number of crucial points: (1) the same social semiotic (c.f. social and psychological) processes may be embodied by 'lower'-level material processes in a plurality of different ways; and (2) there are multiple social semiotic ways of construing material (c.f. neurophysiological and other) states. Typically, such a conflation entails the following kind of downwards reduction: (a) Social semiotic relations and practices (social)

~ (b) Psychological states and cognitions (mind) ~ (c) Neurophysiological relations and processes (biological organism) The problem with these reductive explanations is that while all social semiosis occurs in accordance with physical-material processes and laws, social semiosis is not completely explainable in terms of them. Again, social meaning-making is not reducible to the intentions and 'inner' cognitive processes of the individual social actor. These processes, too, are constructed and construed out of the same resource systems through which we make and exchange meanings with others in social situations. The fact that these resource systems have been adapted to specialized 'inner' uses does not mean that a separate 'psychology' of the individual is required, seen as explanatorily independent of social semiotic relations and practices. What we call the 'psychological' in Western cultures is a methodological (and political) focus on the individual as an independent domain of cause and effect. The principles of organization of the individual certainly draw on and make use of a folk-theoretical repertoire of (individual) beliefs, intentions, wants, and so on, which form the hypostatic local accounts which actors deploy in order to explain their own and others' behaviour. But these notions are really contextually organized and constructed relations and practices, not reducible to such hypostatic local accounts that are founded on an epistemology of mind, seen as an independent and reified domain of cause and effect. Now, the meta-redundancy relations that constitute the different levels of a contextualization hierarchy are not related to each other by efficient causality (see also Halliday 1992a). As I pointed out above, me ta-redundancy constitutes

32

BRAIN, MIND, AND THE SIGNIFYING BODY

a system of constraints in which all levels operate on and constrain all others. The failure to understand the essentially contextual nature of experience, mental states, and so on, is evident in those accounts of the mind-brain relation that try to account for this in causal terms. For example, Bhaskar, in order to uphold the causal relation between 'mind-states' or 'belief and neurophysiological states or 'matter', argues that: '... such causality as holds between beliefs and matter must be conceived, in the absence of any connecting mechanism, as transcategorial' (1979: 132). Earlier in the same book, Bhaskar had pointed out that this criterion is needed so as to overcome the objections that (1) cause and effect must be embodied in a common medium or share common properties; and (2) the absence of any such medium constitutes an arbitrary break in the chain of cause and effect (1979: 131). Psychologists and others often use such criteria to justify an independent science of psychological or mind states on the grounds that this constitutes an autonomous domain of cause and effect. For example, they may argue that P-states (intentions, wants, etc.) CAUSALLYACT UPON/AFFECT N-states (neurophysiology). Why is it seen as so important to justify an autonomous psychology? According to Bhaskar (1979: 108), the social meaning of an action - e.g. greeting someone - is recognizeable as being an act of a certain cultural type. This is seen as independent of the occasion-specific intention whereby it is performed by a given social actor. This then allows for a causal chain to be reconstructed such that the reasons (beliefs) which lead the actor to want or to intend to perform a particular action may be postulated. The causal chain runs like this: (1) SOCIAL ACTION is caused by (2) ACTOR'S INTENTION is caused by (3) BELIEFS/REASONS LEADING TO INTENTION. Two separate dyadic series are thus postulated. But rather than say that I is caused by 2, and that 2 is caused by 3, we can say, following Halliday and Lemke, that 1 redounds with the redundancy of 2 and 3, in a given context of culture, let us call it O. Note that Bhaskar wanted to separate this out from the intentional action performed by the actor. When we put the ecosocial system back into the picture we can see that: 0 redounds with the redundancy of (1, 2, and 3). For Bhaskar, agency emerges from the psychological links between actors and the locally defined resources they use to interpret each others' intentions. This meets his criterion of synchronic emergence, which correctly recognizes that the higher-order principles cannot be completely explained in terms of the lower-order ones (Bhaskar 1979: 125). Agency, then, is a psychological phenomenon, not reducible to neurophysiological states. But this causal way of reasoning struggles, as we have seen, with the problem of finding some linking mechanism transcategorial or otherwise - which can relate cause and effect. With the notion of meta-redundancy, the problems of form-substance dualism from which this derives are irrelevant. Let us now see why. Earlier, I referred to the complementary dual nature of the relations between the semiotic-discursive and the physical-material domains. Bhaskar attempts to overcome the problem of reductionism with the notions of transcategorial causality and synchronic emergence. His attempt to do so relies on the ontological stratification of the domains of the social, the psychological, and the biological. The resulting localization of these domains, along with Bhaskar's

INTRODUCTION

33

(transcendental) realism, means that these domains are seen as further decomposable into the given reality described and the way this is talked about. The complementarity of our dual perspective means, on the other hand, that there is no essential separation of these domains. Instead, which level of reality is construed depends on which level of contextualization and which operations this performs on the relevant system of relations. This approach accords with Bohr's (1948) principle of complementarity in quantum mechanics: there is no single or global perspective which can reveal an 'objective' description of a complete system. Instead, there are multiple construals and contextualizing principles which operate on different levels of the system, thereby giving rise to a plurality of complementary ways of construing this 'reality', without, however, being reducible to any single principle. Thus, we can say that the social redounds with the redundancy of the psychological with the redundancy of the neurophysiological. The complementary duality of semiotic-discursive systems and practices and physical-material relations and processes does not mean (1) that actions are caused by psychological states (intentions, ete.), which are caused by neurophysiological states, but rather (2) that actions redound with the redundancy of psychological states with neurophysiological states. The emergent properties can be explained not in the language of (transcategorial) causality, but in the contextual language of me ta-redundancy (chapter 6, section 7, pp. 300-1). The units and their relations on the psychological level redound with another set of units and relations on the neurophysiological level, so that the units and relations on the higher-level ecosocial semiotic level redound with the redundancy of the psychological with the neurophysiological. The psychological level of analysis, as McKellar (1990: 337) points out, is really a contextualization of the functional, open, and adaptive interface between the biological (neuroanatomical, neurophysiological, and neuropsychological relations and processes) and the social semiotic relations with which it both eo-evolves and contextually redounds. Thus defined, there is no autonomous or independent domain of psychological theory; the psychological domain is always definable in relation to the two levels of the biological and the social semiotic mentioned above. This does not alter the fact that in Western culture an ontology of the psychological is constructed in and through the social practices of both our official ('scientific') and folk-theoretical (,common-sense') accounts of the individual. But this is a local cultural hypostatization. Where do notions such as agency, subjectivity, and so on, come from? The causal species of reasoning which we have examined here locates this in the (ontological) domain called the psychological, seen as part of a causal chain whereby intentions, motivations, and so on, are derived, explained, and attributed. The ontological stratification of the domain of the psychological in such accounts does not distinguish between the accounting procedures and folktheoretical (common sense) explanations which derive from the local hypostatic culture of the informants, and wider explanatory claims of a truly 'scientific' psychology. In other words, given the fact that informants regularly deploy a repertory of notions such as 'self', intentionality, agency, and so on, what is the relation

34

BRAIN, MIND, AND THE SIGNIFYING BODY

between the use of these terms in the theory and in the folk-theoretical accounts of the culture it seeks to explain? This is a major, though unaddressed problem, in those psychological, philosophical, and other accounts which are based on the kind of ontological stratification that I discussed above. Such theories use a causal-deductive model of explanation, which is not formally recursive. This means that the explanatory bases and me ta-theoretical assumptions of such theories and explanations go undisclosed and are not subject to metatheoretical scrutiny (see Thibault 1991a, 1999a, 1999b for further discussion).

8. Defining and Extending the Notion of Meaning in Terms of the Three-level Hierarchy and the Specification Hierarchy The notion of 'meaning' can be thought about in terms of a system comprising three levels. Meaning is not to be confused with the notion of information in information theory (Shannon and Weaver 1949). The latter notion is concerned with how all forms of information can be quantified in terms of common units such as bits or bytes (see Bateson 1987 [1951]; Wilden 1980 [1972]: 233-6; Lemke 1995c: 169-70). Information theory is concerned with the statistical frequency with which different units and their combinations come about. Information theory is not, therefore, concerned with the interpretation of meaning. Information is independent of particular observer perspectives. When we focus on meaning, on the other hand, we are interested in the differences between the diverse semiotic modalities and the different ways in which their distinctive patterns of difference - whether topological or typological - are organized so as to form a distinctive kind of semiotic system. To go back to our previous example (section 6, pp. 26-30), we are concerned with the different sets of alternatives, the different principles of their combination, the different contexts with which they redound, and so on. Traditionally, semiotics, or the study of signs, has been concerned with the interpretation and classification of signs (Noth 1990). However, the term 'sign' will not be central in the present study. The pervasive and uncritical acceptance of the notion of the sign as something which calls to mind something other than what it is has failed to clarify the ontological status of the concept of the sign or the ways in which this concept relates to processes of meaning-making and their textual and artefactual products (see Bouissac 1998: 737). The fundamental question in the context of the present study has to do with the contextualizing relations that we construct and enact in order to recognize pattern and organization in our experience and in our interactions with the world beyond the self, i.e. the nonself, including other selves. In terms of the logic of the three-level hierarchy, interaction between an interpreter and some portion of the world is the focal level. This is the level on which the self encounters the nonself. The self qua self-organizing system is a firstness in Peirce's sense. A firstness has the characteristic of existing in the world and of having particular perspectives on the world by virtue of its position in the world. But firstness does not exist alone; it is always defined in relation to that with which it interacts in the world, viz. secondness. However, the interaction between

INTRODUCTION

35

firstness and secondness occurs and is made possible by the existence of a higher-scalar system which mediates the relations between firstness and secondness and which makes the interaction between them possible. A higherscalar system of this kind, e.g. a system of interpretance, is an example of thirdness. The relations between firstness, secondness, and thirdness can be described in terms of the three-level hierarchy system of relations. I shall now explain how Peirce's categories can be assimilated to the logic of the three-level hierarchy in order to provide some very general principles for thinking about the notion of meaning. The Italian bio-chemist and semiotician Giorgio Prodi has shown that any interaction between an interpreter and some change in a given material phenomenon presupposes some relation of correspondence between the phenomenon observed and the interpreter. Observation, Prodi (1977: 20) points out, is always a participation in the processes of change: the interpreter, in other words, is always altered by the phenomenon which it observes. This discovery of the mutual interference of observer and observed was first made in the early years of the twentieth century by advances in the theory of quantum mechanics, as developed by N. Bohr and W. Heisenberg. To a given variation in some object, there corresponds a correlated variation in another object, or series of objects. In all such cases, these correspondences occur through the modification of a system of relations, i.e. through exchanges of energy. This basic point is so whether we are talking about a biochemical interaction between two molecules, or the mother-infant dyads in and through which early infant semiosis emerges (see below). Any physical-material perturbation in a given environment constitutes a catastrophic release of a certain quantity of energy. This release of energy may impinge upon the interpretative apparatus of some interpreter. The perturbation in the environment that is entailed by a given release of energy brings about a corresponding change in the interpretative apparatus of the interpreter. The interpreter is biologically adapted to construe certain kinds of perturbation in its environment as being in some way significant to its further development and survival. Observation, so defined, is not 'outside' or 'external to' the observed event. Instead, observer and observed jointly participate in the event, and in ways which change the observer-cum-interpreter. In other words, the processes involved are transactional in character. Prodi also discusses more complex 'situations of correspondence' in which the exchange of energy between, say, A and B is conditioned by the reciprocal adaptation to each other of the two terms in the exchange process. Thus, the material correspondence which triggers the exchange of energy presupposes some more specific relation of 'reciprocal adaptation' between A and B. Prodi (1977: 22) emphasizes two important facets of such processes: (1) the release of energy is conditioned by the existence of a specific (interpretative) 'key'; and (2) the triggering of the exchange of energy is subordinated to its being in contact with this key. It is clear here that A and B, taken separately, are formally equivalent to what Peirce called firstness. The interaction - the reciprocal adaptation - between A and B is a secondness. Moreover, the interpretive 'key' that regulates and mediates this reciprocal adaptation is a thirdness.

36

BRAIN, MIND, AND THE SIGNIFYING BODY

Now, the phenomena that we are discussing here are very general in nature and are common to all biological and social systems. This is indifferently so whether we are talking about two molecules, A and B, in a reciprocal relation to each other, two amoeba, or the two members of a parent-infant dyad in a human social system. The basis of this reciprocal adaptation - the exchange of energy and the establishment of material correspondences between A and B - is the same in all cases. What is different is the level and degree of the complexity involved, as well as the extent of the socio-historical processes, in the case of human, and other, social systems which have further elaborated the dynamics of the exchange processes involved and the ways in which these cross-couple physical-material and semiotic-discursive processes. The prerequisites of this process are as follows. First, there must be two material objects which interact. In so doing, they are transformable. It is not the case that two objects happen to interact by chance, as it were. Again, we can note the principle of the contextual redundancy of the organism-plus-environment system that Bateson pointed out (see section 6, pp. 26-31). The three-level hierarchy is useful in sorting out the levels of relations involved. Thus, the A and B parties to the interaction are on the focal level L. This level is subordinated to some higher-order principle of correspondence, or of complementarity - cf. Prodi's interpretive 'key', or some higher-scalar system of interpretance - on level L+ 1. Such a principle is a contextual relationship which specifies that in a given context-type, A and B select each other as a function of that context. There is, in other words, a contextually governed relation of complementarity between A and B, which, in selecting each other, remain indifferent to other potentially available information in their environment. That is to say, the resulting A-B dyad discriminates and selects contextually relevant information from its environment in species-specific ways, which are relevant to its further development and survival. Secondly, whatever principles of 'order' or 'structure' are derivable from such a me tastable complex of relations are completely immanent in them. These do not simply reside in either A or B, taken 'separately'. Rather, A and B are contextually organized in relation to each other. This obviates the need for some extrinsic metaphysical principle of order (intention, purpose, or plan) which the structure simply contains. Thirdly, the relation of complementarity between A and B is always regulated by material forms (substrates) and thermodynamic potentials which constitute the intrinsic dynamics of A and B and which make the exchange process physically possible. Such material substrates and thermodynamic potentials are the affordances on level L-l that I discussed in section 3 (pp. 11-18). The general principles discussed above can be extended to the human case. To illustrate this, I shall now discuss the dyadic interaction that characterizes joint mother-infant semiosis in the phase that Trevarthen has defined as 'primary intersubjectivity' (1978, 1987, 1992). Trevarthen (1987: 184) describes the way in which newborns seek out eye contact with significant others (parents, caretakers). For Trevarthen, these activities constitute the very earliest stages of dialogic interaction. Such expressive movements on the part of the newborn 'trigger' reciprocal and corresponding acts on the part of the caretaker, who construes the newborn's

INTRODUCTION

37

movement as having some meaning relevant to the emerging dyad. For this to be so, it means that the caretaker must construe the newborn's movement perhaps the establishing of eye contact - as having a form which regulates the exchange. The form which the caretaker construes in the newborn's movement elicits a complementary action on the part of the caretaker. The relevant form emerges out of the joint, or dialogic, act of meaning, in and through which forms are construed in the material processes (bodily movements and orientations) which always implicate the release of energy. This process is always a semiotic one. The two participants in the emerging and developing dyad are dialogically oriented to each other insofar as the dyad is governed by higher-order contextualizing principles which regulate the exchange. The child turns his or her gaze towards the caretaker. This action constitutes a release of energy which impinges on the consciousness of the senior member of the dyad. In so doing, the caretaker construes this release of energy in ways which bring about complementary changes and responses on the part of the caretaker. As embodied beings, A and B are entities whose materiality, along with the exchange of energy in and through the interaction of A and B, provide the only basis whereby a given form may be construed. The embodied beings A and B enter into a structured relation of complementarity which cannot but exist without a particular spatio-temporal configuration and a particular material form. In the case of the early mother-infant dyads studied by Kaye, Trevarthen, and Halliday, the dialogic exchange process means that the two members of the dyad engage in a reciprocal exploration of and a selective attention to the environment in which the dyad is formed. In the first instance, this means the reciprocal exploration and selective attending to each other as embodied beings in the space-time of the dyad. In this way, as Halliday (l992a) shows, the earliest notions of 'you' and 'me' emerge. It is only after this stage that the two members of the dyad jointly explore and attend to the physical-material environment, of which they are always a constituent part, in any case. What is important to emphasize here is that this exploration, too, can only occur through the resources provided by the dyad. Thus, the two members of the dyad, in the process of being constituted as 'you' and 'me', are reciprocally oriented to each other in a process of mutual exploration and interpretation (see also Bates 1979). The child's attention-seeking gaze is an act of exploration. It is both a physical-material act performed by an embodied being in space-time, as well as being a release of energy. Such an act is not random; it is an exploratory act which is oriented to getting a significant response from the other member of the dyad. In this sense, it is a 'reading' of reality, because it is oriented towards selecting a significant response from the other. The newborn explores the reality of his or her Umwelt through the dyad and in so doing he or she also seeks a complementary response from the other. When that happens - when A's dialogic orientation to B produces a significant complementary structure or orientation in B (a response, in other words) - then a specific change may be said to have been induced both in the overall organization of the dyad and in the internal structural organization of its members. A's dialogic act is a specific intervention

38

BRAIN, MIND, AND THE SIGNIFYING BODY

in and an orientation in the space-time of the dyad. While it is true to say, as Halliday does, that the resulting interact has no content in the adult sense, it does, nevertheless, constitute a judgement on - a proto-modalization of - the space-time of the dyad and on the nature of the relationship between the two parties to the transaction. It both intervenes in and comments on that relation. As an intervention, it plays a significant part in the co-ordination and channelling of matter, energy, and information flows in the dyad. As a commentary, it provides an implicit judgement - a meta-communication - about the relations between the two members of the dyad. It is the exchange process which both enables and facilitates the progressive self-organization and individuation of both members of the dyad. The members of the dyad, as I noted earlier, are embodied social beings. However, this does not mean that they reduce to mere physical points in Newtonian space-time. Rather, they are also internally complex beings (Ham'; 1983: 75), capable of further internal elaboration and self-organization. This point is not in contradiction with the fundamentally dyadic and dialogic nature of the organizational principles involved. The dyad and the exchange processes it enables are fundamental. Without these, there would be no exchange processes, no principles of self-organization, no 'individual' members. In other words, the processes, relations and structures involved are fundamentally transindividual and ultimately socio-cultural in nature. This does not imply an opposition between the biological and the social dimensions of these processes. There is no contradiction here because the biological processes are themselves, at all levels of organization, attuned to and consistent with the socio-cultural processes to which they are integrated and entrained. Thus, any equation of the biological organism with the 'individual', and of the social with the 'collective', is false because there is no contradiction, or discontinuity, between the two. Our biochemical, neuroanatomical, and neurophysiological makeup predisposes us to participate in complexly interrelated processes of matter, energy, and information exchange at all levels of organization (Prodi 1977; McKellar 1987, 1990). Now, the last example discussed here most clearly applies to the specifically human systems in which meaning is made, stored, and interpreted. But if we compare it to the two examples that I have derived from Prodi, we can conclude that meaning, as seen from our human perspective, is a more highly specified subclass of progressively less specified subclasses that stretch all the way back through the symbolic capacities of bonobos such as Kanzi and Panbanisha (Savage-Rumbaugh et al. 1998; Benson et al. 2002, 2004; Thibault 2004a) to the proto-language of other mammals such as dolphins, cats, and dogs, right back to the 'situations of correspondence' between, say, an amoeba and its food source. The notion of meaning is a highly specified category which we normally reserve for the kinds of semiotic transactions with others and the perspectives these afford that characterize our semiotically mediated transactions with the non-self (cf. secondness), including our conspecifics. However, the progressively more general, or less specified, cases mentioned above show, albeit to differing levels of specification, that our human notion of meaning (semiosis), rather than being something which is unique to and criterial for our perspective, is, in actual fact, a subclass of a very large number

INTRODUCTION

39

of progressively less specified subclasses. In saying this, I am adopting the logic of the specification hierarchy, as defined by Salthe (1993: 52-74; see also Lemke 1995a: 93-6; Thibault 2000a: 298-300). Thus, each level in the specification hierarchy is what Salthe (1993: 7Q-4) calls an integrative level: more-specified levels integrate less-specified levels to their own perspective, as well as being informed by the lower-level perspectives which make up the whole hierarchy. This means that meaning, as understood from the specifically human perspective, includes and integrates to its own perspective, progressively vaguer, more general classes which we might refer to, starting with the human case, as working backwards, as follows: meaning ... [meaning} ... IImeaning}} ... IIImeaning}}}, ete. The unbracketed term 'meaning' here applies to the most specified case, viz. human semiosis: the use of single, double, triple, ete. curly brackets serves to designate progressively more general, vaguer analogues for this term as we progress towards the most general, least specified analogues of meaning. Similarly, we might say that the reciprocal adaptation of A and B in the example discussed above is a less-specified case of IIldialogue}}}, as seen from the perspective of human serniosis.

9. The Signifying Body: Rethinking the Stratified view of Semiosis in Terms of the Three-level Scalar Hierarchy The notion of the sign has been central in serruoucs for many centuries. Generally speaking, signs are defined in terms of their stable and unchanging properties, rather than in relation to the processes in which they participate and through which they are defined. That is, signs are defined as stable things which simply are. This fact is evidenced in the ways in which signs are classified according to properties of their substance (acoustic, manual-brachial, visual, and so on). In such a substance-based view, the fundamental unit of analysis is some kind of entity or particle. This view mirrors the concerns of the reductionist project of the classical view of how the world is structured and came into being. For example, the model of hierarchical constituency in language reflects the spatial metaphor according to which larger-scalar or higher-level units (e.g. sentences, discourses) are de-composed and analysed in terms of interactions among smaller-scalar units and subsystems. The spatial modelling of these relations views them as a hierarchy of relations among units on progressively smaller (or larger) scales such as sentence, phrase, word, morpheme, and so on, on the grammatical level. Analogous hierarchies are postulated for the phonological level. In general, smaller-scalar units are seen as the building blocks out of which larger-scalar units are assembled in a bottom-up, aggregative fashion. Process and time are largely absent in this view. Such views of the sign are not concerned with the dynamics of time-bound processes in and through which 'signs' emerge in semiosis. In a dynamical, process-oriented view, meaning-making (serniosis) can be understood as the integration of faster, smaller, more local processes into slower, larger and more global ones (Salthe 1993: 46; Lemke 2000b: 275; Thibault 2003a: 137-8). The linguistic notion of the stratification can be refor-

40

BRAIN, MIND, AND THE SIGNIFYING BODY

mulated in the light of this understanding. But first, a few words concerning the way in which stratification is usually understood in linguistics with respect to spoken language. The stratified model of the language system takes a given linguistic form as comprising various interconnected layers of organization. For example, it can be shown that the phonological organization of / dogz/ is a single syllable comprised of four phonemes, but that this does not correspond to its grammatical organization in any direct way. Grammatically, it is a morphemic word comprising the two morphemes [dog] + [z]. The various layers of organization - phonological and morpho-syntactic in the present example - are called strata, following Hjelmslev (1954), Lamb (1966), Halliday (1979), and Martin (1991). The various strata are related to each other by means of realization relationships (themselves a sub-class of the more general meta-redundancy relations that occur among the diverse levels of a given contextualization hierarchy). Realizatory relationships functionally relate the selection and organization of units on a given stratum to the strata 'above' and 'below' it. Realization relationships are interstratal coding relationships. Each stratum also has its own principles of organization for the units and structures that occur within that particular stratum. These are referred to as tactic relationships. In stratal terms, we can define a given unit in functional terms along three different dimensions: (1) its tactic relations with other units on its 'own' level; (2) its realizatory relationships with the stratum 'above'; and (3) its realizatory relationships with the stratum 'below'. Realizatory relations are not one-way or deterministic, but are best seen as a two-way productive dialectic, whereby units and relations on a given stratum are constructed and construed in relation to principles of organization on some other stratum. Interstratal realizatory relationships are both dynamic and open. For example, the morphosyntactic structure [[dog] + [z]] realizes the meaning - the semantics - of this composite grammatical structure at the same time as the given meaning is realized by the grammatical structure in question. Likewise, the morphosyntactic structure is realized by a given phonological structure at the same time as the phonology realizes the relevant grammatical structure. Rather than falling back on shop-worn definitions of the sign that have been refined and reinterpreted according to the assumptions and requirements of various philosophical and theological discourses since classical antiquity to the effect that the sign is something which calls to mind something, or that it is something which stands to someone for something (see Bouissac 1998: 737), we can reconsider the stratified view of semiosis in terms of the three-level scalar hierarchy view of complex, dynamic open systems (Salthe 1993: 36-46; Lemke 1995b: 106-12; Thibault 2000a: 296-8). Thus, a spoken utterance consists of patterns of neuromuscular (vocal tract and other) activity whereby the distinctive sound patterns of the particular language are produced. These patterns of neuromuscular activity take place on a very fast timescale relative to the words, phrases, clauses, and so on, on the next higher level of the proposed hierarchy ofrelations (see also Cowley 1998: 547-51). For example, Lemke points out that the articulation of a typical English vowel takes about one-tenth of a second (Lemke 2000b: 276). Below the level of artic-

INTRODUCTION

41

ulation, there are still smaller, faster timescales such as the activation of the neuronal networks that co-ordinate muscular activity on the timescale of tens of milliseconds, and below this there is the timescale of the neurotransmitter flows where events have a duration that can be measured in milliseconds (see Lemke 2000b: 276). The events that I have referred to here occur on a timescale that is below the level of human awareness. Events on these timescales cannot be observed by humans except by technologically enhanced means. The point is that events on these very small, fast timescales do not belong to the environment which we inhabit and in which we interact with others. They are not 'cogent moments' on the ecosocial scale where our own observer perspectives are to be found. At all levels of organization both 'above' and 'below' the focal level of a given analysis, there is organization of process on different space-time scales, though there is no reduction of processes on higher levels to localized and atomized substances with causal powers on lower levels of organization (Bickhard and Campbell 2000: 330-1). Meaningful pattern and organization on higher levels is not supervenient on some ultimate, substance-based level of atomized causal factors. Instead, it is the reorganization across scalar levels of pattern and organization of lower levels such that what constitutes meaning to our observer perspectives on our scale - seconds, minutes, hours, days, weeks, ... years - is pattern or organization of process that is relevant to our own scale. Consider the following example: L+1: discourse, dialogic exchange L:

dialogic exchange units

L-1: vocal tract articulatory (neuromuscular) activity Following the logic of the three-level hierarchy view, the middle level of any given triplicate of levels is the focal level (i.e. level L). The focal level is the level on which the processes which represent the focus of interest in a particular analysis occur. In the present example, level L consists of dialogically coordinated utterances, realized by lexicogrammatical units of such as words, phrases, clauses, and so on. However, the lexicogrammatical patterns that are observable on level L are themselves made possible by and depend on processes on level L-l. The neuromuscular patterns of activity on level L-1, as Lemke (2000b: 277) points out, constitute affordances for processes on level L. By the same token, level L processes are constrained by longer-timescale processes on the next higher level L+1. In the present case, level L+1 represents an emerging pattern of discourse-level activity. The emerging pattern on this level is able to influence the probability of patterns and processes on the next level down, thereby acting as a set of boundary conditions or constraints on the lower level. The three-level hierarchy principle, as applied here to linguistically mediated semiosis, illustrates the principle whereby processes on different timescales become cross-coupled in more complexly interdependent systems with many interacting variables on many different levels of organization. In the newborn's vocal tract, there are potentially very many degrees of topological freedom such

42

BRAIN, MIND, AND THE SIGNIFYING BODY

that, within the constraints afforded by biology, different patterns of neuromuscular activity are possible. Over time, the infant develops an increasingly large repertoire of possibilities of the co-ordination of vocal-tract muscular activity by neuron impulses. The co-ordination of these possibilities acts as catalysts for still more patterns of activation. A situation is then reached in which a relatively restricted set of patterns of activity in the form of regular, repeatable vocal-tract gestures emerges (see also section 11, pp. 49-54). Consequently, other possible patterns that are potentially afforded by the neuromuscular substrate tend to wane as the activation of the substrate is entrained to the newly emergent level of organization. The newly emergent pattern of vocal-tract gestural activity is afforded by the topological variety of the neuromuscular substrate, though it is not determined by this. The newly emergent level of gestural activity takes place on a longer, slower timescale with respect to the neuromuscular patterns of activity on the next-lower scale. This means that patterns of activation on the lower level are filtered or buffered by the newly emergent level of organization. Information on the lower-scalar level of neuromuscular patterns can only be meaningful to longer-scalar L+1 processes such as the dialogic exchange of meanings between mother and infant after being filtered by the newly emergent level of the vocal gestures on level L. Thus: L+1: protolinguistic dialogic exchange L:

vocal gesture repertoire

L-l: neuromuscular patterns of activity In infant semiosis, the emergence of level-L vocal gestures alters the probability landscape of the lower-level neuromuscular patterns by entraining them to its own dynamics. By the same token, the emergence of proto-linguistic vocal gestures in the infant on level L itself provides a basis for the emergence of the new intermediate level of organization of lexicogrammar between the previously existing levels L-l and L+1 of the proto-linguistic stage. In this way, previously discrete functional components - cf. Halliday's (1975) discrete micro-functions in infant protolanguage - become co-ordinated and reorganized into a more complex and internally differentiated system of multiple, overlapping functional components which greatly enlarge the phase space of the system and hence the variety of contexts it can access on an increasingly diverse number of space-time scales beyond the here-now interactional event (see also Silverstein 1985). The two examples discussed above show the timescales that operate above and below the focal level of utterance and proto-linguistic gesture, respectively. Each level in the hierarchy represents the dynamical processes and the timescale that are characteristic of a particular type of event or activity, relative to its scale. With respect to focal-level processes on level L, the processes on the next level down, viz. level L-l, provide principles of organization that make possible the patterns of organization on the next higher level L. At the same time, the patterns of organization on the still next higher level L+1 constitute higher-scalar levels of organization which provide the context for lower-scalar processes and their

INTRODUCTION

43

interpretation. Higher-scalar processes can thus be seen as deploying lowerscalar affordances in the direction of particular semiotic attractors. Neuromuscular activity is meaningful insofar as it is integrated to and interpreted as part of an utterance, rather than being a mere movement of the vocal tract in its own right. In its turn, an utterance is integrated to and interpreted as part of an unfolding discourse exchange between the interlocutors in that event. The principles that I have outlined above draw attention to the ways in which semiosis, or social meaning-making activity, violates the principle of the separability or insulation of non-adjacent timescales. This is the principle which Lemke has formulated as scalar heterogeneity in contrast to the scalar homogeneity of systems that are 'described solely in thermodynamic or biological terms' (Lemke 2000a: 183). Systems of the latter type are characterized by the way in which the ontological isolation of levels in scalar hierarchical systems ensures that 'dynamics at different levels are screened off from one another, with descriptive variables necessarily representing a single level only, ... ' (Salthe 1993: 45). Each level has its own specific characteristics; hence the ontological isolation of the component levels of the system (Salthe 1993: 46). This reflects the principle of the 'nontransitivity of effects across levels' (Salthe 1993: 45). Lemke (2000a: 182) points out that the properties of scale homogeneity, such that the ontological isolation of levels is the norm, is problematic in the case of human ecosocial systems, which are characterized by cross-couplings of physicalmaterial processes and interactions and semiotic-discursive ones which mediate human activity in the overall ecosocial system. In my view, the structuralist reading of the semiotic notion of stratification has overemphasized the nontransitivity of the relations across different strata. This view had led to the assumption that descriptive variables pertaining to any given stratum - e.g. phonology, morphology, syntax, semantics in traditional accounts - pertain to only one level. In this account, the variables on any given level constrain those on other, especially adjacent, levels, yet the abstractions of phonological and morphosyntactic form, on their particular levels, are not connected to bodily processes on other, lower-scalar levels. The structuralist account of the relation between phonology and morphosyntax is an example. In this account, phonological distinctions, which are not considered to be meaningful in any way, merely function to constrain and specify morphosyntactic distinctions on their level. Thus, human cultural systems cannot be adequately described in terms of scale-homogeneous systems whose regularities can be described in terms of thermodynamic and biological processes per se (see above). The emergence of typological-categorial modalities of serniosis, Lemke (2000a: 183) points out, quickly leads to a situation in which radically different spatial and temporal scales intersect in meaning-making activity. In this way, the here-now event of neuromuscular activity in the vocal tract, for example, implicates meanings on many different space-time scales - e.g. the very fast timescales of neuronal processes, the neuromuscular timescale of vocal-tract articulation, the timescale of the utterance qua meaningful event exchanged between interlocutors, the conversational episode, the intertextual links between today's conversation and the one held last week with the same person, the historical-biographicallifespan scale of the individual, the history of the community in which the individual lives, the

44

BRAIN, MIND, AND THE SIGNIFYING BODY

history of the culture, the language itself and the ecosocial system in which the language is embedded (see Lemke 2000b: 275-9). In semiosis, particularly of the typological-eategorial sort, all of these scales may be implicated in ways that are seamlessly interwoven with the here-now event or occasion of interaction between two or more individuals. The two hypothetical examples discussed above are intended to show how higher-scalar dynamical processes on their timescale have constraining effects on much shorter timescales (Lemke 2000b: 280). For example, the much longer, slower timescale of the emergence of semiotic action formations - i.e. the discourse genres, activity-structure types - in a community have constraining effects on the shorter, faster timescale of the neuromuscular processes of the human vocal tract. The human body has signifying potential because it is a semiotic-material participant in dynamical processes on diverse timescales. For example, fast small-scale neuromuscular activities of the body on, say, the herenow scale of a given interpersonal exchange between two persons are attracted to and integrated with the socio-cultural systems of meanings and practices that pertain to processes on much larger timescales, as compared to the immediate context of the body's material interactivity with its here-now environment (see Lemke 2000a: 194). Human bodies qua material artefacts circulate in social networks, participating in some kinds of activities and not others, according to one's social positioning in these networks and their associated activities on the basis of factors such as sex, age, profession, social class, leisure activities, consumer patterns, lifestyle, and so on. Taken as a whole, the body's participation in various activities in these networks, the movement from one activity to another in the course of shifting from one context to another, amount to a longer-timescale semiotic process, as compared to any of the individual shorter-scale activities that the individual participates in. This longer-timescale semiotic process will be defined in this book as the individual's lifespan or historical-biographical trajectory. The body qua artefact is a material participant in processes - activities - that afford its semiotic integration to and interpretation by higher-scalar systems of interpretance that persist on time scales that go beyond the material body itself. Brains function to co-ordinate, to cross-couple body-brain systems to the ecosocial environment by means of the mediating effects of semiotic artefacts, including objects, textual artefacts, tools, technological aids (computers, calculators, slide rules, OHPs, and so on), architectural features, human bodies (both our own and those of others). It is through such processes of semiotic mediation that the trajectories-in-time of selves and their dialogic intertwining with the trajectories of other selves come into view on various scales - semiogenetic, ontogenetic, historical-biographical, and so on. While I agree with the critique put forward by integrationallinguists such as Harris (l995a) and Cowley et at. (In press) that language tokens are not real mental entities that are 'represented' by neural processes in the brain, I also argue for the need to reframe the counter-elaim put forward by Harris and other integrationalists to the effect that linguistic forms and meanings are 'second order entities arising through the codification (see Harris 1995a) central to both lay and professional metalinguistic analysis' (Cowley et al. In press). We must be

INTRODUCTION

45

careful here not to think that language forms and meanings are mere appearances with respect to some truer or more real underlying reality such as, for example, biomechanical activity. In saying this, I am not denying that both lay and professional me ta-linguistic analyses construct linguistic 'objects', as indeed Saussure (1971 [1915]: 23-7; 1993: 187-8) well understood. My point is that language forms, just like neuromuscular activity, have their own patterns, which are distinctive to their own level of organization, rather than being false to some truer underlying reality. I have suggested that language forms and meanings are integrated with bodily processes and can be understood to be the result of these same processes. At the same time, language forms and meanings have properties and relations that cannot be reduced to or explained wholly in terms of the lower level. Thus suggests that (1) words emerge from neuromuscular activity when neuromuscular activity is entrained to certain patterns rather than others; and (2) the emergent patterns make a difference, i.e. are semiotically salient in some community or social network. Point (2) further suggests that language forms have the potential to make differences that bear directly on our own experience, as well as on the experience of others. In this way, we see that language forms and meanings have the capacity to entrain and direct physical-material processes and flows, including biological ones, by virtue of the properties and relations that are specific to their own level of organization. The fact that the practices - lay and professional - of meta-linguistic codification accord properties to second-order entities such as sentences suggests that lexicogrammatical forms and their meanings are themselves treated as secondorder artefacts that circulate in social networks. This does not mean that these second-order artefacts float free of or transcend their material embodiments in, say, neuromuscular activity or visual tracings on a treated surface (themselves the extra-somatic products of neuromuscular activity) What it does mean is that lexicogrammatical forms and their meanings are more abstract semiotic artefacts which our meta-semiotic practices, in part, de-locate from the full multimodal system in which they are deployed and treat as objects of meta-semiotic reflection and manipulation in their own right. This perspective on multimodality in the mature system may be compared to the early-infant system where the separation into distinct semiotic 'modalities' language, gesture, gaze, movement, and so on - is problematic on account of the inherent topological vagueness and relative indeterminacy of the infant system. Cowley et al. (In press) refer to the infant's 'full-bodied sense making' to capture the fact that in early-infant semiosis this separation into diverse modalities and their combinations and recombinations has not yet occurred. This is so because the lower-scalar affordances of the immature system are sub-personal in the sense that they have not been fully contextually integrated to the perspective of the agent, as in the mature system (see section 3, pp. 11-18). In the mature system, on the other hand, diverse semiotic modalities have become discursively, or semiotically, determinate. Therefore, it is possible to talk about in some meta-serniotic discourse - either lay or professional - language, gesture, gaze, movement, and so on as 'distinct' systems. Here, too, this does not mean that the vagueness and indeterminacy of the earlier system have been

46

BRAIN, MIND, AND THE SIGNIFYING BODY

transcended or superseded. Rather, the properties and effects of the earlier, immature system have receded further down the scale of implication such that the diverse modalities can be distinguished and described according to the requirements of the meta-discourse. Moreover, the increasing determinacy of the diverse semiotic modalities also allows for individuating patterns whereby some (topological) differentiations make a difference concerning individual identity, personality, and so on. The arguments advanced here further suggest that the term 'rnultimodality', as applied to the semiotic resource systems of the mature system, is a meta-semiotic term that best captures the increased determinacy of specific resources in the mature system and, at the same time, the term presupposes the increasing capacity to deploy and to (re)combine specific modalities in ways that display a relative independence and distinctiveness with respect to the primordial vagueness and indeterminacy of the immature system.

10. Systemic-functional Linguistic Theory: Bringing Together the Intra-organism and Inter-organism Perspectives on Meaning-making The present study draws on and is informed by the systemic-functional theory of language which has been developed by Michael Halliday and colleagues over the past forty or so years. A central tenet of this theory is that the internal organization of language - its phonological, graphological and lexicogrammatical form - is shaped by the diverse functions which phonology, graphology and lexicogrammar play in the construction of discourse - spoken and written - in its social contexts of use (Halliday 1994 [1985]; Matthiessen 1995). This claim has entailed a shift away from the sentence-based units that characterized formal grammars, which were mainly based on assumptions about written languages. It has also meant a focus on the fundamentally dialogic nature oflanguage and on the ways in which discourse contributes to and maintains a sense of the semiotic wholeness and unity of particular discourse contexts. In so doing, systemicfunctional theory has helped to redefine what grammar itself is and does. In this perspective, grammatical patterns and relations and grammatical categories are seen as standing in an intrinsic and motivated relation with the activity-structures in which the former are deployed and embedded. Thus, the description of grammar and the description of meaning-making activity are seen as commensurate - the one is responsive to and sensitive to the analytical and descriptive requirements of the other. The relationship between the internal organization of language form - e.g. its lexicogrammar - and meaning has led Halliday and others working within the systemic-functional framework to postulate the existence of a small number of diverse functional regions known as the metafunctions in order to explain the always fluid, dynamic, and contextualized nature of the ways in which language forms relate to meaning in context. What does the term 'rnetafunction' mean? Halliday (e.g. 1979) has proposed that the content stratum of language - viz. its lexicogrammar and semantics - is internally organized into three or four different kinds of very general functional regions which are simultaneously interwoven and configured in the internal organization of this stratum. He has glossed these as the experiential, interper-

INTRODUCTION

47

sonal, textual, and logical dimensions of linguistic meaning, respectively. There have also been further developments and extensions of this principle with respect to semiotic systems other than language (see ]ohnston 1992; Kress and Van Leeuwen 1996; Lemke 1998; Thibault 1994, 2000b, 2001; Van Leeuwen 1999). Experiential meaning interprets the phenomena of the world as categories of experience, as configurations of, for example, clause-level process-types (actions, events, states, and so on), the participants that take part in these, and the circumstances that may be attendant upon them. Experiential-meaning relations are realized in the grammar as particulate or part-whole structures which are based on the principle of constituency. Interpersonal meaning is concerned with the grammatical resources for organizing language as interaction (c.f. speech acts, dialogic moves), the expression of attitudinal and evaluative orientations (modality), and the taking-up and negotiating of particular subjective positions in discourse. Typically, interpersonal meaning is expressed by field-like prosodies rather than particle-like segments. Textual meaning is concerned with the organization of language into semantically coherent text in relation to its context. It is concerned with the distribution of information in text, continuity of reference, and lexico-semantic cohesion. Textual meanings tend to be realized by wave-like peaks of prominence. Logical meaning is concerned with relations of causal and temporal interdependency between, say, clauses. Logical meanings are realized by recursive structures which add one element to another so as to build up more complex structures. In the systemic-functional framework, the language system is a network of value-creating relationships. The meta-grammar is a description of the proportionalities, the paradigmatic systems of options, and the value-making distinctions in and through which the resource systems (phonology and lexicogrammar) for constructing texts are organized. Theory, then, is concerned with how these realize, construct, and construe social reality. The grammar is a resource - a meaning potential - for making meanings in and through texts, which are produced and assembled in meaning-making activity. The grammar is semantically oriented and internally organized along multifunctionallines, as in Halliday's (l978a; 1979) metafunctional hypothesis. The relevant epistemology is one of 'complexity': lexicogrammatical forms realize a complex interplay of multiple semantic factors, aspects of which may be selectively foregrounded in the real-time process of meaning-making, so as to construe some dimension of material-phenomenal reality, which is being attended to in the process of social interaction. The multiple strands of meaning potential in linguistic form eooccur in a subtle and indeterminate relationship of 'complementarity', which can never be reduced to a single or determinate meaning on account of the dynamic and open nature of the interacting functional semantic subsystems involved. Systemic-functional linguistic theory provides useful guidelines as to how the semiotic principle of value as first developed by Saussure (1993) can be reconnected with an account of meaning-making activity (see section 11, pp. 49-54). By and large, systemic-functional theory has been seen as a socially based semantic theory of lexicogrammar and its role in the construction of discourse

48

BRAIN, MIND, AND THE SIGNIFYING BODY

(see Halliday's 1978a notion oflanguage as a form of social semiotic; see also his distinction between the 'intra-organism' and 'inter-organism' perspectives on the study oflanguage in Halliday 1978: 10; see also Wells 1999: 10-11). Biological discourses have not played a prominent role in the development of systemicfunctional theory (see, however, Halliday 1995; McKellar 1987, 1990; Lemke 2000a; Thibault 2000a). This may be due to the conviction - perfectly correct that social semiotic phenomena are not reducible to biological phenomena. However, more recent developments in the theory of complex dynamic open systems show the importance of developing a theory of social semiosis in which the socio-cultural and the biological domains of inquiry are brought into a new dialogue with each other (see also Wilson 1998). It is now becoming possible to make a start in the process of building the theoretical bridges between the 'intraorganism' and the 'inter-organism' perspectives on meaning-making activity such that there is no contradiction or dichotomy between the two. I define my own starting point in the present study as follows. The purpose of language and other semiotic modalities is to guide and co-ordinate our interactions with the non-self and to integrate us with our ecosocial environment across space-time scales that go beyond the here-now scale of the biological organism's material interactivity with its immediate physical environment. Given this premise, it seems logical to say that the intrinsic organization of language has evolved in the species (phylogenesis) and develops in the individual (ontogenesis) so that it cross-couples both with the biological architecture of the bodybrain system and with our ecosocial semiotic environment in ways that closely relate to the kinds of social activities that humans perform and the meanings they make in and through these activities. The functional and contextual basis of systemic-functional theory will prove to be an ideal conceptual and analytical tool for developing these links. Rather than the frozen 'syntactic islands' that are studied by formal theories of language, the emphasis here is on discourse as a form of activity that is richly constrained by both society and culture, on the one hand, and by the language system qua meaning-making resource and by the emerging constraints of a particular discourse context, on the other (Halliday 1978a; Beaugrande 1997: 48). Rejecting the view that a universal grammar enables language users to represent the world through relations of 'truthful' correspondence between the syntactic forms of a language and an objectified external world, the importance given in systemic-functional theory to the interpersonal and textual dimensions of meaning in discursive activity, in addition to the experiential dimension, draws attention to the role of discourse in mediating and enacting body-world relations, rather than seeing language as a symbolic-referential means for revealing and transmitting disembodied 'inner' thoughts, intentions, and symbol tokens. The metafunctional basis of meaning-making activity and the semiotic resources that are eo-deployed in discourse contexts therefore suggest a number of parameters in terms of which both body-environment transactions and body-brain functions are co-ordinated and organized without in any way suggesting that either body-environment transactions or body-brain functions are explanatorily or causally reducible to principles of semiotic organization or formal structure per se.

INTRODUCTION

49

However, the emphasis suggested here does require us to take the intrinsic organization of semiotic activity seriously not as an end itself, but as a guide to the ways in which semiosis enables body-brains contextually to integrate with their ecosocial environments across potentially very many different space-time scales (see chapter 5). In this perspective, body-brain organization can be conceptualized in terms of the way it affords possibilities for the contextual integration of persons into meaning-making activity, rather than in terms of putative internal mental states and competences that 'cause' semiotic activity. The alternative is to remain in the old mentalist paradigm according to which language and other semiotic modalities are, above all, formal windows on internal mental states, rather than modes of embodied meaning-making activity-in-time which directly contribute to the self-organization of neural processes at the same time as the latter - neural processes - directly participate, on their own scalar level, in semiotic processes, rather than lying behind these as the efficient causes of semiotic activity. In this view, discourse, irrespective of the specific semiotic modality or combination of modalities, is a dynamic, time-bound interface between socio-cultural and material processes and constraints in the world and the body-brain processes and functions that afford the production and reception of vocalizations, gestures, and so on. Moreover, the dialogical basis of all forms of meaning-making, including the very earliest infant-parent dyads, suggests that meaning-making is inseparable from the forms of semiosis that are specialized to the inner realm and usually described in terms of a separate language of 'thought' and 'cognition'. As Vygotsky has shown, interaction with others both precedes and guides and shapes the transition to the forms of 'inner' dialogue that we conventionally call 'thought'.

11. Reconnecting the Semiotic Concept of Value to the Body-brain System and to Meaning-making Activity Saussure (e.g. 1993: 357-66) theorized the language system (la langue) as a system of differences (see Thibault 2004b). The value of a given term in the language system is defined in terms of its differential relationships with the other terms with which it contrasts in that system. The language system is a (decontextualized) system of differences, abstracted from actual uses of this system in contexts. Saussure's theory was by and large eo-opted by structuralism. In the structuralist view, there is no real place for time or for change. Structures are unchanging and immanent potentialities that are not linked to real-time activity, or to developmental and evolutionary processes on their timescales. They are seen to pre-exist their manifestations in texts and social practices, rather than being emergent properties of self-organizing processes on various timescales. The alternative view that I put forward here seeks to connect the concept of value with the theory of complex self-organizing dynamic open systems. Time and activity are central to this account. In this account, the language system is not a pre-existing code, but is emergent from the real-time meaning-making activities that individuals engage in.

50

BRAIN, MIND, AND THE SIGNIFYING BODY

On the cultural level, a given language community displays considerable diversity and variability in its patterns of meaning-making both on the individual level as well as on the level of particular social groups and the heteroglossic relations of alliance, cooperation, competition, conflict, co-optation, and so on, among these. This diversity is manifested in the range of social meaning-making practices and their distinctive discourse voices that constitute the system of social heteroglossia of a given community. A language system qua system of differential values is an n-dimensional semantic space (Halliday 1991; 1992a: 30). Such a space is a purely hypothetical region encompassing a very much wider range of possibilities than those that are typically or actually used. The system of social heteroglossia refers to the range of typical discourse voices and their associated evaluative viewpoints, their relations of alliance and conflict with other voices in the same community. Typically, the system of social heteroglossia activates only some of the semantic regions of the n-dimensional semantic state space because only some possible combinations from among the very many theoretically possible ways actually get made and recognized in a given community. However, a language system is always also embedded in a higher-scalar ecosocial semiotic environment, which it in part constitutes. Changes in the ecosocial system - e.g. changes in social institutions, social practices, heteroglossic alignments of discourse voices, matter-energy flows, and so on - can lead to changes in the attractor spaces of the language system of a given community. Similar observations can be made concerning the individual level. In the infant's semiotic development, the primordial richly interconnected network of neurons is a very large state space which gets channelled along more specific pathways and more specific functional differentiations in the course of its development. As the infant engages in meaning-making activity with others, this rich initial vagueness will, in time, shift to functionally more specific semantic regions, depending on the type and the frequency of the various discourse experiences that the infant participates in. Each individual has a unique trajectory through this semiotic state space because individuals can occupy a range of diverse semantic state spaces as well as adopt unique pathways for getting from one to another. By the same token, individuals also converge on similar solutions to the kinds of problems that they are required to resolve in their discursive interactions with others. This convergence can be explained in terms of the operations of constraints working on various levels: (1) the constraints of the language system itself; (2) the constraints of the discourse genre or social-activity type which is perceived to be in operation; (3) the constraints of the particular situation or context as it emerges in real-time activity; (4) the constraints of the specific meanings that participants are negotiating or think they are negotiating; and (5) the constraints of individual experience and one's personal historicalbiographical trajectory. Rather than saying that a given grouping of individuals 'shares' an already pre-existing language system, it is more logical to say that individuals are constrained by factors operating on various interrelated scalar levels of organization - e.g. phylogenetic, ontogenetic, and semiogenetic - such that they converge on solutions which are construed as being the same or similar and, at the same time, there is considerable diversity and variability in the ways in which they may arrive at these solutions.

INTRODUCTION

51

The differences that characterize a given language system are the semiotically salient differences that are recognized and used by the members of a given language community. It will be clear that I do not in any way assume that the system of differences is identical or homogeneous for all the individuals who make use of its resources. It is not therefore a uniformly 'shared' system. As an abstraction, it refers to what some theoretical viewpoint constructs as the regular and typical patterns of language that are recognized and used by a given set of language users, however large or small this set may be. The system of differences refers to the possible kinds of meanings that language users can make in the various types of social contexts and social relationships which language is used to enact and mediate. This system of possible kinds of meanings can then be connected to the various ways in which meanings are actually made, in which contexts, and by which social participants. The system of differences that constitutes a given language system is not valuefree in a very non-trivial sense. The semiotically salient differences that are recognized in a given language community are a system of typological-categorial distinctions whereby the users of the resources of that system can orient to, categorize, evaluate, act in, and interpret the world in and through their participation in discourse. Rather than seeing the brain as a sort of digital computer which operates on and manipulates abstract symbol strings inside individual heads, the notion of value, as discussed here, shows how we can connect biology and society in a conceptually unified way through the notion of activity. Both the primitive, biologically intrinsic, pre-semiotic value biases in the newborn, as discussed by Edelman (1992) and Thelen and Smith (1994), and the full-fledged systems of ecosocial semiotic values characteristic of semiotic systems such as language, gesture, facial expressions, depiction, and so on, afford ways of orienting to, interpreting, and acting in one's material and social environments. They do so in ways which are always dependent on activity in real-time. It is in this way that prior, more primitive value biases afford access to more complex discriminations in and through the time-bound activities that these values motivate infants to engage in. This is where value connects with activity. The activities in which the infant participates enable the child's engagement with other persons, sounds, faces, texts, tools, artefacts, symbols, pictures, and so on. These activities loop from the world to the child's intrinsic body-brain dynamics in ways which shape and reorganize them. There are no symbols, words, signs, texts, and so on, in the individual's head. Rather, symbols, words, signs, and texts are dually semiotic-material artefacts that exist in the external world - the ecosocial environment - by virtue of somatic or extra-somatic processes of production and manipulation. It is their dual status as material and semiotic artefacts which affords their integration both to the here-now scale of body-brain activity and to other processes, activities, happenings, and so on, on other space-time scales, beyond the here-now one of the text as material artefact. In this way, semiotic-material artefacts such as texts are extra-somatic resources that enable the participants in meaning-making activity to co-ordinate and integrate their contributions to that activity. Texts qua semiotic-material artefacts are functional components of a meaning-making (semiogenetic) trajectory that extends from the organism's central nervous system through its bodily (sensori-

52

BRAIN, MIND, AND THE SIGNIFYING BODY

motor) activity and into its ecosocial environment and then loops back again to the organism. Meaning-making is distributed among all the components of this trajectory-in-time. Participants in meaning-making activity must lock into and draw on the resources afforded by texts in order to co-ordinate their activities across a diversity of space-time scales. When Saussure points out, in his famous discussion of linguistic value, that 'the purely conceptual mass of our ideas, the mass separated from the language system represents a sort of unformed cloud' (1993: 362), he is showing how a system of typological-categorial differences - i.e. the terms and their values in a given language system - emerge from and, in turn, reorganize and give more specified meaning to the topological-eontinuous substrate of what he calls 'ideas' (see Thibault 1997a: 164-73). The principle of value therefore has the potential to contextualize the phenomena of human experience in and through the system of semiotically salient differences that characterize a particular language system. However, Saussure did not take the further step of showing how signs contrast with each other in ways which also depend on the discourse contexts in which they function in the making and negotiating of meanings in human life. Saussure's observation is not unlike those made by Edelman and Kinsbourne that the primitive state of the brain in the infant is a richly interconnected network of neurons that differentiates or 'crystallizes' into a more clearly defined set of functional distinctions through processes of activation and entrainment of neural networks into more highly specified functional sub-regions by virtue of the initial value-biases that set these processes in motion in the first place. Edelman showed that the values placed on early infant behaviour lead to some activities being preferred rather than others. It is these biologically intrinsic values which, in the first instance, motivate infants to attend to their surroundings, to cry when some need has to be satisfied, to seek warmth, food, and affection, to orient to and discriminate faces, to lock into and to engage with adult meaning systems, and so on. Thus, babies attend to the faces, the sounds, the movements of adult caretakers, in ways that enable them to integrate their activities with those of more senior others. They do so, in the first instance, on the basis of their sensori-motor capacities for interacting with and integrating with their immediate material environment. The resulting dialogic closure (BratenI992; Thibault 2000a) of the human dyads so formed leads, in time, to the transformation of the infant's body-brain system and the building up of increasingly more specified and differentiated semiotic takes on the world. ]ohnson and Morton (1991) have postulated that the initial value orientation that infants have to faces is based on an initial discrimination between a number of contrasting 'blobs', corresponding to the location of the mouth and eyes (chapter 4, section 12, pp. 209-12; see also Thelen and Smith 1994: 315). This primitive distinction is based, initially, on topological-continuous variation, rather than on the typological-categorial distinctions of the kind that are found in the lexicogrammar and semantics of natural languages. The discriminations that the newborn begins to make are motivated in the first place by a value on the infant's self-organizing activity such that the infant's intrinsic dynamics are biased towards preferring to orient to his or her caretaker's face. This preferential bias occurs in synergy with the dynamics of other interacting subsystems such

INTRODUCTION

53

as, for example, the need - itself a value - to lock into and obtain information from one's caretakers. The initial value bias discriminates between 'orienting to faces' versus 'not orienting to faces'. In time, this leads to the further discrimination between the 'blobs' corresponding to the eyes and the 'blob' corresponding to the mouth, and so on. These early primitive differentiations themselves become the topological substrate for increasingly more complex and highly specified differentiations whereby the faces of others serve not only to index individual identity, but function as a complex semiotic system of facial expressions involving many interacting subsystems, including, for example, the position and movement of the eyebrows, the eyes, the mouth, the lips, the nose, and so on. Each of these subsystems, along with the possible interactions among them, as well as the interactions with other semiotic systems such as language, means that the face is a semiotic-material artefact. It is capable of making a complex set of semiotically salient distinctions that can be interpreted, both in relation to other semiotic modalities and in relation to the overall situation in which they occur, as having particular meanings. Adults smile and pull faces at infants in ways that treat the baby as a full participant in the interactions between them. They do so partly on the basis of the infant's own repertoire of behaviours for eliciting responses from adults. Babies are thus entrained to the idea that faces are an important source of information about the others' meanings, including their intentions and feelings. The complex systems of differentiations that can be made by the different sub-regions of the face, both separately and in relation to each other, are semiotic values - i.e. semiotically salient distinctions - which can be used both to modulate and to orient to activity and to one's interpersonal engagements with others. The initial value bias to orient to faces is biologically driven and is founded on the intrinsic biological dynamics of the newborn. In time, however, the increasingly complex range of discriminations that can be made by the face in partnership with the child's increasing attunement to the potential significance of these in discursive interaction entrains the system to a more highly specified system of differentiations that has social semiotic significance in a particular community. The processes described here for the face are also relevant to the protolinguistic sounds that infants produce in their first nine months or so. In time, the infant's vocalizations are entrained to the articulatory practices that subtend the phonological system of a given language system. At the same time, they learn to integrate the phonological units and structures of the language with its lexicogrammatical and semantic structures in contextually relevant ways. The infant's vocal-tract activity is a complex topological space that is capable of discriminating, initially, a far wider range of phonetic distinctions than those that are recognized by the system of phonological distinctions that are salient in a given speech community. Through processes of mutual activation and entrainment over time, the child's body-brain, and more particularly its vocal-tract activity, is shaped so as to recognize and use a more limited set such that other possible distinctions wane and fall into disuse. The very primitive value biases that function to initiate and guide the infant's earliest activities along some pathways rather than others can therefore be seen

54

BRAIN, MIND, AND THE SIGNIFYING BODY

as the vaguer, less specified precursors of the more highly differentiated and more specified systems of distinctions that are characteristic of full-fledged semiotic systems such as language. Following the logic of specification hierarchy thinking (section 8, pp. 34-9), this does not mean that the earlier, vaguer, more general stages are transcended as later, more specified ones emerge along the individual's ontogenetic trajectory. Instead, the earlier stages are integrated to and reorganized by the emergence of the later stages. The earlier stages can be said to prepare the emergence of the later stages insofar as the earlier stages constitute the substrate from which the later stages arise as newly emergent intermediate levels of organization. Thus, the presence of the more specified later level implicates the presence of the less specified earlier level, but not the reverse (Salthe 1993: 64). The earliest, most primitive value-biases motivate the infant to lock into and engage with the semiotic and material resources that are afforded by both caretakers and the physical milieu such that their body-brains are entrained along particular developmental trajectories. The semiotic principle of value thus has the potential to contextualize and to entrain both the body-brain and the world to its categories in and through the system of semiotically salient differences that characterize a particular language system. The concept of value, when reconnected with activity in the way suggested here, is therefore an important component in an ecosocial semiotic theory of how body-brain systems are contextually integrated with their ecosocial environments.

12. A Thumbnail Sketch of the Book The book is divided into three Parts. Part I consists of the general theoretical overview that was presented above in the present chapter. Part II focuses on the expression stratum - the phonetics and phonology - of spoken language in order to explore the principle of scalar heterogeneity in semiotic systems. Chapter 2 takes as its point of departure Hjelmslev's account of the stratified nature of language as a semiotic system. In this chapter, Hjelmslev's insights concerning the ways in which the expression stratum interfaces with and construes the 'phonetico-physiological sphere of movement' (1961 [1943]: 54) are explored. With reference to Lernke's (2000a) notion of 'scale heterogeneity' or 'scale mixing' in ecosocial semiotic systems, Hjelmslev's seminal insights are theoretically reconstituted in order to shed light on the ways in which the articulatory processes of the vocal tract in vocalization are cross-coupled with higher scalar systems of meaning relations and practices on the ecosocial level. Chapter 3 begins by considering the ways in which action and meaning emerge from the time-dependent cross-eoupling of a diversity of systems on different space-time scales. The Principle of Alternation (Lemke 1999) is introduced and its implications for how the continuous topological variety of the sounds produced at level L-I of the perceptual-motor activities involved in the production and perceptual pick-up of speech sounds is reorganized and interpreted as discrete phonological categories at level L+I, corresponding to the phonological system of a language. In chapter 3, I also explore the iconic,

INTRODUCTION

55

indexical, and symbolic dimensions of both the expression and content strata of language. The cross-scalar nature of both expression and content is discussed in terms of a nested specification hierarchy consisting of iconic, indexical, and symbolic layers of organization. Edelman's (1992) account of the brain as a selective recognition system which recategorizes sensori-motor routines as conceptual ones is related to the cross-coupling of expression and content. In turn, the above considerations are related to the ways in which individuals are integrated to their semiotic trajectories by virtue of the ways in which diverse timescales are integrated across expression and content. The three chapters in Part III address different aspects of how brain processes and functions relate to meaning-making on multiple timescales. In chapter 4, I begin by rejecting the view that consciousness can be explained in terms of lower-scalar neural processes per se. I critically discuss Terrence Deacon (1998 [1997]) on the relationship between subjective experience and brain processes and begin to develop an alternative to the 'theory of mind' that is invoked by Deacon (see also Shanker 1996). I propose that self-organization entails an increase in a system's overall semiotic complexity and specification as a result of the collection of the products produced by its own cascading. This proposal prepares the ground for the central argument of this chapter, viz. consciousness is a structured system of meanings involving all levels of the specification hierarchy of iconic, indexical, and symbolic modalities of semiosis. It is in this way that we organize our experience around a notion of the self along a trajectory. The metafunctional character of semiosis is then related to the ways in which the hemispherical organization of the brain itself suggests ways in which the structure and dynamics of meaning-making are consistent with our neuroanatomical architecture without, however, being reduced to this. A further important development in this chapter concerns the entropic character of meaning. Using Salthe's (1993) theory of infodynamics as my starting point, I suggest that the meaning which is stored along the individual agent's historicalbiographical trajectory increases as the trajectory develops and individuates. The implications of this are further discussed in terms of developmental issues. Chapter 5 takes up the issue of the metafunctional organization of semiosis by suggesting that lexicogrammar is embodied in dynamical attractors or constrained pathways through the multidimensional semantic space that constitutes higher-order consciousness. The question is posed as to how we can talk about the temporal and spatial grounding of consciousness, its richly patterned connectivity that implicates networks of relations across many diverse space-time scales, and the sense of being part of and related to a wider social and perceptual field of relations, which includes others. Some further developmental aspects of this question are discussed in connection with the perceptual-motor phase of categorization in the infant's earliest sensori-motor engagements with the world in the work of Thelen and Smith (1994) and in Halliday's (1975, 1992a, 1993) account of a child's transition from protolanguage to language. These discussions prepare the ground for an extended discussion of the metafunctional shape of consciousness in relation to Darnasio's (1999) theory of the relationship between the experiencing self and the object of consciousness. The dialogical basis of consciousness is also briefly discussed in relation to some connections

56

BRAIN, MIND, AND THE SIGNIFYING BODY

which I postulate between recent work on mirror neurons and interpersonal meaning. The chapter concludes by extending these observations to inner speech and its relationship to higher-order consciousness. Chapter 6 begins by emphasizing the unity and continuity of the physicalbiological and the social-eultural dimensions of our existence. This initial emphasis forms the backdrop for an extended discussion of Flohr's (1991) theory of phenomenal awareness. Flohr is interested in overcoming the prevailing dualism between the physical and phenomenal dimensions of brain states. In my view, Flohr's description of 'activity-dependent self-organization of neural nets' shows that contextualizing relations are relevant on the diverse scales of neural organization that give rise to phenomenal awareness. The principle of me ta-redundancy operates here as well. I then show that the Principle of Alternation provides a further crucial perspective on the brain as meaning-making organ. Thus, the transformation from discrete, local neural firings to global configurations of neural networks on higher scales of brain activity and the transformation of such global configurations to objects of conscious experience within the brain constitute a semiotic transformation, such that conscious experience emerges in the perspective of the self.

Part 11

This page intentionally left blank

2 Sensori-motor Activity, Movement, and Social Meaning-making: Rethinking the Expression Plane of Semiosis I. Some Early Signposts from Saussure and Hjelmslev: The Expression Plane as Embodied Articulatory Movement Saussure's notion of the 'signifier' and Hjelmslev's concept of the 'expression stratum' of semiosis have often been mistakenly interpreted as if they referred to a carrier, vehicle, or medium that is distinct from the level of meaning. Phonological or graphological forms, rather than contributing to the overall process of meaning-making, are simply seen as the means for communicating a meaning which is separate from them. Thus, form and meaning are dichotomized. In this mistaken view, phonological and graphological forms are not considered to play an active and constitutive role in the process of meaning-making. In some accounts, they simply serve as the means for differentiating meaningful distinctions in the morphology and syntax of the language in question; in others, they are no more than the means of transmission of a meaning which is separate from them. In both versions, form and meaning are separated. Furthermore, such accounts, in virtue of a strict dichotomization of phonological form and phonetic substance, fail to account for the ways in which material (e.g. bodily) and semiotic processes and their dynamics necessarily intersect in semiosis. In my view, both Saussure and Hjelmslev developed the beginnings of a very different view, one which has to do with the ways in which material, bodily processes and semiotic processes cross-couple and intersect in the dynamics of social meaning-making. In so doing, the two kinds of processes may be said to mutually determine the overall dynamics of the system. In this perspective, a very different status is given to the expression plane of semiosis. Rather than postulating the separation of 'form' and 'meaning', each on distinct levels, there are constant and simultaneous transactions across all levels and in all directions from sensori-motor activity to perception to actions performed in the environment of the self. We act on and explore the world, we orient to others in and through articulatory movements of our bodies, which bring about specific matter-energy effects and consequences in the world, including our interactive partners. In turn, the vocal-tract gestures that we articulate, the sounds that result from these, and their cross-modal couplings with visual or other events in the environment of the participants to discourse events produce reentrant mappings onto the very bodily gestures that engendered these cross-modal perceptions-cum-actions in the first place.

60

BRAIN, MIND, AND THE SIGNIFYING BODY

Saussure's (1971 [1915]: 156--7;see also Thibault 1997a: 166--71) discussion of the way in which language emerges between the two 'amorphous' or topologicalcontinuous substrates of 'sound' and 'thought' can be seen as an early attempt to explain how language and other semiotic resource systems must reduce and entrain the many degrees of freedom - the vague and indeterminate possibilities - of both the bodily processes and dynamics involved in the articulation and production of speech sounds and the stimulus information which the organism picks up in its environment (Gibson 1986 [1979]: 55-7). Saussure put it like this: The characteristic role of the language system [la langue] vis-a-vis thought is not to create a material phonic means for the expression of ideas, but to serve as the intermediary between thought and sound, so that their union necessarily leads to the reciprocal delimitation of units. (Saussure 1971 [1915]: 156; my translation)

B In the first case, a given language system provides the phonological (or other) resources whereby the initially vague or indeterminate potential of the muscles, joints, and so on involved in vocal-tract or other motor functions are entrained and categorized as co-ordinated and patterned bodily movements for the purposes of dialogic interaction. This is the task of the stratum of the signifier (Saussure) or the expression plane (Hjelmslev) of semiosis. Saussure makes it quite clear that, in his view, the signifier is not simply a material carrier of a more abstract and extracorporeal signified. In the second case, the language system provides the lexicogrammatical and semantic resources in and through which individuals construe the phenomena of experience as a more determinate, though never fixed or unchanging, system of semiotic categories. However, these are not two separate activities. Nor are they on two distinct levels of abstraction. Saussure shows that language form entails a 'reciprocal delimitation of units' in both 'sound' and 'thought.' In acting as the intermediary or the interface between the two, language crosscouples and synchronizes the bodily (articulatory) dynamics of the individual with those of other individuals as well as with their shared Umwelt of perceptual information, tools and other environmental affordances. I am using Cibson's term perceptual information rather than the more physiologically based concept of a perceptual stimulus because the former refers to environmental affordances which act as material causes relative to the environment of the individual. A

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

61

perceptual stimulus, on the other hand, is an efficient cause insofar as the application of energy will stimulate a receptor. However, a perceptual stimulus does not specify any information about the source of the stimulus. As a material cause relevant to the organism's behaviour, perceptual information is a relevant or salient aspect of the environment with which the individual interacts in meaningful ways. In this Gibsonian framework, it is possible to re-interpret Saussure's distinction between 'sound' and 'thought' in the following terms. The semiotic entraining of the body's many degrees of freedom, whereby the indeterminate flux of sound is re-construed as articulate speech, is a way of putting the individual into contact with relevant aspects of its ecosocial environment, including his or her conspecifics. Articulate speech and other semiotically entrained motor functions or gestures of all kinds constitute bodily activity whereby socially shareable ways of interacting and integrating one's bodily activity with others are projected into the environment. At the same time, individuals pick up relevant perceptual information by means of their perceptual systems and then semiotically elaborate or re-construe this in their Innenwelt according to the categories of their stored langue interieure: In the first instance, this information provides indexical information about certain relevant aspects of the environment, as perceived by the individual. However, the emergence of a language system between 'sound' and 'thought' means that the indexical relationship of necessity between environmental events and their perceptual pick-up is severed. That is, their semiotic re-eonstrual in and through the resources of a shared language system is based on the fact that the individual's stored langue interieure provides a means for both producing and interpreting such environmental events as symbolic possibilities, rather than as indexical necessities (Salthe 1993: 176). The existence of this capacity presupposes, of course, the ability to recognize that other organisms also share the same system of symbolic possibilities whereby environmental events are interpreted and acted on as signs, rather than as indices of physical events per se in the here-now relation between the organism and its immediate physical environment. In this way, the many degrees of topological freedom of the phenomena of experience - cf. Saussure's 'thought' - are themselves entrained to the semiotic categories of a given ecosocial system. Thus, the sensori-motor functions of the body must be harnessed and coordinated so as to cross-eouple with and function in relation to the extra-somatic world of perceptual information. By the same token, the semiotic activity of construing the world as meaningful 'content' must be cross-eoupled to and matched up with the dynamics of the agent's bodily activity. If this were not the case, then there would be no way in which body-brains could be co-ordinated and synchronized through jointly made interaction procedures for the purpose of social meaning-making. Furthermore, this co-ordination of body-brains shows that motor activity, perception, and meaning-making are all of a piece - they are all heterogeneous dimensions of the one overall self-organizing system in and through which meanings are made in ecosocial space-time. As Saussure and Hjlemslev understood, a language system entails the progressive differentiation and entraining of both sensori-motor activity and its products ('sound') and the

62

BRAIN, MIND, AND THE SIGNIFYING BODY

phenomena of experience ('thought') to the categories of the expression and content strata of a given language system (see Thibault I998a). It makes little sense, then, to think in terms of the phonological realization per se of a particular linguistic meaning, seen as existing on a different level of conceptual or symbolic abstraction. Instead, there is a rich, on-line connectivity between bodily articulation, the acoustic information which is projected into the environment, and its correlations with objects, events, and so on, in other sensory modalities such as the visual, the spatial, movement, and so on. This rich connectivity suggests that a primary function of the signifier or expression stratum of semiosis is the physical-material integration of multimodal perceptual events into a more global meaning which constrains lower-levels. In other words, the expression stratum both (1) indexes its connections with other sensorimotor and perceptual events in other modalities and (2) affords possibilities of its symbolic construal in virtue of its mediation by, for example, the lexicogrammar and semantics of language. In such a view, we may hypothesize that, for example, audible vocal-tract gestures, facial movements, and manual-brachial gestures are connected by re-entry such that the visual, auditory, and movement systems, while independently mapped in the brain (Edelman 1989: 64-90), are also mutually connected so that higher-order or me ta-level relations among them emerge. These higher-order relations, however, do not simply face one way - to the body of the articulator. They are both proprioceptive and exteroceptive - they also face outwards to the world of the non-self. That is, emergent crossmodal connections among sensori-motor activities are directed to and focused on perceptual events in the purview of the participants in some occasion of interaction. This is where lexicogrammatical mediation comes in: in mediating the cross-modal integration of sensori-motor activity and its indexical connections with specific events in the world, the symbolic possibilities of lexicogrammar massively expand and elaborate in culturally specific and shareable ways the meaning-making potential of sensori-motor classification, and in ways that go beyond the here-and-now of perceptual categorization per se (Edelman 1989: 187). Salthe points out that models of complex systems 'change the rate at which the processes we are modelling occur so as to bring them into line with our own observation rates' (1993: 48). Insofar as language and other semiotic systems model and construe the world, we can say that such systems are smaller-scale models of the higher-scalar ecosocial system that we are embedded in. The models of the world afforded by language and other semiotic modalities belong to our own scale, i.e. the world in which we live and in relation to which we share perspectives with others. Since the reference scale includes our own embodied sensori-motor activities, this suggests, for example, that the phonological entraining of vocal-tract gestures as a semiotically formed expression substance is a means whereby the many degrees of freedom or difference - the vague and indeterminate possibilities - of the bodily processes and dynamics involved in the articulation and production of speech sounds constitute a means of bringing the phenomenon which is modelled into line with our own embodied observation rates. Thus, factors such as intonation, rhythm, tempo, and syllabification on the expression stratum entrain vocal-tract dynamics at the same time as they

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

63

construe the lexicogrammatical units and structures on the higher-level content stratum. It is the latter stratum which, in turn, interfaces with the world so as to construe the phenomena of experience in and through the semantic categories provided by the language system. The content stratum thus construes the phenomena of experience with which it interfaces as meaning. Halliday (I 992a) points out that for meaning to be made there is necessarily a second interface viz. the expression stratum - whereby meaning as construed on the content stratum is transduced into bodily processes of articulation. In Halliday's own words, 'it is the transduction of the phenomenal back into the phenomenal via these two interfaces of content and expression' (1992a: 22). The two 'outer' faces of content substance and expression substance embody what Halliday calls the 'contradiction between the material and the conscious' (1992a: 22). The relationship of realization between the expression and content strata thus shows how our semiotic models of the phenomena of experience are self-referentially linked to the observational rates and dynamics of our own bodies. The two outer faces of this relationship both work to complete or integrate their respective lower-level phenomena. The expression stratum thus works to complete the physiological activity of the vocal tract by entraining and integrating it into the meaningful categories of the phonological system of a given language. Likewise, the content stratum integrates and completes the phenomena of experience by placing them within some system of interpretance such as a given language system so as to give them a meaning. Generally speaking, the stratal organization of language is represented as a series of levels, or strata, which are re cursively related to each other. However, this way of conceptualizing stratification does not tell us how the system of relations between strata is self-reflexively connected to the embodied agents who deploy the system in discursive interaction. In other words, where is the language user situated with respect to the (potentially) infinite regress of levels that the recursive nature of the relations between levels involves? Given that stratification is always relative to some system of interpretance, an agent can be said to constitute an interruption or break in the recursive relations between strata (see Salthe 1993: 51). That is, the agent is located between or interfaces with both the expression and content strata, as shown in Table 2.1. The location of the agent between these two interfaces thus constitutes a break or interruption in the stratified hierarchy of recursive relations. The agent's interfacing with the two strata of expression and content has two main consequences. First, it provides a means whereby the agent is attuned to the meaning-making resources of the higher-scalar ecosocial semiotic system. That is, the agent is sensitive to the contextual constraints which emanate from this level. Secondly, the agent has internal dynamics which buffer the system from disturbances which would otherwise compromise, destroy or overwhelm it. In this sense, an agent is said to be robust (Salthe 1993: 51; Juarrero 1999: 249-50). Thus, the agent maintains its structural integrity and its agentive determinativeness - i.e. its embodied presence and agentive effects - in relation to the system of relations in which it is embedded. This system of relations is itself determined in and through the agent's interactions with other agents who share the same perspectives and viewpoints in some system of interpretance. Agency is

64

BRAIN, MIND, AND THE SIGNIFYING BODY

Table 2.1: The robustness of the agent and its interfacing with the expression and content strata of language

Content Purport

Vague topological-continuous; many degrees of difference

Content Substance (construal of phenomena of experience in 'outer' and 'inner' domains Content Form

SOCIAL CONTEXT OF SITUATION Field

Tenor

Mode

DISCOURSE SEMANTICS metafunctional interface LEXICOGRAMMAR (clause to morpheme)

Interface with the Domain of the Phenomena of Experience

~

SELF AS EMBODIED AGENT with its own internal dynamics

~

Interface with the Domain of the Body-brain Expression Form

Expression Substance

Expression Purport

PHONOLOGY

GRAPHOLOGY

tone group foot syllable phoneme

paragraph orthographic sentence sub-sentence orthographic word grapheme

AUDITORY ACT

VISUAL-GRAPHIC ACT

auditory perception vocal tract articulation

visual perception hand-arm-joint-eye kinaesthesis

Neuroanatomical, neurophysiological and neuropsychological capacities of organism as initiating conditions (material affordances) of semiosis

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

65

progressively revealed during the historical emergence of the individual in terms of the way the agent interrupts and interfaces with both the expression and content strata in semiosis. Agents individuate and have effects in meaningmaking activity by virtue of the ways in which both their bodies and the ecosocial environment are mediated by the expression and content strata. Table 2.1 models the robustness of the agent in relation to the two strata of expression and content. Phonologically entrained and habituated vocal-tract gestures enact specific trajectories in the ecosocial environment of speaker and listener. This is a result of the history of the use of such gestures both on the scale of the individual language user and that of an entire speech community. It is because of this history that there has been reentrant mapping of specific gestures with specific perceptions and categorizations of the world. These gestures are, then, more than just body movements per se. As a result of the history of their use, they create expectations which are based on an understanding of the correlations between these and other events in the world. The continuous experience of using phonologically habituated vocal-tract gestures in relation to some classes of perception and action rather than others leads to the emergence of stable attractors. Thus, the same patterns of vocal-tract activity used by different speakers on many different occasions will lead to the same kinds of consequences in the world approximately the same kind of awareness and categorization of objects, events, and so on; the same kind of interactional routines; the same kinds of integration of cross-modal correlations into our unified understanding of the resulting whole as a socially stabilized meaning. There are no stored internal representations of external events in the world which are then matched with the former and interpreted. Rather, patterns of socially entrained bodily activity are stable attractors for the multimodal integration of events and actions, including internal ones in 'thought', as emergent, context-specific occasions of social meaning-making. This requires us to take into account what Lemke (2000a: 181) has discussed as 'scale heterogeneity or scale-mixing' in dynamic systems. In other words, the very different scales of what Hjelmslev revealingly calls the 'phonetico-physiological sphere of movement' (1961 [1943]: 54; my emphasis) in the vocal tract of the speaker and higher-scalar ecosocial systems of meaning relations and practices are strongly cross-coupled to each other. The cross-coupling of the two cuts across and reveals the inadequacy of those views which artificially separate the biological and social-cultural dimensions of meaning-making activity. Rather, meaning-making activity emerges from the interaction in time of a wide variety of diverse elements on different space-time scales. The cross-coupling of expression and content is what makes this scalar heterogeneity - the continuity of time scales - possible in meaning-making activity. Importantly, Hjelmslev, like Saussure before him, also notes the fact that the expression plane of semiosis functions in ways which are precisely analogous to the content plane. Typically, there has been a division of labour between those sub-disciplines - phonetics and phonology - which concentrate on the expression plane of spoken language and those which focus on lexicogrammatical and semantic relations on the content plane - semantics, grammar. Hjelmslev's approach grasps the necessary

66

BRAIN, MIND, AND THE SIGNIFYING BODY

unity of the two in an overall theory of meaning-making. However, this approach has not been widely assimilated into mainstream thinking about language, which continues to operate the divisions and consequent fragmentations of understanding mentioned above. How do Hjelmslev's proposals help us develop a new way of thinking about the role of the body in social meaning-making? Having defined content-form and its relation to content-substance, Hjelmslev, following in the tracks already laid down by Saussure (see above), then proceeds to investigate the precisely analogous functions which expression-form (e.g. phonology) has in relation to expression-substance (e.g. phonetics), as follows: Just as, for example, the color zone or the morpheme zones are subdivided differently in different languages in that each language has its own number of color words, its own number of numbers, its own number of tenses, etc., so we can also disclose, by subtraction from a comparison oflanguages, zones in the phonetic sphere which are subdivided differently in different languages. We can, for example, think of a phonetico-physiological sphere of movement, which can of course be represented as spatialized in different dimensions, and which can be presented as an unanalyzed but analyzable continuum - for example on the basis ofJespersen's system of 'antalphabetic' formulze. In such an amorphous zone are arbitrarily included in different languages a different number of figurze (phonemes) since the boundaries are laid down in different places within the continuum. An example is the continuum made by the median profile of the roof of the mouth, from the pharynx to the lips. In familiar languages this zone is usually divided into three areas, a back k-area, a middle z-area, and a front ?area. If we consider only the stops, however, Eskimo and Lettish, among others, distinguish two k-areas, whose lines of division do not coincide in the two languages. Eskimo places the boundary between a uvular and a velar area, Lettish between a velar and a velo-palatal area. (Hjelmslev 1961 [1943]: 54-5) It is clear on a close reading that Hjelmslev's 'unanalyzed but analyzable continuum' refers to the 'phonetico-physiological sphere of movement' of the organs of speech in the vocal tract. This 'expression-purport' - cf. Saussure's 'sound' - is defined in terms of 'spatialized' or topological parameters in which difference is perceived in quantitative or analogue terms as, for example, more or less degrees of intensity rather than in terms of discrete, digital categories. Therefore, the epithet 'amorphous' must be taken to refer to the analogue or topological-continuous character of this 'unanalyzed' continuum before the entraining effects of a specific system of typological-eategorical semiotic possibilities. It is the typological-eategorical distinctions of expression-form - e.g. the phonology of a given language - which add new emergent properties to the topological parameters of the phonetico-physiological sphere of movement by 'laying down', as Hjelmslev puts it, 'different places within the continuum' as phonologically salient configurations of vocal-tract movements or gestures. For Hjelmslev, the language system does not exist sui generis. Rather, it exists insofar as it is a resource for acting on and intervening in physical-material and social

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

67

reality. Ordinarily, this is taken to refer to the world which we perceive through our senses and which is selectively transduced by the symbolic categories of content-form as a content-substance. However, as Hjelmslev shows, this affirmation applies equally to both the content and the expression planes of linguistic organization. Hjelmslev explains this last, most fundamental point as follows: That a sign is a sign for something means that the content-form of a sign can subsume that something as a content-substance. Just as we felt before a need to use the word purport, not simply of the content, but also of the expression, so here again, in the interest of clarity, despite the time-honoured concepts whose shortcomings now become increasingly evident, we feel a desire to invert the sign-orientation: actually we should be able to say with precisely the same right that a sign is a sign for an expression-substance. The sound sequence [nu] itself, as a unique phenomenon, pronounced hic et nunc, is an entity of expression-substance which, by virtue of the sign and only by virtue thereof, is ordered to an expression-form and classified under it together with various other entities of expression-substance (other possible pronunciations, by other persons or on other occasions, of the same sign). The sign is, then - paradoxical as it may seem - a sign for a contentsubstance and a sign for an expression-substance. It is in this sense that the sign can be said to be a sign for something. On the other hand, we see no justification for calling the sign a sign merely for the content-substance, or (what nobody has thought of, to be sure) merely for the expression-substance. The sign is a two-sided entity, with a Janus-like perspective in two directions, and with effect in two respects: 'outwards' toward the expression-substance and 'inwards' toward the content-substance. (Hjelmslev 1961 [1943]: 57-8) If content-form functions semiotically to construe the phenomena of the world - real, imagined, hypothetical, it does not matter - as instantiations of the categories which belong to content-form, i.e. as a content-substance, then, as Hjelmslev argues, exactly the same reasoning also applies to the phoneticophysiological movements of the vocal tract. These, too, are entrained, channelled, and categorized as a language-specific expression-substance according to the typological-eategorical parameters of some expression-form (phonology, graphology, ete.). In the final analysis, the language system is not detached from the physical-material reality to which it 'refers' on both sides of its internal organization. Rather, it is, through a long process of eo-evolution, derived from it. This is so both from the point of view of the way in which expression-form interfaces with the kinetic processes of the body and its technological extensions, and from the point of view of the way in which contentform interfaces with the perceived phenomena of the world we live in. Rather than say, for example, that the kinetic or articulatory movements of the vocal tract and the system of phonological categories are two distinct scalar levels, we can say that there is a high degree of eo-evolved heterogeneity or intersection between them. Bodily processes are mediated and entrained by the possibilities afforded by the system itself. In other words, bodily processes are shaped by and adapted to a given ecosocial system both systemically and instantially.

68

BRAIN, MIND, AND THE SIGNIFYING BODY

2. Energy Exchange and the Complementarity of Interacting Body-brains The relationship between vocal-tract gestures and the phonology of a given language is not a private or individual matter which refers uniquely to the body of the single speaker. Instead, it is a collective and ecosocial one. The emergence of a semiotically formed expression-substance means that the gestural potential of the body becomes standardized and, hence, reproducible across many different occasions and social agents. It also means that possible bodily gestures may be hypothesized and their correspondence with actual bodily states may then be corroborated. From the evolutionary perspective, the first possibility arises as a result of the second: the standardization of bodily movements and gestures occurred on the basis of a perspective which goes beyond the individual's body per se. Thus, the emergence of a socially shared expressionsubstance is the semiotic-material resource which (1) integrates a multiplicity of individual bodies into the social body of shared bodily dispositions, movements, etc. by construing and entraining these articulatory movements in standardized ways; and (2) affords the possibility of dialogic interaction among them. These two possibilities arise from the fact that semiotically formed expression-substance emerges from and is tied to the body of the user at the same time as it is systemically harnessed and channelled so as to co-ordinate intentionally directed dialogic interaction with others. However, this intentionality of the 'individual' body does not in itself constitute the meaning-making act. The intentionality of the meaning-making body expresses the selective focusing of the system's resources on the specific act of meaning-making; yet, the systemic possibilities for meaning-making exist independently of the specific act. That is, at high-scalar levels of ecosocial organization at the same time that they speak through the individual act and give form to it. This 'giving form' to bodily kinetics as semiotically formed expressionsubstance is always intersubjective rather than subjective in character. It is always formed with a view to (1) mutually synchronizing and co-ordinating bodies for the purposes of dialogic interaction; and (2) selectively fixing the attention of these body-brains on some aspect of the world which is semiotically construed and exchanged in the act of meaning-making. It follows that gestural activity always 'faces two ways': (l) to the body of the articulator insofar as it is the articulation of subjective, embodied feeling states, and so on; and (2) to the body of the interlocutor - the other - insofar as the gesturer's own body is constituted as an 'object' which affords possible dialogic interaction with and for the other. This 'facing two ways' of human gestural activity rests on a very general relation of complementarity between the parties to the interaction. That is, the reciprocal and dialogic orientation of the two interactants is based, in the first instance, on the fact that bodily movements release energy into the environment of the organism. This entails a change in both the gesturer and a corresponding, though by no means identical, change in the interlocutor. In this way, the participants to some meaning-making act impact upon and bring about change in each other. The release of energy by one or the other of participants to the interaction is mediated by the constraining effects of a system of interpretance which connects the participants both to each other as well as to the world which they

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

69

share, i.e. to relevant aspects of their environment - their Umwelten - and their selective internalizations of this as their respective Innenwelten. In such a view, the individual's lang;ue interieure, as Saussure called it, is rightfully restored as constituting possibilities for action and interaction with others relative to a shared ecosocial environment. Thus, signs are assembled in the Innenwelt from the resources of one's stored langue interieure and then projected into the environment as possibilities for social interaction with others. The reciprocal and dialogic orientation of the participants is, then, subordinated to the forming of an emergent, higher-order metastable complex such as, at the very least, an interactive dyad. This occurs on the basis of standardized and reproducible patterns of bodily movement whereby the reciprocal orientation of the participants emerges and is distinguishable from the analog flux of Saussure's presemiotic 'amorphous nebula' (1971 [1915]: 155-6). That is, semiotically entrained body movement transforms the coming together of body-brains in interaction into a process of the selective and adaptive modification of the one in relation to the other. In the midst of the indeterminate - indifferent? - flux of things in general, the body-brains reciprocally engaged in meaning-making selectively orient to each other for the purposes of enacting their jointly made meaning-making activity. Bodily movement is, then, at the basis of the matterenergy exchanges which make semiosis possible. Insofar as such movement conforms to a semiotically formed 'expression-substance', as Hjelmslev would put it, this means that the semiotically formed and entrained movements of participant A are perceived as being complementary to those of participant B. The newly emergent semiotic properties consequently alter and entrain the prior topological dynamics of the interacting biological organisms along socially organized lines.

3. The Stratified Model of Semiosis: The Problem of Conceptual Abstractness and Scalar Homogeneity Typically, the stratum of expression-form is considered to be 'below' that of content-form. In other words, lexicogrammar (morphosyntax) and semantics are assumed to be strata which are 'above' that of the 'lower' level of expressionform (phonology, graphology, etc.), as shown in Table 2.2. According to proponents of the stratified view of semiosis, each stratum has minimal units which are specific to that stratum. These combine to form larger Table 2.2: Stratified model of lang;uage, showing vertical hierarchy of different levels of abstraction Semantics

CONTENT FORM Lexicogrammar Phonology; graphology

EXPRESSION FORM

70

BRAIN, MIND, AND THE SIGNIFYING BODY

structures which are specific to that stratum. The minimal unit of phonology is the phoneme; that of lexicogrammar the morpheme; and the sememe in the case of the semantic stratum. The relationship between strata is seen as a two-way realizatory one. This means, for example, that the sequence of phonemes /k<et/ realizes the lexicogrammatical word 'cat', which in turn realizes the semantic unit [THING; ANIMATE; FELINE]. This encoding perspective may be reversed in the sense that the semantic unit [THING; ANIMATE; FELINE] is realized by the lexicogrammatical unit 'cat', which is, in its turn, realized by the sequence of phonemes /k<et/. That is, the relationship of realization is a two-way and symmetrical one at all levels. The notion of stratification designates a system of nested levels (strata) that continue in both directions from some focal level. The units and the relations these enter into at any given level are parts which are nested in larger wholes which are usually described in terms of a hierarchy of nested constituent units. Further, the relations across levels, or strata, are seen as indirect in the sense that the relations on anyone level do not directly affect those on other levels. For example, there is not for the most part a direct relationship between phonology and lexicogrammar. In this way, the integrity and the scalar homogeneity of the units and their relations on any given scalar level are preserved in the prevailing structuralist reading of the ways in which the various strata are related to each other (SaIthe 1993: 45; Lemke 2000a). In my view, the structuralist attribution of scalar homogeneity to what are, in actual fact, radically scalar heterogeneous relations among the diverse levels of relations involved can be explained as follows. Typically, the stratified character of linguistic semiosis is treated as a formal system in its own right, rather than being seen as the means whereby body-brain and ecosocial system are crosscoupled in ways which enable the continuity and the seamlessness of the diverse timescales that are implicated in the concept of scalar heterogeneity. In such a view, the tendency is to describe the 'lower' stratum of the expression plane as a medium for the 'expression' or the 'encoding' of a meaning or content on the higher strata of lexicogrammar and semantics. In part, this arises from the tendency to describe the relations among the various strata as a series of vertically organized levels ranging from 'higher' to 'lower' or from more abstract to less abstract. For example, systemic-functional linguistic theory distinguishes three symbolic orders of abstraction - viz. in descending order of abstraction, semantics, lexicogrammar, and phonology/graphology - in this sense with respect to language (see for example Davidse 1992: 108 for further discussion of the systemic-functional perspective; see also Whitson 1997, esp. 125-8 for a critique of' conceptual abstractness'). Thus, semantics is realized by lexicogrammar and lexicogrammar is realized by phonology/graphology. In this view, phonology and graphology are seen as a resource for realizing more abstract lexicogrammatical forms as spoken sounds or written notation. The problem with this view is that the more abstract lexicogrammar and semantics are still seen as transcendent with respect to the embodied 'lower'-level activities of vocal-tract or other articulatory movement. Yet this view does not, properly speaking, show how the expression plane of phonology or graphology also looks two ways. It does not tell us how phonology, say, specifies and construes what Hjelmslev had called the 'phonetico-physiological movements' of the speaker's

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

71

vocal-tract gestures. What, then, is the relationship of phonology to bodily dynamics? I shall now discuss this question in the following paragraph. As we shall see, the alternative allows us to see that meaning-making emerges directly from embodied activities. It is not a second-order happenstance or epiphenomenon. If, on the other hand, the expression plane is related directly to bodily movement and activity, it is possible to conceive of this relationship in a quite different way. In this alternative view, A's vocal-tract or other gestural activity is sensori-motor activity which is meaningful for B in the sense that it brings about a complementary change in B. That is, A's body movements constitute a specific material action which intervenes in and exchanges energy with its environment, of which A and B are constituent parts (chapter 1, section 8, pp. 34-9). Further, this bodily action and its corresponding exchange of energy are in some sense organized and oriented by A's relationship to B (i.e., the non-self, the environment, the other). This means that patterns of vocal-tract and other gestural activity qua expression plane depend on their context-specific, moment-tomoment mappings to the external world. It is in the first instance this exchange of energy which establishes the possibility of a reciprocal, dialogical contact between A and B. Moreover, this energy exchange is itself socially organized and directed in ways which allow participants to access and to orient to specific socially defined situations. In this view, form is not something which is separate from matter. Instead, semiotic forms supervene in and organize physical-material, including bodily, states and dynamics for the purposes of social meaning-making. When we say that the sensori-motor movements and actions of A and B are organized in relation to each other, this means that these bodily movements are structured and formed in relation to each other, though always in virtue of some higherorder system ofinterpretance (cf. Peircean thirdness), C. It is C which provides A and B with a shared system of resources for jointly contextualized acts of meaning-making. Thus, form is not something which carries a particular content. I would say, instead, that the recognition of form arises as a consequence of specific matter-energy exchanges between A and B. This means that form is itself a material state as it is interpreted by another material state in relation to some higher-order principle, C, which links A and B to each other. Thus, meaning-making is the process whereby A and B, through their eodeployment of material and semiotic resources, texts, environmental affordances, and so on, act on and bring about socially relevant change both in each other and in their Umwelt. Vocal-tract and other forms of gestural activity are a means of acting on and exploring the Umwelt, including the other persons with whom one interacts. These embodied and socially entrained patterns of movement bring about changes in the world of the non-self (the other) that we can perceive. It is our perception of the consequences of these material transactions with the Umwelt and the reentrant mapping (Edelman 1989) of these perceptions onto the material activity itself which allow semiotic values to be assigned to classes of patterned bodily movement. It is important to emphasize here that such change may refer to the reorganization of neural maps in the brain, the selective

72

BRAIN, MIND, AND THE SIGNIFYING BODY

attending to some object, event, etc. in the material purview of A and B, making some perceptual experience or categorization semiotically salient, a change of embodied feeling state, sensori-motor activity, and so on. This does not means that A and B change in identical ways. The point is that meaning-making is not localizeable in the textual records and products of such exchanges, but must be explained in terms of the dynamics of what A and B do to themselves and to each other in and through such externalized resources. Texts per se are not the same as meaning-making. I would say, instead, that they are the socially organized products and records of our meaning-making activity (Lemke 1984a: 78-80; Thibault 1991a: chapters 2-4, 1994).

4. Blackboxing the Sensori-motor Dimension: Language Seen as Modalityindependent Centralized Processing Mechanism Many linguists' conceptions of language rarely make contact with the neurophysiological, sensori-motor and actional dimension of language as eoarticulated bodily movement or gesture in ecosocial space-time. Many accounts of language in linguists, philosophy, and psychology are based on the notion that language primarily functions to alter the thinking or the consciousness per se of the other. There is correspondingly less attention given to the ways in which language functions dialogically to organize and co-ordinate the sensori-motor and feeling states of the bodies of those who engage in social semiosis. Typically, the sensori-motor dimension has been seen in a somewhat restricted way as the province of the usually highly technical subdisciplines of phonetics and phonology. Over the past decade, there have been interesting developments which have seen a new convergence of the insights of the two subdisciplines. Nevertheless, linguists have been slow to rethink the question as to the ways in which meanings map onto the sensori-motor and actional dimensions of bodily movement or articulation (see, however, Armstrong et al. 1995). In part, this is due to the tendency to view language in mechanistic and formal terms as a bottom-up assemblage of parts into larger wholes. That is, language is viewed as a kind of machine in which the parts are treated as regularities which have to be explained in relation to deterministic rules, seen as efficient and/or formal causes. This sort of reductionist analysis leads to black boxes in which language 'behaviour', motor activity, and so on, are explained in terms of central programs in which either (1) all the spatio-temporal details of articulation are already contained in the program or (2) there is a referent signal or goal-state in terms of which the current behaviour of the system is compared and appropriate instructions are issued. The first of these corresponds to the AI model of computer programming; the second to the cybernetic model. The search for black boxes leads to the postulation of internal mechanisms per se for the generation of linguistic behaviour or for the vocal-tract activity which produces articulate speech sounds. In other words, the sole burden of this causal-explanatory framework is placed on lower-scalar mechanisms which are seen as internal to the mind or the brain of the individual organism. There is little or no accounting for the higher-scalar

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

73

levels of, for example, the ecosocial system in which such behaviour takes place and in relation to which it has the meanings it does. This view may also lead to the conclusion that the 'formal' structure of a language is not required to change 'when "translated" from one modality to another', as Marshall (1980: 277) puts it. In Marshall's view, syntax is formal and autonomous in accordance with the Chomskyan tradition. Thus, Marshall argues that the '(peripheral) medium imposes no constraints upon the (central) syntactic structure of messages' (1980: 276). The latter is autonomous with respect to the former and is in no way shaped by it. In other words, sound and visual images are the physical medium of language, but that understanding language has little or nothing to do with understanding the physical characteristics of the peripheral activities which realized it. Marshall's claims are premised on the view that there is a language faculty which is constrained by universal principles and parameters which are independent of sensori-motor modality of execution and reception. Even allowing for the observed fact that language and gesture are co-ordinated when people interact with each other, Marshall concludes that this is due to 'some central mechanism that keeps mouth and body working together to the same communicative ends' (1980: 284). Marshall's argument is a species of downwards reductionism to lower-scalar causal mechanisms which are blackboxed and located in the mind or brain of the individual. One problem with such a view is that it fails to explain that modalityspecific interactional dynamics are shaped by the activity as it unfolds in ecosocial space-time and that particular semiotic modalities have eo-evolved through their joint use in the making and interpreting of particular social occasions of meaning-making by embodied social agents. In effect, Marshall's argument is representative of that species of view according to which that which needs to be explained in socio-cultural activity is located at the level of the individual biological organism. Thus, meaning-making activity is a mere secondary happenstance - cf. performance - which does not significantly impinge on the structure and functioning of semiotic forms. In other words, there are no significant constraints from higher-scalar levels than that of the individual body-brain complex. There is nothing which might suggest how ecosocial factors have contributed to the emergence of meaning-making activity and its structure and function as an intermediate scalar level which links the individual body-brain to the ecosocial environment of which the individual is a constituent and functioning part. The notion of a central processing and governing mechanism for the co-ordination of 'talk' and 'gesture' assumes a Cartesian-Euclidean-Newtonian model of the world. That is, a unique and absolute space-time in an objective world which is represented in the brain independently of human action and movement. Yet, sensori-motor processes are oriented to action in task-specific ways. The combining of various sensori-motor modalities of action is not controlled by a single principle of the Cartesian-Euclidean-Newtonian type. Instead, diverse sensori-motor modalities may be variably cross-coupled in the production of a movement which is directed towards a specific goal. The various sensori-motor subsystems and their neural substrates are local with respect to their integration into some more global corporeal schema. Rather than universal and absolute

74

BRAIN, MIND, AND THE SIGNIFYING BODY

frames of reference which are independent of bodily experience, this suggests that these subsystems may be differentially integrated according to such factors as the dynamics of the specific occasion of meaning-making and the semiotic modalities used, the memories of previous interactions which are accumulated in the individual body in the unfolding of a specific historical-biographical trajectory, and socio-cultural habitus and dispositions resulting from class, ethnic, gender, generational, and other factors. In my view, the notion of a central controlling mechanism simply serves as an alibi for not explaining the real complexity of the body-brain complex whenever we engage in social meaning-making with others. Rather than the static and reductive model of a central controlling mechanism, we shall need to explore the complex, adaptive, and open nature of the relations between sensori-motor modalities, psychological intentions, and social meanings. We shall need to focus on the dynamic and changing relations between the body-brain and its ecosocial environments in the processes of meaning-making. As a first step, let us reflect on the distinction that Marshall makes between 'talk' and 'gesture', as discussed above. This distinction results from the attempt to understand higher cognitive functions by privileging an autonomous language faculty as the seat of human reasoning. It has its origins in the Cartesian view of language as a uniquely human faculty of reasoning which separates us from the other animals (see Shanker 1998 for an impressive critique). In recent decades, this has lead to a view of the brain as a central processor and governing principle. The brain so conceived is based on the computation of symbol strings in which cognition is seen both as transcending its neuroanatomical substrate and as being the sole property of the individual. The resulting separation of mind and brain and of individual from ecosocial environment serves to recapitulate the Cartesian dualistic separation of mind and body: I think therefore I float free of my material body! What Marshall's distinction between 'talk' and 'gesture' fails to heed is that both are sensori-motor activities of the body. Both are based on the command and control of bodily movements and the inertia and other forces which are opposed to these in the media (e.g. air, water) and surfaces (the ground) through and/or over which we move (Berthoz 1997: 9). The degrees of freedom of the two movements are reciprocally constrained together over the entire bodybrain complex acting in its ecosocial context. In other words, the body-brain complex encounters 'friction' or resistance in the world with which it interacts (see Volosinov 1983: 116; Salthe 1993: 97-9; Thibault 1995: 81-3 for further discussion of the notion of semiotic friction). This friction or resistance, in order to be overcome, or otherwise productively harnessed, requires that work physical and/or semiotic - be done. Thus, talk and gesture are not independent faculties or bodily actions which are co-ordinated by some central mechanism. It is not the case that the person talks and gestures. Instead, he or she engages in a single overall activity of the body-brain-in-its-ecosocial-environment. This means that the body-brain complex reconstructs and anticipates this brain-bodyenvironment-movement relation as a single overall action through the harnessing and co-ordination of many kinaesthetic receptors (Berthoz 1997: 11). These derive simultaneously from the sensory receptors associated with vocaltract activity and hand-arm-joint-body kinaesthesis.

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

75

The coherence I have just referred to derives from the embodied sense of congruence between the various sources of sensori-motor information and the action schema which are stored in memory, and which enable the body-brain to predict the consequences of its actions in relation to the environment. Moreover, this sense of congruence is not simply a moment-to-moment feeling state, but is constituted by the history of the body's transactions with its environment both on the scale of the historical-biographical time of the individual and the yet more extended time-space scale of the accumulated history of such interactions in the life of a given social body. The many degrees of freedom of movement of the body's sensori-motor systems are regulated and constrained so as to produce relatively few degrees of freedom in the form of co-ordinated actions or sequences of actions. Such actions are always in response to friction or resistance - the secondness - that are incurred whenever the organism is required to focus and harness its available energy in response to some environmental event which offers resistance to the organism's completion of some goal or purpose (Salthe 1993: 99). Thus, gestures harness available energy in some situation by means of the regulation of degrees of freedom of movement of the skeletomuscular systems in the service of some intention, need or purpose. All movement is, then, oriented to some generalized principle of otherness - cf. the non-self - which is the purposeful outcome of the body work - semiotic and material - that is carried out.

5. The Intentional Character of 'Inner' and 'Outer' Sensori-motor Activity: Towards a Unified Account The fact that internalized motor images of movement may be activated irrespective of whether the movement is physically executed or not shows that the brain can simulate and, hence, predict a movement without actually performing it (Berthoz 1997: 36). It is the motor dimension of linguistic activity which acts on and produces the peripheral (e.g. auditory and articulatory) sensations. Further, this motor dimension is intentionally directed or focused. This means that language activity is not primarily or only a matter of the transduction of peripheral perceptual and articulatory events by the brain. Rather, internalized motor routines act on and influence auditory and perceptual sensations. I think this is what Langacker means when he argues that both the auditory and articulatory facets of speech sounds are 'conceptual': Even the articulatory facets of speech sounds are properly regarded as conceptual, in the broad sense in which I understand this term. Consider the segment [i]. From the perceptual standpoint, speakers can deal with this sound in either of two ways: they can actually hear the sound as a stimulusdriven perceptual event, or they can simply imagine hearing it, i.e. they can activate an auditory image of it (as in silent verbal thought). Moreover, the auditory image is plausibly taken as primary, in the sense that it is used to categorize acoustic input as an instance of this particular sound. Exactly analogous observations can be made about the articulatory representation of

76

BRAIN, MIND, AND THE SIGNIFYING BODY [i]. A speaker can actually implement the articulatory routine and produce the sound, or he can simply imagine implementing it, i.e. he can mentally run through the motor routine without this mental activity being translated into muscular gestures. Once again the cognitive representation is primary, in the sense that it directs the motor sequence but can also occur autonomously. (LangackerI987: 78-9)

This position reverses the view according to which our perception of sensory stimuli is primary for the organization of action. Instead, it recognizes the very central role of internalized action schemata in the organization of perception. From this point of view, internalized schemata for movement and language - cf. Saussure's langue interieure - are but two specific cases of the notion of action schema. Auditory, articulatory and other sensory images are neural events which may occur without the sensory stimulation of a receptor organ. When this is so, they are no less instantial to their schemata than are those which are evoked by the peripheral stimulation of receptor organs. The same basic principle also applies to articulation. We can for this reason speak, as does Langacker, of mentally 'directing and executing the act of throwing a ball or articulating a certain speech sound' (1987: 112) without the execution of a motor response such that the internalized motor image of these events is not translated into muscular (gestural) activity. Even in the case of inner speech, there is internally simulated cross-modal linking of the movement of the vocal organs with their linguistic meaning as well as between the external actions of the agent and objects and events, and so on that are perceived by visual, haptic, auditory or other means in the stimulus array. In my view, these observations show that gestural activity of all kinds cannot be reductively considered as mere responses to external stimulations. Instead, they are internalized action schemata which are a function of the goals and plans of the agent who actively engages with his/her environment. Rather than responding to environmental stimuli, (internalized) gestural (movement) schemata - cf. auditory, motor, and other sensory images - are a means of actively constituting and orienting to the ecosocial space-time of human interaction. This space-time is actively constituted by the gestural and other bodily movements of social agents as a function of their own projects. The above observations call into question the distinction between the sensory and the motor: an internalized sensory - e.g. auditory or motor - image may be used to elicit either a peripheral response or internal neural activity which is not peripherally connected. The same neural structures are used for 'imagined' gestures as well as for peripherally executed ones. In this way, sensory images show how the brain has a fundamentally predictive function. Sensory images show how the brain takes up and enacts orientations in the world even before the body carries out the designated movement. Mental imagery is implicated in the possibility of not executing a certain gesture. In this way, it provides schemata for orienting the body in the world and for the internal simulation of movements, bodily orientations as a means of modulating bodily movements in response to social meanings. However, the notion of sensory imagery does not presuppose a dichotomy between 'autonomous' and 'peripheral' or between

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

77

'imaginary' and 'executed' movement. The one always implicates the other; execution implies internal simulation (Berthoz 1997: 232). To perceive one's body in the world is to imagine the actions which implicate its use.

6. The Symbolic Possibilities of Bodily Movement Salthe (1993: 97) discusses how the secondness of friction, from the point of view of the production of a specific phenomenon - thought, in his example - may be the energy required 'just to keep the neurons alive, because that is only indirectly relevant (because at a lower-scalar level) to the product of interest (here, ideas).' Friction requires the productive focusing of energy to obtain the right relations between means and end. By the same token, the energy consumed is greater than that which is productively focused on for the attainment of the desired goal. The regulation of degrees of movement in the production of vocal and other gestural activity means that only some of the energy consumed by the brain-body is construed as semiotically salient activity relative to an observer. Such an observer, from our social semiotic perspective, is physically, biologically and semiotically constrained. It is in this way that we can distinguish the meaningful from the meaningless in the body's articulation of a particular (semiotic) project. Thus, some movements (e.g. vocal-tract gestures) are more strongly cross-coupled to the symbolic possibilities of lexicogrammar whereas others are more weakly cross-eoupled. Relative to an observer perspective, the expression plane construes, entrains, and motivates the neurophysiology of body movement as symbolic possibilities for social meaning-making without, however, transcending the materiality of the body. It is important to insist on the symbolic possibilities of bodily movements. Symbolic meaning is not inherent in the movement or gesture per se. Rather, social agents harness the meaning-making possibilities of the body in the service of their own projects. Yet, the body is not a passive medium through which meanings are transmitted to others. The body does not only transmit a force or energy to another body, though it also does this (section 7, pp. 78--81). This would reduce meaning-making to indexical necessity. Symbolic possibility means that others take up and transform the articulatory labour of my body and rework it according to their own projects. In the process, the symbolic possibilities of bodies are continually modified by others who take up and adapt these possibilities for their own purposes. Somatic resources for meaning-making, as distinct from extrasomatic resources, maximally foreground difference, negotiation, and instability (Iedema 1997: 11, 108ff., 222). They are the raw materials out of which social agents fashion the techniques whereby bodies are harnessed and entrained for the purpose of creating and perhaps sustaining associations with other bodies. In this way, as Latour has shown, the social body is a contingent and constantly remade result of the harnessing of such somatic resources for the forming of associations with others (1986: 277). Such associations may be as fleeting as a single dyadic exchange between two individuals or as seemingly enduring as the millions of associations which sustain the social body itself on much larger space-time scales.

78

BRAIN, MIND, AND THE SIGNIFYING BODY

The semiotic phenomenon of language is not reducible to some privileged bio-physical or mental parameter of explanation. This view fails to explain that human language is an ecosocial semiotic phenomenon which transcends any modality-specific physical manifestation such as speech sounds, written notation, Braille, or manual-brachial signing. By the same token, I pointed out in section 4 above, pp. 72-5, this does not mean that these specific modalities reduce to a question of the physical medium of language without also contributing to the architecture and functioning of language. At the same time, language is not reducible to physical modality per se because any such reduction would fail to account for the higher levels of linguistic organization, viz. its lexicogrammar, its semantics, and its discourse-level principles of organization. As Merleau-Ponty points out, without the (human) intention of 'orienting oneself in relation to the possible, to the mediate, and not in relation to a limited milieu; ... ' (1983 [1942]: 176), there would be no phenomenon oflanguage. There can be no reduction to the bio-physical principles of vocal-tract or other articulation without losing sight of the factors addressed by Merleau-Ponty. Furthermore, the fact of the diverse, modality-specific physical manifestations of language means that language can be reduced to no single physical principle of organization. Nor, on the other hand, can language be reduced to some disembodied, modality-independent central controlling mechanism. This latter view assumes that language is somehow above embodied human experience and action in the world. Instead, I posit that linguistic and other meaning-making activity cannot be seen as unified either by some mono-modal principle - e.g. spoken vs. written, visual vs. verbal, etc. - or by domain-specific controlling mechanisms. Rather, it emerges from the interaction of heterogeneous elements in the real-time of human activity. That is, there is no single, homogeneous or unified thing called linguistic, visual or any other form of semiosis. As a form of human intentional activity, language makes both the reality we live in and itself in and through the dynamic and interacting interplay of many factors and resources - perceptual, actional, material, semiotic. Without this form of intentional action, there would be no language. In this perspective, language functions as a higher-order constraint or boundary condition which allows symmetry-breaking relative to an observer perspective to occur. The mediating and categorical functions of language discussed by Merleau-Ponty integrate the environment into the Innenwelt of the organism.

7. Articulatory Movement Seen as Actional Semiotic, Not Physical Behaviour What if both language and movement may be viewed as having both an articulatory and a semantic stratum of organization, i.e. an expression and a content stratum? Typically, language and movement are not seen as being commensurate in this way. Yet movement, like language, may also be treated as involving a realizatory relationship between the semantic and bio-kinematic strata or levels of organization. Until recently, the theory of movement has remained bereft of a grammatical and semantic description. Recently, Radan Martinec (1998,2000)

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

79

has re-explored this problem using the analytical tools and insights of systemicfunctional linguistics. This lifts movement out of the realm of mere physical behaviour and shows that the formal organization of movement is, grammatically and semantically, probably just as rich and complex as that found in language. For example, Martinec shows that movement has a rich experiential semantics and grammar describable in terms of configurations of a semantic Process, the Participant(s) in the process, and the attendant Circumstance(s). In this way, movement is an actional semiotic rather than physical behaviour per se. Like the transitivity system of language, movement is revealed to be describable as a system of semiotically salient differences in which categories of movement are distinguished as the various act-types, event-types, state-types which operate in various contexts of human action. In other words, movement, like language, operates in specific contexts for the fulfilment of specific individual and social purposes and intentions. Human movement is dually a semiotic-discursive and physical-material actional resource for organizing and carrying out our responses to specific problems and situations in the human Umwelt. Further, movement as actional semiotic is functionally organized in ways which relate it to its environments. The cross-coupling of movement and environment mutually constrain each other. Seen 'from below', the spatio-temporal articulation of motor activities on the expression plane is not controlled by a single central governing or causal principle. Articulation is not homogeneous in this sense. Instead, articulation is heterogeneous: the bio-kinematic activities which realize a given action are distributed over many interacting subsystems. This is true of both movementas-action and movement-as-vocalization or vocal-tract activity in speaking. The principle of distributed (not centralized) control means that many different degrees of freedom in Bernstein's (1967) sense may be harnessed and eodeployed to achieve a common, overall purpose. By the same token, the same degree of freedom may be used on different occasions and in different ways to achieve different, local purposes or intentions. There is, in other words, a synergistic entraining of many different subsystems, seen as independent variables. No single subsystem (e.g. muscle linkage) on a lower-scalar level such as the articulatory or expression level of movement can bring about change on some higher, semantic level of organization. If movement, like language, involves higher-order boundary conditions or constraints of a social semiotic nature, this means that the lower-level articulatory stratum and the subsystems that synergistically operate on this level are amplified in ways that are functional according to the semantics of social interaction rather than to physical or biological principles per se. Different principles of muscle linkage comprise an equivalence class not because they all have the same degree of freedom, but because they realize the same actional semantics. This means that the physical movement potential of the body is constrained by factors which are not uniquely physical and biological (Berthoz 1997: 231-2). To be sure, the actional semiotic of human movement emerges from biological systems just as these in turn emerge from physical ones. Yet, there is also what Salthe (1993: 213) refers to as the

80

BRAIN, MIND, AND THE SIGNIFYING BODY

supervenience of, say, the specifically social semiotic level of human action and intention. There would, in other words, be no significant effects on biomechanical processes deriving from the social semiotic level. This may be explained with reference to a discussion in Armstrong et al. (1995: 223) concerning the origin and evolution of language. These authors divide theories of the origin and evolution of language into two broad categories. In the first category, language is seen as 'evolving slowly out of precursors in the communication systems common to the primates' (Armstrong et at. 1995: 223). The view that language derived from prior, gestural forms of communication in the primates belongs in this category. In the second category, a radical break is postulated between language and other communication systems. This is the neo-Cartesian view of, for example, Chomsky (e.g. 1965, 1976) and followers. Armstrong et al. discuss this phenomenon in terms of historical emergence. This is consistent with the Darwinian perspective that these researchers assume. Alternatively, I shall borrow and adapt for my purposes Salthe's (1993: 214-15) notion of developmental emergence as a means of bridging the gap between higher-order social semiotic boundary conditions and the bio-physical processes these constrain and entrain. The emergence of language and other social semiotic resource systems represents, from the evolutionary point of view, a new level of organization which is intermediate between the body-brain complex of the biological individual and the precursor ecosocial system that this is a part of. It is the expression plane which interfaces between the sensori-motor potential of the body-brain complex and the higher-scalar level of the ecosystem. It is possible to conceive of a proto-semiotic evolutionary phase when the relations between our ancestral precursors were mediated by no more than proximate bodily activities. With the emergence of the intermediate level of social semiosis, the body-brain complex is increasingly regulated by its integration into the higher-scalar level of the social semiotic system. Its capacity to engage with distal environmental events on diverse space-time scales beyond the here-now scale is correspondingly enhanced. The newly emergent expression plane of semiosis integrates with sensori-motor processes of the body-brain complex. In the process, it both channels and directs these in ways which go beyond the requirements of neurophysiological functions per se at the same time as these are connected to a still-higher-scalar level - the content plane - whereby the lower level of bio-kinematics of the body as construed and directed by the expression plane can be interpreted in relation to a still wider discourse of social meanings and practices. This shows that there was no prior existing code or system of signs which constituted the intermediate level of organization between body-brain and environment. The system is itself a historically evolving artefact which is, in time, constructed as signs are made and standardized in and through the microencounters between embodied individuals in specific niches of their potentially expanding Umwelt. The latter is the accumulated result of that which is transformed from pre-semiotic indifference into semiotically salient and ecosocially relevant difference. The emergence of a system of signs is the history of the stabilization of the many signifying acts which in time come to be mediated through the emergent thirdness of the system.

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

81

It does not matter whether sensori-matter activity is simulated or imagined, as in inner speech, or peripherally executed as in audible speech sounds. The same neural structures underpin both forms of sensori-motor activity. Inner speech is still internally simulated motor and hence gestural activity even though this activity is not peripherally executed. Thus, the brain may 'imagine' or simulate inner speech precisely because the motor activity may be inhibited at different levels without, however, suppressing the closed loops in which such activity is simulated (Berthoz 1997: 230). The point is that inner speech is still gestural activity which the 'speaker' performs even though the motor systems responsible for the peripheral execution of the skeletomuscular systems of vocal-tract-lipmouth-face-skin kinaesthesis are inhibited.

8. Inner and Outer Body States and Social Semiosis The issues discussed in the previous section raise questions about the 'etc.' function, the unspoken, the background, and so on, which have to do with all those material and bodily processes that defy categorical specification in a semiotic system of typological-categorial differences per se. As I indicated in chapter 1 (section 5, pp. 23-6), any such limitation of our inquiry to the typological-categorial is a restriction on the notion of semiosis which needs to be lifted so that we can take on board a much wider set of phenomenal states and processes and their cross-couplings with the semiotic. How does one talk about the 'inner' visual experience of reading a poem, for instance? Some hints are provided by the earlier work on synaesthesia, but we can probably go further. The kind of experience I am referring to would be a non-linguistic order of contextualization 'above' the denotative level of the linguistic text - a type of non-linguistic 'connotative' semiotic. Needless to say, more work needs to be done on how this is established. Is the evocation of visual imagery when one reads the poem an attempt to create or invent a multimodal coherence for integrating in (internalized) space-time the experience of reading the text? That is, is it a higher-order ('connotative') contextualization which functions to find a solution to the problems of local coherence posed by the text? In this way, the visual imagery which is internally stimulated and experienced in the silent reading of a poem would be seen as internally stimulated perceptions at very abstract cortical levels whose function is to provide solutions to sensori-motor incongruities and incoherences - the 'inexplicable swarm and equivocal generation of motions in our own brains', as Coleridge (1967: 77) expresses itwhich the text generates. The work of Shepard (1984) is suggestive in this regard. I would say that problems of this kind only foreground what is in fact a more general phenomenon whenever we read a text of any kind. Reading is an activity which requires the space-time integration of the reader with the text. It is an ecosocial activity even though (1) physical movement is slight, being limited, perhaps, to the scanning by the eyes of the written notation on the page and the turning of pages by means of a synergy of hand-arm-jointeye kinaesthesis; (2) it is often 'silent' and contemplative insofar as one does not read aloud either to oneself or to others; and (3) much of the reading activity

82

BRAIN, MIND, AND THE SIGNIFYING BODY

takes place on the basis of internalized sensori-motor activity and resulting imagery. But silent reading is no less a multimodal and embodied ecosystemic activity than is, for example, the activity of talking aloud. From the point of view of internalized sensori-motor activity, we know from modern neurophysiology and neuropsychology that a cooperative synergy of sensori-motor modalities is harnessed for the purposes of the space-time integration of participants in some ecosocial act of meaning-making. These entail a multiplicity of complementary corporeal schemata which may be combined and associated in many contextspecific ways for the purposes of such integration. There are, I suggest, nonlinguistic modalities - visual, for example - of 'inner' semiosis which are relevant here. They are, I think, specialized deployments of the same 'outer' resources we use in talking, depiction, movement, and so on. Again we see the principle, discussed above, that simulated inner movement cross-couples with other perceptual and semiotic modalities and helps to guide the emergence of inner semiotic activity. There are, then, different levels of organization - neural, sensori-motor, biokinematic, expression form - each with its own scale of physical size and rate of change. Each in turn is both regulated by the level(s) above it and at the same time regulating the levels below it. In this perspective, there is no abstract central programme or mechanism which must then be translated into muscular movement, and so on. Instead, the four levels of organization mentioned above are organized into larger-scale 'synergies' (Bernstein 1967) which are mutually regulating at all levels. As Salthe (1993: 48-9) puts it, the higher-scalar levels in part 'complete' or 'integrate' the lower ones by conferring on them a local meaning which they would not have on their own, Thus, neural activity is integrated with sensori-motor activity which is integrated with skeletomuscular kinaesthesis, which is integrated with the semiotic categories (phonological, graphological, etc.) of some expression form. The processes of integration described here reflect the interdependence between the topological-continuous properties of the lower-level bio-physical processes and the typological-categorical properties of the expression plane in relation to these. The latter are based on discrete contrasts. Lemke (2000a: 203) points out that the typologizing properties of semiosis tend to spread or cascade throughout the world, invading the world with semiotic categories, particularly when harnessed to some artefact which enhances their use. However, no such process can ever completely exhaust the meaning-making potential of the topological substrate, i.e. the bio-physical dynamics of the body in human social interaction. As I shall argue below, the metafunctional hypothesis provides a way of talking about the various dimensions along which both the typological-categorical and the topological-continuous are integrated with each other from the point of view of the expression plane. Thus, the bio-physical kinematics of the body are not read or interpreted in exclusively typological-categorial terms. The body-in-itself is noumenal or pre-semiotic until it is construed as meaningful by some interpreter. This means that there is a relationship of complementarity between this bio-physical semiotic potential and the interpreter who reads it. In itself this potential is, so to speak, indifferent and depends on a socio-cultural interpreter

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

83

and system of interpretation which can harness and direct its thermodynamic matter-energy flows in socially relevant ways.

9. The Semiotic Mediation and Entraining of Embodied Bio-kinematic Potential As I said earlier, the expression plane is also integrated with the content plane of semiosis. In the case of language, this means its lexicogrammar and semantics. An expression without a content is a contradiction in terms and no sign-making can take place in the absence of one or the other of these levels of semiosis. The content stratum is a higher-order level which further completes or integrates the expression stratum. The lexicogrammatical mediation and integration of, say, speech sounds means that the bio-kinematic interface is partially integrated with a system of interpretance which links embodied socially shareable meanings to the wider ecosocial system and its interpretation. By the same token, the lowerlevel interface between the body and the expression plane is not simply completed by the higher-level content stratum, but is also a model of it. This helps us to see more clearly that the expression plane is itself metafunctionally organized (see section 12, pp. 90-4). From the point of view of articulation, we have seen how the bio-kinematics of the body are brought into play in order to realize a given act-type. In the present perspective, this may, indifferently, refer to vocal-tract activity in the act of speaking or limb, muscular and other systems which I use in order to carry an object across the room. In both cases, the proximate bodily activities of respiration, muscular and other activity in the vocal tract, visible lip and other facial movements (in the case of speaking) and the interplay of anatomical (muscles, nerves, limbs, bones, joints) and time-bound constraints that selectively modify degrees of freedom of movement according to a trajectory in the case of locomotion provide lower-level constraints - 'from below' - which are the system's initiating conditions (Salthe 1993: 214). It is the interaction of the two sets of constraints - time-dependent or dynamical and time-independent which allows the system to anticipate further possibilities and, hence, to act in determinate contexts. Thus, the lower scalar level of the body-brain bears the corporeal traces - the . memories - of previous social interaction (Threadgold 1997: 97-103). These memories serve both to maintain the overall stability of the system in the face of external fluctuations and to provide internal models of the body and its relations to its Umwelt (Berthoz 1997: 7). In this sense, individual organisms have a repertoire of movement schemata which are activated and deployed according to the dynamical requirements of the movement as it takes place in time and space. Salthe makes the further point that such models or schemata '... are lower-level to the organism as a whole, and certainly their dynamics are vastly faster than the environmental dynamics they could be relevant to, thereby allowing anticipation of environmental changes' (1993: 215). Rather than saying that movement is physical, whereas language is cognitive or psychological, we may see how both movement and language are forms of actional semiotic which

84

BRAIN, MIND, AND THE SIGNIFYING BODY

directly mediate and entrain the lower-level bio-kinematic systems which are their expression planes. However, the expression plane is not an objective or ostensive domain in the reductively Newtonian sense. Instead, it specifies the bio-kinematic potential of bodily synergies to be harnessed and channelled in specific directions for the purposes of social meaning-making. This means, in other words, that the neurophysiological and motor activities that are variously involved in both movement and speaking are constrained by higher-order boundary conditions, acting as a system of context-sensitive constraints on the activities of lower levels. In this way, a potentially very large number of degrees of freedom of movement are channelled into a relatively small repertoire of gestures which exhibit structural stability from one occasion to another in spite of many situation and performance-specific variations in their use. Furthermore, rather than postulating more and more layers of formal constraints to explain this as in much phonological theory, it is possible to describe a much more direct relationship between motor activity and its mediation by lexicogrammar and semantics. This brings about the possibility of a new rapprochement between the bio-physical and the social semiotic dimensions of language and other modalities of social semiosis, Thus, bio-physical systems are dynamic, open and adaptive systems that engage in constant exchanges of matter and energy with their environments. They are also comprised of nonlinearly related internal subsystems that may stochastically fluctuate. This leads to a transformation to some new global structure when the internal dynamics of the system are coupled to, for example, a critical change in energy flow. The emergence of a new global structure marks a corresponding reduction in the degrees of freedom of the system's internal dynamics. Further, the corresponding increase in stored information means that the informational cross-coupling potential of the many subsystems comprising its motor activity potential are reduced. According to homeokinetic theory (Kelso et al. 1986), this transformation from a high-energy source to a low-energy sink functions to stabilize the system as an ensemble of nonlinear oscillators. This means that the main organizing principle underlying motor activity is that of rhythmical periodicity. Motor activity and the articulatory forms that emerge from this are shaped by the biokinetics of weakly coupled oscillators of varying periodicities, Furthermore, vocal and other forms of articulatory activity are describable in terms of nested cycles of rhythmic periodicities on many different timescales which mutually model and entrain each other. In the case of speech, it is the breath group or the tone group (Trager and Smith 1951; Halliday 1967a; Laver 1970: 68-70; Lieberman 1984: 118-20; Edelman 1989: 174-5; Coulthard 1992) which is the primary unit responsible for the regulation of the energy which is stored, released, and dissipated in phonation. The breath-group theory proposes that the articulatory cycle is organized by grouping around six or seven syllables on average into complete expirations which are further characterized by having pauses at their boundaries (Laver 1970: 68). As Halliday and Coulthard, among others, have shown, these tone-group or breath-group boundaries, which are organized in terms of a respiratory cycle, are functional in the organization of spoken discourse into

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

85

units of information. That is, bodily processes are both integrated into and mediated by lexicogrammatical and discourse semantic factors on the content stratum (see also sections 13 and 14, pp. 94-100). The following short analysis serves to illustrate the structure of the tone group. Following Coulthard (1992), tone units' boundaries are marked by a double slash; the tonic segment is in upper case: Dion: / / what's he SPILLT it / / alREADy? / / look at the MESS / / PAUL / / you're gonna have to CLEAN UP / / in HERE / / it's a PIG STY / / Paul: / / it is RATHer a PIG STY / / /Dion: / YES / / WELL / / it's YOUR ANimal / / The breath group is the major phonological unit which carries both intonation and rhythmic periodicity. Lieberman points out that rhythmic periodicity 'follows from the fact that we can generate a steady subglottal air pressure function throughout most of the expiration' (Lieberman 1984: 119). It is necessary, as Lieberman (1984: 119) further points out, that this subglottal air pressure be stabilized in order to avoid 'uncontrolled variations in perceived pitch and amplitude'. Neonates are unable to achieve this degree of control when they cry because of their inability to control the fundamental frequency of phonation during the cry. At the age of around three months children begin to learn to control 'their subglottal air function throughout the duration of the expiratory phase' (Lieberman 1984: 119). This occurs as the infant becomes more attuned to the intonational and rhythmic regularities in the ambient speech of parents and other caretakers. For example, Vihman and de Boysson-Bardies (1994; see also Vihman 1991) have shown that phonological structure is an emergent effect of the dynamic interaction of many different factors on different scales - the neuromotor and perceptual systems, individual experience with language, and the ambient linguistic influences of caretakers and others. The child acquires new habits as the stored information in this newly emergent level increases. In this way, the child's motor activity is entrained to and becomes more specialized to the requirements of a given phonological system. But I do not see this process as one of learning abstract and formal phonological rules. Rather, I see it as one principle among others for constructing those global and potentially shareable principles of coherence which allow us to dialogically co-ordinate our bodily activities with those of others. Thus, a phonological system is a system of categorial, rhythmic and other regularities in the speech practices of some community which, as seen from the expression plane point of view, synchronize both peripheral and imagined speech motor activities with those of the other or the non-self - real or imagined - in the Umwelt. It is a means of harnessing friction to productive semiotic ends. This occurs against a bodily background of those primal vagaries and indexical necessities that are not

86

BRAIN, MIND, AND THE SIGNIFYING BODY

assimilable to the symbolic purpose to hand. In this way, meaningful bodily movement - vocal-tract activity, manual-brachial gestures, etc. - always entails a constant interplay of the regular and the surprising. The latter interferes with the predictability and stability of the former, in the process contributing to our sense of pleasure in movement based on the interplay of these two elements. However, this process of integrating the vagaries of the child's immature motor activity to the supervenient level of the adult phonological system does not mean that prior, non-phonological dimensions of articulation are altogether transcended (Salthe 1993: 213). In phonation, the breath group functions as a motor schema whereby the brain simulates and predicts vocal-tract activity before execution takes place. It is in this sense that Laver (1970: 68) writes of 'anticipatory adjustments in articulation'. The basic schema thus comprises an articulatory movement sometimes bounded by pauses, and in which there is one prominent syllable characterized by a major change in pitch in intonation. Typically, there is also a fall in fundamental frequency and amplitude at the end of the breath group, though this tendency may contrast with a rising pattern at the end of the breath group (Lieberrnan 1984: 130). The breath group is an articulatory prosody. It is a prime candidate for what Carol Fowler (1986: 5), following in the tradition of Gibson's ecological theory of event perception, calls an environmental event. Such events are sources of environmental information which can stimulate the perceptual systems of the perceiver. They have 'affordances' or possibilities of interaction with the perceiver. In the case of speech, the environmental event is the vocal-tract gestural activity of the speaker.

10. Metafunctional Diversity on the Expression Plane In this section, I shall explore the possibility that the expression plane of semiosis, no less than the content plane, is organized metafunctionally. The metafunctional explanation oflanguage form was briefly outlined in chapter 1, section 10, pp. 46--9. In my view, there are analogues of experiential, interpersonal, textual, and logical organization and meaning on the expression plane. That is, corresponding principles of metafunctional diversification operate on the expression stratum in relation to the topological-eontinuous substrate of the articulatory dynamics of the body which these entrain and construe as expression-substance. I am treating the expression plane as a class of physical-material events which are constrained by the sensori-motor and perceptual activities of interactants as well as being motivated by and mediated by higher-order social semiotic factors, expression-form (phonology, etc.) in the first instance. Rather than the abstract formalisms of much phonological theory, with its emphasis on levels of formal rules and the psychological reality of these, a metafunctional description of the expression stratum of semiosis must account for the ways in which the expression stratum interfaces with and construes bio-kinernatic phenomena of the human body in interaction with other bodies on various intersecting scales of spacetime. Thus, it would need to account for phenomena such as the following: (1)

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

87

the spatial displacement along a vector-in-time of the articulators in order to achieve specific co-ordinate structures or gestures and the phonemes, syllables, etc. that these articulate; (2) the ways in which modulated rates of change (in pitch, rhythm, movement, and so on) 'index' feeling states, physical sensations, and so on of the body; (3) the ways in which sensori-motor activities, respiration, muscular activity, limb movement, etc. are harnessed and entrained as body work which is directed towards the making of socially shared signifying acts in contextually relevant and coherent ways; (4) the ways in which such bodily events are recognized as having social semiotic significance by others. There are a number of distinctive ways in which the expression plane constitutes the interface between body and ecosocial environment. I shall develop this point in the next section. I shall return to the question of the metafunctional organization of the expression plane in section 12.

11. The Expression Plane is the Interface between Body and Ecosocial Environment The notion of the expression plane of semiosis as the interface between the body and its environment obviates the traditional split between an objective physical dimension and a subjective sensori-motor dimension. The fact that the expression plane 'faces both ways' - i.e. towards the body and towards the environment - itself provides the solution to this problem. Thus, from the point of view of both articulation and perception/reception, the expression plane is an ecological event which links or cross-couples the body-brain complex to its environment (Thibault 1997a: 158-60). There is, then, no objective input to the organism which is proportional to its behavioural output. Instead, information in the environment of the organism is always relative to the organism. This helps to break down the antinomy between expression-substance (e.g. phonetics) and expression form (e.g. phonology) (see Petitot-Cocorda 1985: 97-102). As an interface between body and environment, the expression plane plays a much more constructive role than that of mere information transfer between an organism and its outside. In actual fact, it is a more highly specified (social semiotic) elaboration of the biological principle that organisms selectively and actively attend to their environments on the basis of their own internal, biophysical organization. As an interface between body-brain and environment, the expression plane construes a relation of 'correspondence' between the two. It is a concrete demonstration of the principle that the organism corresponds 'functionally and morphologically' (Prodi 1987: 2; see also Bateson 1973a: 285-91; Gibson 1986 [1979]: 8) to its surroundings. In the process, the organism modifies the environment in virtue of its active and dialogic engagement with it. It also modifies its own internal organization. Rather than a transfer of information from organism to environment (articulation) or from environment to organism (perception), there is a continual process of 'translation'. For example, a smell or odour in the environment will, when detected by the receptor cells in the nose, provoke a chemical change in these same receptor cells of the organism.

88

BRAIN, MIND, AND THE SIGNIFYING BODY

The construal of an expression-substance in and through its correlative expression form means that an ecological event such as an articulatory act and its reception is always a translation into symbolic signs of the indices of environmental events. There is no necessary or causal relationship between the physical stimuli and the internal changes these bring about in the organism. For this reason, the relationship between the two is one of symbolic transduction rather than information transfer (Salthe 1993: 17&-7). The environmental information afforded by the articulatory event function as indexes of that event. However, the existence of shared models of the world or systems of interpretance which are stored in the central nervous system of the organism constitutes the resource whereby interactants use and construe these environmental indices for higherorder symbolic purposes which are not tied to indexical necessities. These symbolic resources are stored in the body-brain complex as schemata which not only interpret information from the environment (perception) but also functions as the models for predicting future actions and their consequences in the environment as well as acting in it. In part, they do so on the basis of the stored memories of the past consequences of such actions. In this sense, the schemata regulate the sensori-motor activities which project indexical information back into the environment where, however, its lack of any necessary connection to particular external events means it can be interpreted symbolically by suitably equipped agents. This is so in the case of the speaker's vocal-tract activity, which, even while providing important indexical information about the speaker's bodily states, is not interpreted as being uniquely tied to these on account of its further possibilities for symbolic reconstrual. It is always crosscoupled with other modalities of perception-action as well as other scales of ecosocial space-time, along with the meanings that are afforded by these diverse scalar levels of organization. The expression plane is, then, an interface which is characterized by the continual exchange which is activated by bodily processes such as respiration, bio-kinematic movements of articulators of various kinds, their modulations, on the one hand, and the environmental information these produce for suitably equipped perceivers, as well as their possibilities for symbolic reconstrual in ways relevant to the ecosocial context in which they occur. The articulatory act qua ecosocial event specifies its informational structure to the perceiver; it does not have to be enriched or re-elaborated as in the inferential model of perception (Gibson 1986 [1979]: 251-3). However, this does not fully explain the symbolic transduction of this physical event into a social semiotic one. For this, it seems to me that we need some criteria of the values which selectively guide and constrain behaviour. The perception of vocal-tract and other forms of gestural activity does not take place in a value-free vacuum. Rather, criteria of value specify that some events are more important, more relevant, more interesting, and so on, than others. Edelman (1989: 98-100) has shown the importance of biologically in-built criteria of value which selectively channel the organism's activities along some pathways rather than others in the interests of its survival and further development. That is, values enable the organism to make relevant functional discriminations in its environment and to act accordingly. In such a perspective, the functional cross-couplings of classes of vocal tract and other articulatory

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

89

events with their social semiotic values is only a further elaboration of this same basic principle. Values - biological and semiotic - together selectively channel, entrain, and motivate meaning-making activity in certain preferred directions rather than others. In recasting the expression plane as an ecosocial event, it is not difficult to postulate that articulatory act and perception are functionally complementary. There is information which is specific to both. This follows from the fact that such events 'face two ways', viz. to the body of the articulator, as well as to the environment in which they are perceived, including, of course, the perceiver. The information specifying the articulator and the information specifying the perceiver are not opposed. Instead, they are simply two facets of the one overall phenomenon. In this sense, they are complementary. There is no need to resort to two separate descriptive languages to talk about the two as would be the case in the various forms of neo-Cartesian mind-body and subject-object dualisms. Carol Fowler (1986) shows in her discussion of phonetic articulation as distal events that are directly perceived that the perception of such an event entails the apprehending of the invariant structure of phonetic gestures as the vocal tract undergoes spatio-temporal change during articulation. The further question that arises concerns how the spatia-temporal eo-articulation of vocal-tract articulators and the acoustic information that results give rise to specific ecosocial semiotic effects. What are the me ta-redundancy relations (chapter 1, section 6, pp. 26--30) among these ecosocial meanings, the sensori-motor action potential of the body, and their eo-articulation on some particular occasion of interaction? This last question suggests the need to find a unified conceptual framework for explaining the relationship between bodily processes and the social semiotic significance and purpose of these. AIl forms of gestural articulation have spatio-temporal properties. They exhibit properties of both temporal succession and spatial adjacency. A gesture, seen as an ecosocial event, exhibits a change in adjacency structure - viz. the spatial overlap of articulators - as well as the successive ordering of gestures in time. How, then, in the face of such spatio-temporal variability, are structurally stable forms recognized? What about seemingly 'static', object-like sign systems such as Writing? The predominantly spatial character of writing suggests minimal or no change of adjacency structure in time. However, the bodily activity of scanning and orienting to the marks on the page or other treated surface necessarily requires movement as an integrating activity (see also Harris 1995b: 46). In my view, the extra-somatic character of the treated surface on which written notation is inscribed only conceals the event modulation which takes place when a reader engages with the written text. Writing is not an exception to this general point, but, perhaps, a limiting case. I would suggest that bodily movement in one form or another is the fundamental organizing principle whereby interacting bodies are integrated into both semiotic action performances and their textual records and products. The sensori-motor activities of the articulator are constrained to informational invariants-in-change by the perceiver's ability to respond to these. Both articulators and perceivers must learn to distinguish between those forms of event modulation (change) that

90

BRAIN, MIND, AND THE SIGNIFYING BODY

bring about change of meaning and those that do not. In my view, a metafunctional account of the expression plane of such events (section 10) can help us to solve this problem. I shall now return to this question in section 16 below.

12. The Metafunctional Basis of Vocal-tract Articulatory Activity In the case of speech sounds, the experiential (or conceptual) dimension refers to the typological-categorical phonological distinctions made by phonemes and their combination according to phonotactic principles into larger phonological units such as the syllable and so on. This dimension of speech sounds digitalizes the acoustic stream in terms of discrete phoneme categories. Thus, the discrete or segmental character of these phonological categories shows how this dimension of the speech sound is concerned with particulate or constituent-like phonological structures which can be described as a ranked hierarchy of smallerand larger-scale constituents. However, the assumption that phonological units are static and context-free mental entities which are divorced from articulatory and acoustic properties of eo-articulated speech is rejected here (Thibault 1997b: 4-5). Alternatively, Browman and Goldstein (1991, 1995) base their gestural theory of phonetic articulation on the notion of co-ordinative structures, as developed by Kelso et al. (1986) in movement theory. Browman and Goldstein argue that articulatory gestures are defined dynamically as 'the formation (and release) of a constriction within the vocal tract through the movement of (a) a particular set of articulators, (b) towards a particular constriction location, (c) with a specific degree of construction, and (d) in a characteristic, dynamically described manner' (1991: 315). From the experiential perspective, phonological gestures construe and categorize actual vocal-tract phenomena which take place in the space-time of articulation. That is, they construe phonetic events which may be analysed into three main functional components, viz. (1) the dynamical movement in space-time of the articulators, (2) the particular articulators which participate in this movement, and (3) the circumstances - cf. the degree and manner of this movement - which are attendant on this dynamical movement and its participants. Typically, the speaker and the listener experience this event as a single organized whole. However, the gestural schema which the speaker deploys organizes the vocal tract in terms of both the articulatory processes which go on in space-time and the constituent parts of this process. Armstrong et al. (1995: 14-15) also show that the manual-brachial gestures used in sign language realize clause-level semantic structures such as [AgentAction-Patient] (cf. [Actor-Process-Goal] in systemic-functional linguistic theory) directly. That is, the combination of hand-and-arm-movement directly realizes the semantic structure in question by performing it. Thus, a hand may perform the semantic function of agent which moves towards a patient. Likewise, the hand movement itself realizes the verbal process or the action. Such semantic functions are, of course, on the content stratum of organization. However, they further comment in connection with speech that:

R E T H I N K I N G THE EXPRESSION PLANE OF SEMIOSIS

91

It is impossible to see the same interrelation of phonology, syntax, and semantics in spoken language, not only because most of the gestures producing speech are invisible, but also because of the difference between the perceptual and productive modes for speaking and signing. (Armstrong et al 1995: 15) In my view, the point is, rather, that there are complementary principles of (metafunctional) organization on the expression plane. In the first instance, these do not construe the semantic structures referred to by Armstrong et al. Instead, they construe analogous and parallel structures in the expressionsubstance of the speaker's vocal-tract and other articulatory activity. The further question as to how these relate to the content plane, which is the focus in the discussion in Armstrong et al., will be taken up later. In the case of the manualbrachial articulatory gestures discussed by Armstrong et al., the movements are sequential in space-time, as seen from the lower scale of the expression plane, whereas the lexicogrammatical and semantic functions that map onto these are multilayered. This contrasts with the vocal-tract gestures of speech, which are co-articulated or overlapping, rather than serial, from the point of view of the expression plane, whereas the morphosyntactic structures realized on the content plane preserve a fundamentally sequential structure (Studdert-Kennedy and Lane 1980: 35-6). Individual gestures may be seen as typologically similar to phonological segments such as phonemes. However, and unlike segment-based theories of phonology, 'there are no a priori constraints on intergestural organization within the gestural framework' (Browman and Goldstein 1991: 319). The notion of co-articulation refers to the fact that vocal-tract gestures exhibit considerable spatio-temporal overlap. There are, then, differences of both kind and degree of overlap and these differences may be phonologically salient in distinguishing one gesture from another. To explain the role of gestural overlap in the articulation of vocal-tract gestures, Browman and Goldstein have developed the notion of a gestural score, which is reproduced in Figure 2.1.

Figure 2.1: Example of gestural scores for /pcen/ ('pan') and /been/ ('ban'); borrowed from Browman and Goldstein (1995: 189)

92

BRAIN, MIND, AND THE SIGNIFYING BODY

Gestures, in the view of these researchers, are not necessarily confined to discrete phonological segments, seen as the natural place of residence of particular constellations of gestures. Rather, an 'entire constellation of gestures' may be 'phenologically' salient. The point is that such gestural constellations are comprised of various parts, all of which function together, in synergy, to produce the entire gestural constellation. The presence or absence of particular gestures thus specifies the smallest-scale ways in which the vocal tract is modified in the production of particular phonological categories. Gestures, it is important to point out, refer to higher-order semantic construals ofthe actual spatio-temporal movements of the vocal-tract articulators (Browman and Goldstein 1991: 323). They are, then, schematic to the actual movements of the articulators. Thus, a speaker's knowledge of a word also includes 'a specification of its gestures and their organization' (1991: 324). Thus, much of the gestural plasticity and variation which is evidenced in casual conversation arises through the concurrent production of overlapping gestures, rather than the referencing of discrete phonetic rules (1991: 324). The schematic character of gestures means that they exhibit structural invariants irrespective of variations which may occur during any particular instance of their vocal-tract execution. By the same token, the gestural score emphasizes that gestures also exhibit spatio-temporal invariants. That is, either the spatial or the temporal dimension of the gestural score may be modulated in ways that are invariant. In other words, gestural scores have both structural and transformational properties which are schematic to specific instances. The interpersonal dimension of speech sounds is prosodic or field-like. It is concerned with the ways in which prosodic contours deform the acousticarticulatory shape of the utterance as a means of communicating the speaker's attitudinal or affective stance or relationship to the utterance or to one's interlocutor. Intonation is the main resource which has been systematically studied by phonological theory. Intonation is prosodic because its articulatory and acoustic effects are not localized in any specific (phonological) constituent. Instead, intonational prosodies have as their articulatory and acoustic scope the entire breath group or tone group. From the perspective of the expression plane, articulatory prosodies deform or modulate the phonological shape sensa strictu of the specific phoneme selections and their combinations. Bolinger (1985) has written of the 'iconicity' of intonation in this sense. That is, intonation, in Bolinger's account, directly registers changing feeling states in the body of the speaker (see section 14, pp. 98-100). The textual dimension of speech sounds is wave-like or periodic. It has to do with the nested rhythmic periodicities that delimit the beginning and the end of, say, the breath group. This periodicity is temporal. Thus, the rhythmic modulation of respiration in the breath group in terms of a beginning-end cycle marks the boundaries of the modulation itself. Such punctuations of the temporal order of succession may provide a means of temporally co-ordinating 'the reentrant cortical areas mediating the phonological, syntactic, and semantic levels of speech' (Edelman 1989: 179), as well as synchronizing interactants into the spatio-temporal rhythms of the speech event at the level of the timing and the synchronization of turn-taking.

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

93

These rhythmic periodicities have their basis in the respiration cycle and are, for this reason, based on specific rhythmic patterns which are engendered by bodily processes and functions. Rhythm may also be modulated and the principal modulating factor would appear to be velocity or change of rate of, say, breathing in the case of speech, acceleration along a movement vector of a hand or an arm in the case of gesture, and walking, and so on, in the case of locomotion. Change of rate applies to some dimension of the spatio-temporal parameters of articulation of the perceptual event in question. Here, I am assuming that the expression plane of semiosis is always a perceptual event in some sensori-motor modality. This is so from both the execution and reception points of view. In other words, it is the expression plane of semiosis which crosscouples the semiotic act to the perceptual and motor activities of the participants to the act in question. Semiosis is always embodied in this sense. No act of semiosis floats free from or transcends this somatic dimension of its execution and reception because it is the expression plane which serves to integrate embodied participants into the meaning-making act itself. This is so in the sense that the expression plane is the interface which cross-couples the sensori-motor activities of the body (firstness) to the environment or non-self (secondness) in and through the higher-order systemic resources of some semiotic resource system (thirdness) (see chapter I, sections 3, 8, pp. ll-18, 34-9). Finally, the logical metafunction is based on the principle ofinterdependency. Its structuring principle is that of recursion. This follows from the spatiotemporal character of articulation. Probably the simplest form of interdependency is that of succession in time. In speech production, the articulation of phonetic segments is characterized as 'overlapping sets of coordinated gestures, where each set of coordinated gestures refers to a phonetic segment' (Fowler 1986: ll). This view obviates the .requirement that a priori planning or control procedures existing in an ontologically distinct mental realm act as the formal causes of articulation. Instead, planning occurs as part of the on-line and context-sensitive production of phonetically structured articulations. The breath group is an example of an event simplex. However, event complexes may also be built up from such simplexes through processes of nesting, addition, co-ordinating, subordinating, repeating, and so on. For example, rhythmic periodicities of different temporal duration may be nested the one within the other (Martinec 2000). Again, a movement sequence may be nested inside a still larger sequence, and so on. The eo-articulation of articulators in the vocal tract in the performance of a given gestural score involves the co-ordination of various articulators which are independent of each other. Other event complexes may involve the sequencing of one event after another. For example, one flips the power switch before the computer boots up ready for use. The question that arises in all cases is how and to what extent the information which specifies the event is preserved under such conditions of temporal and causal transformation. Lindblom (1991: 19) emphasizes, with his principle of 'sufficient contrast', the plasticity of phonetic gestures rather than fixed criterial properties, which are based on the contrary principle of 'maximal contrast'. As Lindblom points out, the latter notion has dominated in the discussion of the

94

BRAIN, MIND, AND THE SIGNIFYING BODY

perceptual (phonological) salience of phonetic gestures. That is, time-bound 'production contrasts tend to counterbalance demands for perceptual contrast' (Lindblom 1991: 19). Rather than criterial properties, there is a context-specific interaction of many different factors - perceptual, articulatory, etc. - which means that the ecosocial event is an emergent property of the time-bound ecosocial context. They are not structured by fixed a priori perceptual or other properties, but constitute what Lindblom calls a 'variable and adaptive means' (1991: 21) for the purposes of ecosocial interaction. Vocal-tract activity, as I pointed out above, is a form of bodily movement. It involves the harnessing and control of specific skeletomuscular and other bodily systems so that the acoustic information we produce is appropriately and productively deployed as we act in and upon our Umwelt. This emphasis on embodied, physical movement highlights the fact that vocal-tract activity is also a forceful physical-material interaction with our environment, including those with whom we interact. The metafunctional analysis in this section suggests how factors such as the dynamical movement of articulators, intonation, rhythm, and other features are all specific dimensions of the ways in which vocal-tract activity is a form of forceful acting on environmental friction or secondness. Abstract, symbolic notions of language too often lose sight of this fundamental insight. Social agents do not simply engage in abstract symbolic 'thinking' or 'cognition' whenever they talk. They also deploy their bodies and its extra-somatic projections as a means of engaging in forceful interactions with the world of the non-self. They must modulate and control these bodily forces in ways which are not separable from the lexicogrammatical mediation of vocal-tract activity. In the next section, I shall discuss an example in order to illustrate this principle.

13. Subjectivity, Agency, and the Prosodic Realization of Interpersonal Meaning The interpersonal metafunction is a resource for intervening in, acting upon, and interacting in the world. Its mode of expression is not segmental, but prosodic (Halliday 1979; Matthiessen 1990). Interpersonal prosodies are not confined to features such as pitch contour on the phonological stratum, but also occur on the lexicogrammar and discourse semantic strata. There are also visualgraphological prosodies. Matthiessen (1990: 21), drawing on a distinction first made by Robins (1957), identifies two types of prosody: (1) prosodies realized continuously; (2) prosodies realized at boundaries. The first type refers to features whose domain extends beyond the point of realization of the feature, to cover the whole or a part of the whole structure. At the rank of syllable on the phonological rank scale, examples include stress, pitch, and length. The second type includes features which delimit some structure by marking what goes before, and what comes after. In so doing, such features prosodically extend over the structure they delimit. Prosodies are similar to Pike's 'hyper-unit' or field view of sequence: '... a "sequence" of two units is treated as a single hyperunit with no essential requirement of internal specific segmentation or specific peak identification' (1967:

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

95

553). Criteria of segmentation and peak identification correspond to Halliday's experiential and textual metafunctions, respectively. Halliday (1979) and Matthiessen (1990: 22) argue that prosodies, which characterize units such as the clause as a whole, act like a continuous motif which colours this whole. I fully agree with this characterization, but I should like to develop here the question of interpersonal prosodies in specific ways, which are relevant to the central questions of this chapter. Let us consider the following Italian example, as shown in Table 2.3. This exchange was part of a larger sequence during which the mother had repeatedly attempted to convince the six-year-old child to get up in order to get ready for school. The clause initial Vocative element selects for tone 1 (falling). The tone group realizing the pitch contour over the clause in the first part of the exchange selects for tone 2 (rising), which is the typical tone for interrogatives. The child's response selects tone 1. The falling-rising tone of the first speaker (the mother) operates as a single semantic unit, extending over the whole clause; it expresses the contradictory nature of the interpersonal semantics, i.e. compulsion and a degree of uncertainty as to the modal investment in the exchange (,should I be making the child get up, or should the child be acting off her own bat?'). The example cited above also illustrates what Matthiessen (1990: 23) calls a '3rd order or semantic prosody': Ilaria ti vuoi alzare. The grammatically segmental elements in bold type realize a semantic prosody on the discourse stratum. Thus, the proper noun Ilaria, the second-person reflexive pronoun ti, and the morphemic realization of the semantic agent tu, which is fused with the verb vuoi in the lexicogrammar, realize a semantic prosody of this participant's subjective 'presence' in the discourse. In this case, it is a directional vector, whose energy extends from the speaking agent (the mother) to the 'making present' of the child as a (reluctantly, on this occasion!) volitional agent in the mother's discourse. Thus, we see how the first-order (phonological) and third-order (semantic) prosodies are harnessed in the vectorial movement of energy which occurs, as the mother seeks to transfer the agency to the child, or enjoins the child to take up this agency on her own account. In Italian, it should also be noted, this use of the reflexive, where the verb process concerns the whole of the Agent, rather than a part of the Agent (e.g. a part of the body), means that the agency is self-engendered; it originates, semantically speaking, 'from within' the Agent. The co-referentiality of reflexive pronoun (Goal + Experiencer) and subject pronoun (Agent) in such cases suggests a self-monitoring of this 'inwardly' attributed volition, which the English I get up does not realize. In this semantic environment, the reflexive pronouns in Italian are a systemic reactance of a semantic cryptotype of such self-monitored and self-engendered processes. In English, the periphrastic I make myselfget up better glosses the Italian mi alzo. In Table 2.3, the reflexive pronoun conflates the transitivity roles of Experiencer and Goal. I have borrowed the first of these terms from Longacre (1983: 155). Experiencer is similar to Halliday's Medium (Halliday 1994 [1985]: 161-75), and is the participant which is centrally involved in the experience realized by the verb process. In the case of the Italian reflexives, I prefer this interpretation because it better glosses the semantics of the self-reflexive or self-monitoring relation between the participant as both Agent and Experiencer of its own

96

BRAIN, MIND, AND THE SIGNIFYING BODY

Table 2.3: Metafunctional analysis of exchange unit: phonological, lexicogrammatical and discourse semantic strata; tonic segments in upper case

Tonic 1

Pretonic

Tonic 2

ILARIA

Tonic

ALZARE

vuoi ti

NO

ILARIA

YOU

WANT

TO GET UP

NO

Vocative

Complement

Subject + Finite: Modal: Volition

Residue

Minor clause: Negative Polarity

GoalExperienccr

Agent

Process: Material Action

Theme Given Starting Situation (agency not yet imputed)

Rheme New Target Situation (agency imputed to or transferred to addressee by addresser)

Directive/Exhortation to Act

Speaker.: Mother

Noncompliance with Directive Boundary marking change of speaking turn in exchange

Speaker; Six-year-old child

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

97

action. At the same time, the transitivity role of Goal (cf. Longacre's Patient) is conflated with that of Experiencer in clauses of this type. The Goal (or Patient) is the participant that undergoes some kind of change as a result of undergoing the process. It seems necessary to make this three-way distinction in order to describe the transitivity relations in such clauses. Matthiessen (1990: 7) also characterizes prosodies in terms of the directionality of, say, the pitch movement. I would go further. This movement is the dynamic energy which is exchanged, and, in part, experienced as a moving force, whenever a linguistic act is uttered. The giving form to a linguistic act also entails a dynamic relationship between this energy and the other strands of meaning which are configured in the linguistic form (see Cremonini and Frasnedi 1986: 23). Interpersonal prosodies are just one parameter of linguistic meaning that shows the scalar heterogeneity of the bodily and semiotic processes involved. This would not be so if meaning were no more than an abstract formcontent relation. Meaning-making has an undeniable material and sensual quality, which, I think, can be explained by the relations between the energy that is released in the production of the linguistic act as well as by its materiality. This energy, as Cremonini and Frasnedi (1986) point out, has 'directional vectors', which both extend beyond the utterance and into its environment and feed back into it, producing thereby a resonating density of significance. These authors also point out that this energy, in order to be effective, and not merely redundant and 'empty', must be brought under control so that it be effectively harnessed and synchronized with the other dimensions of an utterance's meaning. This is the task of rhetoric; rhetoric has the task of both giving directionality to and anchoring this energy in the dialectic with the other strands of the utterance's meaning potential. Lack of such directionality and control would seem to suggest emotional or other disturbances, pathology, or disorientation in the communicative ecosystemic environment of agents. It is this energy which contributes to that 'making present' of an embodied subjectivity that I mentioned earlier. It is the interface between expression and content whereby meaning comes into contact with the materiality of the body, and its neuroanatomical, neurophysiological, and neuropsychological levels of organization. In so doing, the energy of the utterance gives both presence and force to an immanent subjectivity/agency through this eo-articulation of the phenomenal-material and social semiotic domains. It is in and through the dialectically dual and complementary nature of the relation between the two domains that body rhythms, physical pain, and so on, exchange energy with the semiotic, and in so doing they give presence and energy to the social semiotic voicing of our subjectivity and agency (Bakhtin 1990 [1924]: 292). The energy and vectorial directionality of interpersonal prosodies endow our meaningmaking with the presence of a temporal event and a spatializing value. This is one concrete illustration of the principle that the matter-energy which is harnessed and channelled in meaning-making constitutes a forceful interaction with the other. Human meaning-making activity is a hybrid phenomenon: we do not understand and respond to it solely on the basis of the semiotic relations that we construe between the different parts of the activity in relation to the systems of

98

BRAIN, MIND, AND THE SIGNIFYING BODY

paradigmatic alternatives that are relevant to the contextualization of the unfolding text or discourse event. We also understand and respond to it on the basis of the physical-material relations and processes with which these semiotic relations cross-couple and whose flows they entrain not only in the local context of this here-now event, but also in relation to physical-material flows and processes on other spatia-temporal scales beyond the immediate here-now context. To verbally abuse another person is more than just a 'choice' of one paradigmatic option among other possible ways ('praise', 'encourage', 'comfort', and so on), more than just a component functioning in some culturally recognizable activity-structure (syntagm) such as 'intimidation', 'threat', 'harassment', and so on, more than just an index of some particular contextually relevant value or social situation-type (gender relations, employer-employee relations, and so on) which is invoked as relevant to the meaning of the verbal abuse. It is also to induce or bring about clusters of physical symptoms (e.g. racing heart, tense muscles, trembling, upset stomach, and so on) which individuals associate in complex ways with emotions such as fear, guilt, sadness, and so on, and in ways which, in turn, index social situations of, say,parent--child or gender domination, teacher-pupil control, and so on. In other words, our social semiotic practices also induce bodily sensations - both pleasurable and painful - in others, and in ways which contextually redound with higher-order emotional states that are linked to one's positioning within particular social activity-structures or discourse genres, as well as to the accumulated bodily memories that the self collects along his or her life trajectory as a result of the various contingencies which constitute his or her historical emergence as an individual. The cross-coupling of the material and the semiotic, or, in other words, the constant cross-scalar intermingling of these two facets of semiosis, shows how the individual, in using the impersonal meaning-making resources available to him or her by virtue of his/her belonging to a given ecosocial system, manages to distinguish him- or herself from his/her surroundings in the specific act of meaning-making. In using the standardized resources of the system, the individual is inevitably differentiated from his or her surroundings. He or she inevitably indexes the specific states of his/her brain-body in a given time and place at the same time as the behaviour of the individual is cross-linked to higher-scalar, hence impersonal, levels of organization in the ecosocial environment.

14. Vocal-tract Gestures and Grammar: Symptom and/or Supervenience? Bolinger views intonation as iconic or symptomatic of emotions and their degrees of intensity. In other words, intonation is directly 'presentative' of embodied feeling states of the speaker (1985: 98). Bolinger explicitly argues against the 'grammaticalization' of intonation, claiming as his starting point that 'high pitch symptomizes a condition of high tension in the organism, low pitch the opposite' (1985: 99). In this account, intonation and its meaning has its origin and its basis in 'a gestural complex whose primitive and still surviving

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

99

function is - however elaborated and refined - the signalling of emotions and their degrees of intensity' (1985: 98). I agree with this claim entirely as far as the relationship between speech sounds and the speaker's embodied feeling states is concerned. Nevertheless, I do not think that Bolinger provides an entirely satisfactory account of the ways in which intonation is also integrated with and mediated by lexicogrammar. Intonation is not simply a 'primitive' gestural residue of speech which, phylogenetically speaking, preceded language and then merged with it. That is why the term 'paralinguistic' is, in the final analysis, misleading: intonation does not simply accompany language, but is integrated with it. This would not in itself explain the teleological function of intonation in, say, the interpersonal dimension of meaning. Intonation is often the crucial distinguishing factor in construing what kind of dialogically co-ordinated move the speaker is uttering (Halliday 1967a: 21-2). The biological differentiation which Bolinger sees as fundamental to intonation itself undergoes significant reorganization as the social semiotic dimension of intonation emerges more clearly. Lukacs most insightfully explains this general process in the following terms: The division of labour is originally based on the biological differentiation of the persons who form the human group. The receding of the natural barrier as a consequence of the social being's becoming ever more strictly and purely social is revealed, above all, in the fact that this principle of differentiation which is biological in origin, assumes in itself ever more numerous moments of sociality and ends up acquiring in itself a first order importance, downgrading to secondary status the biological moments. (Lukacs 1981: 138; my translation) In this quotation, Lukacs addresses the question as to the ways in which a social system emerges from a biological one. His argument expresses the logic of the specification hierarchy of integrative levels (chapter 1, section 8, pp. 34-9). Bolinger's perspective clearly does not deny the emergence of intonation from biological systems in the individual organism. He argues against those who see intonation 'as part of the abstract code' (1985: 99) on this very basis. What he does not explain is the effect on intonation of its integration with the content stratum of lexicogrammar and semantics in language. If intonation had simply remained 'symptomatic' of the feeling states of the speaker, then there would be no significant traces of its reorganization as a consequence of its integration with the linguistic semiotic. If, on the other hand, the division of social labour entails, as Lukacs argues, the increasing specialization of action for social purposes, then this would suggest that with its progressive integration into the linguistic semiotic, intonation itself becomes more specified as a semiotic resource which is able to realize teleological acts the aim of which is to bring about in others 'the will to carry out determinate teleological positions' Lukacs (1981: 139). Certainly, the crucial role that intonation plays in many languages for distinguishing various categories of dialogically co-ordinated move would seem to bear this out. As Lukacs further points out, such acts, if they are to prove effective, require

100

BRAIN, MIND, AND THE SIGNIFYING BODY

that other persons in the human group recognize their significance and teleological intent. That is, there are higher-order boundary conditions of a social nature which provide the conditions whereby the otherwise purely symptomatic signals from the individual's body are amplified and adaptively modified as socially shared resources for making meaning. The social power of a given body lies in the ways in which others take up and further amplify and modify in socially entrained ways and on many different scales the effects of the bio-kinematic energy, rhythms, and movements of 'powerful' individuals. This does not mean that the prior 'symptomatic' layer of bodily significance has been superseded or transcended. Rather, intonation both retains important traces of its earlier ancestral inheritance and at the same time has undergone significant restructuring and modification in the course of its integration into the more highly specified linguistic semiotic. Only in this way can we explain how intonation, from the systemic point of view, has itself emerged as a highly specified system of value-producing, or semiotically salient, differences in a given language system. How otherwise might we explain the complex distinctions in meaning that intonation makes in relation to grammatical, facial, kinesic, musical, and other semiotic modalities?

15. The Intentional Character of Articulatory Activity There has been considerable discussion in recent years as to whether the object of perception in speech sounds corresponds to vocal-tract articulatory gestures or to the acoustic signal which results from this. This discussion hinges on the further question as to whether vocal-tract activity is merely the surface manifestation of pre-programmed and invariant control structures for phonetic gestures, as in the motor theory of Liberman and Mattingly (1985), or is itself linguistically motivated gesture. In the motor theory, the control structures are abstract invariant properties that control articulatory movements. It is these control structures, rather than the movements themselves, that correspond to the speaker's intentions (Liberman and Mattingly 1985: 23). The control structures, which are said to be encoded as abstract neural structures in the brain, fail to explain that speakers' intentions cannot be seen as a priori abstractions of on-line vocal-tract activity. Rather, intentionally directed sensori-motor activity always takes place in a dynamic and constantly changing context which itself shapes and modulates the speaker's activity. Intentions are a function of many interacting contextual factors rather than an abstract cause of these. Motor acts qua intentional acts are understood, as Merleau-Ponty put it, 'in reference to the aims oflife' (1983 [1942]: 163). The structure of the acoustic array is the result of vibratory resonance propagated from a mechanical source. Thus, the source of the acoustic signal the distal event in the environment - is strongly cross-coupled with the energy which is propagated. It is this which provides the listener with information about the distal event. The listener samples change in the acoustic array which provide him or her with information about the distal event. This event is the source of the disturbance in the ambient acoustic array and of any changes in its dynamics.

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

101

In other words, the acoustic array which the listener samples specifies the environmental event - e.g. the speaker's vocal-tract gestures - to the perceiver. In this way, the listener uses this information to orient to the speaker for the purposes of dialogic interaction. It is not the physical properties of the acoustic signal per sewhich are relevant to the listener. Rather, the linguistically motivated character of the speaker's vocal-tract gestures - Le. the physical properties of the distal event itself - is itself revealed to the listener in and through the information which is structured in the acoustic array. This means that the perceiver directly accesses the gestural properties of the distal event of speaking through the information structured in the array. Fowler and Rosenblum (1991: 36) cite evidence on the perception of intonation (among other factors) to show that the listener's perception of intonation peak corresponds not to a perception of the 'objective' or 'absolute' rate of opening and closing of the vocal folds during the production of the intonational melody, but to those modulations of the rate of opening and closing brought about by gestures which the speaker intentionally uses to modify the intonational melody. The point is that listeners selectively sample the acoustic variables in the acoustic array as a guide to the linguistically organized and semiotically motivated gestures of the speaker. There would also appear to be a value component in this process (Edelman 1989: 152). The selective nature of this orientation to the array suggests that the listener's response to these variables is an adaptive one. In establishing specific links between categories of acoustic variables and their linguistically motivated gestural values, the activities of speaker and listener are reciprocally modified and co-ordinated. For proponents of the motor theory, the abstract control structures which are hypothesized correspond not to the speaker's articulatory movements but to his or her intentions (see above). This leads to the 'encoding' view of the relationship between the control structures and the eo-articulation of phonetic segments in articulation (see Fowler and Rosenblum 1991: 47 for critical discussion). The view of articulation-as-encoding-of-phonetic-segments assumes that speech sounds are segmental. On the other hand, the boundaries between discrete (phonetic) segments are distorted by their encoding into the continuous and overlapping movements of the articulators in real-time vocal-tract activity. In the gestural view oflinguistically motivated vocal-tract activity, on the other hand, eoarticulation is seen as the functional cross-coupling of articulators which are temporally heterogeneous. Rather than the presumed homogeneity of a serial ordering of phonetic segments which is then distorted by its encoding in eoarticulation, there is a complex layering of gestures that sometimes overlap and sometimes do not in time (Fowler and Rosenblum 1991: 47). The lowest scalar level of phonetic organization, in this view, is not comprised of abstract phonetic primitives. Instead, it is comprised of gestural prosodies which are modulated by higher-scalar levels in the ecosocial semiotic system that speaker and listener share. This is implicit in the following criticism of the motor theory of phonetic gestures made by Fowler and Rosenblum: Do listeners need an innate vocal tract synthesizer to recognize acoustic reflections of phonetic gestures? Although it might seem to help, it cannot be

102

BRAIN, MIND, AND THE SIGNIFYING BODY

necessary, because there is no analogous way to explain how observers recognize most distal events from their optical reflections. Somehow the acoustic and optical reflections of a source must identify the source on their own. (Fowler and Rosenblum 1991: 48) Fowler and Rosenblum draw attention to the ways in which the motor theory black boxes the question of how the vocal-tract gestures identified in the distal event functions on the higher-scalar level of ecosocial organization (see also section 4, pp. 72-5). The speaker's intentions can only be recognized as functioning in and through phonetic gestures because these very small-scale phenomena directly implicate much larger-scale processes on the ecosociallevel (see Lemke 1997 for critical discussion of this important point). This is what the notion of an ecological event in Cibson's (1986 [1979]: 93-110) sense in fact entails. The black boxing that I referred to above occurs, for example, when an innate vocal-tract synthesizer is postulated, as in the motor theory of phonetic gestures, as the explanation for the ways in which interactants recognize and interpret gestural activity in distal events such as the speaker's vocal-tract activity. Thus, the black boxing of the higher levels in the form of a presumed vocal-tract synthesizer reduces this interplay of scalar levels to a single level of preprogrammed central-processing units, seen as hard-wired modules in the human brain. Consequently, it fails to show how these smaller-scalar events in the vocal tracts of speakers are directly implicated in larger-scalar dynamics which may include the individual's own individuating trajectory, the speech practices of the community, the shared interactional practices of particular speaking dyads, and so on. For a theory of vocal-tract gestural activity, this has a number of implications. First, the question of the speaker's production of phonetic gestures and the listener's perception of these in the distal event are seen as complementary dimensions of a single larger-scalar ecological event that obviates the need for any reductively bottom-up causal explanations from within the individual organism. The ecosocial character of the event means that both speaker and listener perspectives are necessarily implicated in it. Secondly, the mediation of phonetic gestures by their higher-order lexicogrammatical and semantic meaning relations which construe the larger-scalar phenomena of the world in which speakers and listeners live means that there is a high degree of direct, unmediated and fully motivated relations between the smaller-scalar dynamics of the former and the meanings these transparently have for interactants. That is, higher-scalar semantic meanings and the large-scalar historical and cultural realities these index are directly implicated in lower-scalar phonetic gestures for both speakers and listeners. Indeed, Studdert-Kennedy and Lane (1980: 36-8) advance arguments to show that both phonetic gestures in speech and manualbrachial gestures in sign language exhibit serial properties of motor execution that are directly implicated in the realization of morphemic distinctions. This is so in spite of the parallel processes of eo-articulation which are also evidenced.

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

103

16. Embodying the Metafunctions: The Example of Vocal-tract Articulatory Activity My focus on the metafunctional dimension of the expression stratum highlights the embodied, physical-material basis of meaning-making. The vocal tract is a topological-continuous dynamical system which admits of complex attractors, as discussed by Kauffman (1993: 177). Thus, the metafunctional organization of the expression plane provides a framework for describing the multiple and variable ways in which trajectories within the topological substrate of the vocal tract may converge on a set of states over the space-time span of the articulatory act. Is it possible to describe the four metafunctions as trajectories which flow through particular basins of attraction in the limit cycle around which the movements of the articulators flow during articulation? That is, are the metafunctions comparable to sets of variables which exhibit their distinctive repetitive oscillations (particulate, wave-like, field-like, recursive) around the limit cycle of the system? If the answers to the questions posed in the previous paragraph is 'yes', then this would mean that the four metafunctions represent basins of attraction to which different functional couplings of articulators converge in the overall articulatory act. I am making this suggestion on the basis of evidence provided by Fowler and Rosenblum (1991: 48-9) that listeners 'are sensitive to the distinct gestural sources' that affect fundamental frequency in different (metafunctionally motivated or entrained) ways. Consider the following research findings that are discussed by Fowler and Rosenblum in support of the above cited claim: What guides the listener's factoring of converging effects of Fa [fundamental frequency, ~JT]? Presumably, it is the configuration of acoustic products of the several gestures that have effects, among others, on Fa. Intonational peaks are local changes in an FO contour that are effected by means that, to a first approximation, only affect Fa; they are produced, largely, by contraction and relaxation of muscles that stretch or shorten the vocal folds ... In contrast, declination is a global change in FO that, excepting the initial peak in a sentence, tracks the decline in subglottal pressure ... Subglottal pressure affects not only FO, but amplitude as well, and several researchers have noticed that amplitude declines in parallel with Fa, and resets when FO resets at major syntactic boundaries (e.g. Breckenridge, 1977; Maeda, 1976). The parallel decline in amplitude and Fa constitutes information that pinpoints the mechanism behind the Fa decline - gradual lung deflation, incompletely offset by expiratory-muscle activity. That mechanism is distinct from the mechanism by which intonational peaks are produced. Evidence that listeners pull apart the two effects of Fa (Pierrehumbert, 1979; Silverman, 1987) suggests that they are sensitive to the distinct gestural sources of these effects on Fa. (Fowler and Rosenblum 1991: 48-9) The 'distinct gestural sources' of these various effects on Fa echoes the distinct gestural effects of the relatively discrete functional couplings of articulators that

104

BRAIN, MIND, AND THE SIGNIFYING BODY

give rise to vowels and consonants in eo-articulation. Thus, vowels are produced by slow movements that are finalized towards global configurations of the vocal tract; consonants are movements on a faster timescale that effect local restrictions of the vocal tract. It is the interplay of these two heterogeneous timescales which allows for the phenomenon of eo-articulation, Rather than a corrupted or degenerate encoding of static and abstract phonetic features, eo-articulation shows that the topological state space of the vocal tract is, in eo-articulation, divided into what Kauffman calls 'disjoint basins of attraction' (1993: 176-7). In this way, the greater number of degrees of freedom of the vowel-state functional couplings of articulators need be harnessed only initially such that only a smaller number of parameters need be harnessed subsequently. The more global, slower-scale movements required by vowels constitute the initial state of a trajectory to which the smaller, faster-scale movements of consonants flow. The distinct gestural sources of intonational peaks and declination of vowels and consonants suggest that the vocal tract is a topological substrate (c.f. Hjelmslev's expression purport) which is, in the space-time of articulation, organized in terms of complex interactions of attractors which lie on different trajectories. These attractors are the diverse principles of metafunctional organization to which the distinct gestural sources are attracted in the unfolding speech event. This would suggest that over the time-span of, say, a single breath group, different metafunctional parameters are held constant on some timescale(s) of gestural activity. These parameters are the values which organize, for example, the various effects on FO into metafunctionally distinct regions of gestural activity. For example, the local effects of the contraction and relaxation of muscles in the stretching and shortening of the vocal folds on intonational peak indexes feeling states of the speaker; the gradual lung deflation which is associated with decline in both amplitude and FO indexes the rhythmic periodicity associated with breath-group boundaries; the differing timescales required for the global versus local restrictions of the vocal tract may be associated with the categorical distinction between vowels and consonants as the parameter space is divided or 'analyzed' (Hjelmslev) into discrete regions corresponding to phoneme categories. These three examples would, on a first approximation, seem to suggest some of the ways in which listeners may distinguish the distinct gestural sources of various metafunctional values in the expression plane of speech. Thus, the first example relates to the expression stratum analogue of interpersonal meaning; the second to textual meaning; and the third to experiential meaning. We have here a concrete illustration of the functionally diverse ways in which meaning interacts with and entrains the biological dynamics of the individual's relations to its environment. Further, feelings and cognitions are not separate or autonomous functions. Rather, the metafunctionally diverse-yet-unified nature of meaning-making shows that semiosis - both 'inner' and'outer' - cross-couples the individual body-brain complex to its environment along a number of distinct dimensions simultaneously.

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

105

17. The Metafunctional Basis of Space and of Bodily Movement in Ecosocial Space-time All gestural activity unfolds in space. The notions of space and of spatial relations are crucial for our understanding of the ways in which the body relates to, orients to, and is integrated to its ecosocial environment. I believe that this is so both of gestural activity which is executed, as well as 'inner' gestural activity, of which inner speech is but one specific case. All modalities of human social meaning-making utilize, or in some way depend on, the movement - simulated or real - of bodily articulators. Thus, the receptors which are implicated in gestural activity coincide with those which take part in our sense of movement. These are (1) the visual receptors; (2) the vestibular receptors in the inner ear; (3) the muscular and the muscular-articulatory receptors, both of which measure movement of the various parts of the body; and (4) the cutaneous receptors of the skin which measure changes in pressure or temperature caused by contact either with the parts of the body or with phenomena in the (extracorporeal) external world (see Berthoz 1997: 33). Space is, then, a fundamental organizing principle which may well underpin the metafunctional basis of semiosis itself. There are typological-categorical spatial categories (e.g. in front, behind, up, down, and so on) which enable us to perceive as invariant certain relations between objects and the parts of our body. There are also topological-continuous relations such as spatial distance from the body, which are quantitative and variable rather than invariant and categorical. Kosslyn et al. (1995) propose that the two kinds of relations are coded differently in the neural architecture of the brain. But there is no direct or causal relation between these lower-scalar neural arrangements and higherscalar ecosocial semiotic ones. We can see in these two different kinds of relations a possible lower-level integrative constraint on the ways in which semiotic modalities are themselves organized. Categories such as 'in front', 'behind', and so on are digitalized units of information which specify the given object system relative to an observer perspective. In terms of the body's spatial relations to its environment, it defines the informational co-ordinates of the system appropriate to the scale of its interactions with its environment. Thus, the conceptual categorization of these spatial relations takes place on the level of the sensori-motor samplings the body makes of its surroundings. On the other hand, topological-continuous factors such as spatial distance raise two orders of questions. First, spatial distance raises the question as to which resources are best harnessed in meaning-making activity so as to overcome the problems - perceptual and physical - that distance poses for human interaction. Both visual and auditory resources for interaction are subject to constraints imposed by the material environment on their efficacy (see also Studdert-Kennedy and Lane 1980: 37). For example, visual gestures require close face-to-face contact between interactants as well as an optimal level of ambient light; acoustic signals decay with increasing distance. Secondly, spatial distance between bodies functions as a lower-level integrative constraint on social and interpersonal relations (cf. closeness or intimacy versus distance and remoteness). We may see here an analogue with the metafunctional

106

BRAIN, MIND, AND THE SIGNIFYING BODY

organization of semiotic modalities. Space and its metaphorical transformations would seem to be a fundamental constraint on human social meaning-making. What, then, is the relationship with the metafunctions? What relations in the ecosocial environment and therefore at a higher-scalar level than the individual body-brain favoured the further channelling and development of bodily movements and gestural activity as modalities of social meaning-making? It would seem to me that the neural architecture referred to above constitutes lower-level initiating conditions on the further transformations referred to above (see Salthe 1993: 216). With the emergence of higher-level ecosocial constraints, these possibilities for relating body and environment - self and non-self - incorporated more and more information of a social nature from the higher levels of ecosocial organization. This increasing specification and elaboration of the new emergent possibilities of the system for interacting with its con-specifics meant that determinate semiotic modalities with a full-fledged metafunctional basis arose. Researchers such as Griisser (1991) and Berthoz (1997: 108-9) have suggested that the space in which the body moves is articulated into a number of zones or subspaces in relation to the body. There is, according to these researchers, a personal space, an extra-personal space, and a distant space, to use the terminology proposed by Griisser. Furthermore, these distinctions appear to have a neural basis (Berthoz 1997: 108). Personal space, which is located within the limits of one's own body and perceived by the internal senses, is the space of the self. However, we can also perceive our own bodies as an external object as, for instance, when I look at my own body, or some part of it, as another external object in the world. The space of grasping - both oral and manual - means that the perception of one's body may be extended by means of a tool or implement. Berthoz (1997: 108) points out that in such cases the body is in effect extended such that the body is integrated with the physical object which it grasps or with which it comes into contact. An example would be the use of a pen, crayon, or other writing tool for the purposes of tracing on a treated surface. In this way, the body is extended by the tracing implement so that the body's contact with the treated surface paper, ete. - is felt at the point of contact - the tip - between pen, say, and surface. This shows how haptic sensations are fundamental in the integration of the body and its extra-somatic extensions with physical-material objects and processes which are external to the body. Importantly, Berthoz (1997: 109) draws attention to the fact that the tool so used is not perceived at the point of contact with the body but at the place it occupies in extra-corporeal space. The bodybrain complex thus constructs or projects a spatial extension of itself. Our perception of space is organized in two main ways. The first - called egocentric - is in relation to one's own body. That is, one's own body is the reference point in relation to which objects and their relations in the external world are observed. The second - called allocentric - uses the relations between objects or in relation to some reference point which is external to one's own body. This second way of perceiving space, which appears to be confined to humans and other primates, means that relations between objects can be constructed and perceived without recourse to one's own body as a permanent

RETHINKING THE EXPRESSION PLANE OF SEMIOSIS

107

reference point. In this way, allocentric perception is detached from the movements of my own body and allows for the internal mental simulation of spatial displacements which occur independently of my own body. The distinction between egocentric and allocentric perceptions of space may, in my view, underpin the elementary indexical distinction between 'here' (cf. egocentric) and 'there' (cf. allocentric). This may be compared to the distinction Halliday (1992a: 20-1) makes between the 'two primary modes of experience', viz. 'out there' and 'in here' in relation to which consciousness itself is organized. In early infant semiosis, this elementary semiotic distinction appears to be a precursor of the textual metafunction and its further elaborations and expansions. In this case, the basic distinction may be something like that between 'proximate' and 'distal' spatial relations. However, Halliday's distinction is also experientialized as the first, basic distinction between 'material' processes in the external world and 'conscious' processes that occur within the self. Interpersonally, this same distinction constitutes the basis for enacting interactions between an emergent 'self and 'non-self. To date, most of the effort has gone into the study of the signified or content stratum - e.g. the lexicogrammar and semantics oflanguage - of social meaningmaking. Further, phonology and the much less practised science of graphology have tended to remain separate subdisciplines which have not been integrated into a truly ecosocial theory of the body and its role in social meaning-making. Both phonology and graphology have categorical-typological resources such as phonemes and graphemes; they also have field-like prosodies which are strongly interpersonal-orientational, having to do with affective investment, emotional commitment, ete.; and they also have wave-like or periodic beginning-middleend type structures, boundaries between units, and so on. All of these interface with and construe bodily processes and/or extra-somatic extensions of these. They also project these into the ecosocial environment of the interaction, contributing to the construction of discursive space-time and its felt movement and rhythms.

3 Body Dynamics, Meaning-making, and Scale Heterogeneity: Expression and Content as Cross-scalar Semiotic Processes Embedding the Body-brain in its Ecosocial Environment Language, like personality, is a binder of time, of the past and future in 'the present'. J. R. Firth, 'The semantics oflinguistic science' (1957a [1948]: 142)

1. The Dynamical Character of Expression and Content and the Crosscoupling of Diverse Scalar Levels of Semiotic Organization Action and meaning are emergent properties of the time-dependent crosscoupling of a number of different systems on diverse space-time scales. Meaning emerges in and through the interaction of diverse semiotic modalities and the physical-material world. In the real-time of the unfolding activity, various semiotic modalities: (l) selectively map salient features of the material world to their own activity; (2) at the same time, they map selected features of other semiotic modalities to their own dynamical processes; and (3) they map selected features of participants' perceptual-motor activities to their own activity. It is the dynamical cross-coupling in time of a number of these heterogeneous systems that produces the meaning-making event. These systems include the various semiotic resources that are deployed; objects, events, and so on, in the material world; and the perceptual-motor activities of participants. Action emerges in and through the cross-coupling and interaction of diverse semiotic-discursive and physical-material systems. Genre-specific meaning-making activities occur because the cross-coupling dynamics of all these systems create, in time, an internalized attractor space. It is in this space that participants' experiences prior to the here-and-now event, the previous stages of the same unfolding event, and here-and-now responses to perceptual stimuli at any given moment all act in synergy to produce genre-specific semiotic performances and their resulting object-texts. Meaning-making is, then, a dynamic process which is determined by a number of perspectives, all of which interact together to produce the occasionspecific activity. First, there is the perspective of what it is possible to mean in terms of the intrinsic characteristics and dynamics of any given semiotic resource

BODY DYNAMICS, MEANING-MAKING, . . .

109

system. Secondly, there is the sequential unfolding of events in real-time, the logogenetic selecting and deployment of semiotic options and their crosscoupling with specific external events, as well as changes in cross-coupling strengths in moments of transition from one phase of the activity to another. Thirdly, there are the sensori-motor reactions of participants to what they see, hear, feel, touch, grasp, point to, and so on, in their spatio-temporal purview at any given moment. With these general considerations in mind, this chapter will focus on the question of how the organization of language as expression and content functions to embed the body-brain in its ecosocial and bodily environments. As we shall see below, this requires us to examine the diverse timescales and their cross-couplings that are implicated in the intrinsic organization of both expression and content.

2. The Expression Stratum and the Principle of Alternation Lernke's (1999) Principle of Alternation is a useful tool for theorizing, in terms of the three-level scalar hierarchy, both how semiotic functions are mapped onto dynamical scalar levels, and the reorganization of continuous variation into discrete variants. In the act of speaking, for instance, continuous quantitative variation in sound is related to continuous quantitative variation in the sensorimotor activities which produce speech sounds in the process of articulation. However, the processes of both producing and comprehending speech sounds require that speakers and listeners are able to articulate and perceive those features which are salient or criterial for determining which words are uttered and/or heard on a given occasion. In other words, the continuous topological variety of the sounds produced at level L-I of the perceptual-motor activities involved in the production and perceptual pick-up of speech sounds is reorganized and interpreted as discrete phonological categories, which are typological in character, at level L+I. In the case of speech sounds, this level is the phonological system of a given language. In Table 3.1, I suggest some ways in which the expression stratum of spoken language can be reconceptualized in terms of the Principle of Alternation. In accordance with this Principle, the emergence of a ranked scale of phonological units and their structural-functional relations in a given language constitutes a new intermediate level N which functions semiotically to reorganize the continuous quantitative (topological) variety of units and interactions at the level (L-I) of sensori-motor activity in articulation as discrete, typologicalcategorial phonological units and relations for level (L+I). Level L-I interactions are the material, topological ground of the typological distinctions which emerge at higher levels. Level L+1 comprises the system of phonologically salient distinctions which function in a given language to symbolically construe differences in lexicogrammatical form on the content stratum. These distinctions develop and are conserved in the long-term memory of individuals as a system of phonological values or differences on the basis of the material and semiotic interactions of individuals with their ecosocial environ-

110

BRAIN, MIND, AND THE SIGNIFYING BODY

Table 3.1: The expression stratum of spoken language in relation to the three-level hierarchy

Scalar level

Difference

L+l

System of phonological differences as SI

L

Phonological rank scale comprising hierarchy of ranked units: tone group; foot; syllable; phoneme

L-l

Continuous quantitative variation in sound mapped onto continuous quantitative variation in sensori-motor activity in articulation

Discrete typologicalcategorial distinctions; difference of type

Topologicalcontinuous variation; difference of degree

Symbolic

Phonologically salient distinctions symbolically construe lexicogrammatical forms on content stratum

Indexical

Contrasts in rhythm, prosody, intonation perform indexical grounding functions which give phonological categories a point of reference in the here-now intraand inter-body dynamics of the I-you and other aspects of the here-now speech event

Iconic

Continuous quantitative variation of body dynamics

BODY DYNAMICS, MEANING-MAKING, ...

III

ment along their historical-biographical trajectory. In this way, the phonological possibilities for meaning-making are developed in individuals at the same time as they are transmitted to other individuals in and through ontogenetic and other processes of recruitment to and apprenticeship in the practices of the social group (Vihman et al. 1985; Vihman and de Boysson-Bardies 1994). The phonologically salient distinctions made on the expression stratum are not restricted to the organismic scale of the individual; they also link with other, larger-scale dimensions well beyond the here-now scale of the organism's material interactions with its immediate environment. Semiotically salient distinctions made on the expression stratum interface with kinetic (bodily) processes on level L-l. In this way, bodily dynamics are strongly cross-coupled to dynamical processes existing on potentially many different space-time scales. The phonological system of a given language is a system of interpretance whereby the sounds we produce and hear are classified in terms of a relatively small number of discrete and contrasting equivalence class items. These class items are, for example, the phoneme type-categories of a given phonological system. In this case, level L is that of the phonological forms, usually described as a set of hierarchically related units in a phonological rank scale (see below). The L level has the function of filtering phonologically non-criterial characteristics of speech sounds. Non-criterial characteristics include aspects of the sound which relate to the subjective state of the speaker, the specific circumstances in which the sound was uttered and/or heard, the differences between the speaking voices of individuals, and so on. All of these aspects are non-criterial for the assignment of the sound to a given phonological category. Thus, the phonological contents of symbolic neural space constrain and entrain both vocal-tract gestural activity and our perception of the resulting sounds in terms of a system of phonological distinctions (cf. values) on the higher-scalar level of the SI which is in operation. Thus, physical-acoustic properties of the sound such as variation in fundamental pitch, which is iconically related to the rate of vibration of the vocal chords at the time of speaking, may not be in any way salient or criterial for the categorization of a given sound as belonging to this or that phonological equivalence class (see below). Such non-criterial acoustic-physical properties and their related articulatory processes occur on a much smaller, faster timescale than do higher-scalar phonological units. The learning, producing, and perceiving of the latter by the central and peripheral nervous systems take place over a much longer timescale. Moreover, a system of phonological units and their relations requires a far greater number of interconnected neuronal networks in order to maintain it in long-term memory, as well as to activate it in the production and perception of speech sounds. Level L can be modelled as a hierarchy of vocal-tract trajectories that entrain level L-I dynamics so as to give rise to three different kinds of phonological units. The notion of trajectory serves to remind us that we are not talking about static structural units and relations, but about time-bound trajectories on different scalar levels that are identified with the phonological rank scale, comprising, in the case of English, syllable, foot, and tone group. Importantly, each of these phonological ranks and the units specific to that rank are seen as

112

BRAIN, MIND, AND THE SIGNIFYING BODY

dynamic, time-bound processes of cyclic or periodic vocal-tract behaviour with their own specific rates of completion. Thus, smaller-scale trajectories operate on faster timescales than do higher-scalar ones. Furthermore, the notion of a vocaltract trajectory refers to the temporal cohesion of a given kind of phonological entity (e.g. syllable, foot, tone group) from the perspective of a cogent moment which spans the entire (temporal) existence of the trajectory. Syllable, foot, and tone group refer to such cogent moments of different cycles of vocal-tract activity on their respective scales. Each of these cogent moments specifies the beginning of the trajectory in lower-scalar neuromuscular activity on level L-I and terminates with the completion of a cycle of activity which results in the entraining of vocaltract activity to a recognizable phonological entity in the given language. The neuromuscular substrate on level L-l is the dissipative structure which is the momentary material embodiment of the trajectory (Salthe 1993: 260). However, it is the vocal-tract trajectory on the phonological level L, rather than on the material level (L-l) of its embodiment, which semiotically connects us to the trajectories of others, intertwining and overlapping with them in the processes of logogenesis. The three phonological ranks of syllable, foot, and tone group exhibit this property of temporal existence. On the other hand, the phoneme is a discrete packet of paradigmatic features which serves to specify a particular point in the unfolding phase space of a trajectory by categorizing that point as the instantiation of that particular packet of paradigmatic features, i.e. as an instance of this or that phoneme type-category. The phoneme is, as Cleirigh (1998: 41) argues, an expression of the probability of the speaker or listener locating that particular packet of features at that point. Cleirigh (1998: 38-9) has defined the three phonological ranking units named above as periodic cycles of vocal-tract activity in ways that fit the above definition of trajectory. Thus, the first (lowest) scalar cycle is defined as a cycle of opening and closing the vocal tract 'to dampen phonation in the production of syllables' (Cleirigh 1998: 38). The syllabic trajectory consists in the temporal oscillation between the assignment of heavy and light syllabic weight. This gives rise to a structure consisting of Onset and Rhyme. The units comprising the Rhyme are potentially moraic, i.e. they have syllabic weight (moraicity) while the Onset consists of units that do not have syllabic weight. The latter are, therefore, non-moraic. In the discussion which follows, the clause complex I bought this VJ and I sailed it will serve as the focus of our illustrative discussion. With reference to the syllable sail, Figure 3.1 shows the assignment of syllabic weight. Thus, the constituent comprising the Onset is non-moraic, while the constituent comprising the Rhyme is potentially moraic and can therefore function as a timing unit on the next-highest level, viz. the foot. Figure 3.1 shows the alternation of non-moraic and moraic elements in the unfolding temporal trajectory of the syllable sail from the word sailed in our example. The next-highest scalar level is the trajectory of vocal-tract activity that produces rhythm. The rhythmic trajectory is formed, in stress-timed languages like English, by the alternation of salient and non-salient (weak) syllables. Salient syllables are so defined by virtue of the fact that they are assigned stress through varying topological factors such as loudness and duration. The temporal trajectory which arises on this level is known as the foot. Thus, a foot is a

BODY DYNAMICS, MEANING-MAKING, . . .

113

Figure 3.1: Syllable structure of'sail', showing morale and non-moraic elements

trajectory of vocal-tract activity which, in English, results from the periodic (cyclic) assignment of salient and weak stress to syllables such that each cycle each foot - consists of one stressed syllable and an unequal number of unstressed ones. M / bought this / Vee Jay / A and / sailed it In the above example, the feet are marked off by slashes; the syllable immediately following the slash is the salient syllable, or the Ictus. The carat sign ' A > indicates a silent Ictus. Figure 3.2 shows the wave trajectory of the alternation of stressed and unstressed syllables as the syntagm unfolds in time. The highest scalar trajectory that I shall consider here is that which is concerned with the creation of tonic prominence or tonicity through changes in major pitch. This trajectory is concerned with intonation. The intonation trajectory consists of a cycle which alternates between 'switching tonicity on and off, as Cleirigh (1998: 39) expresses it. In our example, the intonation cycle is instantiated as follows: //i I bought this / Veejay //i and / sailed it // The wave trajectory of tonicity in this example is shown in Figure 3.3. In this way, we see phonology as consisting of time-bound trajectories on different scalar levels. Each scale has its own characteristic temporal cycle or rate of completion and its own characteristic entities. Furthermore, the trajectories on their

114

BRAIN, MIND, AND THE SIGNIFYING BODY

Figure 3.2: Trajectory of wave of stressed and unstressed syallables in clause complex, showing alternation of stressed and unstressed syallables as syntagm unfolds in time

Figure 3.3: Trajectory oftonicity wave in clause complex

respective scalar levels are integrated to each other in the production of speech sounds. This integration of diverse scalar levels leads to the production of an increasingly complex speech signal in which the higher level of generality characteristic of moraicity is reintegrated to the increasingly more specified levels of stress and tonicity. This process of integration across scalar levels increases the amount of information that is stored in the trajectory and hence the agentive determinability of the trajectory (Salthe 1993: 261). It is not difficult to see here how the further integration of expression-stratum trajectories to content-stratum ones amounts to a further increase in the semiotic complexity of the trajectory and therefore of the amount of information-meaning that is stored along the duration of the trajectory. It is the content stratum which accesses stored meaning on the ecosocial scale and which entrains lower-scalar expressionstratum trajectories to its own higher-scalar dynamics. The emergence of the phonological rank scale on level L therefore provides stable parameters such that particular vocal-tract trajectories qua phonological types and their structures persist on many different space-time scales and across many different users (speakers and listeners). The cyclic character of each

BODY DYNAMICS, MEANING-MAKING, . . .

115

trajectory, which Cleirigh has identified, draws attention, in my view, to the selforganizing properties of the neuromuscular substrate. This is possible because each scalar level has stored information that is specific to its level as well as a characteristic cycle of activity which defines its cogent moment. For example, the syllabic trajectory both stores information about syllable weight and, at the same time, has a temporal existence definable as states in its overall phase space, viz. opening --+ closing the vocal tract. Furthermore, we shall see below how a small number of principles of formal organization are fractally distributed across trajectories of both expression- and content-stratum activity. The properties of the filtering or buffering system - i.e. the hierarchy of ranking phonological units at level L - is comprised of units which may have symbolic and/or indexical properties and functions on the expression stratum of speech. On level L+1, a given phonological SI filters and, therefore, interprets the topological-continuous variation of speech sounds as phonologically salient indexical or symbolic differences. Differences of this kind are typologicalcategorial in nature. Indexical functions are signalled by salient contrasts in, for example, intonation, speech rhythm, and other prosodic dimensions of the sound stream. These indexical features of the expression stratum are necessarily grounded in the here-now scale of the speech event. They function to draw the listener's attention to some salient aspect of that event either by presupposing its existence in the speech event or by creatively bringing it into existence through the use of the particular indexical sign. (The distinction between the indexically presupposing and indexically creative functions of indexical signs in language can be found in Silverstein 1976.) In the case of indexicals, sensori-motor activity functions to index some semiotically salient dimension of the ongoing speech event by virtue of the perceivable relation of co-occurrence between the indexical sign - intonational or rhythmic contrast - and the indexed object. For example, phonological systems and structures may index a given lexicogrammatical unit on the content stratum by means of prosodic resources which extend over a given lexicogrammatical domain. Alternatively, a phonological syntagm may demarcate a lexicogrammatical unit, for example when syllables indicate morpheme and word boundaries (Cleirigh 1998: 24-5; Firth 1957b [1948]: 122-3). Furthermore, phonological units such as the prosodic spread or extension of tone over a given lexicogrammatical unit may index that unit as having informational salience in the unfolding discourse. Rhythmic organization on the lower rank of the foot can also be used to render specific lexical items prominent as well as to demarcate lexicogrammatical boundaries. Phonologically indexed aspects of lexicogrammatical and discourse organization also include the 'communicative intentions' of speakers. For example, Gumperz, in Prevignano and Di Luzio (2003: 10-14), argues that communicative intentions are inferable on the basis of indexical signs or contextualization cues of this sort. The intentions so inferred do not, in my view, belong to the domain of internalized cognitive processes per se. I do not assume that the index is the external sign of an internal reality beneath the skin of the individual. Instead, I would say that this indexing of communicative intentions is just one dimension of the ways in which meaning flows into and entrains a particular logogenetic

116

BRAIN, MIND, AND THE SIGNIFYING BODY

trajectory on the here-now scale in which the event is enacted and unfolds for its participants. Thus, Peirce's understanding of indexical signs is upheld here. That is, indexical signs are sensori-motor activities that point to the existence of some 'object' on the here-now scale. Such objects may be grounded by the index either 'subjectively' or 'objectively' - as existing in the field of the speaker's body, including the inner domain of thought and feeling; or as existing in the field of that which is external to the speaker's body, Le. in the external objects and events which are part of the context of situation. The indexing of lexicogrammatical domains by phonological units such as rhythm and tone already shows the principle of scalar heterogeneity at work. This is so in the sense that lower-scalar phonological systems and structures on the expression stratum integrate with the higher-scalar lexicogrammatical and discourse levels of organization on the content stratum. The use of prosodic resources such as rhythm and syllable stress may serve to index semantic relationships which are not made explicit in the lexicogrammar. That is, the particular phonological feature may index a discourse-level semantic pattern which the listener is required to retrieve from his or her knowledge of specific intertextual patterns in order to understand the particular semantic relation. Semantic meaning relations are made in discourse through particular patterned combinations of lexicogrammatical selections in texts. There is no one-to-one correspondence between lexicogrammatical forms and the semantic meanings they are used to make in discourse. Moreover, lexicogrammatical forms do not always spell out the full semantic pattern in an explicit way. Prosodic and other phonological cues often provide subtle cues as to how to construe the relevant semantic pattern in discourse. This shows how expressionstratum features may index intertextually recoverable semantic patterns which are necessary and relevant for the interpretation of the particular lexicogrammatical selections that are made. With this in mind, we can say that speakers' intentions are often shown to be analysable as an indexical relationship between prosodic or other cue and an implicit semantic pattern on the discourse level rather than an internal cognitive state of the speaker per se.

3. Stratification in Relation to Expression and Content The notion of the expression stratum that I am using here is a further development of Hjelmslev's (1961 [1943]: 47-60) theory of the stratal organization of the linguistic sign, seen as comprising the two strata of content and expression. Furthermore, both content and expression are internally stratified. Thus, the expression stratum comprises the two levels of expression form (phonology) and expression substance (phonetics). Likewise, the content stratum is internally stratified as content form (lexicogrammar) and content substance (discourse). The relationship between the strata of content and expression is one of realization. That is, content is realized by expression and expression realizes content. Realization is a two-way relationship between these two strata. It is a symbolic relationship. That is, expression symbolizes content and content is symbolized by expression.

BODY DYNAMICS, MEANING-MAKING, . . .

117

In systemic-functional theory, the content stratum is internally stratified into discourse and lexicogrammar (Martin 1992a: 14-21). Discourse, which is linguistically realized social action and text, is realized by co-patternings of lexicogrammatical selections. Discourse is a higher-level reconstrual of lexicogrammatical patterns so as to realize thematic patterns, interactive modes, and cohesive relations in texts. Following Halliday (1978a: 108-9), text is defined as either spoken or written instances of language-in-use. Discourse and lexicogrammar belong to the content stratum because they are based on various types of semantic-pragmatic meaning and social (inter)action. I take a view similar to that originally espoused by Firth (1957c [1951]: 220), viz. meaning cannot be defined in terms of any single stratum: Language text must be attributed to participants in some context of situation in order that its modes of meaning may be stated as a series of levels, which taken together form a sort of linguistic spectrum. In this 'spectrum' the meaning of the whole event is dispersed and dealt with by a hierarchy of linguistic techniques descending from social contextualization to phonology. (Firth 1957c [1951]: 220) In some respects, the terms expression and content are unfortunate ones for they may suggest the idea of a material means of expressing meaning, seen exclusively in terms of content. In my view, such a conception reflects a fundamental misunderstanding of Hjelmslev's notions (see chapter 2, section 1, pp. 59-67). The stratal organization oflanguage into expression and content reflects diverse principles of semiotic organization on different levels. Meaning is made through the contribution of all the levels involved in the logogenetic process of producing text or discourse. In the case oflanguage, this means that the phonological or graphological, lexicogrammatical, and discourse levels of organization all simultaneously make their distinctive contributions to the overall meaningmaking event. The expression stratum is based on sensori-motor activity. An example is the vocal-tract activity whereby speech sounds are articulated. Expression is internally stratified in terms of phonetics and phonology. I shall use the term phonetics to designate language-specific classifications of articulatory repertoires independently of their linguistic function. Phonology, on the other hand, refers to the higher-level organization of such repertoires into language-specific functional systems and structures. As we shall see below, this means that phonological units and relations, in the logogenetic process of creating discourse or text, are both integrated to and constrained by higher-order discourse and lexicogrammatical patterns of organization on the content stratum. Moreover, phonology is not simply a vehicle whereby such content is 'carried' or 'conveyed'. Instead, it is a full-fledged level of linguistic organization whose functional structures and systems make their own specific contribution to the meaning and organization of the discourse event. This contribution will be discussed in the following section in terms of the iconic, indexical and symbolic functions of phonological forms in the meaning-making process. The phenomena of experience that are construed on the content stratum are

118

BRAIN, MIND, AND THE SIGNIFYING BODY

not perceived. In listening to speech sounds or visually scanning a sequence of graphemes on a treated surface, what is acoustically or visually perceived are not the phenomena of experience that are construed by the content stratum. By the same token, in listening to or in reading say a sequence of graphemes, we do experience the phenomena that are construed by the lexicogrammatical- and discourse-level units and relations. However, such phenomena, unlike expression-stratum units and relations, are not present in the immediate environment as acoustic or light energy which structures the perceiver's point of observation. The acoustic or light energy which structures the perceiver's point of observation plays a causal role in acting on the perceptual systems of the observer. However, the flow of symbolic experience which is also apprehended in the sense of what the discourse is about does not causally act upon the perceiver in the way acoustic and light energy does. In the process of articulation, for example, vocal-tract activity physically modifies the acoustic stream in such a way that the sequence of perceived phonological invariants redounds with a concomitant flow of experienced phenomena such that, in addition to hearing the sounds produced by the speaker, symbolically construed phenomena of experience are also apprehended. The stratification ofianguage into expression and content means that, when one attends to the acoustic stream of speech sounds, one attends to two sets of informational-semiotic invariants: (I) a set of phonological invariants that is specific to the stream of the speech sounds; and (2) a set of invariants that is specific to the semantic categories (persons, places, actions, events, and so on) that are construed by the lexicogrammatical and discourse units and relations. The first involves perceptual pick-up; the second involves the resonance of what is picked up with symbolic categories of experience that cannot be perceived. Perceiving the acoustic stream dually involves a symbolic experience of phenomena as a result of the ways in which the acoustic flux structures the perceiver's point of observation according to the principles outlined in section 2 so as to be recategorized as symbolic content.

4. The Integration of Iconic, Indexical, and Symbolic Modes of Meaning in Phonology The expression stratum is itself a cross-scalar phenomenon which simultaneously links multiple space-time scales. These diverse scales can be described in terms of a nested specification hierarchy which consists of iconic, indexical, and symbolic modes of meaning-making. 4.1 The Iconic Mode ofPhonological Organization Following Salthe's (1993) notion of the specification hierarchy (see also Lemke 1995b: 113, 2000a; Thibault 2000a: 301-3, 2003b), the iconic dimension of speech sounds is the least specified, innermost level of such a nested hierarchy. Many phonologically non-salient features of the stream of speech sounds such as the quality of the speaker's voice tell us that the speaker is sad, anxious, ill, and so on. In this case, we are dealing with continuous quantitative variation in the sound stream which directly - iconically - maps onto continuous quantitative

BODY DYNAMICS, MEANING-MAKING, . . .

119

variation in the speaker's neurophysiological dynamics. That is, they are iconic signs of body states and feelings. They can be interpreted as signs of affective and other subjective conditions of the speaker. These iconic signs are confined to the immediate here-now scale. For this reason, they closely relate to the topological material ground of our being in the world. However, even at this level, scalar heterogeneity is evident on account of the fact that these iconic signs extend beyond the organismic level and are able to be interpreted as such by others. 4.2 The Indexical Mode ofPhonological Organization Phonological units and relations on level L also have indexical functions. This means that the system of phonological possibilities - its categorial distinctions is also grounded in the here-now act of speaking and listening qua sensori-motor activity. Indexical properties add further levels of complexity and specification, along with the new semiotic functions of these properties. The innermost iconic level is integrated to and modified by the newly emergent indexical level. This does not mean that the iconic level is transcended; rather, the indexical level entails the emergence of typological differentiation from topological variation on the basis of the increasing semiotic differentiation of vocal-tract gestural activity. In this way, vocal-tract activity is used to intentionally 'point to' contextual values beyond the body-scale. Many 'non-linguistic' contrasts in the speaking voice such as loudness may have such indexical properties and functions. Thus, contrasts in rhythm, intonation, and prosody can function indexically to give phonological categories a point of reference in the act of speaking by specifying, for example, its interpersonal or textual significance. For example, in English, the assignment of pitch prominence to a given syllable in discourse is an independent variable which interacts with, though it does not define, the phonological type-category which the particular syllable instantiates. Rather, the speaker's choice of pitch prominence indexically grounds the syllable in terms of factors such as its interpersonal-interactional or informational relevance in the here-now of the speaking event. However, it is not salient for the purposes of distinguishing, say, the phoneme categories /b/ and /p/ whereby the lexicogrammatical distinction between the words bat and pat is symbolized. Contrasts in rhythm, tone choice, pitch prominence, and so on, are potentially indexical because they can point to or indicate contextually relevant values which are closely tied to the immediate here-now scale of the act of speaking. That is, they are interpretable as indexes of values which are recoverable from the perceptual purview shared by the participants in the event. Phonological patterns also interact with lexicogrammatical patterns on the content stratum in ways that serve indexical functions of creating texture as well as linking to aspects of organization on the higher-level lexicogrammatical stratum. In this regard, Cleirigh (1998: 57) points out that systems of phonological cohesion may function to index 'the syntagmatic extent of lexicogrammatical units'. Following Firth (1957b [1948]), Cleirigh identifies three such strategies, demarcation, integration, and concatenation, as follows:

Demarcation is the phonological delimiting of a lexicogrammatical boundary: initiality or finality. Integration is the phonological consolidation

120

BRAIN, MIND, AND THE SIGNIFYING BODY

of a lexicogrammatical domain as a single unit. Concatenation is the phonological annexation of two or more lexicogrammatical units within a larger (lexicogrammatical) structure. (Cleirigh 1998: 57) All three strategies can be seen as having both intra- and extra-phonological indexical functions. For example, syllable boundary positions can create (1) endophoric ties that link either backwards or forwards to previous or following syllable peaks or (2) exophoric links to lexicogrammatical units that lie outside phonology on the lexicogrammatical stratum. Thus, we can see here how the creation of 'phonological texture' (Cleirigh 1998) functions to punctuate or digitalize the topological flow of speech sounds into discrete events separated by boundaries. Such events have semiotic salience because they are interpretable according to social conventions which enable their interactional significance to be recognized. The phonological resources referred to here, in punctuating the temporal flow, constitute differences that make a difference to that flow. They are indexical resources which mark out a piece of the temporal flow of articulated sounds as a lexicogrammatical event. In a given discourse context, such events can be interpreted by the parties to the interaction as having a certain meaning within the discourse. Indexical functions like these thus contribute to the perception of the sound stream as an event, or a series of nested and interrelated events on various scales, rather than mere physical sensations (see Hande11989: 183).

4.3 The Symbolic Mode ofPhonological Organization The symbolic dimension of the meaning of phonological forms is not the only function they have in the temporal unfolding of speech events. The symbolic possibilities that such phonological distinctions have for construing or symbolizing lexicogrammatical distinctions on the content stratum means that the sounds produced in the act of speaking can be linked to the meaning-making possibilities of the lexicogrammatical and discourse levels of organization. Expression itself is linked to many different space-time scales that go beyond the here-now scale in which speech sounds are articulated and heard. The most specified, outermost level adds further layers of symbolic complexity in the form of a full-fledged system of phonological contrasts which cross-link to the content stratum of lexicogrammar and its semantics. Once again, the iconic and indexical levels are not transcended, but are integrated to and consequently modified by the symbolic level. This is evidenced by the fact that phonological units and relations have both indexical and symbolic functions in discourse.

5. Rhythm and the Foot The creation of a phonogenetic field in the dynamic processes of articulation is asymmetric in time (Cleirigh 1998: 53). For this reason, it is irreversible. In terms of biological initiating conditions emanating from level L-l, this is explainable by the fact that speech 'primarily exploits egressive airstreams' (Cleirigh 1998:

BODY DYNAMICS, MEANING-MAKING, . . .

121

53). The initiating and terminating boundaries of a syllable on level L cannot be reversed because the principles which relate the boundary conditions of the syllable to its internal organization are irreversible. The initiation of a syllable cannot be reversed to become the terminating position for each has consequences for the instantiation of paradigmatic features in the asymmetric temporal flow of articulation (Cleirigh 1998: 53). At the same time, the directionality of the phonogenetic field of articulation is constrained by boundary conditions operating on level L+1 of some phonological SI. Vocal-tract activity is integrated to and entrained by this level such that particular phonological units and relations are recognizable. Semiotically, the field perspective refers to the phase space, the organized field, which affords the emergence of the wave and particle perspectives. Waves and particles are organizations of quantum field processes (Bickard and Campbell 2000: 331). The vocal tract is a topologically organized continuum in which vocal-tract gestural activity takes place. It is the material ground in which this activity takes place. As such, the topological-continuous space of the vocal tract is a phase space or an organized field in which certain kinds of organized process perturb this field so as to give rise to various principles of organization. These diverse principles interact during the temporal unfolding of vocal-tract articulatory activity. Meaning-making is a probabilistic system for construing and acting on the phenomena of experience. Cleirigh (1998: 41) has shown how the Copenhagen interpretation of quantum mechanics can be applied to phonology, interpreted as a wave/particle complementarity in terms of probability. The application of this interpretation to phonology 'means interpreting waves of moraicity, stress and tonicity as measures of the probability of the observer - speaker or hearer finding the respective syllable, foot or tone group particle at that particular point' (Cleirigh 1998: 41). For example, the syllables comprising the Ictus and Remiss phases of the foot are comprised of clusters of vocoids and contoids. Abercrombie (1967: 85) points out that vocoids and contoids are phonetic segments which occur in particular articulatory environments. These entail a good deal of phonetic variation that is not phonologically salient and which is consequently filtered at the phonological level L+1 where phonetic segments are construed as instantiations of particular phonemes. Such bundles of features correspond to the phoneme type-categories of a given phonological system. Phonemes specify phonological type-classes of consonants and vowels which are specific to a given language system. Phonemes categorize discrete regions of the articulatory flux as particles. A particle corresponds to a point in the unfolding wave to which a specific packet of paradigmatic features is assigned such that the region of the wave movement corresponding to the particle is construed as an instance of a particular phoneme type-category. Thus, phonemes paradigmatically specify a given region of the flow of articulation - i.e. the muscular movements involved - as invariant irrespective of variations in pitch, loudness, and duration (Gibson 1983 [1966]: 93). As Gibson points out, 'the stimulus information for detecting them [phonemes, P]T] is invariant under transformations of frequency, intensity, and time' (1966: 93). Phonemes therefore construe articulatory

122

BRAIN, MIND, AND THE SIGNIFYING BODY

processes corresponding to paradigmatic bundles of distinctive features which are perceived as invariant under change. An analogous and parallel process occurs on the content stratum. Thus, experiential particles such as processes and participants in the clause, or Thing and Event in the nominal and verbal groups, respectively, symbolically re-construe phonological particles on various scalar levels as discrete experiential semantic categories such as Actor, Material Process, Goal, and so on. These experiential particles are invariant under topological transformations of the stimulus flux. In turn, these particles of content construe the continuous flux of the phenomena of experience as instantiations of paradigmatic categories of content. In the following section, I shall focus on the metafunctional organization of the foot in English phonology with a view to investigating how the principles outlined above can be applied to the metafunctional organization of phonological form.

6. The Metafunctional Organization of Phonology as seen from the Perspective of Rhythm and the Foot The syntagmatic structure of the foot is comprised of the two elements Ictus and Remiss. These two elements constitute two phases in the rhythmic element called the foot in the phonological rank scale. Ictus and Remiss are themselves comprised of syllables deriving from the next-lower rank in the phonological rank scale. The foot hierarchically dominates a given number of syllables in a sequence and imposes varying degrees of stress on these syllables, 'according to their location in the foot' (Catford 1998: 345). Syllables produced with greatest intensity and duration are described as [salient]; syllables with less intensity and duration are described as [weak]. The foot is the locus of rhythmic organization in the phonology of English. Rhythm can be viewed from four metafunctional perspectives, all of which constitute simultaneous and overlapping dimensions of the organization and meaning of the foot. The four different modes of phonological realization iconically map onto the four different ways in which the expression stratum entrains and construes vocal-tract dynamics in the act of speaking. The relevant environment of the expression stratum is the body. The iconic dimension of phonology is the least specified, innermost level of a nested hierarchy of iconic, indexical, and symbolic modes of entraining and construing vocal-tract dynamics. 6.1 The Textual Metafunctiunal Textual structure is periodic; it is not based on constituent structure. Periodicity is a dynamic, time-bound process. Rhythm can be viewed as a perturbation in the form of a wave-front which is propagated through an articulatory field. The periodic nature of textual meaning shows rhythm as a periodic sequencing of strong and weaker elements, seen as alternating phases of Ictus and Remiss. Ictus and Remiss are alternating phases in the dynamic unfolding in time of a rhythmic field (Cleirigh 1998: 80). English is a language of the kind identified

BODY DYNAMICS, MEANING-MAKING, . . .

123

by Abercrombie (1967: 97) as having stress-timed rhythm in contrast to syllabletimed languages such as Italian. In the latter, periodic rhythmic movement is produced by the syllable-producing process: 'the chest pulses, and hence the syllables, occur at regular intervals' (Abercrombie 1967: 97). In languages having stress-timed rhythm, periodic rhythmic movement is produced by the assignment of stress. Figure 3.4 shows the organization of the rhythmic field of the two-syllable word sailed in terms of the Ictus and Remiss phases.

Figure 3.4: Rhythmic field of the word sailed, showing Ictus and Remiss phases of its temporal trajectory The foot is the parcelling-up of stronger and weaker syllabic elements into a higher-order unit. The periodic, wave-like perspective allows rhythm to be seen as a dynamic, undulating movement of a particle from a strongly stressed element to a more weakly stressed one (Cleirigh 1998: 80). These elements are the boundaries of the temporal whole which is constituted by the foot. In this way, rhythm structures the temporal dimension of events. The foot may be seen as the phonological 'atom' out of which larger-scale rhythmic structures are assembled on the expression stratum of speaking. From the perspective of the textual metafunction, as Cleirigh observes, rhythm is 'a process of periodically switching on stress' (1998: 81). In this sense, rhythm is viewed as a perturbation of a rhythmic field which gives rise to alternating Ictus and Remiss phases (Cleirigh 1998: 344). 6.2 The Logical Metafunction Rhythm, Cleirigh (1998: 345-6) further points out, can be viewed from the point of view of the logical metafunction as a form of hypotactic interdependency in which the Remiss is dependent on the Ictus. The Ictus is the dominant or a phase of the foot; the Remiss is the dependent or (3 phase. Rhythm, as Cleirigh (1998: 81-2) has pointed out, can therefore be viewed as modification. That is, rhythm is 'the modification of articulation by initiator power' (Cleirigh 1998: 81). Stronger or weaker stressed syllables impact on articulation in ways which affect factors such as aspiration, posture and degree of closure (Cleirigh 1998: 81). In this perspective, we are concerned with the ways in which (articulatory)

124

BRAIN, MIND, AND THE SIGNIFYING BODY

phenomena relate to each other, rather than with how the different parts constitute some larger whole. For example, we are not concerned with the way in which Ictus and Remiss together make up a larger whole - the foot - but with the relationship of interdependency between these two units. Dependency relations are relations between units such that a Remiss can be dependent on an Ictus and hence can be modified by it, as shown below in relation to the word never in Figure 3.5.

Figure 3.5: Dependency relations between Ictus and Remiss phases in hever' 6.3 The Interpersonal Metafunction Interpersonally, rhythm can be seen as the creation of a charged vibratory field which becomes operational when a listener is within range of this field and able to perceptually pick up its effects. Abercrombie (1967: 97) has referred to the experience of speech rhythm as a 'rhythm of movement' which both speaker and listener feel and which affords what Abercrombie has called 'phonetic empathy'. The 'phonetic empathy' referred to by Abercrombie is founded on the joint perception by speaker and listener of the periodic movement of the muscles of the breathing apparatus. Thus, rhythm can give rise to a jointly perceived interpersonal charge which has a range of potential emotional-affective values depending on articulatory variables such as rate of tempo and degree of syncopation (Handel 1989: 458). The point I wish to emphasize here is that interactants may physically feel the bodily enactment of interpersonal meanings which may not necessarily correlate with any specific item on the content stratum (see Cowley 1998: 565-6 for a similar observation regarding pitch fluctuation). 6.4 The Experiential Metafunction Experiential structure is particulate. In this perspective, the foot is a particle interpreted as the peak of a stress wave. As Cleirigh (1998: 42) points out, the peak of the stress wave represents the most probable location of the foot in the unfolding rhythmic syntagm. That is, the peak of the stress wave represents an increase in the probability of the observer's locating the foot at that particular point in the overall wave. Each stress in the unfolding syntagm is thus interpreted as a localized particle to which the listener assigns a discrete foot. The particle perspective results in a tendency to isolate the syllable on which the

BODY DYNAMICS, MEANING-MAKING, . . .

125

strong beat occurs - i.e. the Ictus - as determining the location of the foot. This makes the foot seem to be more determinate than it actually is because of this perspective's emphasis on static segments or points, rather than on the dynamic, temporally unfolding phases of a process and the fuzzy or blurred boundaries between segments (Pike 1967; 1982: 12-13). Again, the focus is on the probabilistic assignment of the foot in relation to the point where stress is strongest. Moreover, the particle perspective privileges a model of linguistic structure which is based on the notion of a constituent hierarchy. That is, larger-size particles are comprised of smaller-size particles (see section 8, pp. 134-9). On analogy with part-whole principles of organization in the grammar (e.g. the experiential structure of the clause as a figure consisting of functionally related roles [cf. particles] such as participant and process), the foot can be seen as consisting of lower-level syllabic constituents which function as parts in the larger whole defined as the foot on the level above that of the syllable. Thus, Ictus and Remiss are particles of, respectively, strong and weak stress. Experientially, the foot is defined as the functional relationship between particles of strong and weak stress which are realized by lower-ranking syllables. These particles of stress are integrated to the higher-ranking foot in which they function as parts in this larger whole. Stress is produced in the vocal tract by varying and modulating factors such as loudness and duration. Loudness and duration are quantitative values, based on difference in degree or topological-continuous variation. Degree of stress is therefore iconic with respect to degree of loudness and/or duration. From the perspective of articulation, both loudness and duration are iconic with respect to (1) 'the degree of force with which air is expelled from the lungs by the pulmonic air-stream mechanism while the vocal cords are in vibration' (Abercrombie 1967: 95) in the case ofloudness; and (2) the temporal duration of segments, depending on factors such as the length of time that a particular vowel posture is maintained (Abercrombie 1967: 82). At all levels of organization - syllable, foot, tone group - phonology is structured in terms of waves of vocal-tract activity. Waves of strong-weak rhythmic pulses constitute a fundamental organizing principle of the expression stratum. Furthermore, rhythmic waves at the level of the foot interact with waves at other levels of phonological organization such as the syllable and the tone group. The fact that periodic, wave-like behaviour is a fundamental organizing principle of the expression stratum of speech suggests that it is adapted to the generating of highly patterned flows of behaviour which cross-couple with the flows of other systems (other speakers) in ways that synchronize with each other in stable ways during interaction. Thus, the feeling of 'phonetic empathy' or of being 'inphase' or 'out-of-phase' with one's interlocutor for the duration of a conversational event, or some part of it, is a product of the ways in which interacting wave variables at many levels of organization are attuned to the many micro-level fluctuations in the wave-like behaviour of one's interlocutor. These fluctuations may be amplified so that they have larger-scale effects on the organization of the flow of meaning-making activity. This further suggests that interaction is an emergent property of the interaction of the waves produced by the different interactants (e.g. speakers).

126

BRAIN, MIND, AND THE SIGNIFYING BODY

7. The Integration of Iconic, Indexical and Symbolic Modes of Meaning in Lexicogrammar Hjelmslev (1961 [1943]: 54-6) pointed out that the internal organization of the content stratum parallels many of the principles also found on the expression stratum, relative to its own scalar level(s) of organization (see chapter 2, section 1, pp. 59-67). Expression and content in language are describable in terms of a few underlying principles of formal organization which pervade both of these strata. As we saw in the case of phonology above, these principles correspond to the periodic, particulate, wave-like, and chain-like modes of organization which are characteristic of the four metafunctional regions on both strata. Furthermore, the iconic, indexical, and symbolic modes of meaning which are internal to the content stratum are describable in terms of a nested specification hierarchy, as outlined above. According to Deacon, Peirce's typology of icon, index, and symbol designates a hierarchy of different modes of reference (Deacon 1998 [1997]: 73; see Thibault 2000a: 298-303 for some further proposals). In my view, Peirce's threeway distinction is not restricted to the referential hierarchy proposed by Deacon, but can be used to explain the various ways in which lexicogrammatical form relates to the plurality of overlapping semantic functions (experiential, interpersonal, textual, logical) that are expressed by forms. Reference is a discoursepragmatic function of language-in-use. It cannot be explained in terms of any single metafunction; instead, reference depends upon the synergy among different metafunctional areas in lexicogrammatical and discourse. My interest here lies in the ways in which iconic, indexical, and symbolic properties can be used to explain various facets of the realization of the semantic metafunctions in lexicogrammatical form. Therefore, my focus in this section is on the intrinsic metafunctional organization of lexicogrammar rather than on extrinsic referential uses of these intrinsic functions. My interest lies in drawing attention to some of the ways in which the intrinsic metafunctional organization oflexicogrammar can be seen as a specification hierarchy which extends from the most general kinds offunctional relationship between form and function (iconic) to the most highly specified (symbolic). The logic of the specification hierarchy further entails that iconic properties of lexicogrammatical form-function relations are retained when indexical and symbolic properties are progressively added. This means that symbolic properties implicate indexical ones which implicate iconic ones. The relations between levels in the implication hierarchy are transitive (Salthe 1993: 64). This means that what is logically true at any given moment of the system is true of the system through all of its moments. For this reason, lexicogrammar has symbolic, indexical, and iconic properties, as reflected in its intrinsic metafunctional organization. The more general iconic properties probably also reflect earlier conditions of the system prior to the emergence of the hierarchy of symbolic, indexical, and iconic properties postulated below. This does not mean that the iconic characteristics of precursor semiotic systems to language have necessarily been retained in their original form. Rather, the addition of indexical and symbolic properties in the further development of the

BODY DYNAMICS, MEANING-MAKING, . . .

127

system necessarily entails the reorganization of the lower (innermost) integrative levels as higher (outermost) levels are, in time, added. The emergent system of levels 'reflects the categorial reach of some observer, extending from the most general types relevant to some classification to the most highly specific (highly specified)' (Salthe 1993: 65). 7.1 The Iconic Mode ofLexicogrammatical Organization The iconic dimension of lexicogrammar and its discourse-level patterning is revealed in the ways in which the four different modes of meaning - the metafunctions - proposed by Halliday are predisposed to different forms of lexicogrammatical realization. Thus, experiential meanings are realized by particulate or constituent-like modes of lexicogrammatical realization; interpersonal meanings are realized by field-like prosodic and scopal ones; textual ones by wave-like or periodic ones based on peaks of prominence and beginning-end boundary markers; and logical meanings are realized by chainlike relations of interdependency between items such that one item modifies another (Halliday 1979; McGregor 1997: 77-80). The four different modes of lexicogrammatical realization iconically map onto the four different ways in which the content strata of lexicogrammar and discourse construct and construe the phenomena of experience - real, imagined, remembered, and so on - in their ecosocial environment. The iconic dimension oflexicogrammar is the least specified, innermost level of a nested hierarchy of iconic, indexical, and symbolic modes. Each of the four modes of lexicogrammatical realization maps onto the continuous topological variation of the phenomena of experience. Experience is transformed as categories belonging to linguistic content along these four dimensions, which are intrinsic to the organization of lexicogrammar and discourse. This does not mean that these different modes of meaning - the experiential, interpersonal, textual, and logical metafunctions - simply reflect natural kinds of categories 'out there' in the real world. Rather, they constitute a first principle of their semiotic reorganization according to the internal organization of the content stratum. In terms of the three-level hierarchy, the iconic dimension of metafunctional organization can be seen as the mapping of topological-continuous variation in the phenomena of experience (level L-I) to topological-continuous variation in the modes of realization of the metafunctions in lexicogrammatical form. The different modes of formal realization in lexicogrammar iconically resemble different semantic regions. This is so in the sense that topological variation in lexicogrammatical form maps onto topological variation in the corresponding semantic region. The three-level hierarchy can be used to model the relations across levels described in this paragraph as follows:

Level L+1: Level L: Level L-1:

metafunctional semantic regions diverse modes of realization in lexicogrammatical form phenomena of experience (topological)

128

BRAIN, MIND, AND THE SIGNIFYING BODY

I shall now consider each of the four metafunctions individually in order to see how the particular principles of structuration associated with each of the metafunctions iconically resemble a particular kind of meaning. 7.1.1 The Experiential Metafunction Experiential meaning focuses on the position of particles and the part-whole analytical relations that exist between these in configurations of functionally related process-participant relations. Thus, the constituent-like or particulate dimension of grammar is iconically related to the way in which experience is analytically segmented into a number of component parts which together form a larger whole. For example, a clausally realized situation comprising one or more participants, a process, and so on, all of which perform different functional roles in the whole. Consider the following example:

I bought this

YJ and I sailed it

This clause complex realizes a sequence in which two clausally realized figures are related to each other by the paratactic conjunction and. The first clause experientially construes a figure in which a process bought relates two participants I and this Vj. The figure which results from this relationship analyses the given phenomenon as a number of discrete, functionally related particles. The functional relations among these experiential particles comprise a figure whose experiential grammar can be schematized as Actor/-Process: Material Action/-Coal. A particular experiential function is mapped on to each of the three particles such that the given phenomenon is analysed by the experiential grammar of the clause as consisting of three discrete components that interact with each other. This also means that the users of the language probabilistically recategorize, on the content stratum, the continuous waves of the acoustic flux as quanta of discrete lexicogrammatical particles corresponding to experiential categories of participant, process, and so on. They do so by analytically segmenting the continuous movement of the wave, or what is more likely, complex patterns of cycles of overlapping and interacting waves deriving from different parameters of the vocal-tract articulatory activity operating on diverse temporal cycles, as peaks to which the assignment of particle position is most probable. The particle perspective on the clause focuses on the wave peak as a static position or point, rather than as a wave in constant movement. In so doing, the emergent particulate organization of the experiential categorization of the phenomena of experience maximizes the degree of certainty of interpretation of some phenomenon. 7.1.2 The Textual Metafunction Periodicity waves ofTheme-Rheme and Given-New in the textual meaning of the clause envisage the clause as a dynamic movement in time comprised of various phases. Periodic phases of clausal activity can be modelled as waves of content. That is, the clausal wave is interpreted probabilistically in terms of the likelihood of the language user finding the respective information quantum at a given

BODY DYNAMICS, MEANING-MAKING, . . .

129

position in the overall wave. In the clause, this means that waves of 'thematicity' and 'newness' are interpreted as indicators of the probability of the language user finding the respective quantum corresponding to Theme, Rheme, Given, or New at a given point in the clause. It is in this way that the waves of ThemeRheme and Given-New flowing through the clause can become quantitized as localized particles occurring at, say, the beginning and end points of the clausal wave. For example, the thematicity wave in the English clause consists of two peaks, corresponding to Theme and Rheme, respectively. These two peaks represent the most probable points in the clause for the assignment of the functional elements Theme and Rheme. In the following example, the Themes are indicated by bold type; the Rhemes are underscored with a double line. maybe his fears came out so he opposed it violently and . . . but mum saved some money up and she lent me some money to buy a boat

Figure 3.6 models the thematicity wave in this syntagm.

Figure 3.6: Thematicity wave in the English clause complex, showing Theme and Rheme as two peaks of information quanta 7.1.3 The Interpersonal Metafunction

The prosodic or scopal character of interpersonal meanings can be explained in terms of the general tenets of field theory. In physics, a field is 'a region under the influence of some physical agency' (Pitt 1977: 149). On the content stratum, an interpersonal field is the domain surrounding a given interpersonal selection - e.g. mood or modality in the clause - such that the given selection syntagmatically extends over and in some way modifies or deforms (influences) some other feature within the given syntagmatic domain, which it holds in its scope. The given interpersonal feature influences and shapes (deforms) the given syntagmatic domain in order to achieve a particular interactive purpose. In this perspective, the process of meaning-making can be seen as one of continually assigning quanta of interpersonal meaning to some syntagmatic domain so as to 'energize' it for specific interactional purposes and effects. Thus, a given

130

BRAIN, MIND, AND THE SIGNIFYING BODY

interpersonal quantum, such as a modality or mood choice in the clause, is a quantum of energy which acts on or extends over and therefore influences a clausal field. Interpersonal quanta such as mood or modality are energetic components of an overall interpersonal field. In the following example, the projecting clause I think and the modal adjunct probably are modalities which act on and extend over the entire declarative proposition in double square brackets so as to give the speaker's interactive perspective on that proposition. I think [[probably [the first thing that gave me an opportunity of understanding what I was going through was my mum saved up she made string bags]]] The projecting clause I think holds the projected clause in its scope in the sense that the modality which is expressed in the projecting clause extends over the modalized field of the proposition in the projected clause. In so doing, the speaker provides his subjective (first person) modal assessment concerning the possibility of the proposition that is expressed by the projected clause. The semantic extension of the modality can be modelled as the field within which the modality exerts its influence. Such a field can be seen as a modally charged semantic field. Figure 3.7 illustrates the full complexity of the choices in modality and mood which operates in this syntagm on various levels, as well as showing the field over which each choice's modalizing influence extends. On the outermost layer, there is the mood choice of the projecting clause. The mood choice holds the proposition in this clause in its scope and indicates that it is being asserted as a proposition that can be argued about in the discourse. On the second layer, the projecting clause is a modality of the kind Halliday identifies as having the feature 'subjective: explicit'. This kind of modality reveals the inherent self-reflexivity of human language. That is, human language has resources for acting on and reflecting upon itself such that the modality in this case involves me ta-linguistic awareness. In this case, the speaker's use of the projecting clause shows a reflexive self-awareness of his own processes of modal evaluation. The third layer consists of the comment adjunct probably, which indicates the speaker's evaluation of the content of the proposition in the projected clause. Finally, the fourth or innermost layer consists of the mood choice of the projected clause. Once again, the choice of declarative mood indicates that the proposition in the projected clause is being asserted by the speaker as something which can be argued about in the discourse.

7.1.4 The Logical Metafunction Logical relations in the lexicogrammar are concerned, for example, with the ways in which clauses combine with other clauses. The resulting combination of clauses is called the 'clause complex'. The logical relations between clauses iconically reflect the ways in which people use language to construe relations of temporal and causal dependency between the experienced phenomena in clauses (see chapter 4, section 6, pp. 195-8). In the following sequence, the speaker's use of the conjunction and iconically reflects the way in which the

BODY DYNAMICS, M E A N I N G - M A K I N G , . . .

131

Figure 3.7: Multiple modalizing fields acting on the syntagmatic domain of a proposition, showing the scope over which each field's modalized influence extends

experienced phenomena in each clause are related to each other. For example, the logical relationship of addition between there are some of us who are thin and some of us who are thick and we all have a little problem iconically reflects the relationship of extending the information in the first clause by adding further information to it in the second clause. In this example, the relationship of'extension' is signalled by the conjunction and. The logical relationship between I ah I ah was going very well actually and until about January the 4th and we started the election campaign is indicated by when. In this case, the relationship is between a given situation in the first clause and a temporal circumstance in the second clause. 7.2 The Indexical Mode of Lexicogrammatical Organization The four iconic modes referred to above are integrated to the indexical properties of lexicogrammatical signs. The indexical properties of lexicogrammatical signs include the Finite element in the verbal group, which functions indexically to ground the clause in relation to a reference point in the

132

BRAIN, MIND, AND THE SIGNIFYING BODY

temporally unfolding discourse event. The Finite element achieves this indexical function through either primary tense or modality. Primary tense is indexical insofar as it involves temporal proximity deixis: it situates the process instantiated by the verbal group in time relative to the moment of utterance. Modality is indexical in the sense that it involves what might be called attitudinal or evaluative proximity deixis: it situates the process instance in relation to evaluations of its probability, usuality, certainty, and so on, relative to the speaker in declarative clauses and the addressee in interrogative clauses. Davidse (1997) further points out that the process instance is also grounded in terms of person deixis by the grammatical Subject - it is linked to grammatical person (first, second, third). In this way, both Subject and Finite perform indexical grounding functions which link the clause to the I-you and here-now of the discourse event. At group rank, the Thing instance, which is realized by the Head of the nominal group, may be grounded in terms of spatial proximity by means of the demonstratives or in terms of person deixis by means of the possessives (Davidse 1997). Whereas the iconic properties of lexicogrammatical signs that I discussed above reflect general, probably universal design principles that relate specific modes of meaning to their particular modes of realization in the lexicogrammar of all languages, the indexical and symbolic properties of lexicogrammatical signs are concerned with increasingly specified principles that characterize individual languages. For example, the grounding function of the Subject is an indexical property of English, though not necessarily of all other languages. In any case, both indexical and symbolic properties of lexicogrammar are organized in terms of a rank scale of lexicogrammatical forms on level L which have the function of filtering non-criterial continuous topological variation in the phenomena of experience on level L-l for the purposes of its construal by the semantic categories recognized by a given SI on level L+1. 7.3 The Symbolic Properties ofLexicogrammatical Signs

Lexicogrammar is also a system of symbolic possibilities for making meaning by virtue of its systems of typological-categorial distinctions between, for instance, different experiential categories of process (material, mental, verbal, ete.) and their associated participant roles; between different interpersonal categories of mood (declarative, interrogative, oblative, imperative) and their associated possibilities for enacting different interactive positions (addresser, addressee, giver of information, seeker of information, and so on) in the negotiation and exchange of meanings in discourse; and between different textual categories of, for instance, deictics. A symbolic system is not necessarily tied to its environments in any specific way. This is so because the specifically symbolic properties of, for instance, language consist of systems of categorial distinctions. Such distinctions are defined by the value which each distinction is assigned by virtue of its relations to all the other terms that constitute a given system of relations. These systems of semiotically salient differences in some community are organized as language-specific functional systems and structures which symbolize experiential, interpersonal, deictic, and other categories. This is true for phonological systems and structures as well as lexicogrammatical ones. It is by virtue of the functionally organized paradigmatic and syntagmatic environments in which

BODY DYNAMICS, MEANING-MAKING, . . .

133

particular distinctions (cf. Saussure's values) occur that a given lexicogrammatical form can operate in very many different contexts rather than being tied to a given context. For example, the Head element in a nominal group, e.g. a very good sailor, symbolizes an experiential type-category of Thing, viz. [SAILOR] in the present example. It does not refer to a specific sailor in a particular real-world context. Similarly, the adjective good symbolizes a type-category of quality just as the determiner a symbolizes the deictic category 'non-specific'. Each of these items derives its symbolic potential from its position in a system of contrasting terms whereby the value of each and every term is defined. For example, a contrasts with the in the sense that both are more delicate instantiations of the schematic category [DETERMINER]. The more delicate (specific) instantiations constitute a system of contrasting terms in which the meanings qua categorial distinctions between a and the are defined. Figure 3.8 shows this contrast. SPECIFIC

the

DETERMINER NON-SPECIFIC a Figure 3.8: Simplified determiner system, showing the categorial distingtion betwen the values 'specific' and 'non-specific', as symbolized by English 'the' and 'a'

More precisely, the linguistic form a symbolizes a category of determiner which can be more delicately subcategorized as [DETERMINER: NON-SPECIFIC; NUMBER: SINGULAR: COUNT]. Thus, a is used in the grammatical environment of singular count nouns, as in the example above. The properties of a under discussion here are symbolic properties that derive from (1) the position of this lexicogrammatical form in a system of paradigmatic contrasts with other forms such as the; and (2) the possibilities of this form for operating in particular types of grammatical structure such as the nominal group. The nominal group is a particular subsystem in the language with grammatical units and structures which are specific to that level. These units and structures are specific to the functional tasks of the grammatical subsystem which is the nominal group. Specifically, the grammatical resources of the nominal group function to ground by deictic and other means the Thing element as an instance of the type-category which is symbolized by the Thing. In the nominal group a very good sailor, the determiner a functions deictically to specify the Thing element as an arbitrary or non-specific instance of the type-category of Thing in question. If determiners such as English a and the were pure indexicals, then they could not be freely combined with other items such as, for instance, adjectives and nouns by virtue of the grammatical resources of the nominal group. That is, they would be necessarily tied to the specific contexts in which their indexed entities occur. However, it is the symbolic categorial-coding potential of these forms which

134

BRAIN, MIND, AND THE SIGNIFYING BODY

enables them to be used in very many different contexts. That is, the categories of deixis and number that are symbolized by these forms are not tied by indexical necessity to any specific context, but constitute a systemically defined meaning potential which can be used and played with across very many lexicogrammatical and discourse contexts. The observations I have made thus far refer to the semantic functions of items such as determiner, adjective, and noun in the grammatical subsystem 'nominal group'. Nominal groups are also integrated to the higher-scalar grammatical rank of the clause such that the nominal group realizes a particular participant role within the grammatical structure of this higher ranking system. In the present case, the clause from which the example is taken is: I wasn't a very good sailor. In this clause, the nominal group a verygoodsailorfunctions as Attribute in a relational attributive clause comprising the following functional roles: I (Carrier) wasn't (Process: relational: attributive) and a verygoodsailor (Attribute). It is only by virtue of this nominal group's integration to the higher ranking unit of clause that it becomes possible to see how the symbolic properties of the nominal group interact with its indexical meaning in the specific discourse situation. Thus, the speaker in the situation of utterance of the clause, as indexed (exophorically pointed to) by the first-person pronoun I, is also indexed, by virtue of the semantic relationship of attribution that links I to a very good sailor in this clause, as a non-specific instance of the type-category of Thing in question. Thus, we see how, in the particular context, it is in the interaction between situational and lexicogrammatical (clause and nominal group) properties and relations that the combined indexical and symbolic properties and functions of these lexicogrammatical forms are revealed. The non-specific deixis of the nominal group means that the nominal group cannot in itself have any referential function of picking out a specific entity for the purposes of referring to it and subsequently tracking it through the discourse. Instead, it is the clause-level relation of attribution which creates the link between indexed speaker and nominal attribute that is assigned to speaker. Table 3.2 summarizes the relevant properties of the iconic, indexical, and symbolic dimensions of semiosis, as discussed in this section.

8. Language as Particle, Wave, and Field The tripartite perspective on language as particle, wave, and field first proposed by Pike (1967) and subsequently taken up and adapted by Halliday (1979) shows how both phonological and lexicogrammatical form are organized by a number of distinct, though complementary, principles which ramify across all levels of linguistic organization. These diverse though complementary principles of organization illustrate two fundamental principles concerning the organization of language form. First, the intrinsic organization of language cannot be reduced to any single principle of organization. Secondly, the complementary modes of organization identified by Pike and Halliday show how the two external 'realities' with which language form interfaces - viz. the body of the articulator

BODY DYNAMICS, MEANING-MAKING,

135

Table 3.2: Semiotic properties oJ icon, index, and symbol Type of Semiosis

Properties

Scale

mapping of topologicalcontinuous variation to topological-continuous variation through crosscoupling of variation in body dynamics to affective and other internal states of organism

non-intentional herenow sensori-motor dynamics of bodily states which cross boundary between internal states of system and immediate environment

topological-continuous variation in physicalmaterial dynamics of body iconic

index

spatial-temporal extension to and intentional projection into immediate environment which can be sampled and individually remembered

symbolic

space-time scales beyond here-now involving history and cultural memory

typological-eategorical difference

and the phenomena of experience - are selectively transformed as functionally diverse areas of meaning, each of which makes its distinctive contribution to the internal organization of both expression and content. Thirdly, the functionally diverse regions of meaning and their corresponding principles of structuration in both expression and content implicate iconic, indexical, and symbolic modes of meaning. As we have already seen, the iconic mode reflects the most general principles of categorization and the symbolic the most highly specified. The tripartite perspective of particle, wave, and field in the clause shows that meaning construal is not localizable at a single point in the clause. Instead, it is the time-bound interaction of all three perspectives as they are dynamically distributed over the global totality of the clause that yields a probabilistic interpretation of a given reality. Thus, the experiential categorization of phenomena, their grounding in a dynamically charged interpersonal field, and their

136

BRAIN, MIND, AND THE SIGNIFYING BODY

quantitization as bits of information relative to the temporally developing discourse context are all necessarily present in the semiotic process of interpreting and intervening in the phenomena of experience. It is the language user's probabilistic relationship to the wave/particle/field complementarity of any given quantum of meaning in the clause that affords the interpretation of reality. Moreover, this complementarity is a contextual one in which the internal organization of language as expression and content, 'reality' in the form of the articulatory (e.g. vocal-tract) and ecosocial dynamics modelled by these two strata, and the observer are all internally connected to each other. The stratal organization of language as expression and content reflects the way in which, at their respective scales, there is no ultimate or fundamenta1level of reality which is supervenient to the rest. Language is not epiphenomenal to some more basic level of organization. Instead, both expression and content are describable at all levels as quantum processes which exist as patterns of particle, wave, and field (Bickard and Campbe1l2000: 331). Complex interacting patterns of semiotic processes involving 'what is observed', 'who observes' and the 'means of observation' lead to the emergence of both articulatory phenomena and the phenomena of experience on the body and ecosocial scales, respectively. The intrinsic metafunctional diversity of both expression and content means that language and other semiotic modalities are not supervenient to any single underlying principle. Rather, the complementarity of the particle, wave, and field dimensions of language form enables the many degrees of topological variety on level L-1 of, for example, the neuromuscular substrate of vocal-tract activity to be entrained to the phonological trajectories of a given language. Moreover, the particle/wave/field perspective means that the principles governing their emergence can be precisely formulated. On the other hand, the majority of currently available models of meaning-making, representation, and information processing are founded on one version or other of conceptual or representational atoms or primitives, each having its own representational content (Bickard and Campbell 2000: 325). Such reductionist accounts are founded on principles of upward causation which seek to explain, usually in some sort of causal-explanatory framework, how such representational atoms are assembled to form larger wholes. The tripartite perspective of language as particle, wave, and field suggests that language, at all levels of its organization, cannot be reduced to fundamental representational atoms or conceptual-semantic or phonetic primitives. Language at all levels is organization of quantum field processes (Bickard and Campbell 2000: 334-5). Particles or constituents are no more than local stabilizations on a given level of principles of organization of underlying field processes. For example, the segmentation of the continuous flow of vocal-tract activity in the production of speech sounds into discrete phonemes amounts to specifying positions in the flow where systems of paradigmatic features are localized as probably occurring at that point. How do we explain these stabilizations against a background of endless flux? Typically, linguistic theories have modelled language in bottom-up terms as comprising constituent particles and the structural relations among these. The resulting constituent structure is postulated as a supervenience base on which everything else depends. For

BODY DYNAMICS, MEANING-MAKING, . . .

137

example, the linguistic entity 'sentence' is seen as being supervenient on certain classes of lower-level constituents standing in certain kinds of relationships with each other. Formal models of language, for example, are not concerned with relations outside those between 'sentence' and its lower syntactic constituents and the formal rules for their combination. Such a system of relations is closed to the external environments in which language operates. On the other hand, if language is seen as a form of dynamic open system, then it is, by definition, engaged in transactions with its environment over the time in which the system persists. In this alternative view, constituency (segmental organization) is seen to be the consequence of a particular and partial way of relating to the wave. The wave is the organizing principle, rather than constituency as such. Without such system-environment transactions, the system cannot and does not exist. It follows from this that the characteristics which language has as a dynamic open system cannot be derived from closed-system properties and relations such as those seen as dependent on a supervenience base. Instead, such properties and relations can only be defined in terms of language qua dynamic, far from equilibrium system. In the terms of the present study, language, rather than being supervenient on a causal base of fundamental properties, persists in time and maintain its dynamical structural integrity and intrinsic properties and relations in time because of the transactions that occur between its internal organization and two environments, viz. the body-brain complex and the ecosocial environment, with which it is cross-coupled. There is no supervenient substance-base on which language is ultimately founded. The reason for this lies in the way both of the interfaces referred to above are themselves persistences in the organization of bodily (neuromuscular) and ecosocial processes. Instead of being explainable in terms of fundamental constituent particles, such local stabilities or patterns of organization are explainable in terms of the patterned nature of the organization of the relevant system - environment transactions that are constituted by the expression and content strata. The Principle of Alternation proposed by Lemke (1999) allows us to view this question in a different perspective. Quantum field processes relative to the space-time scales of the body and the phenomena of experience self-organize into higher-scalar semiotic levels, relative to the observer perspective in some SI, where they are construed as in some way semiotically salient or meaningful. For example, the topological-continuous space-time of vocal-tract activity on level L-l is an organized field (a phase space) of articulatory processes which are reorganized as complex interacting patterns of phonological particles, waves, and fields for level L+I by virtue of the filtering or buffering function of the intermediate level L of phonological units and relations functions. In this way, emergent principles of phonological organization, themselves no more than a further reorganization of quantum field processes on their own level, are made meaningful for higher levels of organization, viz. lexicogrammar and discourse. Language and other semiotic system-processes are not supervenient. Their existence is dependent on a number of environmental factors which can be thought about in terms of Aristotelian causality. First, they depend on biological initiating conditions - e.g. neuromuscular processes - such as those which enable vocal-tract gestural activity and their effects to take place (efficient

138

BRAIN, MIND, AND THE SIGNIFYING BODY

causality). Thus, waves of vocal-tract activity may be said to cause (i.e. efficiently) the compression of sound waves which are propagated from a source. Secondly, they depend on material environmental affordances such as available physicalmaterial resources - e.g. the air as a medium for the propagation of sound waves, treated surfaces such as paper, the nature of human perceptual systems, its amplification and diffusion by culturally evolved technologies such as writing, the Internet, and forms of social organization (material causes). Thirdly, there are principles of organization internal to language such as its dependence on a semiotically salient system of differences and combinations of typical patterns of phonological, grammatical, and discourse systems and structures (formal causes). Fourthly, final causes are those whereby the system anticipates and is oriented to its own future along a temporal trajectory, which is the only way in which the system as such exists. The organization of ecosocial semiotic processes and the perceived stabilities or regularities in these processes can only take place in the presence of appropriate forms of organization of all four factors on various space-time scales. Language is a transindividual system organized on many different space-time scales. It cannot be appropriately reduced to units or relations on anyone scale. Moreover, language is an instance of the general class of dynamic open systems which maintain their existence and integrity in time. In this way, the causal consequences of a given semiotic system also persist in time, again on a multiplicity of intersecting space-time scales. For example, the processes that operate on the level of expression - e.g. vocal-tract activity - are strongly constrained by processes on the content stratum. This is consistent with the view expressed above that meaning flows from an intentional source and constrains lower-level sensori-motor systems in the process of putting into effect a particular action. Meaning-making can thus be seen as the entraining and reorganization across levels of processes of expression and content by higher-order proximate intentions in symbolic neural space. Action is the entrainment of such processes along a trajectory which has causal, though not efficiently causal, consequences. This is consistent with the view that the brain recategorizes sensori-motor activity as conceptual routines, and so on (see section 13, pp. 162-8). Thus, the storing and maintaining of a stable - not static - language system comprising both expression and content on both the ecosocial and individual (organismic) levels means that the causal consequences of that system persist on many different levels. For example, the entraining of meaning along an action trajectory, with its consequent cross-coupling of both semiotic-discursive and physical-material relations and flows, makes an asymmetric difference in the world if the emergent meaning-making trajectory that results is able to flow into a sink of semiotically mediated activity that corresponds to the proximate intention of, say, the speaker (Juarrero 1999: 193). Signs may point to (index) and/or symbolically construe a given contextual value as the purpose or goal of the action trajectory. This means that the intending agent must construct the indexical or symbolic link between sign and intended goal or purpose in order for the latter to be the object of a particular course of action. Thus, intentionally directed action or goal-seeking activity which is directed to some purpose are made possible by processes of semiotic mediation that link what is inside the organism to what is outside it.

BODY DYNAMICS, MEANING-MAKING, . . .

139

Purposes, goals, objects of desire, and so on, are not inherent in the phenomena of experience per se. They are emergent consequences of the semiotic mediation between proximate intentions and relevant contextual values. The neuro-semiotic recategorization of expression as content indicates the validity of the hypothesis that content is an emergent level of organization relative to the lower-level expression stratum. In this view, such properties and dynamical relations as emerge on the higher level are no less real or causally effective than are properties and dynamical relations on the lower level. It thus follows that the properties and relations of the higher level will possess causal properties that the lower level does not possess. Indeed, the scalar heterogeneity of language implicates that emergent indexical and symbolic properties of language ramify across and have effects over potentially very many space-time scales beyond the immediate organismic one. The hierarchical organization of both the expression and content strata illustrates how the many degrees of freedom on level L-l constitute enabling conditions in the form of very many degrees of continuous topological variation that a dynamically emergent system can access through the buffering or filtering functions of level L. Constraints emanating from level L mean that dynamical relations and properties on level L-l are entrained to these higher-level constraints. For example, articulatory repertoires on the phonetic level of vocal-tract activity are reorganized and entrained to the patterns of organized behaviour vocal-tract trajectories - that are afforded by the phonological systems and structures on level L. The constraints arising from this level allow vocal-tract activity to self-organize into more complex forms of activity that would not have been possible if the many degrees of topological variation existing on level L-I were not subject to the filtering functions of level L. In this way, level L-l provides a large repertoire of potential information which the higher level can access and then reorganize (section 2, pp. 109-16). The entrainment of this variety to newly emergent patterns of phonological activity on level L increases the organism's meaning-making potential. This increase occurs by integrating the organism to a contextual level comprising indexical and symbolic semiotic modes. It is these modes which give the organism access to scalar levels beyond the immediate body-scale along with new possibilities of interacting with scales beyond the herenow scale. In other words, in integrating the organism to higher scalar levels, level L enables the organism to be functionally part of ecosocial relational dynamics and structures which the level L-l alone could not bring about.

9. The Brain as Selective Recognition System: Language and Edelman's Theory of Recategorization Edelman (1992: chapters 8-9) has proposed that the brain is a selective recognition system equipped with a capacity to memorize what it recognizes. In both infant protolanguage (ontogenesis) and in the evolution of language in the human species (phylogenesis), phonetic repertoires gradually become linked to conceptual repertoires through processes of association. That is, some patterns of sensori-motor vocal-tract activity are selectively cross-coupled to some patterns

140

BRAIN, MIND, AND THE SIGNIFYING BODY

of concept recognition, in the process leading to the making of even finer sensori-motor discriminations or differentiations on account of the reorganizing effects emanating from the higher, conceptual level. ... to build syntax or the bases of grammar, the brain must have reentrant structures that allow the semantics to emerge first (prior to syntax) by relating phonological symbols to concepts. Because of the special memory provided by Broca's and Wernicke's areas, the phonological, semantic, and syntactical levels can interact directly and also indirectly via reentrant circuits that are formed between these speech areas and those brain areas that subserve valuecategory memory. When a sufficiently large lexicon is collected, the conceptual areas of the brain categorize the order of speech elements, an order that is then stabilized in memory as syntax. In other words, the brain recursively relates semantic to phonological sequences and then generates syntactic correspondences, not from preexisting rules, but by treating rules developing in memory as objects for conceptual manipulation. Memory, comprehension, and speech production interact in a great variety of ways by reentry. This permits the production of higher-order structures (such as sentences in a grammar) and obviously helps with the elaboration of lowerorder sequences (such as phrases). Of course, once achieved, the sequencing becomes automatic, as do many other motor acts. (Edelman 1992: 130; italics in original) The re categorization of conceptual repertoires as lexicogrammatical systems cf. Edelman's 'syntax' - paves the way for a massively expanded meaning-making potential on account of the qualitative differences that the global semiotic reorganization brought about by the new level entails. The recategorization of conceptual systems based on sensori-motor routines as lexicogrammatical ones means that the organism has dynamical properties it would not otherwise have. In turn, these lexicogrammatical systems which are memorized and elaborated in the brain in the course of the organism's development and individuation along its historical-biographical trajectory are integrated to still-higher-scalar patterns of organization by way of the semantic registers, discourse genres, social activity structure types, and so on, that are the typical ways of making meaning in a given community, i.e. its SI on level L+1. This emergent hierarchy of dynamical neuro-semiotic relations means that progressively higher levels selforganize. Each newly emergent level gives rise to dynamical relations and properties specific to its level and, at the same time, the overall system's degrees of freedom - its possibilities for social meaning-making - is expanded. By the same token, the higher levels impose their own constraints on the lower ones. That is one good reason why discourse and lexicogrammar are two distinct levels of meaning-making within the content stratum in systemic-functional linguistics: each level makes its own distinctive contribution to the meaning of the whole, and at the same time is contextually integrated to the other levels. Nevertheless, I would argue that Edelman tends to privilege a two-level model comprising sensori-motor routines and conceptual repertoires in order to explain the emergence of language in the individual. Conceptual repertoires

BODY DYNAMICS, MEANING-MAKING, . . .

141

obviously depend on the organism's time-bound perceptual samplings of the environment in the form of the extra-somatic outside world as well as the individual's internal milieu. I do not think there is a simple linking of sensorimotor routines to conceptual ones. Both phylogenetically and ontogenetically, the cross-linking of the two is always mediated by the higher-scalar ecosocial environment in which the organism's perceptual sampling of the environment and consequent elaboration of conceptual structures is embedded. As Lemke (2000a: 210-11) points out, there is not a simple 'upwards' progression from simpler to more complex forms of organization. Rather, there is always already a higher form of organization, or an organized field of possibilities, on higherscalar levels which mediates the body-brain's transactions with its environment. Thus, the emergence of the content stratum of language as a recategorization of lower-level conceptual structures is the result of the emergent self-organization of a more specified intermediate level coming between the conceptual structures elaborated in the brain and the ecosocial environment. As I have argued elsewhere (Thibault 2000a), Edelman's focus on the individual organism tends to downplay the ways in which the emergence of language, both phylogenetically and ontogenetically, in the individual bodybrain takes place in the context of higher-scalar ecosocial systems which are themselves continually evolving. Edelman is certainly right to emphasize that the dynamics of language in the individual does not depend on a genetically endowed knowledge of language which predicts from the outset the 'growth' of language in the individual. Instead, the dynamics of language in the individual is entrained, in and through social interaction, to the dynamics of higher-scalar ecosocial processes which themselves only have their existence along a timebound trajectory. Precisely the same argument also applies to the sensori-motor routines mentioned above. These, too, only exist along a self-organizing timebound trajectory of vocal-tract gestural activity. Such a trajectory is based on non-linear and irreversible cycles of neural and muscular activity, which are the micro-eomponents out of which vocal-tract activity is flexibly soft-assembled along a time-bound trajectory (Kelso et al. 1986; Goldfield 1993: 53). The assembling in time of such micro-components leads to the emergence of a coordinative structure: many interacting functional variables behave as a single unit of activity. In this way, very many degrees of freedom - i.e. of continuous topological variation - on the lower level of the body are transformed into meaningful information for higher levels of organization. The self-organizing emergence of phonological systems and structures which recategorize vocal-tract activity is a more specified intermediate level between these sensori-motor routines and the higher level of the content stratum. On both levels, lower-level noise is amplified and transformed into information which is meaningful for higher levels. The quantal theory of phonology developed by Stevens (1989) constitutes one of the important pioneering attempts to explain how macroscopic categories emerge from the microscopic detail of the articulatory and acoustic phenomena involved in the production and reception of phonological categories. The processes of re categorization discussed above depend on the system's capacity for discrimination or differentiation. For example, the sensori-motor

142

BRAIN, MIND, AND THE SIGNIFYING BODY

interface between organism and environment relates properties and processes internal to the individual subsystem to its higher-scalar environments: it provides the means whereby such properties and relations relate to the contexts in which such transactions take place. Sensori-motor operations are operations of discrimination or differentiation. The two-way nature of such operations means that they determine both the structure of the relevant system and the way it models its environment. Lorenzana (1993: 295) has pointed out that evolutionary systems [ESs], in general, 'need to build diversity from their simple set of gauges ... this construction of diversity from an initial undifferentiated unity is an essential characteristic of ESs, imposing a condition of gradualism'. In the light of this observation, we can understand the recategorization of sensorimotor repertoires as conceptual repertoires to be an instance of the emergent, self-organizing construction of a new scalar level of organization which allows the system to interact with increasing environmental diversity on increasingly larger scales. The emergence of the content and expression strata of language represents a further increase in the system's capacity to access its potential environment. The emergence of these two intermediate levels of organization constitutes an increase in the system's discriminatory power, thereby allowing the system to access and interact with increased environmental diversity across potentially many different scalar levels. Thus, the stratified nature of semiotic systems and their emergence as intermediate levels of organization between prior, less discriminatory systems is a specific instance of that type of hierarchy known as a 'combinatorial hierarchy' (Lorenzana 1993: 295). In a combinatorial hierarchy, discriminatory operations construct increased typological-categorial differentiation and diversity from simpler, prior conditions of topological-continuous unity. Lorenzana points out that the 'chain of levels' in combinatorial hierarchies is limited due both to specific contextual pressures and to system-environment relations. For example, the further stratification of language in terms of register and genre (Martin 1991) is the result of pressure from the ecosocial system. This means that principles of social organization, the participation of different categories of social agents in diverse social networks, the division of social labour, and so on, constitute further principles whereby language is organized in terms of the diverse semantic registers, discourse genres, and so on, which are functional in different domains of social life. Figure 3.9 models the basic principles of the combinatorial hierarchy with respect to the expression-stratum dynamics of speech. Each level in the hierarchy is formed out of combinations of elements on the level below. Level 1 refers to the oral cavity as a three-dimensional physical space, characterized by undifferentiated topological-continuous variation. Level 2 shows the reorganization of Level-l dynamical processes as a multidimensional physiological space. At this level, new couplings from different dimensions not congruent with physical space per se become possible. Level 3 shows how these multidimensional physiological couplings give rise to language-specific phonetic repertoires independent of phonological function. Level 4 shows how these phonetic repertoires are further reorganized as the phonological systems and structures of a particular language.

BODY DYNAMICS, MEANING-MAKING, . . .

143

Level 4

Level 3

Level 2

Level 1

Figure 3.9: Expression stratum as combinatorial hierarchy, showing reorganization of higher levels as recouplings of elements on lower levels thereby allowing for the emergence of new systemic possibilities and, hence, new system-environment couplings

An analogous and parallel system of levels can be proposed in relation to the content stratum and the way it interfaces with the ecosocial environment. In this case, Level 1 designates the topological-continuous flux of ambient stimulus energy in the environment. Level 2 refers to the cross-modal perceptual sampling and pick-up of this flux as information about events in the organism's environment. Such information necessarily correlates in the brain information from a diversity of perceptual modalities. In this way, new couplings occur which combine inputs from diverse channels not congruent with a single physical parameter. Level 3 shows how the resulting perceptual categorizations are further filtered and reorganized by the systems of semiotically salient differences - the semiotic values - specific to the particular semiotic resource systems of some community. Level 4 shows the further reorganization of these as the lexicogrammatical and discourse systems and structures of a given language. The emergence of these levels on the content stratum of neuro-semiotic organization is not without consequences for the expression stratum. Thus, the initial linking of vocal-tract sensori-motor routines to pre-linguistic conceptual ones and the consequent finer discriminations made by the former also brings about the emergence of specifically phonological systems and structures at level L on the expression stratum. A language system stored in the individual's brain is a self-organizing and distributed global neural space comprising both expression and content. These two levels are interdependent and, at the same time, each has its own internal

144

BRAIN, MIND, AND THE SIGNIFYING BODY

principles of organization and integrity. Expression faces 'inwards' to the body of the articulator and 'outwards' to the content stratum; content faces 'inwards' to the expression stratum and 'outwards' to the ecosocial environment comprising the phenomena of experience. As the three-level hierarchy principle has shown us, each stratum is comprised of both initiating conditions and higher-scalar constraints in the form of an SI specific to that stratum. By the same token, the higher-scalar content stratum exerts downwardly causal constraints on the expression stratum. The principle of the selective recategorization of sensorimotor routines by higher-order conceptual ones and the further recategorization of these by linguistic ones (phonological and lexicogrammatical) means that content imposes constraints on expression such that expression is selectively and functionally adapted and integrated to the requirements of content. Overall, the number of semiotic differentiations made and their possible combinations is far greater on the content stratum with respect to the comparatively much smaller number of phonological systems and structures in a given language. Thus, the learning and interpreting of higher-scalar content units and relations takes place over a much more extended timescale such as an entire lifespan trajectory at the same time as it requires a far greater number of interconnected neural networks in the long-term memory of symbolic neural space than is the case for units and relations on the expression stratum. Children develop the phonological structures and systems of a given language over a relatively much shorter timescale compared to that of their development of lexicogrammatical and discourse systems and structures. The principle of realization, which nicely resonates with Edelman's notion of selective recategorization, means that units and relations on the expression stratum are selectively and mutually contextualized with respect to units and relations on the content stratum. Thus, the sensori-motor activity which is entrained and organized according to the phonological systems and structures of a given language restrict the degrees of freedom of the body's dynamics. That is, body dynamics are entrained to the phonological systems and structures of that language. This entrainment is functional for the purposes of making and negotiating meanings with others in social contexts on diverse space-time scales. This means that human bodies are now integrated to the higher-scalar ecosociallevel of organization where sensorimotor (e.g. vocal-tract) activity is reorganized so as to take on new functional possibilities on this level. Moreover, the emergence of properties and relations specific to the content stratum in symbolic neural space is constitutive of our phenomenology of lived experience in ecosocial space-time. Meaning is not a mere epiphenomenon of experience. For example, the very fast scale of processes on the sensori-motor and eNS levels is integrated to and entrained by the somewhat slower scale afforded by the rhythmic and other patterns of temporal organization of a given language's phonological systems and structures. In turn, these are integrated to the still-slower patterns of rhythmic organization characteristic of socio-discursive patterns of organization evident in texts and social action (see Togeby 2000: 273-7). In this way, to take just one example, our experience of time is structured according to modes of organization that are endogenous to the dynamical

BODY DYNAMICS, MEANING-MAKING, . . .

145

relations existing on the ecosocial scale rather than being referable to lowerscalar sensori-motor dynamics perse, seen as their biological initiating conditions. These higher-scalar phenomenal relations are not exogenous factors with respect to the body. Instead, in integrating the body to them, they become a part of the body-brain's internal organization so that scalar heterogeneity is seen as a fundamental property of the meaning-making body-brain. Thus, we can see how, at all levels of organization, language is not composed of isolable functional components assembled 'from below'. Rather, the metafunctional diversity that is attributed to unfolding acts of meaning-making in logogenesis reflects organismic and ecosocial constraints that influence the time-bound development of the system without recourse to pre-existing causal factors to explain the system's behaviour. Expression and content eo-evolve and eo-develop in the sense that the neuromuscular dynamics of the body are entrained in and through their interaction, both phylogenetically and ontogenetically, with higher-scalar content-stratum processes and their functionings in a still-largerscalar ecosocial environment. At its particular level of organization, vocal-tract gestural activity can be described as a vector field having the properties of both direction and magnitude (Norton 1995: 50; Cleirigh 1998: 52). Relations both within and between the expression and content strata are non-mechanistic informational constraints. Final causality operates here because top-down relations between, say, the content stratum, comprising lexicogrammar and discourse, and vocaltract trajectory function both to select and to constrain the lower level at the same time as they also act to entrain it into its future trajectory (Salthe 1993: 270). This is so because the cogent moment - the temporal cycle - of the higher level of meaning on the content stratum is greater such that the flow of meaning along its trajectory continues well after the completion of the much faster temporal cycle of the articulatory trajectory at the level of the syllable, say, on the expression stratum (section 2, pp. 109-16). Informational constraints also operate on the relations between levels within both the content and expression strata. The relationship between phonology and phonetics that I discussed above can be seen in this light: phonological systems and structures are macroscopic relations and dynamics; phonetics refers to microscopic relations and dynamics at the fine-grained level of articulation (Browman and Goldstein 1995: 180). The constraints do not operate in just one direction, but instead are reciprocal, acting in both directions. On the content stratum, lexicogrammar and discourse similarly operate a system of reciprocal constraints between the microscopic and the macroscopic levels of description. Moreover, the self-organizing properties of lower-level relations and properties such as those evident in articulation on the phonetic level give rise to emergent patterns of organization at higher levels such as the phonological level. By the same token, the emergent higher level downwardly constrains activity on the lower level. The realizatory relationship between content and expression can now be seen as a system of reciprocal linkages between dynamical relations and processes on a number of different levels of organization, ranging from the microscopic level of articulation on the expression stratum to the macroscopic level of discourse on the content stratum.

146

BRAIN, MIND, AND THE SIGNIFYING BODY

In this way, the (physical) act of speaking - vocal-tract activity - is shown to be a single complex system of factors interacting in a non-linear fashion across all of the levels involved.

10. The Integration of Individuals to their Semiotic Trajectories A social system, in the theoretical perspective of the present study, is less specified than an ecosocial semiotic system: the former exists at a lower integrative level, A social system is less specified than an ecosocial semiotic system because the latter can access symbolic, and not just indexical, modes of semiosis, The historical emergence of the integration hierarchy of icon, index, and symbol in human social meaning-making could only come about with the development of new forms of social life and therefore new ways in which humans relate to each other and organize their social relations on different space-time scales. Thus, semiotic evolution emerged as part of the individuation of developing social life. See Bogdan (2000: 40-1) for some relevant observations on the socialized minds of nonhuman primates, based on indexical modes of social interaction, as an evolutionary prelude to the 'unsituated' (symbolic) minds of adult humans. Beginning with Edelman's view that conceptual repertoires are recategorizations of sensori-motor ones, we can say that these two levels were present in the brains of our hominoid ancestors well before social semiosis as we know it today came into existence. Conceptual repertoires and sensori-motor repertoires may be described as proto-content and proto-expression, respectively. Moreover, these two levels were functional at the interface between body-brain and environment. In turn, forms of social life emerged between these two less specified levels. This development required the co-ordination of human activities on scales that went beyond the body-brain's pick-up and sampling of information from its immediate physical and interpersonal environments. Body-brain and environment may thus be seen as lower- and higher-scalar levels in a semiotic differentiation model of the eo-evolution of body-brain and ecosocial semiotic relations and processes. The fact that recategorization entails more fine-detailed specification of the sensori-motor level means that the degree of specification and differentiation of this level is amplified, giving rise to the emergence of the phonetic and phonological levels. Parallel processes simultaneously operate with respect to conceptual repertoires. Consequently, lexicogrammatical and discourse structures and systems emerge in response to an expanding conceptual repertoire along with the need to pool and access the expanding pool of conceptual-semantic categorizations in socially co-ordinated ways. Ecosocial semiotic systems may be viewed in this perspective as dissipative structures regulated and maintained in a constant me tastable dialectic of stability and change (Lemke 1995b) by processes of meaning-making along the logogenetic trajectories of the members of that system. Individuals are connected to each other in social semiosis through their logogenetic trajectories. Logogenetic processes of meaning-making can be seen as just such a process of trajectory connecting. An example is when two individuals

BODY DYNAMICS, MEANING-MAKING, . . .

147

jointly construct some conversational event. In this way, individual trajectories intersect and, at the same time, a new, perhaps transient, higher-scalar trajectory comes into view in the form of the jointly constructed and enacted meaningmaking event and its entextualized records and/or products. The individuation of social semiosis along an ontogenetic trajectory means that the individual trajectory's meaning-making potential expands as it individuates. Thus, the transition from proto-language (iconic-indexical) to language (iconic-indexicalsymbolic) means that the individual's agentive powers increase: he or she is able to access and relate to an expanded range of meaning-making possibilities. The integration of, say, the microfunctions of the indexical phase to the metafunctions of the symbolic phase entails an increase in the amount of stored information along the individual's trajectory (Salthe 1993: 262; chapter 4, section 17). This has clear enough implications for human consciousness, as we shall see in chapter 5. Symbolic meaning-making in particular and its associated practices of entextualization mean that core consciousness and extended consciousness pace Damasio (1999) are extended and integrated by symbolic consciousness. Thus, individual agents can construct trajectories across many different space-time scales beyond the here-now one of the body-brain in its immediate physical environment. This helps us to think of the emergence of symbolic meaning-making from indexical meaning-making, along with the integration of the latter by the former, along an individuating historical-biographical trajectory as an increase in the individual's agentive powers or semiotic causality. This goes hand-in-hand with an increase in the system's stored information, as illustrated by Salthe (1993: 262). As I said earlier, it is this increase along a self-organizing individuating trajectory which enables individual organisms qua dissipative structures to relate to other dissipative structures, human and otherwise. Now, 'relate to' has been by and large formulated in conceptual, cognitive or ideational terms - call it what you will - such that the accessing of systems of symbolic possibilities in language, depiction, mathematics, and so on, is understood as the capacity of the system to store and to relate to more and more conceptual categories in the process of expanding its Umwelt. The transition from an inward-looking dyadic perspective of interpersonal transaction with the mother to a concern for the world outside the mother-infant dyad, along with the concomitant emergence of experiential systems of naming and classification, illustrate this move. In actual fact, the engagement with phenomena outside this dyad is a qualitative jump in the notion of 'relate to', entailing new ways of constructing one's being in the world. At the same time, Salthe's notion of integrative levels reminds us that later (outer) levels always integrate earlier (inner) ones. That is, the symbolic integrates the indexical, which integrates the iconic. Higher-order or symbolic consciousness integrates primary consciousness, which integrates the topological material ground of our being in and relating to the world. The latter notion is akin to Damasio's proto-self, which is immersed in the topological-continuous flux of our material (not semiotic) interactivity with the material world. This is essentially how we begin our being in the world. Moreover, it is the basis on which a person's self-organizing individuating trajectory begins to relate to the

148

BRAIN, MIND, AND THE SIGNIFYING BODY

surrounding world, including the first primordial differentiation of self from non-self. This initial differentiation is topological, not typological. Nevertheless, it constitutes the ground for later integrative levels whereby interpersonal relations are constructed between increasingly differentiated selves and others, in an expanding diversity of social contexts, including those afforded by the interpersonal resources of language and other semiotic modalities. Figure 3.10 provides a schematized model of this process. The topological ground of our initial material interactivity with the world constitutes the vaguest, least differentiated basis for our understanding of and our establishing relations with others. It is a very wide and shallow attractor basin of vague possibilities which was our originary primary being in the world - 'being' before 'categorizing' and 'relating to', ete. differentiated out. It is the primordial basis of our capacity to enter into relations of empathy with others on the basis of feeling states of the body (Damasio 1999: 279-95). This least specified integrative level of initial at-oneness with the world is, however, integrated to the expansion of information-meaning at more specified levels. It gives rise to an increasing 'potential scope of sympathy riding on that same growth of information' (Salthe 1993: 263). Thus, the earliest experiences that the infant has of his/her embodied being in the world and the socio-affective exchanges characteristic of primary intersubjectivity in early mother-infant dyads are the deep roots of our being and as such are shared by others. Salthe's 'potential scope of sympathy' refers to the topological-eontinuous or analogic character of our most primordial forms of communication. Such communication is concerned exclusively with patterns of relationship, e.g. between organism and environment and/or between organism and others (Bateson 1973c [1972]: 340-4; Wilden 1980 [1972]: 164). Analogic communication may, therefore, be considered as a kind of Ur-interpersonal function from which all subsequent functions are logically derived. As the individual's trajectory expands into more specified levels of meaning-making on the indexical and symbolic levels, it also integrates the topological material ground of its primordial iconic being in the world in order to attain an understanding of how to engage in and maintain more specified (and specific) interpersonal relations with an increasing diversity of flesh-and-blood others. The increasing tendency in advanced capitalism to commodify and sell these same properties as 'synthetic interpersonal relations' in the market economy reveals the extent of the market's categorial reach - right back to the very topological material ground of our being in the world.

11. The Intrinsically Time-bound Nature of Semiosis: The Integration of Temporal Dynamics Across Scales of Expression and Content The tripartite perspective on language as particle, wave, and field allows us to clarify the nature of language as a dynamical system. The complementarity of these three perspectives illustrates the time-bound nature of language. As a dynamical system, language undergoes a constantly changing series of states in relation to its ecosocial and bodily environments. The wave/particle/field

BODY DYNAMICS, MEANING-MAKING, . . . Stage 1: being

o

149

Stage 2: feeling states; socioaffective dyads

Stage 3: conceptualizing categories and relating to

Stage 4: microfunctions (indexical)

Stage 5: metafunctions (symbolic)

o

Figure 3.10: The integration of initial at-oneness with the world to the expansion of information-meaning at more specified levels of semiosis along an individuating trajectory

perspective is a means of modelling the way in which the various dimensions of the system change in time. In logogenesis, the behaviour of the system as it unfolds in time is the changes it undergoes as the system changes from one state to another. At any given moment, the system is in a number of different states, corresponding to different overlapping metafunctional regions. The overall number of states in which the system is in at any given moment defines its state space. This means that the dynamical process of meaning-making can be seen as

150

BRAIN, MIND, AND THE SIGNIFYING BODY

simultaneous configuration of wave/particle/field in its state space. The diverse metafunctional regions interact with each other such that dynamical changes in one region affect and depend on the other regions. Furthermore, the intrinsic metafunctional organization of language has arisen in order that language may be selectively co-contextualized with phenomena in its ecosocial and bodily environments. Such phenomena are not external to the relevant system of relations, but are a part of that same system's intrinsic organization. It follows that ongoing, time-bound changes in the contextualizing dynamics that are in operation between unfolding text and its 'external' environment entails changes in the internal configuring of the relations among the diverse metafunctional components which are at play at any given moment in some meaning-making event. This holds for dynamical processes on both the expression and content strata. The evolution of the meaning-making act in time corresponds to a series of points and periodicities which define the trajectory of the system in its phase space. Expression and content are coupled such that dynamical relations and processes on one stratum mutually influence relations and processes on the other stratum. In turn, each of these strata is coupled to the body and to the ecosocial environment. The meaning-making body-brain does not interact with the external world by means of symbolic inputs and outputs. Rather, expression and content are dynamical subsystems of a larger whole. The former is based on neuromuscular activity; the latter on the recategorization of this in the CNS as semantic-conceptual content. In the computationalist model of cognition, on the other hand, the brain is a central control unit which is housed in a body, which is, in turn, located in an external world. The brain interacts with the outside world via the body's direct contact with the outside world. Thus, sensori-motor transducers 'translate' the data received from the outside world and the body, as well as the symbolic states which are the product of the brain's cognitive activity. This means that perceptual organs convert physical stimulation into symbolic representations of events in the body and the external world. The motor system converts the brain's symbolic representations of events into muscular movements. Cognition is a continuously cyclic and sequential process. First, sense data is received by some cognitive system; secondly, the given cognitive system algorithmically manipulates symbols so as to produce an output; thirdly, this output causes bodily movement. The cognitive system is specialized to symbolic representations. As such, it is seen as separate from the body and the environment. Cognition is disembodied and de-contextualized. On the other hand, time is a critical factor in the dynamical approach. Meaning-making occurs in time. Therefore, time itself must be seen as a central component in the explanation of meaning-making. This goes hand in hand with the view that all aspects of the relevant system are simultaneously changing in time. In logogenesis, it is the overall state of the system of relations which is changing from one moment to the next. Meaning-making as it unfolds in time exhibits properties of both continuity and discreteness. Moreover, diverse metafunctional regions are simultaneously in operation as well as interacting with each other within the phase space of the system. The distinctive principles of structuration which characterize the different metafunetions are properties

BODY DYNAMICS, MEANING-MAKING, . . .

151

which emerge over the time-bound trajectory of the system; they are not static givens. Moreover, meaning-making processes occur on many different timescales such that events on different timescales interact at the same time as these are embedded in a brain-body complex and in an ecosocial environment. Real-time is a topological-continuous flow. It is possible to plot a potentially infinite number of points along this flow. Each point corresponds to a state of the system. The neuromuscular activity that takes place in articulation is a process which unfolds along a temporal trajectory. Nevertheless, at each of the points in time from the beginning to the end of the activity there is a position which the articulators occupy in space. In section 2, pp. 109-16, the phoneme was shown to be a point in phase space in this sense. The continuous topological processes of articulation can be broken down into finer and finer points such that there is a state of the system at every point. This fact is a consequence of the bio-physical nature of the processes that take place in the vocal tract. The timing of events on diverse scalar levels in the unfolding meaning-making event is, therefore, of central importance in understanding the nature and functioning of such events. For example, the ranked scale of phonological units and relations, viz. syllable, foot, and tone group, entail different scalar levels of organization, each with its different rates of completion, its different periodicity cycles, and the different ways in which one scalar level temporally integrates with other scalar levels. As I pointed out above, vocal-tract activity occurs along a temporal trajectory in which the system of relations is required to spend a certain period of time in the phase space that is relevant to any given scalar level. Moreover, the timing of events on diverse scales in vocal-tract articulatory activity is cross-coupled with and integrated to dynamical processes on still larger timescales such as those implicated in the content stratum, along with still wider ecosocial processes and relations. The rank scale of both phonology and lexicogrammar should have much to tell us about the temporal nature of meaningmaking on the different scalar levels which constitute such events. Changes in variables on the content stratum depend on and are cross-eoupled to changes on the expression stratum. (See for example Martinec (1998) on the ways in which rhythmic waves of expression correlate with rhythmic waves of content.) Linguistic semiosis is best seen as being of mixed-mode character (Lemke 2000a; Thibault 2003b). This means that language has both typological and topological properties and relations. Change of state in the meaning-making process can be either continuous or discrete. It is essential that the facts of continuity and discreteness can be accounted for in a unified framework. Change from one discrete state to another and continuity of state (not time) are both essential characteristics of linguistic processes. For example, the very fine neuromuscular activity involved in articulation takes place in complex and changing environmental conditions with respect to factors such as localization of interlocutors, distance between them, movement towards or away from each other, the blending of different sound sources, physical features of the environment on different scales, and so on. The terrestrial scale on which speech sounds are produced and received is that of metres rather than kilometres or millimetres unless amplified by technological means. This scale is that of the physical environment which humans typically inhabit.

152

BRAIN, MIND, AND THE SIGNIFYING BODY

In the first instance, the wave train of airborne compression waves produced by the vocal tract is specific to the source of a mechanical disturbance caused by the neuromuscular activity of the vocal tract. The resulting wave train 'is specific to the kind of mechanical event at the source of the field' (Gibson 1983 [1966]: 16). The field of speech-sound waves specifies information about its physical source, i.e. the articulator. Such information constitutes a perceptual stimulus which is causally related to the environmental event which produced it. Information of this kind is iconic and/or indexical in character. On the other hand, the symbolic information which is specified in the sound event is not causally tied to its source in the same way. Instead, it depends on symbolic meaningmaking resources and their associated conventions existing on many different space-time scales that go beyond the causal linking of a sound event to its physical source. The neuromuscular substrate of vocal-tract activity must be flexible and able to adjust to many different environmental conditions. In other words, its own activity must be capable of flowing into and occupying a richly varied continuum of changing environmental possibilities on diverse semiotic and material scales.

Table 3.3: The metafunctional organization of bothphonology and lexicogrammar on the expression and contentstrata illustrating the operation of mixed-mode semiosis, combiningboth discrete typological-categorial distinctions and continuous topological variation on both strata Phonology

Experiential

Lexicogrammar

Difference of degree: quantity-X

Difference of type: type-X

Difference of degree: quantity-X

Difference of type: type-X

Degree of conformity to type-category; fuzzy sets; phonological topology; smearing of probability distributions

Particulate analysis of consonants and vowels as corresponding to discrete phonological typecategories (e.g. phonemes) as specific paradigmatic states corresponding to discrete articulatory categories seen as ensemble of distinctive features and part-whole combinations of these organized as a ranked hierarchy of phonological particles

Degree of conformity to type-category; fuzzy sets; semantic topology; indeterminacy of complementary transitive and ergative perspectives

Particulate analysis of phenomena into parts and wholes corresponding to discrete categories of experience, e.g. participantprocess relations

BODY DYNAMICS, MEANING-MAKING, . . .

Phonology

153

Lexicogrammar

Interpersonal

Field-like prosodic and scopal relations; cumulative, amplifying effects; degree of deformation, e.g. pitch shift

Tone system in tone group comprising a primary system of five contrasting simple tones and two compound tones

textual

Wave-like peaks Tonality and tonicity of prominence in tone group; rhythm in foot as defining boundaries; indexes oflexibegin-end cogrammar elements structures; tran- as units of informasitions; tion; culminative; demarcation of word dynamic periboundaries; odicities of word emphasis; tonicity, stress, contrast between and moraicity +stress/-stress; seen as waves +moraicity/of continuously moraicity; +tonicity/varying tonicity amplitude functioning as extension prosodies of variable length to integrate or concatenate lexicogrammatical domains

Field-like prosodic and scopal relations; cumulative, amplifying effects; degree of deformation

Mood, modality, lexis as categories of (inter) action and orientation

Wave-like peaks of prominence defining boundaries; begin-end structures; transitions; culminative

Theme, rheme; given-new

Likewise, the experiential resources of the clause, say, must be able to model the constant flux of the phenomena of experience as instantiations, to varying degrees, of the categories which are intrinsic to a particular language system. The tripartite perspective on lexicogrammatical form as particle, wave, and field emphasizes the dynamical character of the system. Instead of discrete, static categories, the language system must be able to adapt to the continuous variation of the flux of human experience. The mixed-mode character of language shows that language embodies both discrete, categorial distinctions and continuous topological change and variation. A better understanding of language and its workings is obtained once it is understood that both continuity and discreteness are intrinsic to its dynamics. Table 3.3 sets out the properties of the expression and content strata of language in order to illustrate the mixedmode character of language.

154

BRAIN, MIND, AND THE SIGNIFYING BODY

12. Scalar Heterogeneity and the Phonological and Lexicogrammatical

Rank Scales The existence of a rank scale on both the expression and content strata can be understood in relation to the diverse timescales which are simultaneously implicated in the meaning-making process. These timescales range from the milliseconds of neuronal processes, the fractions of a second in perceptual processes, the seconds of co-ordinated movement, the minutes or hours of an unfolding textual performance (e.g. a conversation), the months and years involved in learning the phonological and grammatical systems of a language, the entire lifespan trajectory of the individual's movement through a given ecosocial system, the history of an entire ecosocial system. Diverse timescales are not sealed off from each other. Instead, they are interrelated and integrated to each other such that the dynamical processes on one timescale affect those on other timescales. For example, the timescale involved in the articulation of syllables creates the patterns of strong and weak syllables which allow for the emergence of the timescale of rhythmic patterning in the foot. In turn, the patterning of Ictus and Remiss in the foot allows for the emergence of a still larger timescale at the rank of the tone group. Variations in loudness and stress produce rhythmic patterning. Rhythm has the function, in English, of making content words prominent; it also creates the potential for tonicity and tone on the level of the higher-ranking tone group (Halliday 1994 [1985]: 10; Cleirigh 1998: 12). The tone group is the only phonological unit in the rank scale in English which realizes a unit on the content stratum, viz. the information unit (Halliday 1994 [1985]: 295). In this way, phonological form is integrated with higher-scalar units of lexicogrammar and discourse on the content stratum. 12.1 The Lexicogrammatical Rnnk Scale The lexicogrammar of English is comprised ofa rank scale ranging from highestranking to lowest-ranking units, according to their constituency potential. Systemic-functional linguistic theory postulates the following ranks for English: clause, group/phrase, word, and morpheme. The number of ranks and the ways in which various systems are distributed across ranks differ from language to language. However, such details need not concern us here. The clause is the highest rank. It is not a constituent of some higher-ranking unit. According to systemic theory, the relations between clauses in discourse are semantic in nature, rather than lexicogrammatical. This is so because combinations of clauses constitute semantic relations at the level of text. The clause constitutes what Halliday has called the 'gateway' to text (see Thibault 1987: 611). On the other hand, clauses are made up of constituents from lower-ranking systems at group/phrase level. In turn, groups and phrases are made up of words; words of morphemes. The lexicogrammatical rank scale serves to integrate smaller-scalar lexicogrammatical units into larger-scalar ones. For example, the clause functionally integrates lower-ranking nominal, verbal, and adverbial groups into the functional structure of the clause. Rather than assuming that lower ranks are

BODY DYNAMICS, MEANING-MAKING, . . .

155

simply the smaller-scalar building blocks out of which higher-ranking units are assembled, it is no less relevant to see how higher ranks provide integrative contexts for units on lower ranks. In this perspective, we see more clearly how lexicogrammatical units at all levels are integrated to and have their function in the making of text in its context of situation. For example, nominal groups are integrated into clauses as various categories of participant roles. By the same token, the deictic and instantiating functions of the nominal group qua group rank have grounding functions whereby the Thing which is realized by the Head noun is instantiated as an instance of that Thing by grammatical systems and functions operating at nominal group rank. The lexicogrammatical rank scale serves to integrate lower-ranking units to higher-ranking ones according to a scale of indexical-symbolic integration, as follows. Rank: Morpheme

In English, morphemes specify semantic information in the form of semantic dictionary entries whereby a given item can be integrated to a particular worddistribution class at the next-highest rank. For example, consider the two morphemes in the word buyer. This item comprises the two morphemes [[buy] + [er]]. The two morphemes may be characterized as follows: Morpheme: buy> verb tr, [process: material action: disposal: acquisition: transfer of Medium Thing to Agent on payment of money; non-iterative; beneficile: potential: +benefactive] e.g. he boughtJohn a new boat Medium [Thing: alienable: transferable; material/immaterial] Morpheme: -er> nominalizing suffix [INSTIGATOR: AGENT; INSTIGATOR: INSTRUMENT], subcategorizes Thing in derived verbal noun as Agent responsible for the instigation of an action [Instigator: Agent] or for the instigation of a process [Instigator: Instrument] e.g. teacher, buyer, voter, ruler, lawn mower, bread slicer Rank: Word

Words specify type-specifications of Things, Processes, Qualities, and so on according to word-distribution class. In this respect, the semantic information that is integrated by the two-morpheme combination [[buy] + [er]] tells us that the resulting word belongs to the word-class noun, symbolizing a semantic typespecification of Thing, viz. [THING: HUMAN: PERSON/INSTITUTIONAL; AGENT: ACQUIRER OF GOODS/SERVICES IN ECONOMIC TRANSACTION ON PAYMENT OF MONEY]. Rank: Group

A type-specification of a Thing (or Process, Quality, and so on) is not tied to specific instances of the type. The common noun buyer is simply the name of a semantic class of Thing, as shown above. The resources of the nominal group

156

BRAIN, MIND, AND THE SIGNIFYING BODY

enable the type-specification to be instantiated as a specific instance of the given type-eategory. Nominal groups, as Langacker (1991: chapter 2) has shown, allow specific things to be specified in some discourse context according to four parameters. The four parameters are as follows with respect to the nominal group those two enthusiastic buyers: 1. 2. 3. 4.

type specification, expressed by the Thing element in the nominal group and non-determining modifiers: enthusiastic buyers; instantiation, expressed by determiners: those two; quantification, expressed by quantifying items: two; grounding, expressed by articles, demonstratives, and possessives: those.

Rank: Clause: Major

At clause rank, nominal and verbal groups form configurations of functionally related items which experientially construe a given scene or situation in terms of a number of interrelated transitivity roles. These roles consist of different categories of processes and participants according to the type of process in the clause, e.g. participants such as Actor, Agent, Medium, Senser, Sayer, and processes such as Material: Action, Mental: Perception, Verbal, and so on. For example, the nominal group those two enthusiastic buyers has the potential to realize a participant role in a clause, e.g. those two enthusiastic buyers went to the auction. In this clause, the given situation is experientially construed as a configuration of the following transitivity functions: [ActorI\Material: Action/-Circumstance: Location]. The clause is the rank where all four metafunctions come together to realize a semantic unit, called the message unit by Hasan (1996a: 119), which functions in discourse. In contrast to message units realized by minor clauses such as hello, good day, damn, and so on, a message unit which is realized by a major clause brings together in the one structural output the full array of metafunctional meanings. Minor clauses, on the other hand, are primarily interpersonal in function in the sense that they may function as both initiating and responding moves in discourse as well as realizing attitudinal/affective meanings. However, they do not exhibit clearly definable transitivity roles whereby an experiential situation, seen as comprising particular classes of process and participants, is construed. Moreover, they do not have the feature of finiteness, which is means by which the clause is assigned a point of reference in the here-and-now of the speech event through the resources of primary tense and modality. Minor clauses do not fully participate in all of the systems which operate in the lexicogrammatical rank scale. For this reason, they are, in some respects, not unlike child protolanguage in the sense that their status is primarily interpersonal-indexical (see Hasan 1996a: 118-19; Thibault 1992a). Hasan accordingly assigns the feature [punctuative] to minor clauses. On the other hand, Hasan assigns the feature [progressive] to major clauses because the full range of metafunctional options which these clauses realize means that they participate fully in the development of discourse and its entextualized products and records. It is in this sense that Halliday has referred to the clause as the 'gate-way' to text; it is the means whereby systems and

BODY DYNAMICS, MEANING-MAKING, . . .

157

structures which operate on different ranks are integrated to and realize a semantic message unit which can contribute to the development of discourse. As we shall now see, it is discourse constructed in this way that exhibits to the full the properties of scalar heterogeneity. Let us reconsider our earlier example from this point of view. The morpheme suffix -er classifies the root morpheme, comprising the de-verbal process noun buy, as the Agent/Instrument of the process named by the root; the word buyer symbolically names a type-category of Thing, viz. the agent or instrumentality which instigates the process buy; the nominal group those two enthusiastic buyers specifies an instance of the semantic category of Thing with reference to a given discourse context; in turn, the clause subcategorizes this instance of the Thing as instantiating the participant role Actor in the transitivity structure of the experiential scene realized by the clause. In combination with the other metafunctions at play in the clause, this experientially construed scene has the potential to participate in and contribute to the development of some discourse about persons, actions, and locations which exist on some very different spacetime scale with respect to the moment-to-moment unfolding of the discourse itself. It is the clause which integrates lower-ranking units to discourse. Morphemes and words do not as such directly realize discourse meanings. Rather, they are mediated by the systems and structures operating on the higher ranks of group and clause, which integrate them to the discourse level of organization. This is no less the case for a clause realized by a single morpheme, e.g. Go! The preceding discussion shows how, at any given rank, the system changes state according to the variables in operation at that particular rank. State changes at a given rank occur in relation to the more stable parameters operating at the next-highest rank. In the case of the expression stratum, where phonological systems and structures occur, these variables have to do with the neuromuscular dynamics of vocal-tract activity. For example, the foot, in relation to the lower-ranking syllable, establishes a set of parameters on a slower timescale which influence and regulate changes in the state variables on the faster timescale of the syllable. In this way, slower timescales on higher ranks modulate the temporal dynamics of faster timescales on lower ranks. By the same token, the faster dynamics of the lower-ranking unit can also affect the slower dynamics of the higher unit. On the content stratum, the lexicogrammatical rank scale is not directly implicated in body dynamics in the same way, but in ecosocial ones. In this case, the integration of lower-ranking units to higher-ranking ones means that the categorial reach of lower-ranking units is extended by their functional integration to higher units. This was shown above in the discussion of the way in which the integration of morpheme to word and word to group, and so on, entails an extension of the contextual reach of a given item. If, for example, language were limited to lower-ranking morphemes and words per se, its contextual reach would be much more restricted for the reasons already mentioned above in relation to minor clauses and protolanguage. The lexicogrammatical rank scale is, therefore, a resource which extends the contextual reach of language such that diverse ecosocial space-time scales are integrated by the interplay of indexical and symbolic factors which the multiple metafunctional development of

158

BRAIN, MIND, AND THE SIGNIFYING BODY

discourse, as afforded by the clause, makes possible. The criterion of stability mentioned in the previous paragraph applies to the lexicogrammatical rank scale as well. For example, a morpheme as such symbolizes some semantic unit, however small, such as the meaning [Agent/Instrument] discussed above in relation to the nominalizing suffix -er. However, it is the contextual integration and mediation of this element by higher-ranking units which provide the more stable semiotic environment enabling its grounding in a specific discourse context. Table 3.4 presents a three-level hierarchy view of the organization of the content stratum of language. Table 3.4: The content stratum of language in relation to the three-levelhierarchy Scalar level

Difference

L+l

System of semantic differences as SI

L

Lexicogrammatical rank scale comprising hierarchy of ranked units:

Discrete typologicalcategorial distinctions; difference of type

Symbolic

Semantically salient distinctions symbolically construe experience

Indexical

Grounding of clause by Finite element in verbal group in terms of temporal/modal proximity (tense/modality) and in terms of person deixis (Subject); grounding of nominal group in terms of spatial proximity (demonstratives) or person deixis (possessives); both give the process and the participant(s) in the clause a point of reference in the unfolding (con) text by means of intra- and extratextual indexical signs

Iconic

Particulate, field-like, wave-like, and recursive modes of lexicogrammatical realization iconic to ways in which diverse metafunctions semantically organize experience

clause; group/phrase; word; morpheme

L-l

Continuous quantitative variation in the phenomena of experience mapped onto continuous quantitative variation in sensori-motor and CNS activity

Topologicalcontinuous variation; difference of degree

BODY DYNAMICS, MEANING-MAKING,

159

12.2 The Phonological Rank Scale Vocal-tract gestural activity has both spatial and temporal organization (see section 2). The creation of emergent spatia-temporal organization occurs along a particular trajectory. Vocal-tract activity has its existence only along a temporal trajectory. Thus, the syllable is a unit of articulation which cannot be defined in terms of discrete and static segments or positions in the overall flow of articulatory events. The dynamics of articulation are extended along a temporal trajectory and can only be defined as such. This means that the syllable is, on its particular space-time scale, a dissipative structure which is entrained by its trajectory (Salthe 1993: 260). Furthermore, the existence of a phonological rank scale shows that phonological trajectories exist at a number of different spatiotemporal scalar levels. As a dissipative structure, the reorganization of micra-level phonetic processes as macro-level phonological ones means that the lower-scalar or microscopic phonetic processes of articulation are entrained into the macroscopic organization of phonological relations and processes. This requires an input of energy along with specific dynamical laws so that the macroscopic order emerges. An articulatory trajectory entails that external entropy increases as the second law is maintained. By the same token, the emergence of macroscopic phonological order means that the entropy production of the structure afforded by the trajectory is reduced. It is this fact which enables local sinks of lowdimensional or macroscopic order - i.e. phonological form in the present case to appear without violating the second law. Organization emerges when the properties and dynamics of the oral cavity qua physical entity, whereby vocal-tract properties are unco-ordinated, to a state in which phonetic properties are coordinated in a stable regime such that the activity of a given feature depends on the activity of other features with which it is co-ordinated. In this way, each microscopic phonetic feature is entrained to the principles of organization of the higher-order articulatory trajectory of the syllable. The neuromuscular activity involved in articulation is thus entrained by trajectories existing at a number of different scales such as those suggested by the phonological rank scale. I would suggest that the notion of trajectory is the fundamental entity in meaning-making. The neuromuscular activity of articulation is a dissipative structure which functions as a material sign of the existence of our and other's meaning-making trajectories (Salthe 1993: 260). It is what provides the basis for their material connectivity. What brings about the emergence of such trajectories? What enables their form and their dynamics to endure over extended periods of time? If meaning has properties of organization in both space and time such that sustained meaningful interaction with others occurs, then this suggests that the control parameter which organizes the lower-level neuromuscular activity of the oral cavity as an ordered articulatory trajectory showing properties of macroscopic phonological organization is meaning emanating from higher-scalar levels. Thus, lower-level oral-cavity properties behave differently when entrained to the ecosocial semiotic dynamics of a higher-order logogenetic trajectory than they would otherwise do. Meaning, as we have seen, exists in symbolic neural space in the context of the body and the higher-scalar ecosocial environment. The increasing discrimination or differentiation which is afforded by the reorganization of microscopic phonetic

160

BRAIN, MIND, AND THE SIGNIFYING BODY

properties as macroscopic phonological ones means that the system organizes itself into a new hierarchy of levels such that new functional components come into being. The emergence of phonological order as the reorganization of sensori-motor routines into more finely categorized gestural systems leads to their further recategorization as conceptual and lexicogrammatical systems. According to Edelman (1992: 100-1, 120-1), conceptual routines are based on the interaction between perceptual activity and value systems. This shows that conceptual categories are mediated by information about the external and internal environments of the organism. Thus, the self-organization of the macroscopic phonological level on the expression stratum means that vocal-tract activity is now dependent on the synergy among different component parts of that activity. Furthermore, this activity is itself enslaved to higher-order principles of organization emanating from the content stratum. Therefore, the expression stratum is now seen as being a means whereby the organism systematically extends itself into and is cross-coupled with its ecosocial environment. Thus, the recategorization of sensori-motor routines as conceptual ones and the further recategorization of these as semiotic categories of expression and content, respectively, provides a means of explaining what theorists of dynamical systems refer to as the 'embeddedness' of the cognitive system in a body and an environment. The processes of semiotic reorganization across hierarchical levels that I have outlined here show how the interaction of organism and environment is a supersystem in which the exchanges of matter, energy and informationmeaning constrain the lower-level subsystems (e.g. individual organisms). In this way, both sensori-motor and conceptual routines are reorganized as semiotic expression and content such that the ecosocial level is imported into the organism's body-brain dynamics. The entrainment of the organism's intrinsic dynamics to the ecosocial scalar level amounts to an expansion of the system's phase space by increasing the meaning-making potential and, hence, the semiotic freedom of the organism qua individual agent. The question of the relation of a meaning-making to its environment can be understood in the light of the notion of trajectory introduced earlier. The flow of meaning from an intentional source in symbolic neural space along an unfolding action trajectory is a temporal process. Thus, the flow of meaning, understood here as semiotic content, produces and entrains bodily changes along its trajectory. Moreover, such bodily changes are themselves coupled to contextual ones. Thus, the dynamic temporal unfolding of articulatory gestures occurs at the same time as dynamical processes of neural activity on lower scales and dynamical environmental processes on higher scales unfold in time. As we saw in section 2, the phonological rank scale specifies a hierarchy of cycles of vocal-tract activity (Cleirigh 1998: 38-9). The first cycle is the process of opening and closing the vocal tract by articulation to dampen phonation in the production of syllables. This can be modelled as a binary oscillation between + syllable weight and - syllable weight, i.e. between moraic and non-moraic syllable constituents (Cleirigh 1998: 38). The second cycle refers to the varying of properties such as loudness and duration so as to add stress. This process is termed rhythm and can be seen as 'switching stress on and off' (Cleirigh 1998: 39). The third vocal-tract cycle is the applying of major pitch shifts to create tonic

BODY DYNAMICS, MEANING-MAKING, . . .

161

prominence or tonicity. This process is intonation and consists in switching tonicity on or off. Each of these cycles occurs at a given rank in the phonological rank scale. Thus, the syllable is defined in terms of its relations to lower-level segments as well as to the higher-ranking unit, foot. In turn, the syllable is integrated to the level of the foot in the sense that it receives further levels of contextualization by being integrated to a higher-order unit. The foot itself is integrated to the still-higher-ranking unit, the tone unit. Each rank specifies a scalar level with its own temporal dynamics - faster on lower scales, slower on higher ones. Each rank in the scale may be seen as a level of specification integrated by the dissipative structure - the organism - that materially embodies the trajectory. That is, neuromuscular processes are continually reintegrated under the increasing specification of phonological structure. It is this integration to increasingly higher levels of specification that provides the basis for the trajectory's coherence over time (Salthe 1993: 261). The emergence of macroscopic phonological organization in the form of a ranking scale of articulatory, rhythmic, and tonicity cycles of vocal-tract activity from microscopic phonetic processes illustrates the principle whereby the lower-level physical embodiment of the trajectory self-organizes. What we have here is an illustration of the way in which articulatory activity qua bodily movement is organized on a diversity of timescales. Moreover, this reveals a fundamental aspect of embodied meaning-making: temporal body dynamics on different scales face 'two ways', viz. inwards to the faster temporal cycles of neural activity in the CNS and outwards to the slower temporal cycles of the other systems - the individuals - with which a given system interacts. Meaning-making unfolds in time and it is crucial that there are resources for attuning to the temporal dynamics of those with whom one interacts. Thus, the physical organism qua dissipative structure is semiotically connected to other organisms by virtue of their trajectories (Salthe 1993: 262). As we saw above, the organism is semiotically integrated to a trajectory in the first instance under the increasing specification of the expression stratum. The integration of neuromuscular potential to macroscopic phonological organization connects the organism to others' meaning-making trajectories on the basis of potentially shareable articulatory, rhythmic, and tonic resources such that it can functionally relate to other organisms who make and recognize the same phonological discriminations. Compare this to the newborn child whose vocal gestures are, in the first instance, iconic of immediate body states and feelings in the mother-infant dyad. The infant can only access the lower integrative levels of topologicalcontinuous variation in voice dynamics; he or she does not yet have access to the stored phonological information deriving from higher levels. To be sure, the infant can both produce and respond to articulatory, rhythmic and pitch discriminations, but he or she can do so only on the basis of the lower integrative level of their iconic significance. The expansion in proto-language and later language of vocal activity into higher integrative levels where the indexical and symbolic distinctions of phonology are made thus entails an expansion of the child's bodily potential to relate to the semiotic trajectories of others. A parallel observation can be made with reference to the content stratum. Before the onset of a lexicogrammar, the child's semiotic content is restricted to

162

BRAIN, MIND, AND THE SIGNIFYING BODY

the perceptual purview of the mother--ehild dyad and its immediate surroundings. In a sense, there is discourse without lexicogrammar. It is a discourse level of semantic content which is integrated to its contextual environment on the basis of indexical ties for the most part.

13. The Emergence of Linguistic Categories from the Child's Primary Forceful Interactions with its Environment Seen as a physiological space, the oral cavity is a neuromuscular potential for achieving specific goals. In order to perform a given articulatory gesture, the infant must adjust the parameter values of the oral cavity in order to achieve a particular articulatory gesture (Browman and Goldstein 1995: 181). For example, a particular gesture may involve a constriction in the distance between the upper and lower lips combined with a stiffness setting and damping, which is determined by the amount of time required for the system to approach the goal of lip closure. This process requires the deployment of spurts of neural energy at precisely the right moment and to the right degree in order to activate the muscles required to perform the gesture. Children learn to do so by experiencing the many different values of the oral cavity as produced in their spontaneous babbling and other vocalizations produced in early infancy in order to attain a specific goal (Thelen 1995: 80). The process of developing articulatory gestures involves exploring the range of parameter values in the state space of the oral cavity and selecting those values that selectively co-contextualize with relevant environmental affordances and the goals and needs of the child. Halliday's description of Nigel's use of very high-pitched squeaks at age 0; 6 months (see also chapter 4, section 10, pp. 202-5) in order to draw attention to the scattering of the pigeons is an example of how the system learns through action. The child's vocal gesture - the high-pitched squeak - is an iconicindexical act which points to some phenomenon (the pigeons) within the child's perceptual purview. The vocal gesture is iconic in the sense that it can be interpreted as expressing bodily and feeling states of its utterer; it is indexical in the sense that it both indexes (1) its utterer as the source of the squeak qua acoustic event in the given environment; and (2) the pigeons as the object of interest or attention in the situation. The pigeons constitute an environmental affordance in the sense that the perceptual information which the child picks up about this environmental event affords a contextual value, as determined by the indexical act. This micro-semiotic act lasts mere seconds in the local context in which it occurs. However, the cumulative effect, over time, of many such micro-semiotic acts cascades into the developmental landscape, as Halliday documents. There is another sense in which the real-time of the action is implicated in or interwoven with developmental time. In the time required for the performance of a given articulatory gesture, values such as stiffness and damping ratio are constant over the time-span in which the act is performed. On a higher temporal scale, these values themselves change in the course of development as the neuromuscular apparatus of the oral cavity itself changes. The anatomy of the articulatory apparatus changes as muscles become stronger, more efficient, and so on. The

BODY DYNAMICS, MEANING-MAKING, . . .

163

point is, as Thelen (1995) shows in another context, that diverse timescales are all simultaneously interwoven in the same developmental dynamic. Therefore, scalar heterogeneity is already fully evident in the indexical phase characteristic of children's early protolanguage. There is a further sense in which Halliday's example involves more than the biomechanics of articulation. That is, the development of the articulatory processes whereby the child establishes first-order contextual redundancies between vocalization and perceived environmental event constitutes in itself a fundamental fact about the development not only of sensori-motor processes e.g. the neuromuscular processes of vocal-tract gestural activity - but also of higher-order meaning-making. Vocal-tract gestural activity, as in the example mentioned above, and the perceiving of the relevant environmental event, are cross-modally correlated on the timescale on which the act occurs, i.e. seconds and fractions of seconds. Children vocalize and look at, touch, and listen to perceptual phenomena of many kinds across a large number of specific occasions. In the process of engaging with perceptual phenomena in their surroundings, as well as with other people, infants may engage in a varied repertoire of vocal gestures ranging from babbling to crying to cooing, and so on. In so using the neuromuscular resources of the vocal tract, they therefore learn what it feels like to deliver different levels of energy to the muscles of the vocal tract as well as what effects these have on their surroundings, including other human beings. That is, infants modulate the resources of the vocal tract and in so doing they produce salient changes in their immediate environment. For example, in the instance cited from Halliday the child's high-pitched squeak both directs the mother's attention to the pigeons and elicits a linguistic response from her. The work of Halliday and others on infant protolanguage has shown that vocal gestures such as the one referred to here have both expression and content. The point is that children, through the repeated use of such vocal and other gestures in potentially very many different situational contexts, are learning about both their perceptual-motor systems and how these relate to the world. In other words, their sensori-motor exploration and sampling of the environment constitutes information for the brain both about their bodies and about the world beyond their bodies. The infant is experimenting by modulating the energy to the muscular systems of the vocal tract and seeing what kinds of vocal gestures result, what kinds of phenomena these correlate with, the kinds of responses these obtain from others, and so on. In this way, the infant learns that a given category of vocal gesture such as the high-pitched squeak referred to above can be correlated with a given phenomenon in the world by virtue of the process of recategorization of sensori-motor routines as conceptual routines (Edelman 1992: 89-90, 246-7). This process of recategorization produces a correlation between the conceptual-semantic discriminations the organism makes in its environment and the possibilities for acting on and intervening in the environment. That is, the vocal gesture is a dialogically oriented and coordinated act whereby the infant causally intervenes in the world by, for example, directing the attention of others to some object of interest. In this sense, the act is proto-interpersonal in character.

164

BRAIN, MIND, AND THE SIGNIFYING BODY

By the same token, the correlation of such sensori-motor routines with conceptual discriminations made on the basis of value-laden perceptual activity is proto-experiential. We see here how early perception-action routines involving vocal and other gestures constitute active, movement-based intervention in the environment combined with the perceptual pick-up of relevant environmental discriminations. These activities take place in the here-now scale of seconds and minutes. At the same time, they are a constitutive part of a longer timescale in which language emerges on the basis of these early re categorizations of sensorimotor routines as conceptual ones. Categories emerge on the basis of generalizations made from task-specific activities. Thus, the child studied by Halliday learns that a given category of vocal gesture qua sensori-motor act, involving the control of the neuromuscular systems of the vocal tract, correlates with some aspect of the world outside his or her body. Moreover, vocal gestures exhibit vectorial qualities of direction and magnitude along their trajectory such that they afford orientation both to the source of the event and to the environmental affordance qua stimulus information which is the source of interest or attention. This is so in the sense that the vocal gesture is perceived as being intentionally directed to a given environmental affordance which it indexes as a relevant contextual value. Furthermore, the vocal-auditory act cross-modally mutually correlates with the perception of inputs from vision, movement, and other information sources. The process of mutual correlation between the sensori-motor and the conceptual domains arises on the basis of the brain's functioning as a somatic recognition system (Ede1man 1992: 89-98). The brain recognizes and categorizes events through its own activity, rather than through processes of instruction. Thus, sensori-motor repertoires and conceptual repertoires mutually recognize each other. For example, the child's high-pitched squeak qua sensorimotor act correlates with the micro-functional semiotic category 'demand for explanation'. It is this two-wayprocess of mutual recognition between the sensorimotor and conceptual domains that paves the way for the emergence of the twoway semiotic relation of realization between expression and content in the event of language. Of course, the example itself shows the limitations of the notion of 'conceptual' category as a covering term for the diversity of categorial distinctions that are correlated with sensori-motor repertoires. The term 'conceptual' is ideationally biased. In actual fact, the categories correlated with sensori-motor routines also and most crucially include interpersonal-interactional ones, as in the example. In ways that will be explored in chapter 6, it can be seen that the brain's activity of mutual correlation between sensori-motor and conceptual routines is a contextualizing activity whereby me ta-redundancy relations are created between the two domains. This last point has important consequences for how we understand the relations between language, context, and brains. As we have seen, the metafunctional diversity of language form - both expression and content - means that semantics is intrinsic to the internal organization of language form. Moreover, language form is characterized by the functional overlap of diverse semantic regions. Each of these metafunctional regions makes its specific contribution to the meaning of the whole. The metafunctional organization of

BODY DYNAMICS, MEANING-MAKING, ...

165

language shows how language is internally related to the contexts with which it is integrated in use. Thus, context is not an external happenstance or epiphenomenon which has no systematic functional relationship to the organization of the content and expression strata of language. This has important implications for our understanding of how language is learned and embodied in neural networks. First, the contextual redundancies between language forms and their contexts of use mean that the learning of language incorporates the contexts of the meanings and forms learned. Secondly, neural networks embody the contexts, bodily activities, and social practices in which meanings are learned in the form of the meta-redundancy relations that connect, say, vocal-tract gestures, speech sounds, lexicogrammatical forms, discourse structures, and so on, in larger patterns of contextualizing relations. However, I shall now return to the far more primordial, pre-linguistic forms of interaction that are the central focus of this section. LakoffandJohnson (e.g. 1999) have discussed the relationship between our forceful sensori-motor interactions with the world and the emergence of linguistic categories of goal-directed action. The primordial experiences of kicking, reaching, grasping, pulling, hitting, and so on, in early infancy involve the use of children's bodies in order to bring some aspect of the world in line with their needs and desires. In so doing, the children must harness and reduce the many degrees of freedom of both their bodies and the world so that some kind of match is created between the two in a given context of action. In this way, children learn to generalize categories of embodied-action schemata which they can adapt and use across a very wide variety of different situations. A schematic category of the body's forceful interactions is abstracted from these diverse situations and correlated with a conceptual categorization. Moreover, Edelman, as we saw above, claims that the brain is a selective recognition system that recognizes events by its own activity, including its own activities of recognition. This is important because the recognition of a category such as 'forceful interaction with the world' as a means of bringing about causal effects in the world entails, with the emergence of proto-language, a recognition that sensori-motor acts such as the vocal gesture discussed above can be used intentionally to cause someone else to act in a causal way, both physically and semiotically. In the above example, the child's high-pitched squeak is an intentional act which is dialogically oriented to causing another social agent - the mother - to provide an explanation of the perceived phenomenon (the pigeons). We can see here how the correlation of sensori-motor activity with a conceptual category gives rise to the schematic category [BODY-FORCEFUL INTERACTION-ENVIRONMENT], as described in Lakoff and Johnson (e.g. 1999: 270-6). This may be seen as a kind of Ur-experiential category which paves the way for its further re categorization as more delicate linguistic categorizations in the way described above. The embodied schema described here may be analysed into its constituent parts, as shown in Table 3.5. The schema outlined in Table 3.5 shows that bodily activities such as reaching for, grasping, and hitting objects can function to explore and obtain information about some object through, for example, haptic exploration. By the same token, they can be used to cause the given object to behave in a given way such as

166

BRAIN, MIND, AND THE SIGNIFYING BODY

Table 3.5: Emergence of embodied categoryformation and its differentiation into 'learning about' and 'acting on ' through forceful interaction with environment Body part

Vectorial control of force in formation of movement trajectory towards object

Object in environment

Hand-arm

Reaching movement

Grasp or touch object for haptic exploration

Hand-arm

Reaching movement

Cause object to move in desired way

Agent

Process: action

Affected

making it move in the desired way by hitting or kicking it. Of course, this second possibility also affords perceptual information about particular kinds of environmental events, in particular information concerning the relations between objects and events which are implicated in a causal series. The understanding of the possibility of harnessing causal relations between one's bodily activities and the behaviour of objects in the external world through actions such as hitting, knocking, kicking, and so on, is a forceful interaction between body and world involving physical (efficient) causality. However, infants also learn that their bodily actions can be deployed in order to cause other persons to act in ways which satisfy their needs and desires. In this case, the causality involved is not physical; instead, it is semiotically mediated by means of some elementary sign such as a vocal gesture, facial expression, gaze vector, or other bodily movement. Body-Forceful Interaction-Environment schemata such as those just described emerge from the very first weeks of the infant's life when infants begin reaching towards objects in their environment (Thelen 1995: 83). We can see how the Body-Forceful Interaction-Environment schema can serve as a means of learning about the environment as well as a means of acting on and bringing about causal events in the environment. In the very earliest stages of reaching to touch objects, it is unlikely that the 'learning about' and 'acting on' aspects are differentiated. In time, new forms and functions emerge from the dynamics of the earliest possibilities. Without trying to suggest any kind of precise temporal or developmental sequence, it can be seen how Halliday's (1975; 1978b; 1993: 109) pragmatic ('doing') and mathetic ('understanding') categories, which are two of the discrete microfunctions he postulates as occurring in the early stages of children's protolanguage, can be seen as a further, more specified development and integration of the 'learning about' and 'acting on' functions deriving from the increasing topological differentiation of the originary Body-Forceful Interaction-Environment schema. This most abstract schema therefore provides a model for the later emergence of more specific schemata which are diversified over various

167

BODY DYNAMICS, MEANING-MAKING, . . .

domains. For example, we can see how the most schematic experiential category of [Agent 1\ Process 1\ Affected] in the linguistic domain can be derived from the still more schematic conceptual categorizations of sensori-motor experience. In turn, the most schematic linguistic categories relate to the more specific lowerlevel experiential categories which operate in specific experiential semantic domains such as the material, the mental, the relational, the verbal, and so on (see Davidse 1991: 14, for a detailed development of this principle with respect to material processes clauses in English; see also Halliday 1967b, 1967c, 1968). Similarly, the abstract sensori-motor schema also provides a superordinate model for the emergence of more specific interpersonal action schemata which are diversified over various semantic domains such as 'giving' and/or 'demanding' linguistically construed information or material goods-and-services. The recognition of this category entails both a capacity to discriminate the forceful control of the body or some part of this in relation to desired outcomes in the world and, at the same time, the embodiment of an understanding of the possibilities for the self to act on the world of the non-self, including other agents who can be recruited as eo-agents in the bringing about or satisfying of one's needs and desires. This means that the category is Ur-interpersonal as well. This aspect may be schematized as follows: [[SELF-AS-AGENT] ACTS] DESIRED GOAL]]

-->

[OTHER-As-AGENT2

1\

ACTION

[[child utters high-pitched squeak as 'demand for information'] responds: gives information]]

1\

-->

[mother

Figure 3.11 illustrates the emergence of embodied category formation from the body's primary forceful interactions with its immediate environment. The correlation of the original sensori-motor schema with the conceptual schema 'forceful interaction' is topological in character. It constitutes a very wide attractor basin representing a very broad - i.e. schematic - category in the earliest stages of the infant's engagements with the world. The basis of this category is iconic in the sense already defined, viz. topological-eontinuous variation in forceful movement of the body selectively maps onto topologicalcontinuous variation in the world. There is differentiation, but it is topological differentiation, based on continuous sensori-motor exploration and adjustment to continuous variation in the stimulus flux afforded by the events in the child's environment. The experiential and interpersonal functions in the linguistic sense are not differentiated. Instead, the broad attractor basin is differentiated on the basis of more specific bodily engagements with the world such as touching, grasping, kicking, pulling, and so on. The onset of protolanguage represents a dynamical shift to a new state space, itself brought about by the entraining of the child's trajectory to the higherscalar trajectory of the ecosocial system in which he or she lives. In this way, the prior attractor landscape is altered by emerging changes in the relations among its component parts, along with changes in the boundary conditions of the system as it moves along its time-bound trajectory. In this context, the prior

168

BRAIN, MIND, AND THE SIGNIFYING BODY

phase of topological-continuous differentiation is reorganized, giving rise to new patternings and, hence, new possibilities for acting and meaning in the world. The destabilization of the Ur-category described above as Body-Forceful Interaction-Environment gives way to a shift into new attractor states such as the microfunctions in the indexical phase of protolanguage. The appearance of the protolinguistic microfunctions represents increasing semiotic differentiation which is now increasingly typological in character, and at the same time the emergence of indexical probability, which integrates iconic necessity, leads to extended adaptive capacity over increasingly larger space-time scales.

Figure 3.11: Linguistic categorization and its emergence from pre-linguistic forceful interactions between body and environment BODY - FORCEFUL INTERACTION-ENVIRONMENT

t

BODY PART - VECTORIAL CONTROL OF FORCE IN FORMATION OF MOVEMENT TRAJECTORY TOWARDS OBJECT - OBJECT IN ENVIRONMENT

'Act on'

'Learn about' Protolinguistic microfunctions:

t

Linguistic metafunctions:

t

M'~~";' TRANSITIVITY + MOOD

~~

Proposition/Proposal

[Agent/Process/Affected]

Transitive

Indicative

Ergative

Imperative

Actor-Process-Goal

Instigator-Process-Medium

GoalGoaldirected achieveing + intention -intention

Event Actiou instigation instigation + event + action

Declarative

Interrogative Oblative

Imperative

The flood burst the dam

Heeut the cake

Did he eutthe cake?

Cut the cake!

He cut the cake

The car struck the tree

He walked the dog

Shall I eutthe cake?

Part III

This page intentionally left blank

4 The Semiotic Basis of Consciousness

1. First-person and Third-person Accounts of Consciousness

Psychologists and neuroscientists agree that conscious experience is a function of brain processes. Furthermore, they agree that conscious experience is directly accessible only to the individual who is undergoing that particular experience. This general consensus has led to the formulation of two main hypotheses concerning the nature of conscious experience. According to the first hypothesis, only the subjective verbal reports of the person who undergoes the conscious experience can be taken as a reliable indicator of that experience. This experience may be 'reported' by linguistic means to someone who is extrinsic to the reported experience; however, the experience itself remains the unique possession of the person who experiences it. According to the second hypothesis, consciousness may be explained just like any other observable physical process. That is, consciousness is, in the final analysis, explainable in terms of neural activities in the brain. These neural activities are accessible to an external observer who can observe these independently of the subjective reports of the person who undergoes the conscious experience. In this second view, consciousness is reducible to and is, therefore, explainable in terms of the physical (neural) processes that take place in the brain. However, clinical researchers (e.g. Libet 1996: 101) point out that studies of brain activity using the technology of positron emission-tomography (PET) and magnetic resonance imaging (MRI) can tell us where, in the brain, changes in activity take place. These technologies cannot, however, describe the specific neuronal activities that may be involved. Nor can they tell us anything about the relationship between these activities and our consciousness of our own or others' mental events. A further set of questions arises at this point. The first concerns the unitary and integrated nature of conscious experience. The human brain possesses some 100 billion neurons; each neuron may have thousands of connections with other neurons. Moreover, many brain functions are localized to specific areas of the cerebral cortex in the form of 'specialized columns of neurons' (Libet 1996: 114). Yet, whatever neural activity in the brain does reach the threshold of conscious experience is experienced as both unified and integrated (Libet 1996: 114). The second question has to do with whether mind or conscious mental activity exerts any causal influence on physical brain processes. It is worth quoting Libet at length on the two approaches which have predominated in attempts to answer these two questions:

172

BRAIN, MIND, AND THE SIGNIFYING BODY

On issue A, Eccles (Popper and Eccles, 1977: 362) has proposed, based on a dualist-interactionist view, that 'the experienced unity comes not from a neurophysiological synthesis but from the proposed integrating character of the self-conscious mind'. This view had, in principle, been expressed by Sherrington (1940), and also by Sperry (1952, 1980) and Doty (1984). For Sperry and Doty, however, the mind doing the integrating was viewed in monistic terms, as an emergent property of brain processes. On issue B, both Eccles and Sperry have proposed that the 'mental sphere' could influence neuronal function but in dualistic vs. monistic terms, respectively. The view held probably by most neuroscientists (and perhaps modern philosophers) is a monist-deterministic one, in which conscious mental experience is simply an 'inner aspect' of brain function (identity theory); it is fully determined by knowable physicochemical processes in the brain, and its apparent ability to influence brain function is a subjective illusion with no actual causal powers. Although each has explanatory power and each can be shown to be compatible with (not falsified by) the available evidence, none has been subjected to appropriate or adequate experimental testing in a format that could potentially falsify it. (Libet 1996: 114) Libet then discusses the notion of 'binding' as a possible solution to the first of the two questions discussed above. The notion of binding is based on the discovery of global patterns of synchronization of neural activity, which some researchers see as 'the neural coding for a unified mental image in an otherwise chaotic background' (Libet 1996: 114). However, Libet cautions as to the need to distinguish between "binding" at a cognitive level (that mayor may not involve conscious experience) and the binding that refers to unity experienced in conscious awareness' (1996: 114). Libet hypothesizes the notion of a unified 'conscious mental field' (1996: 115) which would solve the question of both the unity and the integration of conscious experience. Furthermore, the conscious mental field, produced by diverse neuronal activities, can causally affect and change neuronal function (see Thelen and Smith 1994: 189 for an alternative account based on direct cross-modal communication; chapter 5, section 3, for further discussion). Many brain processes are not conscious even though they relate to functionally differentiated brain functions which have to do with memory, perception, learning, and so on. However, the recognition that there are both conscious and unconscious brain processes has not led to any scientific consensus concerning the nature of consciousness. The reductionist view that consciousness can be explained in terms of physical brain processes remains unable to relate these to personal experience or to explain the link between such experiences and the physical brain processes to which these are causally related. Bateson (1980 [1979]: 39-40) points out in this regard that we are aware of the products of perception; however, we are not aware of the neurophysiological processes which make perception possible. Bateson's observations can be extended to conceptual and semiotic, including linguistic, processes. Thus, we are aware of the thoughts that we attend to in inner awareness whereas we are not aware of the neuronal activity which realizes these thoughts. Moreover, many aspects of linguistic

THE SEMIOTIC BASIS OF CONSCIOUSNESS

173

processing in the brain are not conscious. This fact has long been recognized by linguistics. Typically, the participants in linguistic interaction are consciously aware of the speech sounds which they attend to, as well as the sense of a meaning which they understand and attend to as the interaction proceeds. However, many aspects of linguistic processing are not normally available to conscious awareness and would greatly impede the process of conscious meaning-making if they were.

2. The Representation of Subjective Experience in Consciousness in Relation to the Higher-scalar Environment of the Individual Theories of consciousness are concerned with the problem of how we represent our own subjective experience as well as that of others. Deacon formulates this problem as follows: The problem with other minds is that the glimpses we get of them are all indirect. We have a subjective experience of our own thought processes, but at best only an imagined representation of what goes on in others' subjective experience. When we speculate about others' 'inner' states, the only data we have to go on are what they tell us and what we observe of their physical states. Like the subject in the Turing text, we are forced to make assessments on rather limited and indirect data. We can, it seems, have direct knowledge only of ourselves. In philosophy, this argument is aptly termed solipsism (from the Latin solus, alone, and ipse, self). (Deacon 1997: 424) ... the problem of representing the subjective experience of another and the problem of representing one's own subjective experience both entirely depend on the nature of the representational processes involved. The problem is not whether some knowledge is representation and some is direct and unrepresented. The problem is, rather, what sort of representation is involved, and what knowledge this provides of our minds and the minds of others. (Deacon 1997: 425) Deacon argues that both first-person and third-person perspectives on one's own and other's experiences are both dependent on 'representational processes'. The notion of representation, in contrast to the notion of 'direct and unrepresented' knowledge (and experience), raises the further question as to how consciousness can be theorized as a highly specified meaning system on all integrative levels (iconic, indexical, symbolic) in the inner perspective of the self who contextualizes and interprets the given experience as meaningful in the perspective of the self. Consciousness is based on internalized self-self transactions that have emerged from and are a further reorganization of self-other transactions that the self has engaged in with others (see Thibault In press. This suggests that, rather than representation, dialogue and/or its less specified analogues on the iconic and indexical levels are fundamental here. As we have seen, Libet, Bateson, and others point out that our consciousness of our

174

BRAIN, MIND, AND THE SIGNIFYING BODY

thoughts, perceptions, and so on, does not amount to consciousness or perceptual awareness of the neurophysiological processes that underpin and sustain these. Instead, we are conscious of the 'products' or the phenomenological manifestation of the experience itself. This fact suggests that the central issue here is, once again, that of the interpretation of meaning in the perspective of the self. The observations of Libet that I discussed in the previous section suggest that consciousness entails the global reorganization of neuronal activity in the brain across many different scalar levels of organization. Emergent, higher-scalar levels of neural organization reorganize the information on lower levels as meaning for the self. The higher-scalar levels constrain the lower-levels of the body-brain complex to its own dynamics. Moreover, the highest scalar level, which constrains and interprets the lower levels, is the self. The self is realized by the most complex, most global (higher-scalar) level of neuronal organization in the individual's brain. The self can be seen as a system of interpretance that interprets and enacts meaningful experience at the same time as it constrains and modulates sensori-motor activity along its trajectory within the perspective of the self. Consciousness, in my view, just is a highly specified system of meanings in this sense in the brain of the individual. However, this does not mean that the self is reduced to (higher-scalar) neuronal organization per se. The point is, rather, that very complex global levels of neural organization provide persons with the neural resources to contextualize and, therefore, to give meaning to experience. They give meaning to experience in ways that may have consequences for the signs which the individual outputs into the environment as specific action trajectories. The self is the embodied person who acts and means and is recognized by others as acting and meaning in ways that can be related to the self as the source of these actions and meanings. Selves are integrated to their contexts in ways that are interpretable as meaningful from their own selfperspective as well as from that of others. Deacon (1997: 426) alludes to the notion of 'theory of mind' which has been developed in cognitive psychology to explain how one learns, though enculturation, to view things from another's perspective. The 'theory of mind' first emerged in discussion concerning ape language and cognition (Premack and Woodruff 1978). It then became a dominant theme in developmental studies of human infants (e.g. Astington, Harris and Olson 1988). It has also been influential in autism research (e.g. Baron-Cohen 1989a, 1989b). For a critical reflection on this entire debate, see Battacchi, Battistelli and Celani (1998). According to Deacon, it is the development of 'symbolic referential abilities' which enable humans to interpret 'information from another's behavior to be interpreted as representing another mind' (1997: 426). Species that lack such symbolic abilities are limited to the representation of indexical associations between stimuli from both self and others. Deacon explains this symbolic referential ability as follows: As novelists and poets amply demonstrate, the range of personas and experiences that can be conveyed through symbolic media is unbounded. In a very real sense, this gives us the ability to share a virtual common mind. Because

THE SEMIOTIC BASIS OF CONSCIOUSNESS

175

symbolic representation maintains reference irrespective of indexical attachment to any particular experiences, when an idea or a narrative or someone's experience is reconstructed by another, it can be regrounded, so to speak, by interpreting it in terms of the iconic and indexical representations that constitute the listener's memory. Symbolic reference is interpreterindependent, because each interpreter independently supplies the nonsymbolic ground for it. (Deacon 1997: 427) Deacon adds to the standard account of 'theory of mind' the notion that this is made possible by the development of symbolic referential abilities. The various steps in Deacon's line of argumentation may be summarized as follows:

1. Each person's mind is separate from and independent of every other person's mind; 2. Access to another person's subjective experience - i.e. to that person's mind - is limited and indirect; 3. Minds represent subjective experience; 4. Minds are possessed by self and other; 5. Representation is the means whereby we know about our own mind and the minds of others; 6. In order to have access to others' minds, we must be in possession ofa theory of mind which is based on symbolic referential abilities. Deacon assumes that representation is the key to unlocking the subjective experiences that are housed in someone else's mind. Moreover, the level of first focus in this account remains that of the individual mind-brain. However, individual organisms are not separate from other organisms; they are constitutive parts of higher-scalar systems of interpretance. A system of interpretance is a means of deciding whether information deriving from either self or nonself is potentially meaningful to our own observational perspectives, as well as to others who potentially share these same observational perspectives. Individual body-brains do not exist as independent entities. They are always integrated with higherscalar systems and, at the same time, constitute the interpretative framework for smaller-scale ones. This is so both biologically and socially. In such a view, the relevant focus of study is not the individual body-brain per se, but the way it is embedded in and is a constitutive part of its higher-scalar ecosocial environments. The individual body-brain complex is a system of meaning-making potential relative to its higher-scalar environments. Moreover, other body-brain complexes with which the individual interacts are integrated to and embedded within the same overall supersystem. In such a system, the individual subsystems contribute to the meaning which is stored in a given subsystem through interaction with it. Individuals learn about other individuals who share the same system of relations when, on the basis of dialogically co-ordinated interaction between two individuals, the individual system's Innenwelt is changed. The individual subsystems do not directly interact with each other; the interaction between them is always mediated by and entrained to the dynamics of higher-scalar systems of

176

BRAIN, MIND, AND THE SIGNIFYING BODY

interpretance and their associated practices. Change in the individual's Innenwelt is a result of the meta-redundancy relations - the patterns of contextualizing relations - that are constructed between individual (self) and other individuals (nonself) on the basis of such transactions. Thus, meaning is stored, not at the level of the individual per se, but at the level of the contextual configurations e.g. the dyads - which integrate individuals to their ecosocial environments and, therefore, to the systems of interpretance that are embedded in these. The notion of self-organization (chapter 1, section 2, pp. 8-11) provides a means of overcoming the tendency of the representational theory of mind to overemphasize the individual's experience of his or her environment. Selforganization entails an increase in a system's overall semiotic complexity and specification as a result of the collection of the products produced by its own cascading (Salthe 1993: 176). Following Halliday's (1993) notion of the 'interpersonal gateway' to meaning-making in the process of the young child's 'learning how to mean', we can see how the emergent self is dialogically linked to its Umwelten in such a way that perceived events in the child's purview impinge on the child as indexes, in the first instance, of environmental events. These indexical signs of environmental events are, in turn, built up and elaborated in the child's Innenwelt in the form of linguistic and other semiotic models as to how one can act on and construe the world. In this way, there emerges a developing system of possibilities for action which afford the child a repertoire of possibilities for acting on, intervening in, and constructing his or her world. As noted before, these processes always occur in and through the mediating influence of the dyads which organize the child's interactions with the nonself. The child responds to events in the world, including the activities of other conspecifics, with an expanding set of possibilities for action. Dialogically organized interactions with others lead to the development of models of the world which mediate the child's subsequent interactions with the world. Such models include the linguistic system which is elaborated and stored in the individual's Innenwelt in the course of development and individuation.

3. Locating the Seat of Consciousness Where is the seat of consciousness? According to both dualists and reductionists, it is in the brain. But this conflicts with our consideration of self-organization, as explained above. Higher-order consciousness entails the emergence of a sense of self (Edelman 1992: 165-72; Edelman and Tononi 2000: chapter 15). This is dependent upon the emergence from initial proto-serniotic vagueness and indeterminateness of ever more specified and determinate symbolic possibilities for acting and meaning. That is, the individual is defined by the entire life-span trajectory along which the self unfolds and individuates. (For related notions see Harre's (1979: 312-34) theory of the individual's social trajectory or moral career; Sartre (1969 [1943]) on the project; Salthe (1993: 180-5) on the individuating ontogenetic trajectory; Riegel (1979) on developmental lifespan psychology.) The projects to which the individual is committed are more highly specified according to the higher-scalar meaning systems, or systems of interpre-

THE SEMIOTIC BASIS OF CONSCIOUSNESS

177

tance, which are available to the individual in his or her ecosocial environment. In the human case, the self of sense emerges in and through the individual's participation in dialogically organized exchanges between self and nonself. In this way, the self is constructed, elaborated and maintained in the brain in the form of higher-order, self-referential meta-representations such that the system constructs meanings about its own internal states and its relations to others (Flohr 1991; in this volume, chapter 6, section 2, pp. 283-9). The central argument of this chapter is that consciousness is a structured system of meanings whereby we organize our experience around a notion of the self along a trajectory (Harre and Gillett 1994). It is the most global manifestation of neuronal self-organization. Because of this, it has the capacity both to constrain and to interpret the activities and experiences of its own lower-scalar components in its own perspective. The self is, then, a self-organizing system of meanings in and through which the individual interprets the effects of its own self-organisation (Salthe 1993: 158). Harre (1983: 146) talks about the self as constituted 'in a discourse made up of speech acts having the form of self predications'. I would rather emphasize the building up of meanings along a trajectory through dialogic exchanges with other selves with whom one's trajectory intertwines. The point is that consciousness, in the view I am putting forward here, is necessarily founded on dialogic exchange processes which make possible the emergence of self-referential perspectives. It is these self-referential perspectives which enable experience to be sourced at or centred on a determinate 'self'. Consciousness, in particular higher-order consciousness, entails the self-organization of perspectives which, when viewed globally, are seen as having relative coherence and order. However, the local manifestations of consciousness are not necessarily orderly. Consciousness is itself an emergent consequence of different levels of semiotic organization - both scalar and integrative. The further question arises at this point as to the eo-evolution oflanguage and other semiotic modalities and the human brain. Given my premise that consciousness is not reducible to its neuroanatomical enabling conditions, I argue that the various levels of human consciousness qua system of meanings are structured according to metafunctional principles of organization or their vaguer, less specified analogues on lower integrative levels. As we shall see in more detail in chapter 5 in relation to Damasio's theory of consciousness, the metafunctional organization of consciousness qua internalized semiosis in the perspective of the self is an appropriate means for the structuring of the experiences which are sourced at a given self. The metafunctional structuring of consciousness is therefore able to account for the necessarily dialogically constituted nature of the relations between self and nonself, as well as the embodied grounding of the meanings attributed to a self relative to its surrounding milieu. The notion of an intentional 'consciousness of' relation which holds between a person and an intentional object (see Harre 1983: 148) is a meaningful relation. We shall explore in chapter 5 how this relation takes the form of the metafunctional semiotic organization postulated here. For such a relation to occur, it follows that the person concerned has the requisite modes of semiotic organization stored in his or her central nervous system. Moreover, the objects of conscious awareness and attention on all levels in the postulated integrative hierarchy of iconic,

178

BRAIN, MIND, AND THE SIGNIFYING BODY

indexical, and symbolic modes of semiosis (see chapter 1, section 8; chapter 3, section 7) are experienced as either topological-eontinuous variation or typological-eategorial distinctions which make a difference to the person who has the conscious experience, relative to a given system of interpretance. Thus, one does not have conscious awareness of the brain states and processes - the neural activity - but of the meanings (iconic, indexical, and symbolic) which are variably realized by these and which are made available to conscious awareness in the interpretive perspective of the self (see above). This point applies indifferently to phenomena of experience of all kinds, whether they exist in perceptual awareness or are activated by non-perceptual awareness in the processes of imagining, hypothesizing, pretending, and remembering. The relational structure of consciousness is therefore shown to have what I believe to be revealing parallels with the metafunctional organization of semiosis. This does not mean that all consciousness is able to be 'reported' by linguistic means. Moreover, the distinction between 'reportable' and 'nonreportable' experience (Harre 1983: 155) does not mean that that which is nonreportable by linguistic means is outside the realms of conscious experience and awareness. This would be to underestimate the importance of perceptual-motor forms of categorization at the iconic and indexical levels of the integration hierarchy of semiosis. Furthermore, the assumption of non-reportability would disallow the importance of topological-continuous variation as a means of makings differences that make a difference to some individual. Not all consciousness is necessarily 'reportable' in terms of the typological-eategorial semantic distinctions made by the lexicogrammar of natural language. So called 'non-propositional' awareness and knowing (Harre 1983: 155) is all about this kind of non-symbolic construal of conscious experience. In any case, the notion of 'reportability' suggests that there is something empirically prior which language simply packages up and reports as the more essential experience. Instead, the metafunctional form of consciousness at all levels in the integrative hierarchy shows that the act of consciousness constitutes the object of consciousness relative to the categories of the system of interpretance which is used to contextualize the experience. The dialogic exchanges between self and nonself are possible because, in the first instance, the biological organism belongs to that class of physical systems which are dynamic, open systems (Prigogine and Stengers 1985 [1984]: 143; Lemke 1984a, 1984d, 1995b; Wilden 1981: 2). Such systems exchange matter, energy, and information-meaning with their environment and, at the same time, maintain their specific identity in relation to their environment by virtue of these same system-environment transactions. However, these exchanges of matter, energy, and information-meaning do not mean that the organism is simply 'in' an environment with which it exchanges matter, energy, and information. It is through these exchanges that the organism participates in the active constitution of its environment at the same time as it is constituted by these same selfenvironment transactions. It is in this way that organisms expand their Umwelten (Harre 1990: 301). In expanding their Umwelten, they define their own identity and structural integrity in and through these exchange transactions. In these exchange processes, human individuals exchange both structure and meaning

THE SEMIOTIC BASIS OF CONSCIOUSNESS

179

with their environments. The intrinsic organization of these self-environment transactions and the ways in which they integrate both with the body-brain complex and with the higher-scalar ecosocial environment can be explained in terms of the metafunctional shape of consciousness that I first postulated above. The environment is, in the first instance, perhaps better formulated as the nonself in relation to the self. The self selectively orients to, engages with, interacts with, and interprets the nonself. The nonself includes other persons and objects, events, and so on. In exchange, the structure so exchanged is implicit; verbal meaning is, in large part, explicit in fully contextualized occasions of discourse, though non-verbal forms of semiosis such as facial expression, body movement, and so on, may be much less so. Importantly, meaning is distinguished from information by that fact that meaning is always construed in relationship to the observational viewpoints and perspectives of the participants on the ecosocial scale in question. Information is not dependent on such viewpoints, but is a statistical measure of the variety in relation to which a given event occurs (Wilden 1981: 17-18). Again, the importance of meaning and the interpretation of experience in the self-perspective as meaningful for conscious experience are suggested by the relevance of viewpoints and, therefore, interpreters to meaning as distinct from information. On the human scale which is of concern here, meaning requires human perspectives and viewpoints which can be shared with others on the basis of the dialogically co-ordinated exchanges between self and others. Dynamic open systems, of which human individuals are a specific case, maintain their structural integrity through the processes of obtaining required matter, energy and information with a high order of structure from their environments and returning low-order matter and energy to the environment in the form of disorder. In the human case, higher-scalar ecosocial structures and processes integrate and extend the structures and regulatory principles of the individual. In the early mother-infant dyads characteristic of primary intersubjectivity (Trevarthen 1987, 1992), dialogic exchange processes integrate the child into the meaningmaking practices and the systems of symbolic possibilities of the group. Thus, the individual's emergent symbolic or higher-order consciousness entails the building-up and elaboration of internal structures of meaning and interaction. This has the dual function of (l) integrating individual consciousness into the social semiotic system and (2) providing individual consciousness with symbolic resources which expand and extend the individual's own possibilities beyond the here-now scale of its material interactivity with its environment. Integration into the symbolic possibilities of the social group is therefore necessary for the emergence of higher-order consciousness in the individual (Edelman 1992: 131-6). By the same token, higher-order consciousness in the individual is also essential for the social group. This is so for three reasons. First, the symbolic meaning-making resources of the social group afford the continuing integration of apprentices (the new-born, new members, etc.) into the group. Secondly, it is also the means whereby consciousness is made shareable with other conspecifics, and in ways which ensure the maintenance of a higher-scalar collective consciousness in historical time. Thirdly, these resources make possible the social

180

BRAIN, MIND, AND THE SIGNIFYING BODY

semiotic construction of individual historical-biographical coherence with reference to the individual's unfolding lifespan trajectory in some community. These three points illustrate the need to view higher-order consciousness in the individual as an integral component of a still-higher-scalar system of ecosocial relations and processes which exist on much greater scales than the individual biological organism. The integrated nature of conscious experience, as discussed above in relation to Libet, can be seen as an instance of the non-linear operations of a complex system whose closed in ternalloops of neural activities and functions on very many complexly interrelated scalar levels of neuronal organization give rise to metastable states or fields of consciousness. Consciousness is a metastable state of very high complexity which enables strong cross-couplings between the individual body-brains and selected aspects of his or her inner and outer environments. Such cross-couplings can only occur through semiotically mediated transactions between individual and environment. Moreover, rather than seeing symbolic consciousness as the mere bottom-up aggregate of many individual body-brain complexes, it is possible to conceive of the full-fledged emergence of symbolic consciousness in the human species as the evolutionary consequence of the historically emergent differentiation of symbolic modes of social meaning-making (language, gesture, depiction, etc.) from the iconic vagueness of prior systems. The higher degree of specification of symbolic modes of meaning-making can be seen as the result of the increasing differentiation of the topological-continuous variation that is characteristic of the prior system as it differentiates into a number of more specified subsystems. In this way, the prior system constitutes the immediate environment of the newly emergent system and therefore provides the initial systemic conditions whereby the increased specification of the symbolic mode can emerge (see chapter 3, sections 4 and 7). This suggests that the evolution of symbolic consciousness in the human species arose from the increasing differentiation of vaguer prototypes of exchange between organism and environment. The symbolic brain, in this view, is not the unit which generated language, but is the evolutionary product of the further differentiation of previously existing semiotic modalities in their contexts of use. Symbolic modes of meaning-making entail a higher degree of explicitness and specification than do the iconic and indexical modes from which they emerged and which they contextually integrate to their own (symbolic) level (chapter 3, sections 4 and 7). The emergence of the symbolic level in no way implies that the iconic and the indexical levels have been transcended. Linguistically mediated discourse is fully contextualized language in operation in relation to other contextual features, including other semiotic modalities which are relevant to its co-contextualization. It is a context-specific restriction of the meaning potential which is stored and elaborated in the brains of individuals at the same time as it is, to varying degrees, shared by them by virtue of their being embedded within the same system of interpretance in a given ecosocial semiotic system. This meaning potential is stored in the individual's brain in largely unconscious ways until is selectively activated according to the requirements of some context. Thus, explicitly meaningful and consciously enacted discourse can be seen as a critically important resource for selectively cross-coupling the

THE SEMIOTIC BASIS OF CONSCIOUSNESS

181

individual to the ecosocial environment. Discourse is a powerful agency both for recruiting individuals to relevant social practices and for integrating consciousness with its contexts in relevant ways. This is equally true of both 'inner' and 'outer' forms of meaning-making. Linguistically mediated social meaningmaking is always to some extent explicitly meaningful for the participants in the exchange, whereas the iconic-indexical modes of posture, gesture, facial expression, and so on are much more implicit and, hence, 'unconscious' to a greater extent. Higher-order consciousness is characterized by the use of symbolic modes of meaning-making such as language. From the perspective of the individual subsystem, it entails the capacity for the linguistic and other semiotic mediation and construal of experience. I have argued that this process crucially depends on the integration of the individual organism into higher-scalar ecosocial processes and trajectories. At the same time, it is clear that some of the semiotically mediated exchanges that take place within the individual organism in the form of 'inner speech', linguistically mediated 'thought', internalized visual semiosis, and so on, are specialized to the 'inner' realm of the individual and set against transactions that occur in the 'outer' realm such that the former are seen as private or individual. However, it is important to recognize that even those forms of meaning-making which have been specialized to the inner realm are definable by and integrated into the semiotic resource systems of the higher-scalar ecosocial system. If conscious experience is mediated by and integrated into a higher-scalar system of interpretance, then it seems legitimate to ask how such semiotic resource systems, which are stored at the higher-scalar ecosocial level, give shape to consciousness itself. The representational view of consciousness focuses on the individual organism which represents the world as conscious experience. This has resulted in an overwhelming emphasis on the bottom-up brain-mind mechanisms and processes which 'cause' conscious experience in individuals. Similarly, those approaches which focus on the ways in which our sensori-motor activity and perceptual systems constitute the body images and schemata in and through which we construct the world continue to emphasize the individual organism as the focus of theoretical inquiry (e.g. Lakoff and Johnson 1999). Alternatively, I have argued that consciousness cannot be characterized in terms of individuals per se. Instead, it requires a shift in focus to a three-level scalar hierarchy view of the systems relations that are involved (Salthe 1993: 36-46; Lemke 1999; Thibault 2000a). That is, it requires us to ask how the intrinsic dynamics of higher-scalar semiotic resource systems such as language and the social practices in which these are embedded and deployed play their part in the shaping of consciousness. In terms of the principles of scalar hierarchy theory and scalar heterogeneity (chapter 1, sections 3, 9), it is possible to re-think the locus or seat of consciousness in relation to the body-brain complex of the individual organism. Biologists such as Salthe (1993: 181) have pointed out that the molecules and cells which make up our bodies are constantly replaced. From the perspective of the biological organism, we are not the same organism that we were just a few years earlier or even a year ago. Nevertheless, we construct a sense of continuity over

182

BRAIN, MIND, AND THE SIGNIFYING BODY

time so that our consciousness of our selfhood is maintained and enhanced in spite of these constant changes to the organism. In the process, we encounter new historical contingencies, which provide us with new memories and experiences. Salthe poses the question as to the locus of selfhood and agency in this perspective. This locus is the entire ontogenetic trajectory of the organism, starting with birth and continuing until death. Any moment-by-moment action of the organism is no more than a local manifestation of this trajectory (1993: 181). To quote Salthe: It would be convenient for present purposes to describe our own ontogenetic trajectories from a scale delivering a cogent moment of about one of our lifetimes. In that case the entire trajectory is present simultaneously as a single object. This object can be taken to be the seat of self-organization, agency, and selfhood because these inhere in its ontogenetic trajectory rather than in the organism, which is continually changing (besides being at any moment merely a mechanism, which can't have those properties). (Salthe 1993: 182) The entire trajectory is a 'cogent moment', as viewed from a still-higher-scalar perspective, with respect to the very different scale of the moment-by-moment existence of the individual body-brain complex. The latter is a lower-scalar entity whose lower-scalar dynamics operate on a much faster timescale than those of the higher-scalar dynamics of the trajectory as a whole. Each of the two scales has properties which are ontologically specific to its own scale and the respective entities and processes on that scale. It is symbolic or higher-order consciousness which makes the emergence of such a trajectory possible. Moreover, the organism which sustains the trajectory is a dissipative structure which is characterized by the historical emergence and accumulation of meanings along its entire trajectory, Increased dissipation to the environment goes hand in hand with increasing structural complexity such that fluctuation to higher-order states is increased, thereby allowing the evolution of new metastable states (Lemke 1984c: 29). The trajectory is a self-organizing system. Self-organizing systems are defined by the fact that they exchange matter, energy, and meaning with their external environments in wayswhich bring about the spontaneous emergence of order and pattern in the system (see above). Self-organizing systems have history and individuality (Prigogine and Stengers 1985 [1984]: 176; Salthe 1993: 142; Lemke 1995b [1993]: 112-13). Consciousness is, then, a trajectory-in-time. It is the trajectory which is the seat of selfhood and agency (Salthe 1993: 184). The trajectory of consciousness is both meaning-based and regulated by memory. The body-brain complex of the individual is the moment-by-moment realization of this trajectory. The two - Le. consciousness and the body-brain - exist on different scalar levels. The link between them is one of realization, rather than causality. The relationship of realization specifies (1) that the trajectory of consciousness is realized by the body-brain complex; (2) that the body-brain complex realizes, or embodies, consciousness, therefore the body-brain complex is a constitutive part of the context of consciousness on account of the fact that consciousness is always embodied in the body-brain dynamics of individuals. In other words, the body-brain is part of

THE SEMIOTIC BASIS OF CONSCIOUSNESS

183

and is embedded in a still-wider system in and through which meaning is made. The body-brain constitutes and embodies the lower-scalar enabling conditions of consciousness as well as being the lower-scalar environment of the trajectory of consciousness. In a sense, the trajectory of consciousness can be thought of as the meaning - the content stratum - which is realized by the expression stratum of the body-brain complex. The self-organizing character of this trajectory depends crucially, as I pointed out above, on the ongoing exchange of matter, energy, and meaning between the system and its environment. In the case of symbolic or higher-order consciousness, this is founded on the dialogically organized meaning-making exchanges that take place between self and others who share the same ecosocial perspectives and viewpoints. The trajectory of higher-order consciousness is, therefore, constituted against a background of the dialogically organized meaning exchanges in which the individual has participated along the course of his or her trajectory. The temporal organization of the trajectory is thus made possible by its dialogic synchronization with the trajectories of others with which it engages (Riegel 1979: 167). The evolution of new metastable states of consciousness occurs on the basis of conflict or contradiction which must be resolved for progression to a new state to occur. Dialogue is driven by the negotiation of such difference or contradiction, which the participants in the dialogue seek to resolve in some way. Resolution, or what Bakhtin (1986: 76-7) called 'finalization', can never be achieved in any ultimate way. The dialogic nature of consciousness means that new questions, new doubts, new differences will always arise such that the trajectory of consciousness remains open-ended, therefore constantly engaging in {dialogic} interaction with others. Physical brain processes and states are inadequate to explain consciousness precisely because they cannot deal with the principle of final causality, which is necessarily entailed by the dialogic basis of consciousness. Physical brain activities are cause-effect relations within just one level of relations in the scalar hierarchy. Final causality, on the other hand, is a higher-scalar constraint which operates 'downwards' on lower levels. Such constraints, which emanate from the higher-scalar levels, are, increasingly, semiotic, rather than physical, in character. Final causality does not derive from the lower level of physical brain processes, which would be efficient causes in Aristotle's schema. The latter operate on a much faster temporal cycle than does the trajectory of individual consciousness, which spans a lifetime. This means that the much faster rate of the lower-level brain activities in a given cycle of such activities comes to completion while the higher-scalar-level trajectory of consciousness unfolds according to its much slower dynamics, and at the same time continues into the future. Thus, the higher-scalar level of the trajectory, which is the seat of consciousness, operates on and selects the lower-level brain activities at the same time as it pulls these into the future (Salthe 1993: 270). The individual agent is dialogically oriented to the nonself along the entire duration of its trajectory. Therefore, the forever open and incomplete character of this relationship is the means whereby the trajectory of the individual self is pulled into its future by the need to resolve ('finalize') local contradiction and difference through dialogic engagement with the nonself. It is in this way that

184

BRAIN, MIND, AND THE SIGNIFYING BODY

the trajectory self-organizes at the same time as it individuates. In terms of the three-level logic of the scalar hierarchy (see chapter 1, section 3, pp. 14-16), this means that level L - the level of first focus, or the ecosocial scale on which individuals interact with each other and their material and semiotic environments - is constituted by the dyadic character of self/rionself transactions. These transactions can be modelled as a hierarchical triplicate of levels, as follows: L+1: L: L-l:

Historical-biographical trajectory of individual consciousness (historicalbiographicallifespan) Dialogic basis of ecosocial self!other transactions along logogenetic trajectory of unfolding text-time (moment-by-moment) Physical brain activity in individual organism (milliseconds)

4. The Meaning-making Capacity of the Body-brain Complex through the Discrimination of Difference The fact that the world is not something 'out there', an object in itself, having its own reality independent of our ways of making meaning about it, has important implications for the body-brain. The recognition of this fact requires the abandonment of the representational theory of mind and cognition and its replacement with a view in which the world and its meanings are actively produced by us. Further, the systems of perceptually, conceptually, and semiotically salient distinctions constituted by our perceptual systems, the conceptual distinctions elaborated in primary consciousness, and language and other semiotic resource systems in higher-order consciousness are inseparable from the architecture of our body-brains and, hence, the embodied nature of our ways of making meaning (chapters 2 and 3). Edelman's (1989) notion of an 'unlabelled' world and Saussure's (1971 [1915]: 156) notion of the 'amorphous flux' (see also Thibault 1997a: 166-71; 1998a: 25-6) are apt metaphors for expressing the indeterminate nature of pre-semiotic vagueness before it is made into something more determinate in and through our deployment, in context, of the systems of categorization made available to us by some meaning system. In other words, we transform indeterminate vagueness into something more determinate and meaningful for us by actively exploring and intervening in the environment that we inhabit. Organisms produce the world, so to speak, through their capacity for embodied exploration and interpretation of it. In so doing, they make the properties - perceptual, affective, and motivational - of the given aspect of the world which is the focus of their attention and interest emerge. In selectively focusing on and showing interest in some things rather than in others, they implicitly assign value and, therefore, motivational salience to some things rather than to others. They do so by means of the various sensori-motor and semiotic systems which enable them to construe, potentially, very many degrees of difference in the topological vagueness of the ambient flux which surrounds them. Thus, the human hand, in comparison with the hands of other animals,

THE SEMIOTIC BASIS OF CONSCIOUSNESS

185

in conjunction with the system of visual perception, has a greatly enhanced ability to discriminate very many degrees of difference, and therefore fine detail, as compared to other animals. Human and non-human primates have a more developed and therefore more discriminating grip than other animals. This increased capacity for discrimination affords the kind of detailed exploration and differentiation referred to above. Human and non-human primates also have a more highly developed pre-frontal cortex. The combination of these two capacities of the hand and the pre-frontal cortex means that humans and other non-human primates have enhanced capacities both for construing more and more salient and, hence, potentially relevant and meaningful differences - both topological and typological - in the world. The pre-frontal cortex is the most recent product of primate, including human, cortical evolution. It is a further cortical development of the motor systems responsible for direct motor control. This means that it affords abstract symbolic exploration and investigation of the environment in the absence of actual movement. It is this capacity for abstract exploration which, in turn, allows the organism to place itself in the position of others and, therefore, to view things as others do, as well as to abstractly look ahead and reflect on possible courses of action and their consequences before these are put into action as actual movement and behaviour. The interpretation of another's gaze vector is a case in point (see section 9, pp. 201-2). Our actions in the world and our experiential construals and perceptual-motor categorizations of the phenomena of experience through sensori-motor activity and exploration are, then, increasingly abstracted from sensori-motor activity so as to produce symbolic 'thought'. We see, then, that direct sensori-motor exploration, along with exploration which is abstracted from this in the form of 'thought', are capable of acting on and construing and constructing the phenomena of experience in our 'inner' and 'outer' worlds. The greatly enhanced capacity for such exploration in humans on account of the resources of symbolic memory and social meaningmaking means that our very being - at once social and biological - is linked to this capacity. The question of the embodiment of the brain raises the question of the structure of the brain and its relationship both to the body and to the ecosocial environment which lies 'outside' our bodies. The body is represented in the neocortex both in terms of the exploratory motor activity of perceptual systems and the performatory activity of executive systems (Gibson 1983 [1966]: 46; Edelman 1992: 105). With reference to the latter, it is significant that the face and the hand are massively and disproportionately represented in the neo-cortex with respect to other areas of the body (Thelen and Smith 1994: 136-7). The reason for this lies in the fact that a quite large proportion of the motor cortex is dedicated to the control ofthese two parts of the body, both of which are capable of very sophisticated and finely differentiated exploratory (and performatory) activities in comparison with other parts of the human body. They are thus capable of discriminating very many more degrees of difference than are other human body parts. Take, for instance, the crucial case of the human hand with its opposable thumb. This feature of the human hand affords a vast number of possibilities for the exploration and manipulation of objects, for tool use, for the

186

BRAIN, MIND, AND THE SIGNIFYING BODY

symbolic construal (imitation, etc.) of actions and objects which are not present, for pointing and imitation, and for the articulation of gestures and the realization oflinguistic meanings in sign Uohnston 1992; McNeillI992). Similarly, the human vocal tract and its cross-modal connections to facial movements also afford greatly expanded possibilities for articulating very finely differentiated exploratory and performatory activity. In the case of writing, hand-arm-joint-eye kinaesthesis (Gibson 1986 [1979]: 275) enables the writer to interact with others across very diverse spatial and temporal scales. The ability of the hand to articulate complex motor acts or gestures and the resulting graphic traces on a treated surface means that addressers and addressees can integrate meanings across diverse spatio-temporal scales and assimilate them to their own Innenwelt. Moreover, the complex motor activity which is involved in human speech can be seen as a further development and extension of the human ability to analyse phenomena into their component parts and to interpret them. It is the interplay between face and hand, on the one hand, and the brain, on the other, which has enabled human beings to massively expand their meaning-making potential and, hence, to intervene in, shape, and expand their Umwelt across diverse space-time scales (Latour 1996a, 1996b; Lemke 1999; in this volume chapter 3, section 1, pp. 108-9). Symbolic consciousness and the body parts which afford active exploration of the environment enable humans to orient to the indeterminate and the unknown (Prodi 1987: 20). The hands and face are able to discriminate very many degrees of (topological and typological) difference. The delicate musculature of both the face and vocal tract and the hand-thumb system are able to make and discriminate complex systems of differences and their combinations and re-combinations according to the requirements of specific ecosocial contexts. This is so of both the performatory activity of these bodily systems qua executive systems, as well as the exploratory activity of them qua perceptual systems. In this way, social meaning-making becomes possible. From an evolutionary perspective, this discriminatory capacity was further enhanced by the development of a vertical posture: this development allowed for an enlarged visual field, and at the same time the upper body was no longer tied to the requirements of moving around in the environment. Instead, the upper body could dedicate itself more and more to the selective orientation to and exploration of the environment in collaboration with the various perceptual systems such as the haptic, olfactory, auditory, and visual systems (Gibson 1983 [1966]: 72-4). The increasingly delicate and discriminating patterns of difference that could be made in and through these bodily resources enabled emergent meanings to be expressed. These meanings could, in turn, be addressed to and shared with others on the basis of higher-scalar boundary conditions which emerged as a consequence of the interactions between the organism's internal dynamics and its ecosocial environment. In this way, the same patterns of difference are constrained by the same attractor such that they could be taken up and adaptively modified or re-worked by others. The taking up of the same patterns by others meant that the meaning-making potential of these patterns of difference was able to link individuals across diverse space-time scales. This linking of individuals across diverse scales is, in the first instance, possible

THE SEMIOTIC BASIS OF CONSCIOUSNESS

187

because there always were precursor higher-scalar attractors and boundary conditions which functioned to entrain human body movement in socially significant ways. In this way, systems of differences become socially shared and constrained by virtue of their being entrained to the same higher-scalar attractor on the ecosociallevel (Salthe 1993: 270, 277-81; Lemke 1995b: 116). The synergy of hand and eye in hand-arm-joint-eye kinaesthesis meant that humans could explore, act on, interact with, intervene in, and experiment with the emergent possibilities of an unlabelled world. Moreover, the cross-modal synergy of hand and language in both speaking through the co-contextualization of vocal-tract and manual-brachial gestural activity and writing through the cocontextualization of hand and eye in hand-arm-joint-eye kinaesthesis and the visual-graphological traces on a treated surface vastly extended this capacity. The emergent and increasingly discriminatory capacities of this kinetic interface in the physiological processes of articulation in speaking, writing, and gesture are, as I mentioned above, related to the massive representation of the activities of hand, face, and vocal tract in the neo-cortex. The connection of such bodily processes to activity in the neo-cortex, as evidenced by the massive neural activity associated with the use of these body parts, suggests an intimate connection to consciousness, and especially to symbolic or higher-order consciousness. Thus, the ability to intentionally control and modulate the activity of the hand, face and vocal tract goes hand in hand with higher-scalar cortical activity. This, in turn, suggests an intimate tie-up between consciousness and those parts of the body most closely associated with symbolic meaning-making. The very fine discriminations made by vocal-tract and other (e.g. manualbrachial) articulatory activity can be co-ordinated with the articulatory activity of others in both space and time. Furthermore, the integration of these motor functions and the discriminatory potential that they afford to higher-scalar semantics in symbolic neural space means that the exploratory activity of vocal-tract activity, as well as that of the hand, has implications for higher-order consciousness. The very fine discriminations made by vocal-tract activity, the face, and the hand are all forms of expression that can be recategorized as content (linguistic or otherwise) by virtue of the brain's capacity to act as a somatic recognition system. This is no less true of inner speech, which involves the categorization of the brain's own activity as imagined motor routines corresponding to the expression stratum of speech, rather than the categorization of inputs from external sources of auditory and other perceptual information involved in listening to another's speech (chapter 5, section 9, pp. 271-2). In the case of inner speech, this means that the brain's own internal activity categorizes past samplings of speech sounds as specific types of phonological routines that can be activated by the brain's own internal activity and matched with the appropriate structures on the content stratum by virtue of the brain's capacity to function as a somatic recognition system. This capacity therefore applies both to the categorization of the perceptual inputs from external sources of speech sounds and to the phonological routines that occur in different global mappings in the brain as a result of the brain's internal activity. In all these cases, the fine differentiations made by the vocal tract, the face, and the hand in the process of exploring and/or acting in the environment have contextual redundancies with

188

BRAIN, MIND, AND THE SIGNIFYING BODY

categories of content to which they are integrated. In this way, the exploratory potential of these parts of the body has direct implications for the categories of consciousness itself. Thinking, as I pointed out above, is symbolic exploration abstracted from sensori-motor activity. However, the fact that, in thinking, direct exploration of the environment through sensori-motor activity does not take place, does not mean that thinking is not also a form of activity. In my view, thinking is both action and reflection, to use Malinowski's (1923, 1935) terms, which has been specialized to the 'inner' domain. It is a kind of action and reflection which can take place without directly subjecting the body to the hazards and risks entailed by a given course of material action. Verbal thinking - that is, language activity specialized to the inner realm of 'mind' - means that reflection on action and its consequences, the discourse genres of, say, abstract reasoning and argumentation, particular ways of interpreting and understanding, and so on, are not the sole property of a single individual. Luria describes syllogistic reasoning as a form of higher-order rational thinking in precisely these terms: One of the objective devices that arises in the process of the development of cognitive activity is the syllogism - a set of individual judgments of varying degrees of generality in certain objectively necessary relationship to one another. Two sentences, of which the first ('precious metals do not rust') is in the nature of a general judgment ... while the second ('gold is a precious metal') is a particular proposition, are not perceived by the developed consciousness as two isolated phrases in juxtaposition. A human being whose theoretical thought processes are well developed will perceive these as a completed logical relation implying the conclusion, 'hence gold does not rust.' This conclusion does not require any personal experience; it is arrived at through a syllogism created objectively by historical experience. A considerable proportion of our intellectual operations involve such verbal and logical systems; they comprise the basic network of codes along which the connections in discursive human thought are channelled. (Luria 1976: 101) The 'networks of codes' are the discourse genres which give determinate shape to 'discursive human thought'. The 'completed logical relation' between the implied conclusion and the first two parts of the discourse structure is a logicosemantic one which can be glossed as CONDITIONACONSEQUENCE. Thus, the rational thinking which is involved in logical reasoning and inference is mediated by genres structures which become internalized as procedures for higher-order thought. Furthermore, what may appear, viewed in this way, as the monological property of individual consciousness is, in actual fact, a dialogical response to some prior situation or text. The tendency to naturalize these internalized genres as the monological properties of a single consciousness is reinforced by the tendency to consider written texts as the unique property of a single individual. The fact that the first two sentences are put in a particular kind of logico-semantic relation with the third one also shows the internally dialogic nature of the text as a whole. Thus, the first two constitute a particular generalizing stance on a given situation-type while the third sentence is a particular

THE SEMIOTIC BASIS OF CONSCIOUSNESS

189

dialogical response to this situation from another point of view, i.e. from the point of view of the specific characteristics of gold as an instantiation of the schematic category 'precious metal'. Symbolic thinking, in the form of, for example, 'inner speech', links the individual to the accumulated ways of thinking built up over many generations in the course of the constant and adaptive modification of thought on the part of all of those individuals who have contributed to it. Symbolic thinking, which is thinking mediated by a semiotic system of linguistic or other potentially meaningful differences in a given community, affords differentiated thought and thus extends and enhances our ability for active exploration and expansion of our Umwelt. In this view, thinking is a specific use of neuronal capacities for giving pattern and meaning to experience and for organizing one's responses to events in both our inner and outer environments as specific courses of action. As Saussure demonstrated in his discussion of the mediating influence of the value-producing categories of a given language system on 'thought' and 'sound' (chapter 2, section 1, pp. 59-67), such mediation enables new possibilities of meaning to emerge from the vague and 'amorphous' flux of pre-semiotic 'thought' and 'sound'. That is, we produce our world in and through our semiotically mediated interventions in and engagements with this vague and undifferentiated mass of possibilities, which we can never, in any case, know or experience directly, but always only mediately. The projection into the world of the highly delicate differentiations afforded by the sensori-motor activities of the hand and face-vocal-tract systems also means that we endow the phenomena of our experience with value, affect and motivational salience. That is, we assign them with value in terms of what they entail for us as action and interpersonal interaction (Thibault 1992b).

5. Language Functions and the Cortical Organization of the Brain: Implications for Higher-order Consciousness The human brain is organized in terms of two cortical hemispheres. The right hemisphere regulates our responses to the unknown; the left is more concerned with actions which are carried out in response to the known and familiar. Researchers such as Davidson (1984,1992) have shown that the two hemispheres are differentially specialized as regards value and affect in connection with the way the individual orients to the world. Positive affect (e.g. attraction) correlates with increased neural activation in the left frontal cortex. Negative affect (e.g. repulsion) correlates with increased neural activation in the right frontal cortex. This suggests that the two hemispheres differentially regulate our affective and value-laden responses to the world. The right hemisphere regulates our responses to that which has not yet been assigned a specific value or motivational salience. The left hemisphere, on the other hand, is concerned with our orientation to that which has already been assigned a specific value and motivational salience. In the first case, the negative evaluation of the uncertainty and indeterminateness which the individual encounters constitutes the basis on which active exploration may occur. In the second case, we are on familiar, safe

190

BRAIN, MIND, AND THE SIGNIFYING BODY

ground and already know what to expect and therefore how to act. In both cases, value-laden responses are linked to sensori-motor activity and, therefore, to specific courses of action and their likely consequences. The lateralization of brain functions has frequently been cited as evidence for the specialization of language functions to the left hemisphere of the brain. In this view, the right hemisphere is specifically adapted for the production of visual and auditory images, for global pattern recognition, holistic thinking, and so on. Table 4.1 presents the lateralization of brain functions in the left and right hemispheres according to Deacon. The right hemisphere appears to be associated with topological-continuous variation, with the vagueness of the not-yet-fully specified or determinable, thus suggesting what Salthe calls 'signifying possibility rather than necessity' (1993: 143) on account of its indeterminate character. The left hemisphere, on the other hand, is specialized to the typological-categorial analysis of phenomena into discrete elements - that is, with the specification of that which is determinate. Typically, the left hemisphere has been seen as specialized to language whereas the right hemisphere is considered to be concerned with holistic patterns and images. (See for example Corina et al. 1992 for arguments in favour of the view that left hemisphere specialization of language respects a distinction between linguistic systems such as American Sign Language (ASL) and nonlinguistic gestures.) However, Deacon (1998 [1997]: 311) argues that the hypothesis of hemispheric specialization for language is erroneous and that what is really at stake is a question of the way different linguistic and other brain functions are differentially distributed between the two hemispheres (see also jakobson and Santilli 1980; Cowley 2002: 79-80). Deacon sees this as 'another reflection of the role of competitive processes in determining the ultimate representation of functions in the cerebral hemispheres during development' (1998 [1997]: 311). Table 4.1 summarizes Deacon's observations concerning the typical ways in which various language functions tend to be distributed across the left and right hemispheres in the development of perhaps the majority of human beings. A close examination of the various items in Table 4.1 reveals that all of the functions that are presented in this table are intrinsic to language form and function. Moreover, the functions that are prototypically associated with linguistic processing and assigned to the left hemisphere, in actual fact, represent a very partial and distorted view of language. It is a view which is based almost exclusively on formalist criteria, as well as on a narrowly mono-functional view of language as being about information transmission or the cognitive, ideational, or representational function of language. My point is that all of the functions presented in Table 4.1 are relevant to language. It follows from this that language is globally distributed over both hemispheres of the brain. Therefore, the whole brain is involved in the functioning of language in the individual (see also Peng 1994: 110). Table 4.1 shows that a number of different, though complementary, functional semantic domains ofianguage are differentially distributed across the two hemispheres. Given the limitations of the model of language which inspires Deacon's account, it seems necessary to explore even more deeply the implica-

THE SEMIOTIC BASIS OF CONSCIOUSNESS

191

Table 4.1: Lateralization of language functions in the left and right hemispheres according to Deacon Left Hemisphere

Right Hemisphere

Word and phase-level processing and analysis

Large-scale semantic processing of language of larger symbolic constructions that words and sentences contribute to: complex ideas, narratives; and arguments (Deacon 1997: 312); compositional complexity Contextual logic (re jokes, humour) (Deacon: 1997: 312)

Specific details

Implicit organizational logic (Deacon 1997: 313)

Phonological processing

Processing of prosodic features of speech (rhythm, pitch, emotional tone) (Deacon 1997: 313)

Phonemic processing

Correlation of changing pitch, volume, and rate of production with level of arousal; changes in quality of vocalization as indication of type of interaction (hostile, submissive, ete.); overall phrasing with respect to breath control (Deacon 1997: 314)

Ultra-rapid analyses of sound changes and control of rapid, precise, skilled movement sequences tions of his argument that the human brain eo-evolved with language rather than determining the nature of human language. From this perspective, it makes sense to ask how the intrinsic design features of language form and function are reflected in the human brain. In systemic-functional linguistic terms, Table 4.1 suggests that the different metafunctions are differentially specialized to the two hemispheres. Following Deacon, this does not mean that the relationship between linguistic metafunction and brain hemisphere is fixed. Instead, it depends, as Deacon suggests, on developmental factors rather than on in-built genetic ones. Moreover, the evidence he cites concerning bi-lingual hemispheric specialization in trained interpreters also suggests the role of individuation in the way language functions are distributed across the two hemispheres in the brain. Following Deacon's suggestion, we can hypothesize that diverse metafunctional domains are differentially distributed across the left and right hemispheres in the interests of overall processing efficiency in the brain. The right hemisphere is specialized for the interpersonal and textual metafunctions; the

192

BRAIN, MIND, AND THE SIGNIFYING BODY

left for the ideational (experiential and logical) metafunction. The differential distribution of the metafunctions across different cortical hemispheres shows that meaning is a global level of organization of the brain's self-organizing dynamics. Furthermore, the global cortical distribution of the metafunctions constitutes a means for integrating language forms both with the body-brain and with the world in ways which reflect the intrinsic organization of language itself. If language is intrinsically organized in ways which integrate it to the dynamics of the body-brain and to the world, then this suggests that the metafunctional basis of language and other forms of semiosis has important implications for the self-self transactions on which consciousness is founded. This also raises the question as to whether different individuals have different personal dispositions to different metafunctional dimensions of linguistic meaning, along with different dispositions to integrate language with other non-linguistic semiotic modalities, or even different dispositions towards a given semiotic (e.g. visual, linguistic, kinesic) modality with respect to others. In fact, language is never independent of other semiotic modalities. Language and other semiotic modalities are always eo-deployed in an integrated way. Spoken language is combined with manual-brachial gestures, posture, gaze, body movement, facial expressions, and so on. Each of these semiotic modalities co-contextualizes the others in the making of some discursive event. Cocontextualization means that one modality is not simply added to some other. The assumption that they are simply added to each other rests on the assumption that the different modalities are separate and independent to start with. Rather, different semiotic modalities have both eo-evolved with each other on the phylogenetic timescale and eo-developed and co-individuated on the ontogenetic scale. They also co-contextualize each other on the timescale of particular discursive activities and the textual productions that derive from these. The co-contextualization dynamics of the diverse semiotic modalities is shown in the ways in which they form typical eo-occurrence patterns which are recognizable as being typical or regular combinations in some community. Moreover, the combining of an item from one modality with an item from some other can alter the meaning that each item has on its own. Drawing on some observations of Bateson (l973e: 319-20) concerning the combined effects of multiple changes on somatic flexibility, Lemke (1998) has drawn attention to the multiplicative character of meaning-making: in multimodal forms of discourse and/or text different semiotic modalities co-contextualize each other, rather than being merely added to each other. The combined effect of the various modalities is multiplicative rather than merely additive. The multimodal character of meaning-making activities is commensurate with the global organization of meaning in the brain. In this perspective, the global cortical distribution of the metafunctions provides support for the hypothesis of a more general metafunctional basis to all modalities of semiosis and their organization in the brain. In systemic-functional theory, there is a growing body of evidence that the metafunctions constitute the intrinsic design features of other modalities such as depiction (O'Toole 1994; Kress and Van Leeuwen 1996; Lemke 1998; Thibault 2000b, 2001), movement (Martinec 1998), gesture, and sign language (Johnston 1992). This does not imply that the typological charac-

THE SEMIOTIC BASIS OF CONSCIOUSNESS

193

tensncs of language can be used to explain the very different topological principles of organization of, say, depiction. The fact that every discourse event and every text is multimodal derives from the ecosocial reality that meaningmaking events and texts are always simultaneously material and semiotic (chapter 1, section 2, pp. 8-11). Material and semiotic processes are always crosscoupled in meaning-making activity. The distinctions we make in our theories between language, depiction, sound, gesture, and so on, qua distinct systems of differences (typological or topological) are abstractions from the materiality of all meaning-making activity. Abstract systems of differences define and crossclassify types of relationships on the basis of multiple hierarchies of metaredundancy relations (Lemke 1984c: 35-9). Yet, meaning-making is always grounded in material couplings, the many degrees of topological variation of which can never be exhausted by any system of typological-categorial distinctions such as the theoretical abstraction of a language system that is uncoupled from the material. It is this inexhaustibility of the topological-continuous ground of semiosis which allows for new meanings to be made. This is because semiosis is never coupled to the material in entirely predictable ways. Therefore, new eocontextualizing relations across modalities can emerge at the same time as new, previously latent material possibilities can be semioticized. That is, they can become differences that make a semiotically salient difference to the members of a given community. The global integration of meaning across the right and left cortical hemispheres shows that the brain is actively involved in the integration and contextualization of the material-semiotic couplings referred to here. The global, 'imagistic' or topological-continuous character of right-hemisphere processes suggests that this hemisphere is concerned with partly nonsemioticized material couplings whereas the left hemisphere is concerned with the semiotic categories that are involved in their semioticization. The metafunctions, now generalized as an organizational principle that is common to all modalities of semiosis, can be seen as a principle of integration whereby diverse semiotic modalities are integrated and instantiated at the level of situational context or social activity. There is not, therefore, a one-to-one correlation between a given metafunctional selection in, say, language and a given contextual feature. Instead, contexts are semiotic constructs which are defined by global combinations of resources from different semiotic modalities and their material couplings. In this way, social activities and contexts, which are global constructs, are themselves organized according to metafunctional principles. The brain is thus involved in the process of contextualizing the ways in which the metafunctional components of different semiotic modalities are eo-deployed in order to produce a particular action on the basis of specific semiotic and material couplings. Therefore, the ways in which gesture, pointing, gaze, posture, and other bodily activities are integrated with language and with each other in speech can be seen to be a consequence of the ways in which very general semiotic principles of metafunctional organization in symbolic neural space constrain and entrain to specific action trajectories the different components of all the meaning-making and material resources that are eodeployed in the enactment of that activity. Table 4.2 presents four accounts - two recent, two not so recent - of language

194

BRAIN, MIND, AND THE SIGNIFYING BODY

Table 4.2: A comparison offour accounts of language form and function in context, showing an emerging consensus concerning the multifunctional nature of language and its contextual motivation Malinowski

Biihler

Halliday

Silverstein

language as action

symbolic field: (l) expression/ appeal

in terpersonal metafunction

interactional coherence

language as reflection

(2) representation

experiential/logical metafunctions

de notational coherence

context of situation

deictic field

textual metafunction

entextualization/ contextualization

form and function. The four accounts show a striking convergence, in spite of many differences in detail, in their overall conception of the relationship between language form and the meanings realized in its contexts of use. In emphasizing the convergence on the three general types of meaning in these four accounts, I am endeavouring to suggest the possibility of a broader consensus regarding the significance of the four meta functions - experiential, interpersonal, logical, and textual - for the shape of consciousness. Language is a multidimensional semantic space in which diverse functional regions are simultaneously realized in the same lexicogrammatical or textual form. The fact that there is an emerging consensus on the multifunctional character of language, if not on all the specific details of how the various functions are realized in the formal organization of language, has important implications for a theory of consciousness. If symbolic consciousness is semiotically mediated, then it becomes necessary to theorize consciousness on the basis of a semantically enriched view of both language and consciousness. Generally speaking, the models of language which inform the discussion of consciousness, when language is considered relevant at all, are narrowly focused on 'referential', 'cognitive' and 'conceptual' theories of meaning. These theories are founded on the formalism of the sentence and the forms of propositionallogic that are associated with formal logic. But semiotic mediation is a discursive process; it is linguistically mediated social activity and/or text which are integrated to their contexts. Moreover, social activities and the texts that derive from these are shaped by genre forms and conventions according to the requirements of specific social situation-types. This fact requires that the full metafunctional diversity oflanguage form and how this relates to the contexts of both the body (expression) and of our experience of the world (content) needs to be part of the explanation of consciousness itself. In the different, though related ways, and to varying degrees of generality and abstraction, the functional theories of Malinowski, Buhler, Halliday, and Silverstein draw attention to the fact that context can never be defined in terms

THE SEMIOTIC BASIS OF CONSCIOUSNESS

195

of a direct relationship between any given functional component in language and the activities that language and other semiotic modalities enact. Rather, social activities and the situational contexts in which action unfolds eo-deploy resources from diverse semiotic modalities such that a particular experiential selection in language can be eo-deployed with, say, an interpersonal selection in gaze that has implications, not for the given linguistic or gaze selection as such, but for the global meaning of the activity in which these features are combined and embedded. This is possible because particular experiential and interpersonal values from different semiotic modalities cross-classify each other at the level of the activity structures in which they occur. The ways in which selections from different semiotic modalities co-contextualize each other in this way depend on the contextualizing relations that define the activity structures of a given community. The fact that action is enacted by diverse semiotic resources and that these semiotic modalities are globally integrated by general metafunctional principles that are found in different semiotic modalities, albeit according to different principles of formal realization, provides a powerful argument in favour of the view that the metafunctions are homologous with the global organization of meaning in the brain. According to the metafunctional theory of language developed by Halliday and others, language is a resource for construing human experience, for enacting interpersonal relations, for giving coherence and texture to our meaning-making acts and the ways these connect to their contexts, and for construing logical and temporal relations between events. Table 4.2 compares four functionally motivated accounts oflanguage form in order to suggest a basis for such a consensus. The proposal that consciousness is shaped along metafunctional lines draws attention to the fact that a number of interrelated factors are all at play, simultaneously. First, one is aware of some object of consciousness as an object which is construable as belonging to such and such a category of experience. The idea of a category of experience refers to a difference which make a difference - topological and!or typological - at any given level of the implication hierarchy of icon, index, and symbol. There is no suggestion that such categories necessarily correspond to a linguistic term, though they may. The experiential resources of language are a resource for construing phenomena of experience as semantic 'objects' of consciousness. Secondly, consciousness entails an active and intentional orientation to the objects so experienced. Thirdly, consciousness is experienced and felt as a textured whole which relates self to its surroundings in a situated and unified way. These three aspects of consciousness will be discussed in sections 6, 7, and 8 below in relation to the metafunctional organization of language.

6. Experiential Meaning and the Assimilation of the Phenomena of Experience to Knowable Categories A given object of consciousness is not simply a representation of some phenomenon. Perceptual, conceptual, and semiotic categories construe specific instances as being instantiations of this or that category. In the case of language,

196

BRAIN, MIND, AND THE SIGNIFYING BODY

the experiential resources of the clause construe some phenomenon - real or imaginary - as an instantiation of a given category of process-participant configuration in the clause. The term 'construal', as distinct from the notion of representation, therefore refers to the ways in which the phenomena of our experience, including perceived events as well as imagined, remembered, or hypothesized ones, are construed as instances, to varying degrees, of the categories of language or some other meaning system. A process-participant configuration is a semantic figure, as defined by Halliday and Matthiessen, which is realized grammatically by the clause. For example, the clause Two hundred years ago, rococo was all the rage in Europe consists of the Process-ParticipantCircumstance configuration Circumstance: Time-Participant: Carrier-Process: Attribution-Attribute: Circumstantial. Experiential categories such as the ones in this example symbolically construe the phenomena of our experience as instantiations of the categories that belong to a particular language system. The notion of representation may suggest that language represents something which is prior to it. For this reason, the term 'construe' is to be preferred. The experiential metafunction in language is concerned with the symbolic construal of experience as categories and relationships of experiential meaning. This metafunction interprets the phenomena of both inner and outer experience in and through the experiential structures and categories of the clause and clause complex. Halliday and Matthiessen have formulated this relation as follows: A phenomenon is the most general experiential category - anything that can be construed as part of human experience. The phenomena of experience are of three orders of complexity: elementary (a single element), configurational (configurations of elements, i.e. a figure) and complex (a complex of figures, i.e. a sequence) ... (Halliday and Matthiessen 1999: 48) Sequences, figures, and elements are semantic constructs. As Halliday and Matthiessen point out, their typical realizations in the lexicogrammar are as follows: sequence ~ clause complex figure ~ clause element ~ element in structure of clause (group/phrase) Consider the following example: Sequence: When travelling to Fassifern by bus, buy a bus ticket to Fassifern. Figures: When travelling to Fassifern by bus buy a bus ticket to Fassifern Elements: when, travelling, to Fassifern, by bus, buy, a bus ticket, to Fassifern

THE SEMIOTIC BASIS OF CONSCIOUSNESS

197

In this example, the two figures are related to each other in a sequence by the logico-semantic relation of CONDITIONAACTION. The individual figures comprise the following experiential configurations: Actorol'rocess: Material: Action-vCircumstance: Location/Circumstance: Manner (clause 1)

Actor/Process: Material ActionAGoalACircumstance: Location (clause 2) Both of these figures construe the 'you' (the addressee) as the Actor who performs a sequence comprising two actions in determinate circumstances of location and manner. The central construct in the semantics of the experiential component of the lexicogrammar is the figure. A figure is a symbolic construal of some phenomenon of experience in the inner or outer domains - real or imagined - as a configuration consisting of a process, one or more participants, and associated circumstances. This is not a question of some direct relationship of representation or correspondence between phenomena (entities, happenings, events, ete.) in the world and selected aspects of language form. Rather, the phenomena of experience are symbolically construed by the experiential categories according to a relatively small set of process-participantcircumstance configurations in a given language system. For example, in the grammar of English, a very small set of process types - being/having, doing, sensing, and saying - configure with their respective participants to form a cluster of related semantic models of human experience. The figure is based on the following principle: the figure as a whole is comprised of a configuration of parts which function in the whole to which they belong. The semantic figure is thus analysed into its constituent parts and the functional relations which operate both between the parts and the whole and between the various parts. That is, the configuration is comprised of part-whole and part-part relations. The notions of sequence and figure, as defined by Halliday and Matthiessen, show how the constituent elements of, say, a given figure cannot symbolically construe the phenomena of experience unless they are placed in what Biihler (1990 [1934]: 206) called the 'symbolic field' of language. The symbolic field of experiential meaning is constituted by the configurations of figures and sequences of these such that their constituent elements receive their symbolic field value on the basis of the larger-scale configurations and sequences they participate in. The individual elements alone, independent of the symbolic fields in which they are placed, do not symbolically construe the phenomena of human experience. This important fact highlights the misleading nature of the claims as to the symbolic value of words per se. Thus, the noun 'cat' per se does not symbolize a particular phenomenon of experience - i.e. a given instance of the type-category [CAT]; instead, it symbolizes the schematic category [CAT] whereby all instances are already fully specified to some relevant degree of generality. Thus, the noun 'cat' does not refer to or instantiate a particular individual cat in some referent situation. The symbolic construal of the phenomena of experience depends on further specification of the given sign. Such specification is, in the first instance, provided by the symbolic field of the

198

BRAIN, MIND, AND THE SIGNIFYING BODY

figure (and sequence) in which the given element is located and in terms of which it derives its field value. In the case of experiential meaning, the constituent elements of the figure, say, are grounded in their symbolic field values through the transitivity relations of the type of process-participant configuration in which they participate, as we saw above. The symbolic construal of phenomena through the experiential categories of a given language system implicates the processes of naming, classifying, and taxonomizing. In this way, the phenomena of experience are selectively attended to and categorized so that they may be assimilated to the domain of what is known and, hence, in some sense already familiar.

7. Interpersonal Meaning as Exploratory and Orienting Activity in Relation to the Ground However, there can be no symbolic construal of the phenomena of our experience by means of the experiential resources of language without the given construal also being given a determinate point of reference in the here-&-now of the speech event. The here-&-now of the speech event is the ground (Langacker 1991: chapter 6; see also Halliday 1994 [1985]: 75) in relation to which the experiential construal in the clause is given a point of reference. The clausally realized experiential figure is grounded by the resources of the Finite element in the verbal group. In English, the Finite grounds the particular processparticipant configuration in terms of either primary tense or modality. Primary tense is what Davidse (1997) calls 'temporal proximity deixis': it locates the process-participant configuration as an instance in time relative to the ground the here-&-now - of the utterance. Modality, following Davidse, is a form of modal proximity deixis. It grounds the given process-participant configuration with reference to the addresser's evaluation, in the case of declarative propositions, and the addressee's evaluation in the case of interrogatives, as to how likely, certain, possible, usual, obligatory, necessary, and so on, the processparticipant configuration is. Again, these modalized evaluations are made with reference to the ground, as defined above. Moreover, both temporal proximity deixis (primary tense) and modality proximity deixis (modality) ground the proposition (declarative or interrogative) in the way described here in the perspective of the selves who are dialogically co-ordinated by the exchange of meaning which takes place in and through the uttering of the given proposition. In the clause, 'Two hundred years ago, rococo was all the ragein Europe, the declarative proposition is located as being temporally remote from the ground (past tense). Davidse, following Langacker (1991: 93), points out that the Finite codes subjectively. That is, the Finite locates the proposition in terms of the addresser and/or the addressee's subjective orientation to it along the parameters of temporal and/or modal proximity. That is, the addresser and/or the addressee is positioned as being subjectively near to or far from the processparticipant instance. In this sense, the process-participant configuration can be said to be grounded in the perspective of the SELF. The notion of SELF is being used as a cover term here to refer to either the addresser or the addressee in the

THE SEMIOTIC BASIS OF CONSCIOUSNESS

199

exchange. In the above example, it is the writer, in the first instance, who locates the proposition as being temporally remote from him or her, Le. some two centuries ago in the past. However, the addressee also orients to the proposition and re-grounds it in his or her perspective so that you and I qua readers, for example, also orient to the proposition as being about events which are temporally removed from the here-now ground of the act of reading. In this way, the addressee is able to dialogically co-ordinate his or her self-perspective with that of the addresser. This means that the resources of temporal and modal proximity deixis in the Finite enable addresser and addressee to orient to the proposition in terms of the same temporal or modal frame of reference with respect to the ground. Thus, even though I may read the above clause, which occurred in an article in Cathay Pacific's in-flight magazine, in a context which is different from the original one of its writing, I can re-ground the clause in my perspective because I can subjectively orient to the temporal proximity deixis such that the referent situation specified by the process-participant instance is understood as being temporally remote from me as well. This means that I can make that referent situation a symbolically construed object of consciousness in the perspective of my own self at the same time as I dialogically orient to it. The process-participant configuration is also grounded, Davidse argues, by virtue of its being tied to the Subject of the clause. In other words, the clause is grounded with reference to the person deixis of the Subject. It is tied to its grammatical person, which can be first, second, or third. Person deixis links the process-participant configuration to the intersubjectively and dialogically coordinated I-you axis in terms of which the exchange of meanings take place in some determinate context. The I-you axis is, therefore, a further form of ground in terms of which the proposition is referenced. In the above example, the Subject rococo is a third-person participant that is located as being outside the intersubjective I-you relation in terms of which the exchange of meanings is negotiated between I and you. In other words, it is specified as being remote from the ground in the sense that it is a third-person other. Once again, we can see how the grounding function of the Subject takes place within the perspective of the selves who are implicated in the exchange process. In this case, rococo is not a first- or second-person participant in the exchange of meanings. Instead, it is marked as being removed from this, i.e. as belonging to the third-person domain of the other to which I and you orient. The Subject objectively grounds and actualizes the proposition by instantiating the entity - rococo in the above example - in terms of which the proposition is grounded. That entity can be the addressee (I) or the addressee (you), or it can be objectively removed from the I-you ground and grounded as a third-person other which addresser and addressee jointly attend to as the element in which the speaker invests the modal responsibility of the clause. The Subject is the entity on the basis of which the speaker expresses a modalized perspective in the proposition and in terms of which the addresser seeks to influence the addressee's take on the same proposition. The interpersonal metafunction is what Halliday (1993: 103) defines as the 'active' principle of language. It is language functioning to enact interpersonal, dialogically organized interactions between some 'I' and some 'you'. The

200

BRAIN, MIND, AND THE SIGNIFYING BODY

interpersonal metafunction is therefore centrally concerned with language as action. Closely linked to this is the fact that the resources of the interpersonal metafunction are fundamentally concerned with systems of social values whereby interactants orient both to each other in and through their interaction, to the utterance itself, and to still wider systems of social values that the utterance may invoke. In other words, interpersonal meaning is concerned with the link between value and action and the implications which the former has for the latter when interactants orient to and engage with the world of the nonself (Thibault 2002). This further implies that the interpersonal metafunction is concerned with exploratory activity whereby the individual negotiates and engages with the indeterminate, the not-yet-specified, and the unknown. Thus, linguistic propositions (declarative, interrogative) and proposals (oblative, imperative) are a means of enacting this kind of exploratory activity. In the process, the individual generates value-laden hypotheses about the unknown while assimilating it to some more determinate experiential framework (see also Halliday 1993: 102). Thus, interpersonal exploration of and engagement with the unknown, the unfamiliar, or the indeterminate also leads to the increasing specification of the previously vague phenomenon through its construal by the experiential categories of language. If, hypothetically speaking, language was interpersonal interaction and action per se, there would be no resources for converting topological-continuous variation into more determinate and manageable typological-categorial distinctions of the experiential kind. The existence of these typological-categorial distinctions means that diverse experiences can all be categorized as being similar on account of the system of values of the given language system (Thibault 1997a: chapter 7).

8. The Textual Metafunction as Semiotic Means for Giving Unity and Wholeness to Meaning-making The textual metafunction is concerned with the way in which language is organized as discourse which is operational in some context (Halliday and Hasan 1976; Hasan 1980). It is dually concerned with (1) the ways in which the sign-tokens (words, clauses, ete.) which comprise a given text achieve a relational cohesion with one another through such resources as topic maintenance and continuity, continuity of referents, and so on; and (2) the subjective orientation of interactants within what Biihler (1990 [1934]: 117) referred to as the 'deictic field' of language. The deictic field consists, in the first instance, of the herenow-I system of subjective orientation to a co-ordinate source (e.g. the speaker of the pronoun 1) in some context of situation in which the given discourse is operational. More generally, the textual metafunction is concerned with both how the various parts of the discourse are related to each other as a textured whole (a text) such that the resulting whole is perceived as having achieved a degree of relative coherence as a whole and (2) the ways in which the participants in the temporally unfolding interaction manage to achieve systematic and regular links between some emergent context and the unfolding linguistic or

THE SEMIOTIC BASIS OF CONSCIOUSNESS

201

other text. Moreover, it is important that the participants perceive the patterned contextualizing relations between the two as (situationally, generically) appropriate or effective realizations of text-in-context (Silverstein 1997: 270).

9. Experiential and Interpersonal Meaning in Gaze When we attend to someone else's gaze vector, we can see how similar metafunctional principles operate at the lower, less specified integrative level of perceptual awareness. In attending to and attempting to interpret the target of someone's gaze vector, perhaps in order to determine his or her interests or intentions with respect to the target, I implicitly analyse the structure of the other person's gaze in terms of the person who is the source of the gaze (Gazer), the directional vector of the person's gaze, and the Target or goal of the gaze vector, understood as the entity which the Gazer is attending to. In this perspective, the gaze is analysed as a process-participant configuration on analogy with experiential meaning in the clause. Table 4.3 provides an analysis of this proto-experiential dimension of the meaning of gaze. Table 4.3: Proto-experiential meaning in gaze vector Participant

Process

Participant

Gazer

Directional vector of gaze

Target

The gaze is also related to the here-now ground relative to the observer (the self) who interprets the other person's gaze. In this case, the Gazer is the objective grounder or the actualizer of the gaze. The gaze vector that extends from the Gazer's eyes to the target, on the other hand, subjectively grounds the gaze in the here-now of perceptual awareness and perceptual modality (visual); the Cazer's current interests, as interpreted by the observer; distance from the Gazer; and the Gazer's possible subjective orientations to the Target. For example, is the Gazer looking at a member of the opposite sex because of sexual interest in or attraction towards the Target? The Target, on the other hand, does not participate in these grounding functions, which are based on the Gazer-Gaze Vector nexus. Instead, the Target further specifies the overall process-participant configuration by specifying the specific entity that is the focus of the gaze. The Gaze-Gaze Vector nexus is, therefore, similar in function, though on a lower integrative level, to the Subject-Finite structure in the Mood element of the clause. However, gaze is much more immediately tied to its here-now ground in ways that language is not on account of the way in which the interpretation of someone's gaze depends on the interpreter's here-now perception of the other's gaze. Table 4.4 illustrates the grounding function of gaze, as discussed in this paragraph.

202

BRAIN, MIND, AND THE SIGNIFYING BODY

Table 4.4: Proto-interpersonal meaning in gaze vector, showing ground functions Objective Ground

Subjective Ground

Gazer as objective grounder and instantiator of gaze vector

Gaze Vector grounded in here-now of perceptual act; indicating subjective stance on Target (e.g. sexual attraction)

Target as specification of entity that is focus of attention

10. Proto-interpersonal Meaning and the Child's Exploration of Its Environment Interpersonal meaning, as we saw in section 7, is implicated in the initial stages of the exploration of the vague, the unknown, and the indeterminate. It is concerned with hypothesis formation. Consider the following revealing discussion by Halliday: A human infant engages in symbolic acts, which I have referred to as acts of meaning. Children are predisposed, from birth, (a) to address others, and be addressed by them (i.e., to interact communicatively); and (b) to construe their experience (i.e., to interpret experience by organizing it into meanings). Signs are created at the intersection of these two modes of activity. Signs evolve (a) in mediating - or, better, in enacting - interaction with others, and (b) in construing experience into meaning; specifically, in exploring the contradiction between inner and outer experience (between what is perceived as going on 'out there' and what is perceived as going on 'in here', within the child's own consciousness (cf. Trevarthen, 19S0). Thus, typically, at 0; 3 to 0; 5 months (years; months) babies are 'reaching and grasping', trying to get hold of objects in the exterior domain and to reconcile this with their awareness of the interior domain (they can see the objects). Such an effort provokes the use of a sign, which is then interpreted by the adult caregiver, or an older child, as a demand for explanation; the other responds in turn with an act of meaning. There has been 'conversation' before, but this is a different kind of conversation, in which both parties are acting symbolically. A typical example from my own data would be the following, with the child at just under 0; 6 months (Halliday, 19S4a, p. 2): There is a sudden loud noise from pigeons scattering. Child [lifts head, looks around, gives high-pitched squeak] Mother: Yes, those are birds. Pigeons. Aren't they noisy! (Halliday 1993: 94-5) In Halliday's example, it is the sudden loud noise made by the pigeons as they scatter which impacts on the child's perceptual systems. This noise is an index information in Gibson's sense - of an environmental event. As such, it stands in

THE SEMIOTIC BASIS OF CONSCIOUSNESS

203

an iconic-indexical relation of necessity to that event. However, it is not simply a question of the brain's detecting and processing of specific (auditory) sensations which are, in turn, interpreted as representations of the world, and which then inform the mind as to what is going on in the world. The point is that once the information which is carried by the nerve fibres from the peripheral receptors enters the central nervous system, it is no longer locked only onto the stimulus (Wall 1996: 176). In other words, it is no longer a question of a purely indexical relationship between environmental stimulus and the organism's response. As Wall writes, the information which is received by the brain 'depends on the contingencies of other events in the periphery and on the brain state which orders and permits and shapes the arriving messages' (1996: 176-7). Thus, the sensorymotor response which is made is not passively locked into the stimulus, but depends on an active process of selection in the brain (Wall 1996: 177). This process of selection means that the transduction of the stimulus information which is picked up by the receptors results in its translation into various possibilities of sensori-motor response. This can only occur on the basis of a stored system of signs in the Innenwelt whereby the child uses stimulus information about environmental events as signs for interpreting the world. The child's response to an environmental event - his squeak - has potential symbolic value because there is no necessary relationship between this sensory-motor response and the way it may be interpreted by his interlocutors. That is, the squeak is already interpreted as belonging to a system of symbolic possibilities, however elementary. This means that the squeak derives its value from its position in a system of elementary possibilities for meaning-making. Furthermore, the child's vocalization can be seen as a dialogically co-ordinated response to a particular non-verbal aspect of the context of situation. The system of symbolic possibilities which are stored in the child's Innenwelt thus regulates his sensori-motor activity as value-laden possibilities for interacting with and dialogically engaging others. The value-laden character of these possibilities means that they are endowed with actional and affective significance. The squeak qua squeak is in itself an environmental event which generates stimulus information about its source. Again, such information is both iconic and indexical. However, the fact that the child's interlocutor construes the squeak as a 'demand for information', as indicated by her own response, means that the squeak is also construed as having symbolic value. For this reason, the squeak is not necessarily (indexically) locked into specific environmental events; instead, it affords possibilities for symbolic action which can be deployed in other contexts, in response to other environmental events. This is so because the meaning 'demand for information' of the squeak is not locked into a particular here-now context, but derives its symbolic value from its position in a developing system of other contrasting values and their modes of realization. The child's high-pitched squeak, which Halliday glosses as a 'demand for explanation', is a situation-specific instance of adaptive motor behaviour. It is a means whereby the child explores his immediate environment. It is an elementary form of hypothesis, which elicits its further modification on the part of the other (the mother) in the exchange. This modification, of course, comes in the form of the mother's far more highly specified response. In this way, the

204

BRAIN, MIND, AND THE SIGNIFYING BODY

child learns about the world in these jointly constructed acts of interpersonal engagement and experiential construal. Moreover, the mother's response is likely to have affective consequences as well. The assimilation of the unknown phenomenon which had perturbed the child's perceptual purview to known experiential categories is potentially laden with affective valence and motivational significance. Thus, the satisfaction of the child's 'demand for explanation' and the assimilation of the given phenomenon to the jointly shared and sustained experiential categories of the dyad boost positive affect and, hence, motivate the desire for further exploration. The loud noise caused by the scattering of the pigeons is an environmental event which is picked up by the child's perceptual systems. The fact that the child lifts his head and turns around to attend to this event indicates this quite clearly. The head movement constitutes a response to the event, which is seen as being in some way interesting, surprising, and so on, and is, for this reason, tracked by the child's head movement. In this way, the event - the scattering of the pigeons - gets constituted as something which is worth orienting to and responding to. The child's high-pitched squeak, therefore, becomes the child's comment on what he has selected as the focus of his attention. The interesting and revealing thing about this exchange is the way in which the child's high-pitched squeak is recognized and construed by the mother as being a comment about the pigeons. That is, the mother is attuned to the interests and experiences of the child in three specific ways. First, she recognizes that the child's squeak is a particular orientational stance. Secondly, she recognizes that this stance is about something. Thirdly, she provides, in making her response, a link between the symbolic possibilities of the higher-scalar meaning systems of the culture and the mother-child dyad. All three aspects are fully revealed by her response, which shows both that she is tracking and is tuned into the child's stances towards particular phenomena and that she can co-ordinate the child's stance with her own (linguistic) construal of the event. The overall structure of this dyadic exchange may be glossed as follows: PERCEPTUAL PICK-UP OF STIMULUS INFORMATION ABOUT ENVIRONMENTAL EVENT

CHILD'S ATTENDING TO EVENT AS A FOCUS OF INTEREST

CHILD'S RESPONDING TO EVENT WITH PROTOLINGUISTIC SIGN/ENGAGING INTEREST AND ATTENTION OF MOTHER

MOTHER'S RESPONSE INTEGRATES EVENT AND CHILD'S RESPONSE INTO HIGHER-SCALAR ECOSOCIAL CATEGORIES

THE SEMIOTIC BASIS OF CONSCIOUSNESS

205

The child's response to the event in the form of his looking around followed by the high-pitched squeak is a meaningful act which is directly dependent upon the experienced phenomenon. It lacks the symbolic organization of a lexicogrammar, which enables acts of meaning to be freed from direct dependence on the phenomenal (Halliday I992a: 23, 25). Nevertheless, his composite meaning-making act may be seen both as construing experience and, at the same time, his dialogically engaging the mother in his experience, in the process enacting an intersubjective framework of interpretation, which the mother further elaborates. The two-part nature of the child's response - viz. looking around followed by squeak - can be interpreted as a protolinguistic analogue of the interpersonal function of mood, which specifies the speaker's interactive stance on some proposition or proposal in the adult language. Thus, the scattering of the pigeons is constituted as a kind of proto-Subject about which some comment is made in the form of the squeak. The squeak may therefore be seen as a proto-proposition which is made about the designated Subject. Both Halliday's analysis and the mother's response draw attention to the way in which the latter is attuned to both the interactional status of the child's sign as well as to its experiential significance.

11. Bogdan's Theory of Mental Sharing and Topical Predication Bogdan (2000: 78) argues that the infant's selection of a target of attention is a topic and that the representation of the target is a comment. This results in what Bogdan calls 'topic-comment parsing or segmentation and the mental routine that effects the parsing is topical predication, ... ' (2000: 78). Bogdan considers topical predication to be pre-linguistic and based on mental sharing: Work in linguistics, psycholinguistics, and philosophy has shown that topical predication cannot be reduced to logical or grammatical predication (see Clark and Clark 1977 and Ninio and Snow 1996 for psycholinguistic surveys; see also Bogdan 1987 and Dretske 1972 about implications for propositional attitudes). This irreducibility can be grounded in the hypothesis that topical predication is prelinguistic and rests on mental sharing by sentimental bonding and physiological regulation. Also supportive is the fact that autistic people, whose minds are not socialized intersubjectively, can do logical and grammatical but not topical predication. To account for these developmental truths and the reasons for them, it helps to start with the liberal notion that topical predication indicates a mental take (comment) on something of interest (topic). The topic can be whatever is a target of attention or interest - a thing, event, property, relation, scene, sentence, thought, image, picture, sign, or complex structures of such. The topic can be out there in the world or up here in the mind, whether one's own or somebody else's. The comment is some mental take on or mental relation to a topic, expressing an experience, emotion, stance or attitude. Depending on whether the topic expressing an experience is a worldly or mental item, the comment can be a representation of something out there in the world or

206

BRAIN, MIND, AND THE SIGNIFYING BODY

a representation of a representation in the mind, respectively. There are many ways in which topics and comments can be represented - in perception, memory, thought, mental imagery, bodily posture, action, with or without language. (Bogdan 2000: 78) The idea of a pre-linguistic topic-comment structure, with its correlative of the subject's mental takes on the world, suggests that there already exists an at least weak classification of self as distinct from the world, including others. Furthermore, it suggests that there exists a prior, pre-linguistic structure on which linguistic forms of predication are subsequently built. In the dynamics systems view, on the other hand, the differentiation of self and world depends on more than the prior existence of a topic-comment structure. Halliday's notion of the interpersonal gateway to meaning draws attention to the initial primacy of prelinguistic proto-interpersonal or dyadic engagement. The infant's earliest experiences of the world are topological and require a gradual and progressive differentiation from a less differentiated whole. The self is experienced as not yet sharply differentiated from the world, though in some kind of vague relationship with it. That is why I prefer the proto-interpersonal interpretation given above. If, on the other hand, there exist 'prior topic-predication formats' (Bogdan 2000: 78), then this would amount to saying that the child comes already equipped with a primitive means for segmenting the self/nonselfrelationship in terms of some target of selective focus in the world (internal or external) and one's propositional take or attitude on this in the form of a comment. Self and world would already be construed as different in this view. In my view, the construal of self and world (nonself) as different is not already given; instead, it is a form of developmental emergence (Salthe 1993: 244). Thus, the shift from the child's primordial experience of self-in-relation-tothough-not-yet-differentiated-from-the-world to the experience of se1f-asdifferent-from-world involves the forming or coalescing of a higher-scalar entity - e.g. a dialogically organized proto-proposition - around a number of variables that now appear as the lower-order components - neural values, environmental stimulus information, visual tracking by head-turning, protolinguistic gestures and vocalizations, the presence of others, meanings from other conspecifics out of which the new property has emerged. By the same token, there is also a corresponding shift in the observer/agent's attention to the typologicalcategorial distinctions belonging to the higher integrative leveL This means that the newly emergent proto-propositional resources and attitudes mentioned above, with their typologizing of self/nonself relations, are now seen as closer to the observer's scale which is shared by other members of the culture, rather than to the vaguer state of initial topological non-differentiation. The latter, however, remains implicit at the lower scalar levels, which are now constrained by the higher ones. In any case, it is the dialogically organized and therefore socially shared act of proto-meaning which exists and which is most visible in the supersystem transactions between self and other. Nevertheless, the mental and individual bias illustrated in Bogdan's description of topic predication indicates the hankering after inferred objects - mental entities - once the observer's supersystem categories have been described with reference to their leveL

THE SEMIOTIC BASIS OF CONSCIOUSNESS

207

The child's protolinguistic engagements with the world make possible further differentiations of the reentrant loop that maps these engagements onto the child's inner neural activity. The child thus moves from an initial, primordial topological state of non-differentiation (vagueness) between experiencing and meaning, to an experiencing of experiencing, to the progressively more differentiated construal of meanings that are attributed to experience (Lemke p.c.). Bogdan's focus on pre-linguistic topic predication could suggest a built-in telos where none may, in fact, be necessary in order to explain what is going on. His focus on mental sharing, physiological regulation, and sentimental bonding says rather less about the physical-material processes - e.g. stimulus information about environmental events - which inform and constrain the self-organizing properties of the child's neuro-motor system. Such environmental events and the stimulus information they afford the organism are contingencies with which the self-organizing properties of the child interact in ways the probabilities of which are pre-set by biologically in-built criteria of value (section 12 below). In my view, the prior existence of a topic predication structure would be too rigid an imposition and would not allow the child to respond to constantly changing environmental contingencies. As the above discussion of the example from Halliday begins to show, the key to understanding what is going on depends on understanding the ways in which individuals adapt to openness and novelty in complex dynamic systems comprising very many fluctuating variables. In our example, these include the interactions between an environmental event, the child's perception of this, his attending to it in looking around, his squeak, and the mother's response. It is the mother's more specified linguistic response that provides the link to higher-scalar ecosocial semiotic categories and relations. In so doing, she provides the child with access to resources - symbolic possibilities - and entrains him into their contextually appropriate use. The further exploration and entraining of these possibilities brings about the increasing differentiation of topological vagueness and wholeness and the development of an expanded repertoire of relations between self and other. The above analysis serves to show that it is not a question of prior, prelinguistic topic-comment structures which permit 'mental sharing'. The analysis shows that the ability to focus on and form a proto-proposition about some event is an emergent property of the overall activity structure that takes place. As the analysis shows, this may be broken down into at least the four components specified above. The individual is thus regulated by his or her exchange transactions with the ecosocial environment, including other conspecifics. Both the physical environment and the social environment function as structured sources of potentially meaningful information. In the example, the physical environment provides stimulus information about an event - the scattering of the pigeons - to a potential observer; the child picks up and attends to this information, as demonstrated by his tracking the event through head-turning. His squeak dialogically engages the mother, who responds with a linguistic utterance. In my view, the mother's contribution to the exchange provides a structure which functions to complete or further elaborate the child's developing structures. (See also Cole 1996: 187-90 on the notion of routine activities as proximal environments for development.) Thus, the much more

208

BRAIN, MIND, AND THE SIGNIFYING BODY

implicit structure and meaning of the child's contribution to the dialogue elicits the more complete response of the mother. The latter has the function of extending and modifying the implicit structures of the child by elaborating and further specifying them. In this particular example, the mother's utterance, Yes, those are birds. Pigeons, is a declarative clause whose demonstrative pronoun Subject those indexes the same phenomenon which the child had initially responded to. That is, the perceived phenomenon is construed as a linguistic entity about which a proposition is made in the rest of the clause. In this case, the attributive clause functions to specify the type-category to which the pigeons seen by the child belong and, therefore, to classify or name them. Thus, the relatively low order structure created by the child - viz. 'looking around followed by high-pitched squeak' - is itself a kind of proto-proposition in relation to which the perceived event (the pigeons) is the proto-Subject about which some proto-proposition is made. My point is that this proto-proposition is not an internal mental structure already possessed by the child. Rather, its emergent properties are distributed across various components of a muItimodaI social activity. The mother's further elaboration of this structure in the form of an explicit linguistic proposition therefore entails its further elaboration and completion. Table 4.5 provides a multimodal analysis of the transaction considered here. Table 4.5: Multimodal eo-deployment ofperceptual-semiotic resources in the exchange between the child and his mother in Halliday's example Movement

Visual

Auditory

The scattering of the pigeons as environment event

Spontaneous movement of the pigeons as they scatter

Optical information about an environmental event available to a point of observation (e.g. the child and his mother)

Auditory information about an environmental event available to a point of observation (e.g. the child and his mother)

The child's dialogically co-ordinated response

Turns head-body to attend to and visually track the event

Directs gaze vector to event

Utters highpitched protolinguistic squeak

The mother's response

Co-ordinates headbody movement with the child's to jointly attend to event

Co-ordinates gaze with child's to jointly attend to event

Utters linguistic response

Perceptual-semiotic modality The dialogically eoordinated components of the unfolding event

THE SEMIOTIC BASIS OF CONSCIOUSNESS

209

There is no need to insert mental functions between biology and culture in the way that Bogdan does (2000: 31). Here is how Bogdan distinguishes mental functions from neural functions: The contrast need not be between biology and culture, with nothing in between. Culture cannot be grafted directly on unsensitized neural functions. Left out are mental functions, however simple. The problem is not that such simple functions are neural; of course they are, in an implementational sense. The problem is that mental functions cannot be reduced to biological or neural functions, insofar as the former (but not the latter) have psychological or mind-world value. That psychological value concerns the relation between organism and world, both physical and social, and the impact of that relation on behavior. This is how psychology (but not biology or neuroscience) looks at organisms. (Bogdan 2000: 31) The problem in Bogdan's account lies in his assuming that neural functions are neutral or value-free. They are not. Moreover, neural functions are concerned with the relationship between organism and world. There is no need to postulate mental functions, however primitive, which mediate between biology and culture from the outset. Biology is neither value-free nor non-social right from the word go. Value-laden neural functions predispose the child to participate in self-regulating exchange transactions with their ecosocial environment from the outset. The child's high-pitched squeak, in our example, is a dialogic act which is oriented both to an environmental event and to the mother. It shows how the child's active and dialogic orientation to his environment works to seek information and meaning which will affect his further development. The child's proto-linguistic squeak constitutes an active intervention in, rather than a passive response to, his environment. The squeak is fully dialogic, if not linguistic, because it is intended to obtain an appropriate response from the mother or other primary caregiver. In this way, the child's own agency constitutes an attractor space in which the context-bound responses of caregivers are organized and channelled. The attractor space therefore constitutes a domain in which the social practices of attending to and interacting with infants interface with biological values and functions. It is the interfacing of the two which enables the further development of the child's own self-regulating structures. In this way, mental functions such as topic predication are built up on the basis of the reentrant mapping of the child's exchange transactions with its environment onto internal neural activity. They are not part of the mechanism whereby such structures are acquired. The means of acquisition of such structures is better handled by Edelman's notion of value, as we shall now see.

12. Interpersonal Meaning, Value, and Action What is it that impels the infant to ever more differentiated symbolic possibilities? Is it necessary to postulate pre-linguistic topical predication as the key to 'mental sharing'? Pre-linguistic topic predication would point to the existence of

210

BRAIN, MIND, AND THE SIGNIFYING BODY

an already existing in-built initial differentiation between self and other, however primitive. Recent neurobiological evidence does not favour this view. For example, Damasio advances the notion of the 'proto-self, which he defines as a 'preconscious biological precedent' (1999: 153) of the self. I shall return to this notion in chapter 5, section 5. Damasio's neurobiological arguments do not support the notion that there exists an already prior, initial polarization of self and world as being categorially distinguished. This seems to suggest some kind of categorization of this fundamental distinction prior to experience. Edelman has proposed the notion of some very general biological values which are intrinsic to the organism from the start. Edelman's values are not based on such categorial distinctions. Values simply function to provide the initial impetus to the organism's ongoing, time-bound engagements with the world. They do not in themselves provide the organism with ready-made categories for distinguishing self from other through experience or for experiencing one's own experiencing, let alone for giving meaning to experience. The progressive differentiation of self and other and the elaboration of the self's experience of the world is based on epigenetic principles. On the basis of experience, the differentiation of self and other occurs through the reentrant mapping and gradual strengthening of neuronal groups whereby the category of self-other relations and transactions is elaborated. Values function to nudge the developing organism along certain developmental pathways. For this reason, they may be seen as kick-starting the interaction between the organism's self-organizing dynamics and the environmental contingencies it encounters. For example, Thelen and Smith (1994: 315) cite research undertaken byJohnson and Morton (1991) which proposed that 'infants' strong attraction to faces begins with an initial orienting bias in newborns' (1994: 315). Thelen and Smith continue as follows: This bias, the behavioral evidence suggests, favors the arrangement of several high-contrast 'blobs' corresponding to the relative location of eyes and mouth, likely mediated through subcortical pathways. Given that infants see such configurations in real faces close up during nursing, changing, and socializing, even a small initial valence toward that configuration would lead to rapidly strengthened neuronal groups in the presence of faces. As higher brain centers become more involved in the second and third months, the system will already be attuned toward facelike stimuli, insuring that infants will be positioned to acquire more information about faces. Indeed, preference for particular faces emerges by 4 months or so. What is minimally needed to start such a process moving is a sensory system with some relatively unspecific tuned parameters and a value wash that keeps the organism preferring certain stimulus configurations over other values. Such values are not specific tutors. Having a bias in the system that says light is better than no light or human voices are better than auto horns does not endow the system with knowledge modules or conceptual primitives or the understanding of objective properties. Rather, these are valences or trophisms similar to those exhibited by simple organisms and even by animals without nervous systems. That such valences might be contained within the architec-

THE SEMIOTIC BASIS OF CONSCIOUSNESS

211

ture of the system does not put a ghost in the machine, or if it does, it is the ghost of life itself, for similar biases are exhibited even in single cells. (Thelen and Smith 1994: 315-16) The initial orienting bias to the 'blobs' mentioned in the above citation indicates the purely iconic and topological-continuous nature of the child's initial orientation to the other. Thus, the values which favour the active attending to environmental events through the perceptual systems, the participation in dyadic exchanges with mothers, and so on, bias the self-organizing dynamics of the system. The infant participates in activities which pull the infant away from the initial attractor space of relative topological (iconic) non-differentiation between self and other towards the increasingly differentiated indexical and symbolic differentiations afforded by the mother's access to the higher-scalar categorial (indexical and symbolic) differentiations of the ecosocial system. In doing this, the child's initial, less differentiated relations between self and other become progressively more differentiated. The child progresses from protoimperative followed by proto-indicative modes which are tied to the phenomena of experience to the full range of interactional possibilities afforded by the interpersonal systems of the adult lexicogrammar. Figure 4.1 illustrates this developmental emergence as a presupposition-cum-implication hierarchy of icon, index, and symbol with reference to the developmental emergence of interpersonal semiosis. This suggests that the interpersonal metafunction is concerned, in the first instance, with the vague, the unspecified. It deploys prosodic and scopal modes of realization so as to engage with and resolve, through dialogic negotiation (cf. Bakhtin's concept of finalization), that which is not yet assimilated to some determinate classification. In the first instance, the interpersonal metafunction Figure 4.1: Integration-cum-presupposition hierarchy of iconic, indexical, and symbolic modes of grounding Icon

Perceptual-motor grounding of bodybrain in Umwelt

Physical-material Vague, unbounded topological-continuous

Index Contiguity with selectively attended to contextual values in here-now spatiotemporal purview of addresser and addressee

Symbol

Entextualization and contextualization of deictic field

Semiotic-discursive Specified, bounded typological-categorial

212

BRAIN, MIND, AND THE SIGNIFYING BODY

is concerned with the assigning of affective valence and motivational relevance to the not-yet-defined phenomenon. It is concerned, above all, with questions such as 'what value do I assign to this thing?', 'what should I do with it?', 'how should I orient to it?', and so on. In other words, interpersonal meaning orients interactants in terms of the given phenomenon's value-laden salience for action. It is only through acting in and on the world - i.e. the nonself - that we are also able to construe it experientially in ways which make it understandable and, therefore, able to be assimilated to the stock of human social meaning-making. It is the ability, initially, to formulate hypotheses about the unknown and the vague or indeterminate which constitutes the first stage in our adaptation to it. In this way, our modalized orientations to phenomena constitute a structured response which functions to inhibit and channel our responses to the indeterminate at the same time as we elaborate our experiential understanding of it. It is the experiential metafunction which completes and complements this process by abstracting from our subjective investments in particular courses of action so as to create a more explicit experiential framework of knowledge.

13. Procedural Knowledge, Declarative Knowledge and the Semiotic Spiral towards Symbolic Consciousness The distinction that psychologists make between procedural and declarative knowledge is relevant here. Procedural knowledge is concerned with action oriented to intervening in the world and bringing about desired change. It is based on the active sensori-motor exploration of the world in order to develop appropriate and lasting patterns of action for dealing with the world and its contingencies. It is knowledge which is implicit because acted on only in direct interactions with other objects, other people, and so on. The right hemisphere is responsive to and productive of the global or holistic imagery which provide the initial topological-continuous variation - the proto-meanings - on which a text's meaning is built. The left hemisphere re-interprets these values as explicit structure and increasingly abstract taxonomic hierarchies of experiential categories. It is significant, then, that the procedural basis of interpersonal engagement and action emerges before declarative knowledge, both phylagenetically and ontogenetically. Declarative knowledge, on the other hand, is concerned with what is explicit and known; it includes knowledge which is created in and through the ideational resources of language. As Halliday, Trevarthen and others have shown, the child first learns, though protointerpersonal interaction, the procedural knowledge associated with the negotiation of the T I'you' relation in, say, the mother-child dyads characteristic of primary subjectivity. Only later is the infant's attention turned towards the experiential construal of the world which lies outside 'me' and 'you' in the realm of the third person. Thus, early interpersonal interaction and the dyads which give shape to this entail, in the first instance, the entraining and adaptation of sensori-motor patterns of activity prior to the more abstracted forms of symbolically mediated activity that are characteristic of later stages of development. Table 4.6 schematizes some of the salient stages in this process.

THE SEMIOTIC BASIS OF CONSCIOUSNESS

213

Table 4.6: Types of knowledge and stages of semiotic development in early infant semiosis

Time in Months

Type of Knowledge

Semiotic Mode

0.0-0.8; 0.8-0.9

Sensori-motor knowledge; practical knowledge; adaptation of reaching response to object; implicit recognition of self-object distinction

Iconic

0.9-18.0

Representational knowledge; means-end and cause-effect relations; inten tional activity; memory

Iconic-indexical

18.0-24.0

Emergence of abstract me ta-representations; construing remembered, unshared information; emergence of I-me distinction, i.e. child knows himself not only as a point of action (I) acting on others, but as others see him (me)

Iconic-indexical-symbolic

Procedural knowledge, which is based on sensori-motor activity and the protointerpersonal dimension of meaning, is not essentially representational in character. Rather, it is concerned with the enacting - initially in interpersonal contexts - of social activity-structures (cf. genres). However, it is, in its own way, a form of knowledge which is transferred between individuals in and through the processes of exchange. Mimicking and imitation, for example, (Piaget 1946; Battacchi 1999; Perinat and Sadurnf 1999) can occur with reference to other persons as well as in relation to inanimate objects. In play, children may interact with the nonself, which may be another person or some object. In the process of dialogically engaging with the given object, both structure and meaning are exchanged. Initially, the child projects some qualities onto the object of interest in his or her purview. The child then imitates his or her own interactions with these objects; in the process the child ritualizes his or her own activity as he or she adaptively modifies his or her own actions in and through their exchanges with the object. In this way, ritualistic activities emerge as the precursor of symbolic activities (Thibault 1998b; Lorenz 1969 [1963]). That is, genre and discourse structures, perhaps specific to the precursor world created and inhabited by children (Harre 1983: 251-2), emerge and function as extensions of the self.

214

BRAIN, MIND, AND THE SIGNIFYING BODY

Interpersonal moral orders are grounded in procedural activity and only later made more abstract through experiential reflection on action and interaction. Thus, declarative memory produces experiential construals of the implicit logic of interpersonal exchange in the procedural system. The elaboration and storage of such meaning is essential to the formation of interpersonal moral orders in which one's own and others' actions can be explained and accounted for within a network of responsibilities and agentive positions and their assignment to individuals. In this way, interpersonal-procedural knowledge is abstracted and codified as forms of moral reasoning (see Hasan 1992a; Thibault 1992b). The declarative-ideational system has thus elaborated a meta-discourse concerning procedural-interpersonal activity through a process of abstraction from the here-now of interpersonal engagement and exploration. As the work of Halliday, Trevarthen, and other researchers shows, the earliest dialogic acts of infants in the form of sensori-motor acts (gestures, vocalizations) and imitation occur prior to the emergence of any explicit meta-semiotic awareness of the principles which regulate interpersonal exchange. The declarative-ideational system allows for reflection on and further elaboration of the meaning-making processes that occur in early sensori-motor exploration, imitation, and play. The procedural-interpersonal system constitutes a form of memory for action and interaction. This memory is built up and elaborated on the basis of very many individual exchanges which have been integrated into the patterns of interpersonal engagement which are appropriate or otherwise acceptable in a given interpersonal moral order. The integrative nature of interpersonal exchange means that the often competing motivational demands of agents are resolved with reference to the interpersonal moral orders in which agents interact with each other. It also means that individual needs and desires are regulated by social norms and values which the individual internalizes. This means that the system of sometimes conflicting, sometimes aligned values, judgements, viewpoints, and so on, in the system of social heteroglossia, are organized and internalized in terms of hierarchies of values such that the morality of specific courses of action can be evaluated and re-evaluated. As I pointed out above, the enactment of such value hierarchies, which are usually implicit in specific acts of exchange, is developmentally prior to the experiential construal of the actions in which these values are implicit. In this way, explicit experiential reflection on action and value becomes possible, and in ways which can act back on and alter action itself. Explicit experiential reflection means increased symbolic abstraction. With increased powers of symbolic abstraction go the capacities of self-monitoring, self-control, and self-awareness that are defining characteristics of higher-order consciousness. This shows very clearly how both the interpersonal and experiential dimensions of meaning-making are implicated in and are indispensable to the workings of higher-order consciousness. Thus, the emergence of symbolic or higher-order consciousness (Edelman 1989, 1992) comprises a spiral that begins with sensori-motor exploration and procedural-interpersonal engagement with the other before symbolic consciousness emerges. These earlier forms are then experientially construed as symbolic acts which can be shared with others. These observations accord with Halliday's (1993: 103) notion of interpersonal meaning

THE SEMIOTIC BASIS OF CONSCIOUSNESS

215

as the 'gateway' to later forms of meaning-making: new meanings are first enacted interpersonally before being construed ideationally. Symbolic sharing, in turn, constitutes the basis of the further abstraction of interpersonal interaction from the here-now of sensuous and embodied activity. This means that procedure can be dialogically reflected from the social viewpoint of others - real and virtual- then enacted in subsequent interpersonal exchanges. The result is that interpersonal interaction is itself modified, thereby leading to further changes in its experiential construal. This process can take place in the course of interpersonal interaction itself, or internally by virtue of its abstraction from sensori-motor activity and its consequent specialization to the 'inner' realm of semiotically mediated thought, which is itself a specific form of discursive activity (section 4, pp. 184-9).

14. Interpersonal Meaning, Goal-seeking Activity, and the Goal Hierarchy The interpersonal grammar of mood shows that the goal and the means to attaining this are not constitutively separable elements. Each of the basic mood choices, along with the speech functions that the various mood choices enact, entails a present state, a desired future state, and the resources - selective crosscouplings of the physical-material and the semiotic-discursive - that could be used to attain the goal-state, viz. to transform the former into the latter. The grammar and semantics of the interpersonal metafunction shows that what are often called 'desires', 'wishes', 'needs,' 'wants', and so on, and seen as mental predicates in the theatre of the individual mind, can be seen as dialogic interacts (Martin 1992a: chapter 2, 1992b; Bakhtin 1981, 1986; Thibault and Van Leeuwen 1996). The interpersonal grammatical resources discussed above are also deployed in the service of larger-scale goals on different space-time scales. Thus, specific semantic interacts and the clauses and larger units which realize these interacts or moves are seen as nested in this larger-scale structure in terms of relationships of the kind Silverstein designates as 'interactional coherence' (1997). At any given moment in the unfolding interaction, the number of possible variables that can function as means and end are not totally openended, but are limited in number. A particular choice of speech-function move in the interaction entails criteria of inclusion and exclusion by virtue of the dialectical duality of paradigmatic and syntagmatic principles of organization which operate in discourse. However, the open and dynamic character of discursive negotiation (Thibault 1999a) means that the range of possible relevant variables is not fixed, but flexible. The principle of goal-seeking means that we have, to varying degrees, a relative semiotic freedom (Wilden 1981: 25) to make decisions and to choose between possible alternatives. The human pre-frontal cortex, which is fundamental to goal-seeking activity, makes this relative freedom possible by virtue of the way it organizes the temporal arrangement and sequencing of our actions as motor plans. The cortical appendage, the basal ganglia, plays an important role in this sense (Edelman 1992: 105-6). It does so by taking account of relevant contextual variables and matching that information with sensori-motor activity.

216

BRAIN, MIND, AND THE SIGNIFYING BODY

Thus, selections in the interpersonal grammar are, in the ongoing temporal flow of discourse, cross-coupled to the physical-material processes whose flows they entrain and co-ordinate. The basic principle underlying this operation is the way in which the sought-after goal - the desired future - is modally evaluated in relation to the undesirable and negatively evaluated present. The linguistic resources of modalization and modulation play a key role here. In discursive activity, dialogic interacts are organized in relations of interdependency (one thing follows after another, e.g. Question-Answer, and at the same time there exists a relation of semantic interdependency between the two parts of the dialogic interact), as well as hierarchically (some goal-seeking pathways dominate others). Within this complex network of ecosocial relations (Lemke 1995b), consciousness may be seen as an integrity (Salthe 1993: 51) which interrupts the hierarchy of relations in the way I proposed in chapter 2, section 1, pp. 59-67. The logic of scalar hierarchies tells us that one level- the ecosocial one - is the one we engage with most naturally. It is in relation to this level in the overall hierarchy that actions, objects, and so on, are interconnected with other events, objects, and so on. Further, these interconnections constitute an entire system of relations potentially extending over many different spacetime scales. They can function both independently of our body-brains as well as directly in relation to them. Higher-order thinking requires the distinction between self and nonself. Lukacs (1980 [1978]: 26) has described how the separation of subject and object, which emerges in the labour process, entails the 'channelling' of the resulting reflection of reality into consciousness. Thus, 'being' and its 'reflection in consciousness' constitute a break with the prior, unitary form of being. The resulting antithesis leads to the emergence of a qualitatively new level of being, viz. social being. Similarly, Cole (1996: 118-21) discusses the notion of artefactmediated action as first developed by Russian psychologists such as Luria (1976) and Vygotsky (1986 [1934], 1987 [1934]) in the cultural-historical tradition. Cole draws a distinction between 'natural' or unmediated functions and 'cultural' or mediated ones as two ways in which the subject and object (environment) are related to each other. In my view, there is no such thing as an unmediated relationship between subject and object. Rather, the question is concerned, above all, with the different kinds of mediation afforded by the integration hierarchy of iconic, indexical, and symbolic modes of semiosis. This shows us that the purely iconic mapping of topological-continuous variation onto topological-continuous variation of the kind found in perceptual-motor category formation prior to the emergence of indexicality remains linked to the immediate here-now scale. On the other hand, indexical and symbolic forms of mediation progressively free the agent's consciousness from the here-now scale. The so-called 'direct', 'natural' relation is best seen as the most primordial form of iconic relationship between self and nonself in which no meta-level experiencing of experience is apparent. The linguistic resources of modalization and modulation function to ground dialogic interacts in relation to the here-now of the speech event (Finite) and, therefore, in relation to the ecosocial space-time scale of human social meaningmaking. Modal systems such as 'probability' and 'usuality' in English ground the

THE SEMIOTIC BASIS OF CONSCIOUSNESS

217

number of variables that can be contextually managed in order that the number of variables does not run away from us. The point is that modality deals with the contingent and the unknown by allowing participants in discourse to orient to and evaluate it as if it were predictable, known, or possible, and so on. The unknown, the uncertain, the doubtful, and so on, constitute sources of friction. Interpersonal exploration and negotiation is brought into play so as to determine the most appropriate level of analysis for the evaluation and resolution of both semiotic and material friction (Thibault 1995). Moreover, the nesting of discourse moves within dialogic interacts, and the nesting of these in turn with still-larger-scale interactional episodes, means that specific contingencies are not simultaneously evaluated at all levels of the whole hierarchy of relations. The interpersonal grammar oflanguage is organized in such a way that interactants can orient to a goal on a specific level and act as if only that level were contingently relevant. In this way, the values of things in the world can be selectively attended and discursively fixed so that action is possible.

15. Consciousness and Semiotic Stratification I would now like to consider consciousness in relation to the stratified nature of semiotic systems (chapter 1, section 9). Stratification is the means whereby the agent is cross-coupled with and integrated with the ecosocial system (chapter 1, section 9, pp. 39-46; chapter 2, section 1, pp. 59-67). This is so because the expression and content strata interface with and integrate the agent's Innenwelt and Umwelt, respectively. This means that the agent has internal, semiotically mediated, and progressively elaborated and individuated models in his or her central nervous system for acting in the world. Moreover, individuals can intervene in and expand their Umwelten through the externalization of these models in the form of social meaning-making activity. In this way, the social semiotic system is integrated with the small-scale dynamical processes of the agent's body at the interface of the body with the expression stratum. The agent is also integrated with the dynamics of spatial and temporal scales that are greater than those of the agent's own bodily processes. The integration with these much larger scales is possible because of the interface between the agent and the content stratum. It is this second interface, as we have seen, that connects the agent to the phenomena of experience in his or her Umwelt. Higher-order consciousness is semiotic through and through. The two interfaces which are constituted by the expression and content strata are the means whereby the individual organism and its dynamics is cross-coupled with the very different dynamics of space-time scales which can extend well beyond the organism along a semiotic-material trajectory into its ecosocial environment. In this sense, language and other symbolic modes of meaning-making constitute a resource for the integration of the here-now activities of the organism with larger space-time scales. Peng accordingly distinguishes between motor functions and motor activities. The latter may appear, superficially, to 'make use of the cortico-bulbar pathways for efferent neuronal connections to innervate the musculature of the vocal apparatus (including the facial musculature), if

218

BRAIN, MIND, AND THE SIGNIFYING BODY

movements of the muscles are the main objective' (1994: 114-15). The difference between 'motor functions' and 'motor activities' lies in the fact that the former are part of language in the individual, whereas the latter are not related to language activity in the individual organism. Both motor functions and motor activities, as defined here, are performatory activities of the organism's executive systems (Gibson 1983 [1966]: 46). In the case of speech production, the motor functions which are involved in the articulation of speech sounds during phonation are semantically integrated with other facets of language in the brain of the speaker in order to produce a linguistic event. Motor activities are not semantically integrated in this way. Thus, the motor functions of phonation are integrated with informational invariants concerning (1) the sound shape of the word to be uttered, (2) their correct sequencing on the expression stratum; and (3) the lexicogrammatical form and semantics of the word on the content stratum (chapter 3, section 11). Peng points out that the motor functions consist of the mapping of these various components onto each other. To quote Peng: ... when the speaker wants to produce the sequence of sounds, Ip/, lce/, and

I d/, the speaker must have the meaning of the word padin the cerebral cortex first, for which the sequence of sounds is intended, before it is produced; then, the sequence of sounds must have the brainstem function, that is, going through the cortico-bulbar pathways, coupled with the extrapyramidal function and the cerebellar function, when it is being produced. In contrast, however, when the lips are to be closed for the purpose of mastication, in order to prevent food from coming out of the mouth, the closing of the lips is simply a part of the motor activities of the nervous system, distinct in nature from what I have called motor functions which are a part of the specific brain functions pertaining to language. (Peng 1994: 114) The use of the cortico-bulbar pathways in the production of human speech sounds entails a functional loop which connects the cerebrum or neo-cortex to the vocal apparatus through the cortico-bulbar pathways (Peng 1994: 117). It is this loop of functional connections leading from the neo-cortex through the cortico-bulbar pathways to the vocal apparatus which enables the speaker to make the very fine segmentations and distinctions necessary for the articulation of speech sounds. In this way, the speaker produces the invariants of stimulus information which listeners can discriminate from the topological-continuous variety of the acoustic signal. Therefore, the listener is able to tune into these same invariants even when the speaker, the time and place, and so on, of speaking and listening, are different from one occasion to the next (see Gibson 1983 [1966]: 278; this volume, chapter 3, section 2, pp. 148-53). The motor functions, as theorized by Peng, thus integrate the expression and content strata of language within the central and peripheral nervous systems of the individual. By the same token, the individual organism is semiotically integrated with space-time scales which extend beyond the material body. Stratified semiotic systems such as language and depiction face two ways, i.e. both 'inwards' to the body of the articulator of the sign and 'outwards' to the

THE SEMIOTIC BASIS OF CONSCIOUSNESS

219

phenomena of experience in the ecosocial environment. It is from the latter that the articulator picks up stimulus information by means of his or her perceptual systems (chapter 1, section 3, pp. 11-18). In fact, both the body and the world are environments with which the expression and content strata, respectively, interface in the processes of transducing input and output between the body and the ecosocial environment with which the body-brain interacts. These inputs and outputs can be classified according to the schema presented in Figure 4.2. Figure 4.2: Classification of input and output impulses involved in the symbolic transduction of stimulus information and sensori-body activity in semiosis (adapted and modified from Gibson 1983 [1966]: 46) Exteroceptive (stimulus information actively obtained from environment by perceptual systems as proto-meaning) Inputs: content (afferent impulses) Proprioceptive (stimulus information obtained from own body, e.g. inner sensations, feelings, inner visual and other imagery)

Attending to (listening, looking, etc.) meaning-making acts of others by means of active perceptual exploration Outputs: expression (efferent impulses)

Performatory activity of articulators (vocaltract, manual-brachial gestures, arm-handjoint-eye-tool kinaesthesis in tracing, graphic act) in the externalized execution of signs

The inputs and outputs which cross the boundary between organism and external environment are the means whereby matter, energy, and information are exchanged between the two. These inputs and outputs contain different orders of information. These orders can be classified according to my reinterpretation of Deacon's hierarchy of reference in terms of the presupposition/implication hierarchy discussed in chapter 3, section 7, pp. 126-34. This is so in relation to both the expression and content strata. Moreover, the information-meaning which is afforded by these two strata is both exteroceptive and egoreceptive. In providing information to the other (exteroception), the speaker, say, simultaneously provides information to him- or herself about his or her own meaning-making activity (egoreception). This means, for example, that speakers hear their own voices while speaking and at the same time, feel the muscular movements of their face and vocal tract. Gibson (1986 [1979]: 115) points out that all perceptual systems are both exteroceptive and egoreceptive:

220

BRAIN, MIND, AND THE SIGNIFYING BODY

the body of the perceiver is also a source of information about the perceiver's body during the perception of some environmental event. Both environment and self are simultaneously specified by information from the two sources in the act of perception (Gibson 1986 [1979]: 116). The complementarity of these two perspectives is based on the fact that the information for specifying the two is the same. Similarly, the body of the speaker in the process of articulation is both a source of information about the speaker to observers (e.g. the addressee) in the external environment and simultaneously also a source of information to the speaker about his or her own body. As we shall see in chapter 5, section 4, pp. 246--8, the reentrant mapping of the information from these two sources - self and nonself - in the act of perception creates higher-order maps in the brain of these lower-order self/nonself mappings. The resulting higher-order or global mappings constitute a basis for the internalization and consequent reorganization of this perceptual information in ways which are relevant to the structure of (self)consciousness. The local outputs of multiple maps that are connected to the exteroceptive and proprioceptive sources of multimodal stimulus information are reentrantly connected to form a global mapping. This global mapping forms a dynamic loop that correlates the information about the organism (proprioception) to the information about external events (exteroception). In this way, the selection of neuronal groups in particular local mappings results in the emergent distinction between an 'in here' perspective on what is 'out there'. Edelman points out that 'sensorimotor activity over the whole mapping selects neuronal groups that give the appropriate output or behaviour, resulting in categorization' (1992: 90). Appropriateness of output or behaviour is determined by internal criteria of value that constrain the ways in which the organism interacts with its environment. In this way, categorization emerges on the basis of exploratory activity that matches, in biologically appropriate ways, the physiological life-supporting systems of the organism (Edelman 1992: 90-1). The distinction between 'self and 'nonself' is not, as we shall see in more detail in chapter 5, a given; instead, it emerges as a consequence of the ways in which the individual's sensori-motor engagements with the nonself, including most crucially other conspecifics, select neuronal groups that eventually result in the categorical distinction between self and nonself. The fundamental importance of the orientation to the other in the infant's very early proto-semiosis suggests, once again, that this proto-interpersonal orientation is founded on internal criteria of value which define these engagements with the other - most critically the mother or other primary caregiver - as an appropriate domain of activity. That is, as an activity that is essential for the infant's further development. Thus, the infant's internal value criteria constitute the biological initiating conditions on level L-l of a semiotic trajectory on focal level L which has its beginnings in the infant's performing of behavioural routines directed at fulfilling its most basic needs as a newborn - nourishment, warmth, affection, contact with other conspecifics as sources of information that is needed for the further development of its ontogenetic trajectory. The perceiver cannot avoid perceiving, and, therefore, being aware of him- or herself in the act of perceiving something. This means that the perception of

THE SEMIOTIC BASIS OF CONSCIOUSNESS

221

some environmental event (exteroception) and the perception of our own act of perceiving (proprioception) are simultaneously implicated in perception. Moreover, our conscious attention can be focused on one or the other of the two poles of what is now seen as a single perceptual structure, albeit with different possible poles of attention (chapter 5, section 4). In other words, awareness of self and nonself are both implicated in the structure of the perceptual act. This suggests that consciousness, here seen as a meaning structure at all levels of the implication hierarchy of icon, index, and symbol, is based upon the reentrant mapping in the brain of this perceptual distinction between information pertaining to the self (proprioception) and information pertaining to the nonself (exteroception). This means that (1) we are conscious of some phenomenon of experience which we are perceiving, performing, imagining, thinking about, or remembering and (2) we are conscious of that experience as ours, i.e., as being indexically referable to our own self, as designated, for example, by the English pronoun T. That is, we have an experience of something and an experience of our self as the experiencer of that experience. Consciousness is the result of the semiotic mediation and re-structuring on higher levels of organization of this relation. As stated earlier, this semiotic mediation can take place at all levels of the implication-cum-presupposition hierarchy of icon, index, and symbol. Perceptual-motor, conceptual, and symbolic forms of categorization in the brain (Edelman 1992) therefore constitute an implication hierarchy of different semiotic levels. The relational structure of consciousness, as described above, can be integrated to these levels of increasing semiotic specification and consequently reorganized. The self-organizing character of these processes means that the brain does not 'represent' experience in terms of pre-existing, stored categories, images, symbolic tokens, and so on. Meaning at all integrative levels emerges in and through the interaction between the categories in the brain and the phenomena of experience, as mediated by a higher-order system of interpretance. I shall now discuss this in more detail, starting with the expression stratum. First, speech sounds are iconically grounded in the physiological processes of the speaker's body (chapter 3, section 4, pp. 118-20). In this way, speech sounds afford information which specifies the embodied states and feelings of the speaker. In this perspective, the perceived sounds are necessarily tied to or grounded in and provide information about the speaker's body, its feeling states, and so on. Secondly, there is information in speech sounds which indexes the spatial location of the speaker in the environment relative to the listener (Biihler 1990 [1934]: 125), the personal identity of the speaker, and so on. Thirdly, speech sounds contain information which enables the listener to perceive the phonological categories, units, and structures which belong to the sound - i.e. phonological - system of a given language such as English. This kind of information constitutes the symbolic dimension of the expression stratum of spoken language. Whereas the iconic and indexical information in speech sounds is, to varying degrees, causally tied to its environmental source (the speaker) as a perceptual stimulus which it specifies, the symbolic order of speech sounds specifies, on the other hand, the phonological units and structures of a given phonological system. Two important implications derive from this. First, the

222

BRAIN, MIND, AND THE SIGNIFYING BODY

phonemes, syllables, and so on, in a given utterance are instantiations of the phonological categories of a given language system. They are not, therefore, tied to specific speakers' bodily states or identities, but are potentially shareable by all speakers of the given language. Secondly, this order of information in speech sounds symbolically construes or realizes a more abstract level of semiotic organization, viz. the lexicogrammatical units and structures on the content stratum (see also chapter 3, section 9, pp. 139-46, on the brain as somatic recognition system). In Gibson's terms, we might say that this third order of information symbolically specifies the more abstract content stratum of some language system. I shall now turn my attention to the content stratum, again with a view to analytically refracting this through the presupposition-cum-implication hierarchy of icon, index, and symbol that I proposed in chapter 3, section 4, pp. 118-20. First, the ambient stimulus information is iconically grounded in some environmental event which is the cause of the variants and invariants in the stimulus information picked up by the observer's perceptual systems. This information directly specifies that event for the observer (Gibson 1986 [1979]). Secondly, stimulus information in the environment can specify events which are not necessarily present in the here-now of the stimulus information which is picked up by the observer. Such events can be reconstructed or inferred on the basis of the available stimulus information, though not necessarily present to the observer. In this case, the stimulus information indexes some environmental value which may be spatially or temporally displaced with respect to the information picked up by the observer. Thirdly, environmental events which are specified by ambient stimulus information, as well as those which have been extracted from the ambient flux of environmental information, can be symbolically construed in and through the semantic categories of a given language system. In this way, we respond to and orient to the world not directly - Le. perceptually - but indirectly, through the lexicogrammatical and semantic resources of a shared system of social meaning-making. Now, it is important to point out that on both sides of this relationship - viz. expression and content - all three levels of the implication hierarchy from icon, index, to symbol are organized on the basis of degrees and kinds of difference. It would be wrong to equate the iconic level with the 'sensations' which empiricist theories of perception and consciousness consider to be the basis of consciousness. Sensations, in this empiricist view, are devoid of categorial or conceptual value. However, the iconic mode is still founded on topologicalcontinuous (analogue) differentiation (topological-continuous variation) as a resource for discriminating degrees of difference rather than type of difference (Wilden 1981: 20). On the other hand, the symbolic dimension of language is founded on typological-categorial difference, i.e. discrete digital categories of different types. This applies equally to both expression and content. This is very different from the doctrine that consciousness consists of 'thoughts' and 'sensations'. According to this doctrine, 'sensations' are non-conceptual and are merely 'felt' as subjective experience on account of their impact on our sensory systems. It is further assumed by this doctrine that organisms perceive phenomena on the basis of the 'qualities of their nerves and neurons, not with the properties of the external world', as Gibson (1983 [1966]: 56) has eloquently

THE SEMIOTIC BASIS OF CONSCIOUSNESS

223

put it. In this view, pure sensations are, in turn, transmitted to the brain by the nerves and turned into thoughts and ideas. Only the latter are considered to have conceptual value. Alternatively, the semiotic model of stratification, as defined here, suggests that value-laden differences - topological and typological - perfuse all levels of consciousness right down to the iconic level of perceptual awareness. This is equally true for both our experience of phenomena and our experience of our own bodies. This observation also ties in with the intentional character of consciousness, along with the fact that the objects of consciousness are experienced as being instances of this or that category of phenomenal experience. Consciousness at all levels is embedded in higher-scalar systems of interpretance. For this reason, consciousness is not uniquely generated 'from below' by physical states of the brain. The presupposition-cum-integration hierarchy of icon, index, and symbol suggests the need for an internalist perspective on the question of consciousness. In this perspective, we are inextricably inside the system that we are observing. There is no objective viewpoint whereby we can observe and analyse the object system from the outside. Nevertheless, theories of consciousness have tended to relate to the phenomenon of consciousness adaptively, i.e. as something to predict or explain. For these theorists, organization and pattern in the neuroanatomical structures and processes of the brain is the thing to be explained. These theories continue to view consciousness as if it were a system we can observe from the outside. They fail to see that the attempt to find and explain the seat of consciousness as something in the brain is to look in the wrong place. Both the intrinsic awareness of one's own acts of consciousness and the attribution of consciousness to others is a question of a joint, dialogic relation between observer and object system. The emergent order or form of consciousness necessarily requires us to abandon naturalism and eliminative materialism in favour of a perspective based on meaning. Consciousness has been isolated from its contexts on the assumption that those contexts do not impinge on the nature of consciousness itself in significant ways. Moreover, the' source of order and coherence has been located solely in the brain; it is as if other aspects of the world do not have their own order, and in ways which affect the nature of consciousness in substantial ways. If, on the other hand, the content of consciousness is defined on the basis of the meaning which is stored in the system, where meaning here refers to the internal form and structure of consciousness, it is clear that it is the system's internalized encounters with signs and the attempt to discover their systems of interpretance, which specifies the structure of consciousness. The problem for the observer, who inhabits the same ecosocial scale as the individual observed, is one of having to discover and interpret the signs which tell him or her something about the content of the other's consciousness. As we shall see in greater detail in later chapters, consciousness can be theorized as a highly specified meaning system in the perspective of the self on all integrative levels of the semiotic hierarchy of iconic, indexical, and symbolic integrative levels.

224

BRAIN, MIND, AND THE SIGNIFYING BODY

16. An Alternative Reading of Descartes in the Internalist Perspective of Interpretive Activity The observations I made in the previous paragraph suggest an alternative reading of Descartes's famous observations concerning the passers-by whom he observed from his window in the street below. Here is the passage in question from Descartes's SecondMeditation: I am taken aback, nevertheless, when I reflect upon the weakness of my mind, and consider how prone it is to error. For, however I may ponder over all this in the silence of my thoughts, words themselves hold me up, and I am almost deceived by the terms of everyday speech. For we say that we see the same piece of wax if it is presented to us, and not that we judge it to be the same because it has the same colour and the same shape. And from this I would almost conclude that I know the wax by the sight of my eyes, and not by the scrutiny of my mind. I chance to look out of the window upon the passers-by in the street below, I have no hesitation in saying that I see men, although their hats and cloaks may be no more than a disguise for clockwork figures. Yet I judge them to be truly men; and so, what I thought I saw with my eyes, I really know through the sole power of judgement which resides in my mind. (Descartes 1960 [1641]: 114-15) Generally speaking, Descartes's observations have been understood in the light of the externalist perspective (see Shanker 1998). This perspective assumes that the observer (Descartes) and observed (the persons in the square) do not exist within the same system of interpretance. It is assumed that the two perspectives are not commensurate. In the internalist perspective, on the other hand, we can see how Descartes attempts to interpret the people he observes in the square through his own interpretative activities. Moreover, it is possible in this perspective to see that Descartes is enmeshed in a universe of signs. It is these signs alone which provide the interpretative frameworks of his (and our) observational perspectives and viewpoints. In the externalist perspective, the referents of the signs are already given. In the internalist perspective, on the other hand, the world is construed through the systems of interpretance which mediate our encounters with the world and allow us to endow it with meaning. The first thing to note about Descartes's discussion is the focus on meaningful pattern and order in what he observes. Thus, order can be observed in the form of men crossing the square, rather than just automatons wearing hats and cloaks. Descartes attributes this perception to a faculty of judgement whereas I would prefer to say that it is a case of the operation of what Salthe calls 'the constraint of informational closure' (1993: 133). This constraint depends on the observer's choice of categories and the extent of their specification. Salthe draws on Dretske's definition of information as 'the most highly specified digital content an analog percept can have' (1993: 134). The scene which Descartes attends to is an unbounded visual percept. For this reason, it is an analogue percept. As discussed in section 6 above, the experiential grammar and semantics of language are one typological-categorial (digital) resource which affords the

THE SEMIOTIC BASIS OF CONSCIOUSNESS

225

observer a high degree of symbolic specification of the stimulus information he or she picks up in the ambient energy flux. Given that Descartes clearly possesses the digital category 'man', the men he sees crossing the town square are perceived as men, rather than as less specified hats and cloaks which conceal automatons. The point is that the experiential semantic categories that are used selectively by Descartes contextualize the given percept as a grounded instantiation of the given category. The fact that Descartes invokes this latter categorization demonstrates that very many different possibilities, to varying degrees of specification, may be implicated in the interaction between observer and object. This is, of course, the logic of the presupposition-cum-implication hierarchy of icon, index, and symbol. The most highly specified symbolic categories always implicate less specified categorizations on many different levels of specification in the overall hierarchy of relations. This applies equally to the relations among categories within, say, the symbolic mode, as well as to the relations among the symbolic, the indexical, and the iconic modes. Thus, the category 'hat and cloak wearing automatons' could function as the type-categorization of the observer's perception. However, this category is overridden by the more specified category 'men'. The fact of this overriding suggests that the former categorization is a lower-level categorization which relates more closely to the observer's perceptual categories, rather than to the symbolic categories of language. Descartes's observations have been assimilated to, and have been seen as instigating, the modern cognitive science paradigm. According to this perspective, the mind is a field of representations which are delivered to consciousness by perception. Moreover, the radicalism of Descartes's view is upheld to lie in the notion of perception as judgement, rather than the merely passive receiving of sensations, as in the empiricist tradition (Seager 1999: 5). Nevertheless, it is intriguing to consider the fact that Descartes's linguistic construals of both his own activity as perceiver as well as the phenomena he perceives - the men in the square - do not refer to the lower-scalar level of the ambient energy flux which subtends stimulus information, which we do not perceive in any case, or to the activities of the central and peripheral nervous systems. Both ambient energy and the activities of the central and peripheral nervous systems exist on a lowerscalar level with respect to either the 'I', corresponding to Descartes as observer, or to the phenomena of experience that are perceived by the 'I'. Rather, the lexicogrammatical choices in Descartes's discourse construe phenomena on the scalar level of the world which Descartes, along with the writer and readers of the present text, share. This is the scalar level of the town square, the men who cross it, and the human observers who would look at these men from the windows of their apartments. Descartes's own language locates both himself and the phenomena observed on this same scalar level. The experiential semantic categories in and through which Descartes construes this observer-object relation means that the analogue percept which Descartes sees is re-construed as a series of typological-categorial distinctions by the lexicogrammar of the clauses in Descartes's text. This corresponds to the symbolic mode in the presuppositioncum-implication hierarchy of icon, index, and symbol. These symbolic categories are the most highly specified categories with respect to the topologicalcontinuous (analogue) character of the visual percept, which is unbounded. The

226

BRAIN, MIND, AND THE SIGNIFYING BODY

latter is less specified, and corresponds to the iconic level. However, this does not mean that the less-specified iconic level has been replaced by the symbolic one. What it means is that the symbolic level has integrated and subsumed - i.e. 'contextually completed' - the iconic (and indexical) levels. This has the further implication that, in contextually integrating to its level the iconic and indexical levels, the symbolic level contains or implies additional levels of meaning on lower integrative levels which can be accessed by further semiotic (interpretative) labour. Such interpretative labour entails the possibility of interpreting the non-symbolic ground of others' conscious experience from our own perspective. (See the discussion of gaze in section 9 above, pp. 201-2.)

17. The Entropic Character of Meaning It is once again important to point out that meaning, however defined, is not inherent in each of these levels. Instead, it is derived by interpreters who have access to the relevant system of interpretance. In Descartes's case, his interpretation of what he observes from his window is based on linguistic categories available in his ecosocial environment. In Halliday's discussion, as cited in section 10 above, pp. 202-5, the child's categories - e.g. the high-pitched squeak - are considerably less specified, thereby limiting the child's reaction to the noise of the pigeons to the iconic and indexical levels. The mother's response provides the more highly specified symbolic category which the child thus far lacks. Halliday's (1975; 1978b; 1993) research on children's 'learning how to mean' shows that language development in the child amounts to a constitutive, and not merely contingent, increase in stored meaning along the individual's ontogenetic trajectory (see Salthe 1993: 135). This is very different from Chomsky's characterization of the 'underdetermined' and 'fragmented' character of the inputs which the child receives from the outside (1976: 10). The alternative to the Chomsky-type view rests on the view that the informational, and hence meaning-bearing, capacity of a system is entropic. This has been formulated by Salthe as follows: If information is to be taken as entropic, it must follow the equivalents of the First and Second Law of Thermodynamics. That is, in a system with informational closure, (1) what can be converted into information must be found to be conserved, and (2) information itself must increase or remain constant. (3) If macroscopic configurations are to be considered information, we must identify a system in which they so function. These are not insurmountable problems. We can dispense with the first of these three criteria by noting that energy is what is converted into information, either in the formation of a configuration or in its discovery and classification by another system. We need merely show that an object system and an observer exist in the same physical system to satisfy the First Law as well as it is ever satisfied in thermodynamics. To show that macroscopic information in an internalist situation follows the second law, I have proposed the following logical argument (Salthe 1990b): It is sufficient to show that observer and object system(s), Si, are

THE SEMIOTIC BASIS OF CONSCIOUSNESS

227

together embedded in the same complex supersystem, for the following reasons: If we were to decompose total system information (macroscopic information), H, into the potential information capacity of the supersystem, C ma" and that stored in its contained subsystem(s), H, then the information in the global system would be

H must continue to increase, or at least cannot decrease, when we explicitly consider the information, Ho, stored in an observer, 0, as well. Then

H

= Cmax + H', + Ho

describes the total system, in which Ho could also be observed as contributing to H by another observer in the system coordinate with 0, if there were one. (Salthe 1993: 118) Following these important formulations of Salthe in favour of the view that information, and, hence, meaning, behave entropically, it is now possible to say that (1) the meaning which is stored along the individual agent's historicalbiographical trajectory increases as the trajectory develops; (2) meaning is produced on the basis of energy exchanges whereby energy is dissipated along a trajectory not only as heat energy, for example, but also as information which is potentially meaningful, relative to the observer's categories; (3) the trajectory of consciousness emerges, is constructed, and is maintained on the basis of the individual's exchanges at all integrative levels in the semiotic specification hierarchy of icon, index, and symbol with its ecosocial environment. The meanings that are stored in the individual's Innenwelt enable meta-redundancy relations to be constructed between these and selected aspects of the environment. The individual is, therefore, able to selectively attend to the phenomena of experience in his or her inner and outer environments. The observer, therefore, construes the phenomena of experience according to the degree of specification of the signs - iconic, indexical, symbolic - to which he or she has access. Moreover, the notion of meta-redundancy shows that consciousness always entails a patterned relationship of contextualization between internal semiotic configurations which are stored and elaborated in the individual agent and selected environmental features.

18. Consciousness as the Contextualization of Experience in the Perspective of the Self Insofar as language and other semiotic modalities constitute an interface between the central and peripheral nervous systems of the individual organism and the ecosocial environment, such systems are a means whereby 'inside' and 'outside' can 'communicate' with each other. They are the means whereby the external world is assimilated to the central nervous system of the organism so as

228

BRAIN, MIND, AND THE SIGNIFYING BODY

to become an integral part of the organism and its trajectory. It is in this way that higher-scalar environmental constraints impose limits on the degrees of freedom - the many degrees of difference - of lower levels. The constraints emanating from the higher level re-set the probabilities of the dynamical processes on lower levels so as to direct and constrain their behaviour. This re-setting of probabilities applies to all forms of categorization in the organism, viz. perceptual, conceptual, and symbolic categorization. However, it is not a question of the categories which the organism has in its central nervous system 'representing' or 'referring to' an already given external world. Instead, it is a question of selectively attending to and orienting to 'objects' through processes of differentiation. Gibson (1983 [1966]: 270) points out that these processes are learned. The organism learns what to attend to. In the case of the perception of objects, 'it is the detection of distinctive features and the detection of invariants under changing stimulation' (Gibson 1983 [1966]: 270-1). Gibson continues as follows: The exploratory perceptual systems typically produce transformations so that the invariants can be isolated . . . And the action of the .nervous system is conceived as a resonating to the stimulus information, not a storing of images or a connecting up of nerve cells. (Gibson 1983 [1966]: 271) Rather than the brain producing representations of the outside world, the categories in the central nervous system resonate with environmental stimulus information. This is, I argue, equally true of perceptual and conceptual categorizations, which are elaborated biologically (Edelman 1992: 108-10), and the symbolic categories of language and other semiotic systems. The symbolic categories of a given language system are a resource emanating from the higherscalar ecosocial system. Symbolic categories allow individual consciousness to resonate with phenomena of experience which are not necessarily present in the organism's perceptual purview. In this way, the experiential categories which are realized in the lexicogrammar of the clause enable the construal of symbolic objects which individuals can selectively attend to and orient to. The experiential resources oflanguage are not used to 'represent' or to 'refer to' something which is already given, and which these categories then slice up in pre-established ways. Rather, the phenomena of experience - objects of consciousness - are construed in and through the system of categories - the value-producing differences - which are internal to a given semiotic system. The resonance between organism and environmental information of which Gibson speaks is the process whereby phenomena of experience are made to emerge from protosemiotic vagueness. Organisms selectively contextualize the relationship between themselves and the environment by means of the symbolic and other categories which are stored in the central nervous system in and through the epigenetic exchange of information between organism and environment. Individuals can therefore converge on the phenomena of experience and selectively incorporate them into their Innenwelten. In the Western folk-theoretical tradition, consciousness tends to be accorded the status of an internal, subjective reality which can be 'reported' or 'repre-

THE SEMIOTIC BASIS OF CONSCIOUSNESS

229

sented' by linguistic tokens. In this respect, it is accorded the status of an 'object' which can be referred to and predicated ofjust like any other entity in the world. This is a realist view, which is based on the assumption that there exists a 'really real' world either 'out there' or 'in here', independently of the ways in which these entities are contextualized by the systems of meaning of a given ecosocial semiotic system. However, consciousness, no less than any other entity, is an entity in the supersystem transactions of some ecosocial system. It emerges and is contextualized as a phenomenon of experience in and through the systems of contextualizing relations of the ecosocial system. The meanings it has for the self (or for others) are meanings which are construable for the self in and through these same systems of contextualizing relations. Yet, consciousness is usually accorded a metaphysical status as something which exists independently of the possible ways it may be construed by a given meaning system. In ecosocial semiotic theory, the 'reporting of' or 'referring to' conscious experience, or even internalized reflection on it, is always part of a larger pattern of contextualizing relations which includes the experience itself. The notion that consciousness can be linguistically 'reported' implies a referential theory of meaning in which the objects that are 'referred to' are external to the (linguistic) act of referring. In this externalist view, consciousness is a content that exists in its own right and is external to the contexts in which we contextualize it. What is experienced, so to speak, is always imminent in supersystem processes which are selectively contextualized by the meaning system. In this point of view, consciousness is, therefore, a constitutive part of those contexts, rather than being external to them. It thus may be more appropriate to say that consciousness is, from the point of view of the insiders in some social situation, indexically invoked as locatable at a given source - e.g. first-person or third-person. At the same time, it is symbolically construed in terms of the experiential categories of the lexicogrammar of language. The point is that, in this view, consciousness is indexically invoked in the given context from the point of view of the insider-participants in the situation. The linguistic practices of construing consciousness in self and other make no such distinction between what is external to speech and lurking 'behind' speech in the mind of each individual. Rather, consciousness is indexically invoked as being internal to the speech practices themselves in the sense that any internal phenomenon inside the 'head' only has meaning if it is contextualized by the categories of some meaning system. Gibson (1986 [1979]: 255) has formulated a view of non-perceptual awareness which is useful here. Gibson points out how, in the older mentalist psychology, higher mental processes were opposed to sensations, and seen as the operations of higher-order faculties in the mind. In this regard, Gibson mentions such mental operations as remembering, thinking, conceiving, inferring, judging, expecting, knowing, imagining, dreaming, and rationalizing. According to Gibson's ecological perspective, these may be explained as follows: To daydream, dream, or imagine wishfully (or fearfully) is to be aware of surfaces or events that do not exist or occur and that are outside the limits of possibility. These three kinds of nonperceptual awareness are not explained, I think,

230

BRAIN, MIND, AND THE SIGNIFYING BODY

by the traditional hypothesis of mental imagery. They are better explained by some such hypothesis as this: a perceptual system that has become sensitized to certain invariants and can extract them from the stimulus flux can also operate without the constraints of the stimulus flux. Information becomes further detached from stimulation. The adjustment loops for looking around, looking at, scanning, and focusing are then inoperative. The visual system visualizes. But this is still an activity of the system, not an appearance in the theater of consciousness. (Gibson 1986 [1979]: 255-6) The symbolic possibilities of language and depiction, to mention two semiotic systems with fully developed symbolic possibilities, mean that neither of these two systems is tied by indexical necessity to entities already present in the environment. The lexicogrammatical and semantic distinctions on the content stratum of language, for instance, do not refer to the ecosocial environment in any direct way. In learning the contextual meta-redundancies between linguistic forms and particular classes of environmental events, linguistic forms enter into contextualizing relations with invariant stimulus information which is extracted from different kinds of environmental events (visual, kinesic, auditory, haptic, and so on). Moreover, the learning of these contextual redundancies cuts across the stimulus-response dichotomy, 'for they can be vocal-motor or manual-motor' (Gibson 1986 [1979]: 281). Importantly, the learning and further elaboration of these redundancies between classes of linguistic and other semiotic forms and perceptual invariants is a matter of the progressive attunement of the system to finer and finer differentiations. The attunement to these goes hand in hand with the elaboration of a sense of consciousness along a time-bound trajectory. This view suggests that the linguistic and other semiotic resources for invoking consciousness in one's self and in others function both to indexically invoke and to symbolically construe informational invariants which have been, as Gibson puts it, detached from stimulation. In the absence of stimulation from the ambient energy flux, the activity of exploring the environment for stimulus information has then been re-directed 'inwards' to invariants and differentiations to which the system has been sensitized through experience. This last point further suggests that the contextualized uses of these linguistic forms may 'resonate' with perceptual invariants that have been de-located from the external environment and re-located in the individual's Innenwelt. In both cases, it is still a question of the situationally-specific contextual redundancies between such invariants and the semiotic forms used to index and symbolically construe them. The presupposition-cum-implication hierarchy of icon, index, and symbol is relevant here as well. Consciousness is always relative to observer categories. This is so whether the object system is external or internal phenomena of experience, or even one's own activities of consciousness (metamentation). If the categories of the nervous system become increasingly specified as it develops in ontogeny (Salthe 1993: 135), then it seems plausible to suggest that the development of the organism involves the increasing specification of signs from the initial vagueness of the topological-continuous variation characteristic of the iconic to the highly specified typological-categorial distinctions characteristic of the symbolic (see chapter 1, section 5, pp. 23-6). This does

THE SEMIOTIC BASIS OF CONSCIOUSNESS

231

not mean that the symbolic mode replaces the other modes as the organism develops. As we have seen, the symbolic implicates the indexical, which, in turn, implicates the iconic. What it does mean is that the observer has information stored in him or her such that an object system can be specified in terms of the full range of possibilities afforded by the hierarchy of iconic, indexical, and symbolic modes of semiosis. Therefore, at all levels of organism-environment information exchanges and cross-eouplings, from the perceptual and conceptual categorization of primary consciousness (Edelman 1992) to the symbolic categorization of higher-order consciousness, the body-brain complex is embedded in higher-scalar systems of interpretance of varying degrees of specification. It is only through such systems of interpretance that the organism can determine whether information which is obtained from the environment is potentially meaningful for the organism. For example, perception, rather than being the passive receiving of data from the outside, occurs on the basis of a reciprocal adaptation of both organism and stimulus information to each other. A given organism is not attuned to all of the information that is potentially available in the stimulus flux of ambient energy. Instead, it is selectively attuned and adapted to only some of the total possibilities in its environment. In fact, it would be more appropriate to refer to the reciprocal eo-adaptation of both organism and environment (self and nonself) to each other. Thus, the release of energy provides information about an environmental event relative to a potential observer. This means that stimulus information in the environment and observer are both reciprocally conditioned by the existence of a higher-scalar context which links them together.

19. The Embedding of Consciousness in a Higher-scalar System of Interpretance The existence of a higher-order context means that the relationship between two interacting entities, A and B, is not random, but occurs with a certain degree of probability. That is, the higher-scalar level C functions as a boundary condition on the transaction and the signs at the A-B dyad level. In the course of their interaction, A and B reciprocally adapt to each other. This means that they are transformed by the contact between them. Whereas ambient energy about indefinitely many other environmental events is not available to the organism, because it is not evolutionarily designed so as to be selectively attuned to these, the reciprocal nature of the contact between A and B means that there exists a relationship of complementarity between them. This necessarily occurs against a background of other sources of stimulus information which are not selected by the A-B relationship. The situation described here is a very general one, which applies to all biological phenomena. This suggests that what we call 'dialogue' in the specifically human case is a more highly specified subclass of the kind of reciprocal adaptation of A and B that is characteristic of all biological organisms. This general principle can be extended all the way right back to, for example, the trophic orientation of amoebas to light. We do not normally think of amoebas as engaging in 'dialogue'. The 'dialogue' between amoeba and light

232

BRAIN, MIND, AND THE SIGNIFYING BODY

source would be, following Salthe (1993: 18), the vaguest and least highly specified relationship of reciprocal adaptation between some A and B. The specifically human dialogue would be, at least from our human perspective, the most highly specified case - the one for which we normally reserve the term 'dialogue'. The point is, as we saw earlier (chapter 1, section 8, pp. 34-9; chapter 3, section 4, pp. 118-20), that the higher, more specified levels integrate the lower, less specified ways. The latter do not transcend the former, though the coming into existence of new higher-scalar boundary conditions will re-interpret the lower levels in new ways. In the human case, the arguments presented above suggest that both primary consciousness and higher-order consciousness are embedded in systems of interpretance of varying degrees of specification. A conscious observer is always conscious of something, whatever this something might be. However, the observer's consciousness of something can only occur on the basis of stimulus information, whether perceptual or non-perceptual (see above), which constitutes potential information about some 'inner' or 'outer' phenomenon. This means that such stimulus information, relative to the observer and his or her categories, is a potential sign of some phenomenon of experience. It is important to remember here that observer and stimulus information - i.e. potential signs - are reciprocally adapted to each other, for what the information means depends on the observer and the categories stored in his or her central nervous system. The categories stored in the observer themselves derive from a higher-scalar system of interpretance in which both observer and what the observer is conscious of are necessarily embedded. This shows very clearly that the observer is, in part, a result of the signs he or she can orient to and adapt to. By the same token, consciousness of something is not reducible to unmediated contact between the 'mind' and the actualities of the world. In primary consciousness, here-now samplings of the environment and their categorization entail the adaptation of the organism to the iconic and indexical signs which it encounters and interprets. These signs mediate between the organism's activities in the world and those here-now aspects of it that the organism interacts with. This is a less highly specified form of semiosis with respect to the more highly specified symbolic forms of higher-order consciousness. Nevertheless, it remains a form of reciprocal adaptation which is necessarily constrained by higher-order boundary conditions which act as a system of interpretance of these iconic and indexical signs. In the human case, the contextual integration and further reorganization of these levels by the symbolic level of higher-order consciousness vastly expands those aspects of the world with which the organism can interact. This depends on a system of symbolic possibilities which can connect and integrate the individual body-brain complex with space-time scales beyond the here-now one of primary consciousness. Current theories of consciousness tend to emphasize the individual and subjective character of conscious experience. This emphasis downplays or leaves out of the picture the way in which consciousness entails observer perspectives and viewpoints. Moreover, such perspectives and viewpoints are shareable with others who belong to the same system of relations. It cannot simply be a question of a physical system - the brain - representing the outside world. The observer

THE SEMIOTIC BASIS OF CONSCIOUSNESS

233

interacts with the phenomena of experience in and through a given system of interpretance. Systems of interpretance do not only entail a system of categories for construing experience and interacting with it; they also entail systems of social meaning-making practices which determine how these categories are deployed in contextually relevant ways. Theorists of consciousness have debated the question of the observer's intrinsic awareness of his or her conscious acts of experience (see Natsoulas 1998, 2000). In my view, this question closely relates to the interaction between observer and object observed in and through a system of interpretance. A number of levels can thus be distinguished: 1. the environmental stimulus information (the object system) relative to potential points of observation; 2. the object system in interaction with the observer; 3. the observer who observes the interaction between object and observer.

The above schematization once again highlights the fundamental importance of the stratified nature of semiosis and its role in higher-order consciousness. I pointed out in chapter 1 that language is a stratified system relative to an agent who is cross-coupled to the system via the two interfaces of expression and content. The agent constitutes a break or an interruption in the potentially infinite recursivity of the levels of relations involved. It is this robustness which allows agents to construe models of the world, the phenomena of experience, at the same time that such models are cross-eoupled to the bodily dynamics of the observer-agent (chapter 1, sections 1, 2, pp. 3-11; chapter 2, section 1, pp. 59-67). By the same token, semiotic stratification also entails the capacity of the system to be turned back on itself: there is also me ta-semiotic reflection on the system as an object of observation. Given my earlier argument that this always entails the cross-coupling of an observing agent to the system of observation (the system of interpretance), it follows that what is observed in meta-semiosis is not language per se, but the interaction between language and the observer-agent who uses it. This corresponds to level 3 above. The stratified nature of semiosis thus constitutes the permeable interface between body-brain and ecosocial environment. The bodily dynamics of the agent are both kept separate from, at the same time as they enter into communication with, the phenomena of experience through the processes of symbolic transduction discussed earlier. The agents who inhabit a given ecosocial system, the phenomena of experience, and higher-scalar systems of interpretance, are incorporated into a single system of relations on account of the stratification of semiosis into a number of different levels of organization. It is this fact which enables the intersection of radically different scalar levels in symbolic or higherorder consciousness. A stratified semiotic system such as language, gesture, or depiction faces two ways. As we have seen, semiotic systems interface with both the body of the agent and the phenomena of experience along the expression and content strata, respectively. Take something which is, canonically speaking, seen as cerebral, solitary, and disembodied such as reading a book. The categories of written expression (graphemes, ete.) and of spoken expression (phonemes, ete.) emerge as figures

234

BRAIN, MIND, AND THE SIGNIFYING BODY

against a background of topological-continuous variation and vagueness. For example, graphemes and the larger graphological units, such as orthographic words, orthographic sentences, ete. which these combine to form, emerge in relation to the ground of the treated surface onto which someone has traced these visual invariants. The process of visual scanning of these graphic tracings on a surface employs the smallest level of the visual system, that of the foveal field of view of the two eyes. Thus, the hemispherical field of the stationary or near stationary head during the activity of the sampling of the arrested array of visual structures on the page of a book is explored by saccadic movements of the eyes (Gibson 1983 [1966]: 258-9). In this way, the body of the observer is integrated with the specific principles of organization of the expression stratum of the written language by means of his or her active perceptual exploration of the visual invariants which are picked up. However, these visual invariants are themselves internal to a still-larger system of relations on other scalar levels of organization in a still more global system of relations. Furthermore, the visual invariants which are picked up resonate in the reader's brain with lexicogrammatical and semantic categories on the content stratum with which they contextually redound. That is, the specific organizational principles in operation on the expression stratum are symbolically reconstrued in relation to another set of organizational principles on the content stratum (see the discussion in chapter 3, section 9, pp. 139-46, on the brain as somatic recognition system). It is this which permits symbolic transduction between the organism and his or her outside. Meaning is not, then, something which is isolable in just one part of the system. Rather, it is a result of the relationship between the agent-observer and the overall system of relations on all levels. There is no problem as to how a sequence of graphemes can be transformed into a larger unit such as a word, sentence, and so on. As the eyes scan the tracings on the page, they do not make discrete samples of each element (grapheme, ete.) in the series. Instead, the entire sequence as a whole constitutes a structured visual array which is unaltered by the reader's moment-bymoment scanning of it. The central nervous system detects this structure as a whole as it unfolds in time during the scanning process. It is not a question, as Gibson points out, of comparing successive visual sensations with graphic or other images stored in memory: The invariance of perception with varying samples of overlapping stimulation may be accounted for by invariant information and by an attunement of the whole retino-neuro-muscular system to invariant information. The development of this attunement, or the education of attention, depends on past experience but not on the storage of past experiences. (Gibson 1983 [1966]: 262) The point is that the reader's active scanning in order to obtain visual information produces a series of transformations and transitions. Furthermore, the proprio-sensitive nature of the visual system means that the changes specify the activity of visual scanning and exploration, rather than visual stimuli which

THE SEMIOTIC BASIS OF CONSCIOUSNESS

235

passively register on the retina. Memory of previously stored graphic and other images is not required. There is no comparing of present stimuli with such stored images. Instead, the graphic tracings on the page are assigned their meaning through a process of integration between past sampling and present sampling.

5 The Metafunctional Character of Consciousness: Some Correlations Between the Neurobiological and Semiotic Dimensions 1. The Contextual Character of Consciousness Linguistic and other semiotic systems are not self-contained and autonomous entities. While they have their own intrinsic principles of organization and functioning, viz. a relative functional closure with respect to their higher-scalar ecosocial environments, they are also influenced and changed by the ways in which human beings jointly deploy the meaning-making resources afforded by systems of social signs. Social semiotic systems exist and are maintained in relation to higher-scalar ecosocial levels which exercise constraints or boundary conditions on them. Language and other social semiotic resources are open both in relation to the social meaning-making practices of social communities and in relation to the architecture and dynamical processes of the body-brain complex. Moreover, the notion of embodiment, as distinct from the biological organism which subtends our embodiment, has been proposed as an alternative to the current new wave of mechanistic thinking about the body and our relationship to it, as seen in the pervasive and reductive view of the body as a bottomup assemblage of parts (organs, etc.) which can be replaced, exchanged, and even sold on the black-market thanks to modern medical science and technology. In the ecosocial semiotic perspective of the present study, I have emphasized the importance of the expression and content strata as permeable interfaces which cross-eouple meaning-making processes to both the individual's ecosocial environment - his or her Umwelt- and the body-brain complex, i.e. the individual's Innenwelt, or invironment. The early in sights of Saussure and Hjelmslev concerning the stratified nature of the linguistic sign have been theoretically reconstituted in the ecosocial semiotic perspective to this end. Moreover, the two dimensions are not constitutively separable and must, therefore, be seen as components of a single self-organizing system of relations on diverse levels. Modern theories of consciousness, on the other hand, have stressed the autonomy and individuality of consciousness to such an extent that they have failed to throw off the shackles of their Cartesian solipsistic inheritance. Consciousness, which can never be autonomous in this sense, always exists in the context of both the lower-scalar and the higher-scalar dynamical relations and processes which produce and sustain it. Lukacs (1980 [1978]: 105-6) has

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 237 observed that the human body can exist without consciousness, but that consciousness cannot exist without the lower-scalar biological processes that sustain consciousness in the body-brain complex. This shows that lower-scalar physical, chemical, and biological constraints work through consciousness. By the same token, consciousness constrains these lower-scalar dynamical processes. The division of labour between body and soul is, thus, an untenable one; it is one which reflects actual social divisions of labour such as the current one between a medical science and technology of the body qua organism and a religious one of the individual soul that is housed in the body. Lukacs's understanding of the mastery of consciousness over biological processes can now be seen as referring to the ways in which higher-order or symbolic consciousness constitutes the most highly specified level in an integrative hierarchy containing progressively less specified outer levels. Thus, the individual is comprised of a number of different integrative levels (Salthe 1993: 70), which are constructed from the viewpoints of different (theoretical) discourses, viz. physical, chemical, biological, ecosocial semiotic, and psychological. From the perspective of an ontology of social being, to paraphrase Lukacs, each of these levels constitutes a set of constraints, all of which operate on the overall system of relations. With reference to the individual agent, higher-order consciousness is the most highly specified level because it is symbolic in nature and function. It cannot, therefore, be reduced to or explained in terms of the constraints operating on the lower physical, chemical or biological levels. Instead, it contextually integrates these to its own level so as to complete them in the time-bound process of constructing and construing an individual self. Lukacs thus draws attention to the ways in which constraints pertaining to the highest level - viz. the symbolic - operate on the lowest, least specified level of the physical existence of the body-as-thing obeying the physical laws of nature. We now come back to the question of how semiosis acts on and shapes consciousness. This is not, in my view, a matter of linguistic and other semiotic forms causally acting on individual consciousness in a direct, unmediated way. Instead, the lexicogrammatical resources along with their typical patterns of use mediate and entrain consciousness to their patterns. This is so in the sense that their forms and the categorial distinctions these construe have evolved as a result of the teleological positings of individuals. Consequently, these forms are not reducible to the biological mechanisms which sustain them in the individual. Human social semiotic systems have arisen in relation to social practices and orientations, rather than to purely biological ones. For this reason, the use of particular semiotic forms requires that social agents appropriately orient to their social contexts in and through the discursive deployment of these forms, rather than to the forms per se. Semiotic forms of all kinds are mediated by higher-order meaning-making practices whereby agents position and orient themselves to each other and to their contexts by means of specific co-deployments of such forms. (See Lemke 1990; Thibault 1991a for earlier formulations of the relationship between social semiotic resource systems and social meaning-making practices.) The question of how individuals are integrated with space-time scales that go beyond the here-now environmental samplings that are characteristic of primary consciousness require us to return to the following question: what is the nature of the higher-scalar level(s) above the individual? In order to answer this

238

BRAIN, MIND, AND THE SIGNIFYING BODY

question, I propose that we need to investigate the social meaning-making practices in and through which linguistic and other semiotic forms are eodeployed in both social interaction and inner semiosis. Consciousness, I suggested above, entails the recognition of meaning in our experience. Meaning does not refer uniquely to linguistic meaning, though this is clearly an important dimension of symbolic or higher-order consciousness. Primary consciousness, which is based on biologically driven perceptual and conceptual categorization, is meaningful because it involves the perception of pattern in experience and the ability to assign form to that pattern by way of conceptual categorization. Nevertheless, these patterns are not uniquely private or individual experiences that are locked away in the head of the individual. They are always linked to larger-scale supersystem transactions between individuals and their environments. What are often referred to as 'inner' cognitive processes can, therefore, be seen as forms of meaningful pattern recognition on lower integrative levels in precisely this sense. Higher-order consciousness does not mean that primary consciousness and its categorizations are transcended. Instead, the former integrates the latter so as to contextually complete it. Higher-order consciousness involves a higher level of meaningful activity whereby the recognition of pattern and meaning in experience is linked to meanings and contexts which are characteristic of a given culture as a whole. This indicates the need to represent consciousness contextually, rather than seeing it as simply being 'about' something which it represents. Rather than a simple pairing of conscious awareness and the object of this awareness, the approach I am developing here assumes that there is a pattern in this relationship and that this relationship has structure at logically higher orders of abstraction than either the conscious self or the observed phenomenon (object of consciousness). In this chapter, I shall explore the contextual character of consciousness by refracting the questions raised in this section through the metafunctional theory of linguistic and other modalities of semiosis that has been developed in systemic-functional linguistics. In so doing, I shall argue that consciousness at all levels is intrinsically organized in terms of a diversity of (proto)-metafunctional parameters. These parameters help to define the various ways in which consciousness is integrated to both its lower- and higher-scalar environments.

2. The Metafunctions and the Shape of Consciousness A central assumption of systemic-functional linguistic theory is that lexicogrammatical form is only describable with reference to semantics. In both formallinguistics and the information-processing model of cognition, on the other hand, syntax and neurophysiological brain processes are describable without reference to meaning. In both cases, cognition consists in the explicit manipulation of symbols in the mind according to rules based on formal logic. Edelman's theory of neural activities in the brain shows, on the other hand, that meaning is globally organized in the brain in terms of self-organizing topological regions. Edelman's observations resonate well with the view that meaning is a multidimensional semantic space which is informed by a topological perspective

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 239 (Halliday and Matthiessen 1999: 70-1). Halliday and Matthiessen point out that the topological perspective allows for indeterminacy in the sense that different regions of meaning overlap. Lexicogrammar is embodied in dynamical attractors or constrained pathways through the multidimensional semantic space. This observation is supported by the emergence of a lexicogrammar in the transition from protolanguage to adult language (Halliday 1993: 96). Lexicogrammar acts as a dynamical attractor for both bodily processes and the phenomena of experience. This is so by virtue of its interface with both the expression stratum and its interface with the semantics of the content stratum. Thus, lexicogrammatical neural pathways through the multidimensional semantic space entrain both meaning and sensori-motor activity as dialogically organized and oriented meaning-making acts. The higher-order boundary conditions of these dynamics correspond to the discourse genres and social activity-types of a given culture. Contextualized acts of meaning-making are mapped onto a vector space of cooperating neurons which constrains both the sensori-motor activities and the semantic features which are activated. This suggests that neural activities involved in meaning and consciousness can exert downwardly causal effects on bodily processes so as to produce socially meaningful action. The individual agent's internalized meaning potential can thus be seen as a set of attractors which refer to the system's self-organized internal dynamics. However, this internalized meaning potential is neither pre-given in the form of innate rules nor externally imposed by social norms. Instead, it is the elaborated product of the individual's transactions both with the higher-scalar ecosocial system and the specific contexts in which meaningmaking occurs. Meaning is thus embodied in self-organizing neural networks which constitute a multidimensional semantic space. The emergence of lexicogrammar in the transition from protolanguage to language is an epigenetic process based on the child's experience of and participation in the social meaning-making practices of the social group. The developmental progression from protolanguage to language entails that the emergence of language is an additional level of semiotic specification which, however, holistically integrates the prior protolinguistic stage to its own semiotic level. Protolanguage is not transcended when language emerges; rather, it is restructured by the higher integrative level at the same time as it continues to exert its own effects on the higher level (Salthe 1993: 85). The child's entrainment to the practices of the group likewise entrains his or her dynamic neurological organization. The move from the child's first symbolic acts, comprising an expression and a meaning without a lexicogrammar, to the adult language is a catastrophic reorganization of the multidimensional semantic space and the re configuring of the lexicogrammatical pathways through it such that four distinct regions of meaning are simultaneously attracted to these pathways. In the adult language, the four metafunctional domains of meaning are strongly coupled within the grammar of the clause, along with other levels of organization below this (see chapter 3, section 12, pp. 154-8). The increased coupling of the four metafunctions is evidenced in the child's developmental progression from early protolanguage to the adult language, as demonstrated in the longitudinal studies undertaken by Halliday (1975, 1978b, 1993), Painter

240

BRAIN, MIND, AND THE SIGNIFYING BODY

(1984), and Oldenburg (1987), as well as the work of Trevarthen. As these studies show, protolanguage does not exhibit this strong coupling of semantic functions which are intrinsic to lexicogrammatical form in the adult language. The coupling of the metafunctions occurs as the child develops a lexicogrammatical system which reorganizes the prior system, which consisted of vocalizations which realized situationally specific meanings without the presence of a lexicogrammar. Language development is not caused by prior, organism-internal factors, but by epigenetic experiences which foster the child's entrainment into the practices of the adult system of those who surround him or her. The fact that these changes are accompanied by development of the frontal cortex during this crucial early period of language development raises the question as to how the development of the adult language in the individual, with its strong internal coupling of metafunctional components in the grammar, contributes to the shaping of brain activity through the reentrant mapping of external linguistic activity onto internal neural activity. If language is intrinsically organized so as to enable individuals as complex systems to semiotically construe, interact with, and create meaningful ties to their ecosocial environments, then it should follow that the intrinsic metafunctional organization of language form in some way contributes to the shaping of consciousness itself. A number of important implications flow from this formulation. First, the notions of construing and constructing, in place of the idea of representation, indicate that the process of categorization is an active and contextualized one. It is not a matter of acquiring a set of internal symbols which represent external reality, as in the traditional cognitive science account. Secondly, the intrinsic metafunctional organization of semiotic forms further indicates that categorization has frequently been too narrowly defined in terms of the ideational function of language-as-thought. This tendency has been at the expense of seeing language both as action and a means of preserving a wholeness or unity of meaning in context such that individuals and the meanings they jointly make in their interactions with others are felt to be part of a larger organizational whole which links embodied individuals to their surrounding milieu in textured and coherent ways. Halliday has often argued that language and, by extension, other semiotic resources have evolved and are organized as they are on account of the social uses they have been put to over very many generations of evolutionary time (e.g. 1978a: 187). In other words, our semiotic resources have eo-evolved along with the social and biological contexts of their deployment. If consciousness emerges as the higher-order experiencing of one's own embodied experiencing, then language and other semiotic resources can be seen as having eo-evolved along with the emergence of consciousness as a way of mediating the experiencing of experiencing. It follows from this that our semiotic resources and their intrinsic design features are deeply implicated in the very nature of consciousness. This is not the same as saying that all of consciousness can be reduced to questions of language per se. Rather, the implication-cum-presupposition hierarchy of icon, index, and symbol shows that both primary and higher-order consciousness, both of which are modes of experiencing of experiencing, are mediated by the possibilities of iconic, indexical, and symbolic semiosis. Experience itself shows

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 241 multilevel complexity, interactional relationism, and dynamic contingency, and the task of our semiotic resources is to mediate these characteristics in the perspective of the self who undergoes the experience in question. In this chapter, I shall explore the implications of the metafunctions for the structure and functioning of consciousness on various levels of specification. I shall begin this discussion in the following section by first considering some of the vague and very general proto-metafunctional characteristics that are discernible in the infant's earliest perceptual-motor engagements with the environment. As we shall see, the (proto)-metafunctional shape of consciousness entails a contextual, rather than a solipsistic, view of consciousness.

3. Vague Contours of the Metafunctions in the Infant's Early Perceptualmotor Engagements with the Environment Current accounts of consciousness, in reducing it to a question of physical brain states and processes, or in failing to account for language other than as a device for 'reporting' or 'representing' individual, indeed soli psis tic, conscious experience, fail to account for the fact that consciousness is itself context-bound, intrinsically dialogic, and meaningful through and through (see also Volosinov 1973 [1930], 1976 [1927]). It is based on the viewpoints which agents can adopt on account of the semantic phase space of meaningful alternatives that are available to them in a given ecosocial semiotic system. The question arises as to how we can connect individual consciousness to the higher-scalar ecosocial system, and in ways that indicate the complex, open and non-linear dynamics of consciousness. In other words, how can we indicate its temporal and spatial grounding, the richly patterned connectivity that may implicate networks of relations across many diverse space-time scales, the sense of being part of and related to a wider social and perceptual field of relations which includes others? In order to forge the semiotically mediated link between the intrinsic dynamics of agents, the systems of meaningful alternatives available to them, and the higherscalar ecosocial environment, I shall propose that individual body-brain complexes and their dynamics are embedded in and entrained by systems of social meaningmaking practices which characterize a given society or some part of this. Edelmari's theory of neuronal group selection has shown us how the brain, on the basis of its ongoing sensori-motor exploration of the environment, selforganizes as a consequence of its context-bound interactions with the ecosocial and material environments within which it is embedded. In the first instance, the newborn infant builds up perceptual-motor categories of objects, events, actions, and so on, through the multimodal reentrant mapping of his or her time-bound experiences of seeing, listening, touching, moving, and so on (Thelen and Smith 1994: 187). My re-analysis of Halliday's (1975, 1978b, 1993) account of how children 'learn how to mean' drew attention to the intrinsically multimodal character of the dynamics of early language development (Thibault 2000a; and in this volume chapter 4, section 10, pp. 202-5). Thelen and Smith point out that such multimodal correspondences 'are the cause of development rather than a product of development' (1994: 187). Furthermore, Edelman's notion of

242

BRAIN, MIND, AND THE SIGNIFYING BODY

in-built values shows how the newborn is selectively nudged along certain trajectories of multimodal perceptual-motor engagement with the world in order that he or she is entrained into the patterns that are most salient and useful for his or her further development. The values that Edelman speaks about are biological and are part of our evolutionary inheritance. It is the child's spontaneous intrinsic dynamics which interact with and engage the ecosocial environment such that the time-bound activity of the system extends and changes the values available to the child for further exploration and interpretation of its environment. Thelen and Smith have expressed the multimodal basis of the child's earliest experiences as follows: ... infants discover action and object categories through the cross-correlation of multimodal experiences. The disjunctive, heterogeneous experiences of hearing and seeing and feeling and moving are all time-locked and change together as the infants' activity, state and actions change. We proposed that infants discover the category-relroant properties of objects and events invariant cues to shape, color, and texture across changes in lighting, perspective, and distance - through their multimodal and activity-dependent experiences of objects. Likewise, they discover patterns of coordination that provide functional actions through movement and its perceptual consequences. These ideas turn the usual developmental stories backward: multimodal correspondences are the cause of development rather than a product of development. (Thelen and Smith 1994: 187) The early processes of perceptual-motor category formation are not driven by pre-coded genetic patterns which predetermine the course of development. Category formation is a dynamical process of self-organization which occurs as a consequence of the individual's time-bound interactions with his or her environment. Halliday's (l992a: 21) claim that meaning arises 'out of the impact of the conscious and the material, as mutually contradictory forms of experience' supports the claims of Thelen and Smith. The conscious domain i.e. 'in here' - refers to the internal dynamics of the child; the material- i.e. 'out there' - refers, on the other hand, to the environment of objects, events, actions, and so on, which the child perceives and engages with through his or her activitydependent exploration of it. The child's earliest encounters with the world in the first four months or so are essentially iconic or topological-continuous in character. The essential point is that the multimodal stimulus information concerning features of objects and events such as colour, texture, shape, and so on, that infants pick up is necessarily correlated - in and through the child's own active, movement-based exploration - with perceived events in the world. Thus, typical associations are built up on the basis of a regularity in the associations which is perceived through successive explorations over a period of time. Edelman's theory of neuronal group selection explains how such regular multimodal associations of events in the world are reentrantly mapped as patterns of connections in the brain. The further point to be made here is that even in this first, iconic-topological phase of proto-meaning-making, there is a higher-order system of interpretance of matter, energy and information flows

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 243 which regulate the transactions between the child's internal dynamics and the stimulus information which is received about events in the world (see chapter I, section 3, pp. 11-18). The fact that perceptual-motor categories are formed through the selective strengthening of groups of neurons on the basis of such multimodal associations of stimulus information indicates that thirdness - i.e. explicit principles of interpretation and systematicity - is being constructed in the course of the interactions between the child's internal dynamics and the environment. The observations of Thelen and Smith also show, in my view, the existence of a very vague, imprecise outline or contour of proto-metafunctional organization in this earliest perceptual-motor phase of categorization. Thus, we have (l) proto-ideational meanings in the form of the child's multimodal experiences of the category-relevant properties of objects and events; (2) proto-interpersonal meanings in the form of the child's active exploration and engagement with the world, largely on the basis of 'movement and its perceptual consequences', as well as through the socio-affective exchanges between child and mother; and (3) proto-textual meanings in the form of patterns of co-ordination between child and his or her surroundings that provide functional action through movement. At this early perceptual-motor stage, the child's relationship to the world is a topological-continuous one. The child's internal dynamics are probably not strongly differentiated from the external, material world. The one is seen as being in a relationship of continuous variation or merging with the other. This does not mean that there is no distinction; rather, the distinction is not a sharply defined, or typological-categorial, one. When the child begins to move into protolanguage around the age of three to five months, the early protolinguistic signs produced by the child - e.g. the high-pitched squeak in Halliday's example (chapter 4, section 10, pp. 202-5) are the child's first conscious attempts to produce signs in order to engage others interpersonally. This is not to say that interpersonal exchange between child and mother begins with the onset of these protolinguistic signs. As we have seen, interpersonal dyads characterize the very earliest stage of infant-mother interactions. The difference lies in the largely iconic, topological-continuous character of the socio-affective exchanges that regulate the earliest stages on the basis of continuous flows of matter, energy, and information between mother and child. The move into protolanguage is, on the other hand, indexical; these protolinguistic indexical signs are contiguous with the contextual values they index, in contrast with the continuous nature of the iconic mode whereby the intrinsic dynamics of the child and the flux of ambient stimulus information about the external world are only vaguely differentiated from each other as continuous variation, rather than categorial (digital) distinction. Halliday (1975, 1992a, 1993) points out that the simple signs of protolanguage have Just two surfaces, interfaces between the conscious and the two facets of the material (content purport) and expression purport), such that meaning consists in making the transduction between them' (l992a: 22). That is, a simple, situation-specific semantics is directly realized by vocal or other gestural means without, however, the mediation of a lexicogrammar. At this stage, the proto-metafunctions are no longer vague, inchoate possibilities, barely

244

BRAIN, MIND, AND THE SIGNIFYING BODY

discernible in the many degrees of topological freedom of the iconic mode. Rather, their future, fully grammaticalized outlines can now be glimpsed in the proto-metafunctions of this indexical phase. In Nigel's high-pitched squeak, for example, there is (l) proto-ideational meaning, 'attending to phenomenon of experience'; (2) proto-interpersonal meaning, 'dialogically engaging mother/ demanding explanation'; (3) proto-textual meaning, 'linking internal perception to external event within sharable purview of parent-child dyad through use of indexical sign to selectively attend to/point to event'. The movement from the initial iconic stage to the indexical protolinguistic one arises as a consequence of the child's ongoing, time-bound interactions with the environment in the course of which higher-order regularities or principles of interpretation come to be constructed. Thus, we see how the activity of the dyad progressively resets the values of the emerging system as simple, bi-stratal protolinguistic signs appear in the system. This resetting of values shows how the perceptual-motor activity of the child's environmental explorations in the initial system produces positive feedback. The resulting positive feedback, in turn, leads the system far from equilibrium. As a consequence, the newly emergent higher-order constraints of the indexical mode provide new values so that the system is nudged still further along its trajectory. The new constraints of protolanguage, again in concert with the child's internal dynamics, mean that the attractor pattern in which the child is located is re-structured such that the probabilities of the system as a whole are realigned. The move into grammar, as Halliday characterizes the transition to truly symbolic semiosis, involves the emergence of a tri-stratal semiotic system of the kind first outlined in chapter I, section lO, pp. 46-9. The semantics is now realized by a lexicogrammar. The increasing dynamism of the meta-redundancy relations between strata (see chapter 1, section 6, pp. 26-30) means that the values of the system are constantly being re-set from instance to instance (Halliday 1992a: 29). This can only happen because of the essentially open, dynamic character of stratified semiotic systems such as language. Such systems are involved in constant exchanges with their environments through the activities of the agents who use the system for their own purposes in particular contexts. Consequently, the order - the meaning-making potential - of the system increases. Semiotic systems are open, far-from-equilibrium systems. Rather than settling into a state of equilibrium, far-from-equilibrium systems maintain themselves through constant exchanges of matter, energy, and information between the internal dynamics of the system and its external environments. Biological and social semiotic systems are both examples of this kind of system. Such systems exhibit a number of fundamental characteristics. First, they have complexity. That is, the components which comprise the system interact in very many heterogeneous and non-linear ways. Secondly, they are dissipative structures because they maintain their organizational complexity by obtaining matter, energy, and information from environmental sources and by exporting or dissipating some of this energy back into their external environments. Dissipative structures are systems that exist far from equilibrium. Dynamic open systems which both have complexity and are dissipative structures also show another important property: they are self-organizing

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 245 systems. The child's dyadic exchanges with his or her caregivers ensure that his or her internal dynamics are being supplied with a constant supply of potentially meaningful information. The emergence of language and other symbolic modalities out of the prior proto-semiotic modes is a case in point. The system as a whole shifts from one global principle of organization to another. The appearance of lexicogrammar in ontogenesis is a form of emergent organization which has fundamentally different organizational properties from the elements that constitute the prior system and cannot be predicted on the basis of the prior state of the system. In proto-semiosis, the various meanings - proto-ideational, proto-interpersonal, proto-textual - are realized as discrete acts, which Halliday (1975, 1993) has described as 'microfunctions'. On the other hand, the emergence of lexicogrammar entails the slaving of the various functional components to a global principle of organization, which is metafunctional. This metafunctional principle may be considered, following Haken (1977), to be an order parameter. This means that the system as a whole is now constrained by a very few metafunctional parameters on account of the appearance of a lexicogrammar which is 'slotted in' between the phonetic and semantic levels of organization of the prior protolinguistic system. This type of self-organization typifies complex systems which are engaged in constant transactions with their semiotic and material environments. Such a system is, as Halliday puts it, a multidimensional and highly elastic semantic space; it has a very large and constantly evolving meaning-making potential, as well as a large number of potential cooperative modes both with the physicalmaterial world and with other semiotic modalities (e.g. gesture, movement, depiction, and so on). In generating local points of instability, particular eopatternings of meaning-making selections can function to attract possibilities from other modalities. Such local asymmetries are amplified the further the system is from equilibrium. In this way, self-organizing systems are attracted to a preferred configuration out of many possible ones. When language emerges from protolanguage, it does so under the enslaving influence of an order parameter, viz. a metafunctionally organized lexicogrammar. The system is thus attracted to this preferred mode. In this way, lexicogrammar is an attractor state: the system prefers the principles of organization of its state space and its semiotically salient distinctions (values) over others. Lexicogrammar as state space refers to the overall system of values within which the degrees of freedom of the system is embedded in specific contexts of use. Thus, the parameters of this state space are constantly adjusted and redefined according to the contexts which bring the system into contact with its environment. The emergence of lexicogrammar moves the system into an n-dimensional semiotic space (Halliday 1992a: 25) such that the state space of the system is defined as an abstraction of the possible values of the terms in n-dimensional space, where n refers to the total number of terms - the system of semiotically salient differences, or values in the Saussurean parlance - that constitute the system. The iconic, indexical, and symbolic modes may be seen as stable states along the pathway to the emergence of higher-order or symbolic consciousness. In a given phase state, the system exists in a particular attractor landscape (Thelen and Smith 1994: 56), which is defined by the overall system of interacting

246

BRAIN, MIND, AND THE SIGNIFYING BODY

variables that define the system's dynamics and by its initial conditions. In fact, the system is never really static, but is constantly responding and adapting to environmental transactions. The shift from one state to another in ontogenesis - iconic to indexical to symbolic - is a qualitative leap to a new phase state such that the entire attractor landscape is reorganized. The overall dynamical organization of the system is transformed. The shift from iconic to indexical and symbolic modalities of semiosis in ontogenesis progressively means that the dynamics of the body-brain complex are entrained into the global dynamical organization of higher-scalar ecosocial relations and processes. In this way, the neural activity of individuals is entrained to the meaning-making practices and the semiotically salient differences of the ecosocial level of organization. Symbolic consciousness emerges out of and constrains, without ever totally transcending, the less specified perceptual-motor and conceptual forms of categorization that are characteristic of primary consciousness (Edelman 1992: chapter 11). The iconic and indexical modes are always nested within and integrated by the symbolic mode in terms of the logic of the specification hierarchy. It is important to draw attention here to the fundamentally multimodal character of all human meaning-making. The multimodal character of symbolic modalities of semiosis is built upon and does not entirely transcend the multimodal character of the infant's earliest multimodal sensori-motor explorations of his or her environment in the process of perceptual-motor category formation. Moreover, the longer phylogenetic history of pre-linguistic ore-facial and manual-brachial gestures with respect to language does not mean that language transcends or otherwise stands independent of these phylogenetically prior forms. Rather, the historical emergence of language led to its multimodal integration with other semiotic modalities. Rather than impeding the full flowering of language as the ultimate expression of human rationality, this process of integration has afforded the possibility of the evolution of new genres and ways of making meaning including the multimedia and hypertext genres characteristic of the age of the Internet. In the next section, Gibson's ecological theory of perceptual awareness will be examined in order to see what it can contribute to the semiotic theory of consciousness that I am developing here.

4. The Structure of (Self)Consciousness in Perceptual Awareness Gibson (1986 [1979]: chapter 7) shows that the information specifying the self (proprioception) and the information specifying the perceived environment (exteroception) are not two different kinds of information. Instead, they are two poles of attention based on the same kind of information. The perceiving of an environmental event means perceiving that event as occurring 'there', at a certain distance from the perceiver, and so on. Thus, Gibson shows that the perceptual awareness of objects and events in one's environment is also and always self-awareness of one's self as the one who perceives those objects and events in the perspective of the self (see also Battacchi 1999: 57). Reference to perceived objects and self-reference are no more than two poles of attention of

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS

247

a single overarching operation which necessarily always implicates both poles. Gibson's observations show that the act of perception is always grounded in the perspective of the self qua perceiver. Thus, we see how the principle of grounding, which is of crucial importance in the operation of language, also operates at the lower integrative levels of perceptual awareness and pre-linguistic conceptual thinking. Individuals are likewise aware of the fact that the perceptual and mental states which they experience are experienced as their own. That is, perceptual and mental states are grounded in the perspective of the self who has this experience. Battacchi (1998: 6) argues on this basis that consciousness necessarily entails not only an awareness of one's own mental states, but also a concomitant awareness that the mental states are one's own. For this reason, Battacchi proposes the term' (self) consciousness' in order to draw attention to the fact that 'consciousness is always self-consciousness' (1998: 6; see also Battacchi 1996). Gibson's discussion of the two poles of attention implicated in perception is a useful starting point for exploring the question of how others can access the self's perceptual awareness and states of consciousness. Let us start with perception. The first question I shall consider is how and to what extent an individual act of perception, grounded at a particular self, can be re-grounded by another individual who is observing the first individual's act of perception and trying to figure out what the first individual is perceiving. This happens in the case of gaze-following by primates. Bogdan (2000: 110-13; see also chapter 4, section 9, pp. 201-2, for further discussion) discusses how primates recognize the vectorial quality of another's gaze so that they are able to track the direction of the other's gaze. Furthermore, this involves an act of interpretation on the part of the observer such that the observer is able to follow the gaze vector's directionality in order to determine the target of the observed individual's gaze (Bogdan 2000: Ill). From the point of view of the individual who performs the act of gazing, we can say that this individual is simultaneously aware of the target of his or her gaze qua object of attention, along with the fact the individual's perceptual awareness of this target is grounded in the perspective of the particular self who has the experience. This grounding of the perceptual awareness in the perspective of a self is both iconic and indexical in character and is, therefore, tied to the individual's material interactivity with the immediate environment in which the act of gazing occurs. Moreover, the self is aware not only that it is the self who has the given experience, but also the nature of the specific perceptual modality - visual perception, in this case - in which the perceived object is experienced. The two poles of this relationship may be schematized as follows:

[[PERCEIVING SELF --> [PERCEIVED OBJECT]] In my view, the two poles of this experience are precisely what enables the experience to be reconstructed from the perspective of an observer of the self's actions. That is, the observer is able to re-ground the experience from his or her perspective by virtue of the contextualizing relations which the observer construes (interprets) between the observed selfi's gaze vector and the gazed-at-

248

BRAIN, MIND, AND THE SIGNIFYING BODY

object. In turn, the contextual relationship between these two factors is recontextualized on a higher level by the observer, i.e. from the perspective of the observer qua selfs. This re-contextualization shows that even at the lower integrative level of perceptual awareness, both humans and non-human primates have the ability to re-ground the perceptual experience of another and to interpret it from the perspective of their own self. This does not mean that the observer, for example, is able to stand in the shoes, so to speak, of the observed self and have the same experience. The distinction between first- and third-person accounts of consciousness (chapter 4, section 1, pp. 171-3) is something of a red herring. Instead, the observer's ability to re-ground another's perceptual experience shows that even at this level, there exists the possibility of interpreting another's experience from one's own perspective in the way described above. In any case, both the perceptual experience of the observed self and its re-grounding by an observer are tied to a particular perceptual purview on the immediate here-now scale of the body in interaction with its immediate environment. For example, the observer (self,) in our example must be able to observe and, therefore, to visually track the observed self.'s gaze vector to its target. What is important is that self and observer belong to the same higher-scalar system of interpretance. It is the integration of self and observer to this SI and its consequent mediation of the information deriving from them which determines whether this information is meaningful to other selves who share the same SI and its perspectives. Thus we see that the lower integrative level of perceptual experience and the forms of consciousness that derive from it are not randomly different from one individual to another. The perception of environmental events from the grounded perspective of a self and the interpretation of the self's perception from the standpoint of an observer are possible because both environment and the individual conspecifics who act and perceive in that environment are integrated to and participate in higher-scalar relationships which define and regulate possible individual-environment transactions. This means that (1) the individual's sensori-motor interactions with and responses to the environment, in spite of individual differences and variations from one individual to another, reflect patterns of interaction that are typical of all those individuals who participate in the same environment; and (2) the perspectives of self and observer in our example can be co-ordinated in the way described because each individual's sensori-motor patterns of interaction share fundamental characteristics with those conspecifics who have been entrained to the same kinds of self-environment transactions. As we shall see in section 5, Damasio's neurobiological account of consciousness is based on self-object transactions. Damasio's account also begins to reveal the proto-metafunctional character of consciousness. In sections 5 and 6, I shall examine Damasio's theory with this perspective in mind.

5. Damasio's Neurobiological Theory of Consciousness Damasio proposes a neurobiological theory of consciousness which is based on 'two intimately related problems' (1999: 9). Damasio explains these two aspects of consciousness as follows:

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 249 The first is the problem of understanding how the brain inside the human organism engenders the mental patterns we call, for lack of a better term, the images of an object. By object I mean entities as diverse as a person, a place, a melody, a toothache, a state of bliss; by image I mean a mental pattern in any of the sensory modalities, e.g. a sound image, a tactile image, the image of a state of being. Such images convey aspects of the physical characteristics of the object and they may also convey the reaction of like or dislike one may have for an object, the plans one may formulate for it, or the web of relationships of that object among other objects. (Damasio 1999: 9) The second problem is defined as follows: This is the problem of how, in parallel with engendering mental patterns for an object, the brain also engenders a sense of self in the act of knowing. To help me clarify what I mean by self and knowing, I urge you to check their presence in your own mind right now. You are looking at this page, reading the text and constructing the meaning of my words as you go along. But concern with text and meaning hardly describes all that goes on in your mind. In parallel with representing the printed words and displaying the conceptual knowledge required to understand what I wrote, your mind also displays something else, something sufficient to indicate, moment by moment, that you rather than anyone else are doing the reading and the understanding of the text. The sensory images of what you perceive externally, and the related images you recall, occupy most of the scope of your mind, but not all of it. Besides those images there is also this other presence that signifies you, as observer of the things imaged, owner of the things imaged, potential actor on the things imaged. There is a presence of you in a particular relationship with some object. (Damasio 1999: 9-10) Damasio's definition illustrates the principle mentioned earlier that consciousness is an evolved way of mediating the experience of experiencing. Consciousness, Damasio observes, 'from its basic levels to its most complex, is the unified mental pattern that brings together the object and the self' (1999: 11). The bringing together of self and object means that the two interact in some way; self may be generalized as an observer system which interacts with some other system - the object - and therefore forms some kind of interactive relationship with it. The mental images which the self forms of the object constitute meaningful information internal to the self. These images constitute a selective mapping of information in the environments - both external and bodily - onto the observer system's internal structure in order that the system can impose order and pattern on environmental fluctuations. The interactive relation between self and object may be seen in terms of what Peirce called 'firstness' and 'secondness' (see chapter 1, section 8, pp. 34-9). While self in Damasio's account would be a first, the interaction between self and its object is a second. Furthermore, it is clear that the interaction between self and object is not random, but takes place in an organized - Damasios term is 'unified' - field of mental images (of the object)

250

BRAIN, MIND, AND THE SIGNIFYING BODY

which mediate between self and object and which enable the two to be placed in some kind of interactive relationship with one another. Again, we see that we have self-awareness not of the self per se, but of the self-in-interaction-withsurround (i.e. the internal milieu and the external environment). That is, we loop on the whole Batesonian loop, including both efferent and afferent elements, such that the 'out there' that we construe as being different from the 'in here' is re cursively mapped in the brain as an internalization of self-object transactions. The system of relations described by Damasio may be seen in terms of the three-level scalar hierarchy (chapter I, section 3, pp. 11-18). It is important to point out here that Damasio's description of the self as 'this other presence that signifies you, as observer of the things imaged, ... ' does not refer to the brain per se, but to a self-referential perspective which is neurologically realized in specific areas of the brain (Damasio 1999: 25). In terms of the three-level scalar hierarchy, interactions between self and object constitute the focal level L. Below this level, there is the specific neurobiological organization of the individual organism (not the self) which predisposes the organism to certain kinds of interaction with objects in both its inner (bodily) and outer environments and to form mental images of these. This is the L-Ilevel. Above the focal level, there is a system of interpretance, which is comprised of the perceptual-motor interactions between the organism and its inner and outer environments and which forms the basis on which mental images of environmental objects are interpreted and understood. This is the L+I level. The vaguely semiotic appeal, whether intended or not, of Damasio's designation of the self as 'this other presence that signifies you' (my italics) suggests that the self is realized by lower-level neurological events, but is not reducible to these. Overall, Damasios description of the basic components of what he calls core consciousness shows how levels L-I and L are contextually integrated by the higher-scalar level, L+I, at the same time as being self-reflexively connected to the higher-scalar level which provides a theory of the lower levels (Salthe 1993: 49). It is the higher level which interprets - contextually integrates - the lower levels. This last point is illustrated by Damasio's discussion of the importance of mental images in the guiding of action (1999: 24). That is, mental images formed in core consciousness are a means for guiding the self's interactions with the object. Damasio puts it like this: Images allow us to choose among repertoires of previously available patterns of action and optimize the delivery of the chosen action - we can, more or less deliberately, more or less automatically, review mentally the images which represent different options of action, different scenarios, different outcomes of action. We can pick and choose the most appropriate and reject the bad ones. Images also allow us to invent new actions to be applied to novel situations and to construct plans for future actions - the ability to transform and combine images of actions and scenarios is the wellspring of creativity. (Damasio 1999: 24)

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS

251

and Consciousness generates the knowledge that images exist within the individual who forms them, it places images in the organism's perspective by referring those images to an integrated representation of the organism, and, in so doing, allows the manipulation of the images to the organism's advantage. Consciousness, when it appears in evolution, announces the dawn of individual forethought. (Damasio 1999: 24-25) The mental images formed in the brain on the basis of the organism's interactions with its internal and external environments contextually integrate organism and object such that the organism may selectively represent the given object and act in relation to it. Importantly, these image-object relations are symbolic; they are not indexically connected to particular external environmental objects and events. Instead, the mental image symbolically interprets its object. For this reason, the organism is able to exercise choices in action and response from among a repertoire of symbolic possibilities which are internalized in its Innenwelt. Furthermore, these organism-object transactions are, as I have argued, integrated into a higher-order system of interpretance. In one sense, all of the basic components of consciousness occur within the brain of the individual. However, I have shown that the logic of the three-level scalar hierarchy applies here as well. In other words, there can be no reduction to neural events per se. Core consciousness is, in the first instance, a form of protosemiosis whereby the higher-scalar level of the organism-environment transactions - i.e. the system of interpretance - is internalized as the organism's here-now Innenwelt in the form of mental images which are grounded in the perspective of the self who observes and experiences these images. That is, models of organism-environment transactions constitute a system of interpretance in the Innenwelt such that particular interactions between self and the objects of (self)consciousness can be interpreted. The fact that mental images allow the organism to choose among different potential actions implies that mental images are potentially meaningful in some way to the organism. This further implies that a higher-scalar system of interpretance allows the organism to know and act on the here-now scale by formulating contextually appropriate interpretants. Mental images, Damasio claims, are created within the brain in order that the organism may interact with objects 'actually present or recalled' (1999: 25). This suggests that the mental images of core consciousness, rather than simply representing something which is already pre-given, may be said to indexically invoke or otherwise symbolically constrnct their objects. Thus, mental image a is redundant with object b in the sense that they are both contexts for each other's interpretation (see Lemke 1984c: 36 for this formalism; see also chapter 1, section 6, pp. 26-30). Damasio had remarked (see above) that images convey 'aspects' of their objects. That is, the reciprocal process of contextualization is a selective one. However, in terms of the logic of the redundancy relations, the relation between a and b itself minimally requires a further order of contextualizing relations, c, in order to be integrated to or interpreted by some higher-

252

BRAIN, MIND, AND THE SIGNIFYING BODY

scalar system ofinterpretance. That is, for a mental image to be recognized as an image of an object, there must be a further level of contextualization which specifies how the a/b relation is to be interpreted or recognized by the self (Lemke 1984c: 36). This would be suggested by the context-sensitive nature of neurological processes (see chapter 6, section 4, pp. 290-5). In the here-now environment of core consciousness, objects in the organism's external and internal environments form redundancy relations with neurological events such that mental events are realized. These mental events are signs of environmental objects and events such that an image is grounded within the perspective of the self. Consciousness, in this case, would be the higher-order context in which the a/b relationship between environmental events and neurological events is interpreted as a mental image which has meaning in the perspective of the self. Prior to the conscious core self of Damasio's account, there is the nonconscious proto-self (Damasio 1999: 173). Damasio outlines the distinction between unconscious proto-self and conscious core self as follows:

As far as the brain is concerned, the organism in the hypothesis is represented by the proto-self. The key aspects of the organism addressed in the account are those I indicated as provided in the proto-self: the state of the internal milieu, viscera, vestibular system, and musculoskeletal system. The account describes the relationship between the changing proto-self and the sensorimotor maps of the object that causes those changes. In short: As the brain forms images of an object - such as a face, a melody, a toothache, the memory of an event - and as the images of the object affect the state of the organism, yet another level of brain structure creates a swift nonverbal account of the events that are taking place in the varied brain regions activated as a consequence of the object-organism interaction. The mapping of the object-related consequences occurs in first-order neural maps representing proto-self and object; the account of the causal relationship between object and organism can only be captured in second-order neural maps. Looking back, with the license of metaphor, one might say that the swift, second-order nonverbal account narrates a story: that of the organism caught in the act of representing its own changing state as it goes about representing something else. But the astonishing fact is that the knowable entity of the catcher has just been created in the narrative of the catching process. (Damasio 1999: 170; italics in original) The relationship between proto-self and the sensori-motor maps of the objects that cause changes in the proto-self occurs in first-order neural maps. At this level, there is a first, weak differentiation of proto-self and object, but no awareness of this differentiation, as shown by the fact that the proto-self changes in response to environmental fluctuations that cause these changes. Damasio defines the proto-self as 'a coherent collection of neural patterns which map, moment by moment, the state of the physical structure of the organism in its many dimensions' (1999: 154; italics in original). That is, the proto-self experiences, but has no secondorder experience of its experiencing. Furthermore, the proto-self 'has no powers of perception and holds no knowledge' (Damasio 1999: 154). The emergence of core consciousness and its corollary of a conscious core self is based on second-

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 253 order neural maps which realize an 'account of the causal relationship between object and organism' (1999: 170). This is where the experience of experiencing enters the picture. Damasio explains core self and core consciousness as follows: You know that you are conscious, you feel that you are in the act of knowing, because the subtle imaged account that is now flowing in the stream of your organism's thoughts exhibits the knowledge that your proto-self has been changed by an object that has just become salient in the mind. You know you exist because the narrative exhibits you as protagonist in the act of knowing. You rise above the sea level of knowing, transiently but incessantly, as a felt core self, renewed again and again, thanks to anything that comes from outside the brain into its sensory machinery or anything that comes from the brain's memory store toward sensory, motor, or autonomic recall. You know it is you seeing because the story depicts a character - you - doing the seeing. The first basis for the conscious you is a feeling which arises in the representation of the nonconscious proto-seifin theprocess of beingmodifiedwithin an account which establishes the cause of the modification. The first trick behind consciousness is the creation of this account, and its first result is the feeling of knowing. (Damasio 1999: 171-2) The second-order neural maps of core consciousness bring in a further differentiation of the first-order reentrant loop linking proto-self to object. Thus, core consciousness entails a conscious experience of experiencing, as realized by the second-order neural maps mentioned by Damasio in the above citation. Core consciousness is transient; it is linked to the here-now and is non-verbal. Its timescale is fractions of seconds. Beyond the core self, there is what Damasio refers to as the autobiographical self (1999: 172-3). The autobiographical self brings history into the overall picture. It is built on core consciousness, but is distinguished from this by the fact that the transient experiences of the self of core consciousness can now be committed to memory and at the same time conceptualized (Darnasio 1999: 173). The autobiographical self has a past, as well as an anticipated future. Nevertheless, it too consists of nonverbal images. Damasio notes that this memory can be expanded and refashioned along one's life trajectory (1999: 173). The formation of an autobiographical self means that further differentiations of the self and its relationships to objects - past, present, and future - can be made. The autobiographical self is based on a concept 'in the form of dispositional, implicit memories contained in certain interconnected brain networks' (Damasio 1999: 174). The making explicit of these memories at any given moment means that we move from the experience of experiencing (core consciousness) to the memory-based conceptualization of experience. This is what Damasio calls 'extended consciousness' (1999: 195-200). Darnasio's distinction between the unconscious proto-self and the conscious core self and the autobiographical self demonstrates the logic of the specification hierarchy - both synchronically and developmentally. Thus, there is a progression from the most vague and general qualities of the proto-self, which has neither perception nor knowledge, to the transient here-now knowledge that

254

BRAIN, MIND, AND THE SIGNIFYING BODY

the core self has of its mental images through to the more conceptually elaborate and memory-based images of the autobiographical self of extended consciousness. In this specification hierarchy, it is clear that the autobiographical self is both more elaborate and more determinate than the less specified proto- and core selves as a consequence of its memory-based conceptual ability to store information as well as to individuate this information. This specification hierarchy of consciousness begins with the absence of observer self-consciousness (protoself). This is the least specified level. For this reason, it is also the level that is furthest from our experience. The increasingly self-referential perspectives of core consciousness and then autobiographical consciousness, in Damasio's account, are explainable on the basis of the increasing specification of these levels, which are closer to the perspectives familiar to the observer. As we shall see below, the metafunctional organization of language can be see as a more specified mode of organization of higher-order or symbolic consciousness in the form of, for example, conscious, linguistically mediated thinking (sections 9 'and 10, pp. 271-5). However, the forms of consciousness which occur on lower integrative levels can also be made intelligible in terms of the metafunctional principle. This is so for two reasons. First, the principles of organization imposed by more specified levels such as symbolic (higher-order) consciousness impose constraints on lower (less specified) integrative levels. Secondly, statements about the organization of higher-order consciousness can only be made, at least implicitly, because the forms of consciousness on lower levels came into existence before the higher levels. This is so both phylogenetically and ontogenetically. This suggests that less specified principles of organization on the lower levels are integrated to and reorganized on the higher levels according to more specified principles. Importantly, selves can be conscious of mental images which are nonverbal in character. These images are grounded in the perspective of an intentional self who attends to these images in consciousness. The unconscious proto-self is based on first-order neural patterns whereas the core self and the autobiographical self have knowledge of nonverbal second-order mental images. In core consciousness, these images occur in the here-now, whereas in extended consciousness they can be connected to a past and a future. The imagistic character of both core and extended consciousness corresponds, in my view, to what Peng (1994) has called proto-meaning. Mental images are proto-semiotic, although they already begin to exhibit some of the characteristics of semiosis. I believe it is right to see these mental images as proto-meanings because they are contents which are realized by neural events in the brain. One difference with a fullfledged semiotic system like language is that the same mental image can be realized by different patterns of neural activity within the individual on different occasions. Mental images, as well as higher-order nonverbal thoughts, are de-coupled from neural processes such that the content which is realized by the neural processes becomes the object of conscious awareness and attention rather than the neural processes themselves (Juarrero 1999: 90). While there is no conventional coupling of meaningful mental content to their means of expression in neural processes, there is, nevertheless, a principle of realization which stratifies mental contents and their neural means of expression. Thus, the

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 255 flow of non-perceptual sensations which are felt by the proto-self may be seen as a form of pure iconicity; the transient here-now character of the perceptually caused images of core consciousness is akin to indexically presupposing signs; the ability to recall past images, to anticipate the future, and to elaborate mental images in extended consciousness suggests something like Silverstein's (1987a, 1987b) notion of indexically creative or entailing signs. The latter two categories require a self-referential perspective, which would seem to be required by any form of proto-indexical mental imagery. This is so in the sense that indexicality is always related to and grounded at an intentional source which points to (indexes) the indexed object. The three components of the specification hierarchy mentioned above also show how proto-self, core self and autobiographical self are all embedded in an immediate supersystem which contains both (proto-)self and the object in the immediate (internal or external) environment of (proto-)self. This shows how the emergence of consciousness from non-eonsciousness is always guided by higher-order supersystem transactions. Accounts of consciousness which privilege a bottom-up view of consciousness in terms of the organism's neurobiological dynamics may miss the point that consciousness, by definition, always implies higher-order supersystem processes and dynamics (see Salthe 1993: 277-81; Lemke 1995b: 115-16). There is always a stable, higher-order regulatory environment on a slower timescale which regulates the faster-scale neural dynamics within the individual organism. Consciousness in the individual organism and the neurobiological processes which subtend this are always coupled to larger-scale self-object relations and dynamics and in ways which intentionally direct and guide these dynamics. In the case of consciousness, selfobject transactions, as described by Damasio, are the immediate supersystem. The social character of our evolutionary ancestors, as well as the human species itself, means that many different individuals develop under similar environmental conditions, thereby increasing the likelihood of their undergoing similar bifurcations (Lemke 1995b: 115). The emergence of consciousness in an individual has environmental effects that promote the same kind of bifurcation in other individuals. Organism-environment transactions might lead to the formation of neural patterns which reentrantly map these supersystem interactions to an internalized perspective. In this way, a relatively stable, higher-order supersystem of self-object interactions gets internalized in the organism's Innenwelt. This would be formally equivalent to the idea of mind as an individuated system of interpretance which the self uses to attend to, interpret, and respond to events (signs) in the self's internal and external environments. But conscious selves are always cross-coupled to other conscious selves in stillhigher-scalar networks of both physical-material and semiotic-discursive processes and relations on the ecosocial scale. According to Damasio (1999: 185), language converts or translates the second-order nonverbal narrative of images of consciousness (core or extended) such that these 'become verbally present in our minds' in the process of focusing on them (ibid.). I agree fully with Damasio (e.g. 1999: 108) that self and consciousness exist prior to language and that language does not construct consciousness as such. Damasio's core and extended consciousness pre-exist language, as does Edelmari's primary

256

BRAIN, MIND, AND THE SIGNIFYING BODY

consciousness. The neurobiological arguments of Damasio and Edelman are compelling in this respect. However, the suggestion that language merely has a translating function with respect to the mental images of consciousness is, in my view, a seriously misleading notion of the function of language. I shall now turn to this question. I suggested above that consciousness is an internalization - a reentrant looping - of self-object supersystem transactions. This means that consciousness is always the result of higher-order supersystem transactions. Consciousness came to be in the context of such higher-order environments which got internally mapped as the organism's Innenwelt. It did not come into being on the basis of bottom-up neurological dynamics perse. Core consciousness and extended consciousness are founded on biological and thermodynamic processes and transactions between the organism and its material environments. The space-time scale is the immediate here-now one. That is, individual core consciousness is tightly coupled to here-now material transactions; on the other hand, extended consciousness, with its past and future, exhibits the capacity to incorporate diverse space-time scales to a much greater extent. However, the cross-eoupling of consciousness to ecosocial semiotic processes means that the individual's consciousness is now linked to systems of very different space-time scales (see Lemke 2000a, 2000b; Thibault 2000a). In the case of language and other social semiotic resources used to make meaning in a given culture, these form part of a larger-scale ecosocial system which integrates and regulates the smaller-scale subsystems of individuals to its larger-scale patterns and dynamics. The cross-coupling of individuals to these higher-scalar ecosocial relations and dynamics means that individual consciousness, in particular what Edelman calls higher-order or symbolic consciousness, is entrained to the social practices in and through which language is deployed to make meanings in specific contexts. Symbolic consciousness has all the characteristics of core consciousness and extended consciousness, but it is also a more semiotically specified form of consciousness which enables us to give symbolic meaning to experience by organizing and entraining consciousness in relation to the always to-degrees shared social meaning-making practices of a given culture. Furthermore, the connecting of consciousness to a culture's symbolic resources for making meaning means that, in symbolic consciousness, maximally diverse space-time scales may intersect in the individual (Lemke 2000a). Rather than 'translating' nonverbal mental images in consciousness, language, in the process of construing these in terms of its own intrinsic categorial differentiations, crosscouples individual consciousness to the social semiotic formations of a community. On the other hand, to say that language 'translates' nonverbal images in consciousness is, implicitly, to assume that language in the individual organism is the focal level. It is to say that we make sense of language in relation to mental images arising 'from below' in the brain of the individual and which language encodes ready for decoding by someone else. But consciousness is a relationship to phenomena on the human-scale that we share with others. It is a question of the perspectives that the self can adopt on the phenomena of experience and how these experiences may be meaningfully exchanged with other selves who are able to share the same perspectives. This also helps to resolve the dilemma of the third-person and first-person

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS

257

approaches to consciousness. Proponents of the first approach seek to reduce consciousness to neural events per se (chapter 4, sections 1-3, pp. 171-84). This approach is reductionist precisely because it does not account for the humanscale observer perspectives of selves. It fails to see that the models of neural events that are made in this third-person perspective are in fact models of the observer's relationship to the events on this level. On the other hand, firstperson perspectives, in emphasizing the uniqueness of individual consciousness, can border on solipsism in failing to account for the fact that consciousness qua the observational perspective of the self occurs on the human-scale level of other selves which are embedded in a higher-scalar system of interpretance which contextually integrates the perspectives of different selves to its dynamics. In this way, the perspectives of diverse selves can be dialogically co-ordinated and negotiated in discourse. This does not mean that individuality is eliminated. Individual consciousness is robust in its system of interpretance precisely because it leaves signs of its individuality in its interactions with the viewpoints of other selves who are embedded within the same system of interpretance and in terms of which individual viewpoints have their meaning (see Salthe 1993: 51 for the notion of robustness; see also this volume, chapter 2, section 1, pp. 59-68).

6. The Proto-metafunctional Structure of Core and Extended Consciousness Damasio's account of core consciousness and extended consciousness suggests some striking parallels with the internal metafunctional organization of language, as in Halliday's (1979) account. For reasons I shall discuss below, I do not think this is in any way accidental. But first I shall re-examine the principal components of Damasio's theory of core and extended consciousness with a view to seeing how they can be seen as precursors of the intrinsic metafunctional organization of language in higher-order or symbolic consciousness in the way that I suggested above. 6.1 Conceptualizing the Objed of Consciousness: Mental Images and Proto-experiential Meaning Mental images are of some object either in the external environment of the individual or within the internal milieu of the individual organism's body. Mental images are neurally constructed images of objects that are constituted as phenomena of experience for the self. Damasio points out that the brain constructs mental images of objects in both the external environment of the individual organism as well as of objects experienced as being within the internal biological milieu of the organism's body. This distinction shows how the very first level of consciousness - Le. core consciousness - already constitutes a distinction between that which is experienced as existing 'out there' in the external world as distinct from that which is experienced as existing 'in here' in the internal biological milieu of the individual organism. Furthermore, the mental images of the objects of experience are not simply re-presentations of already given phenomena in the inner and outer worlds. Rather, neural processes in the brain

258

BRAIN, MIND, AND THE SIGNIFYING BODY

actively and selectively construct mental images of the phenomena of experience by attending to specific aspects of these phenomena, the emotional state of the individual, and so on. To paraphrase Edelman, there is no pre-labelled world which mental images simply reflect. Mental images are the experiential contents of consciousness and are realized by patterns of neural activity though without being reducible to these. Furthermore, mental images are multimodal in character: they can be formed in diverse sensory-perceptual modalities such as touch, seeing, hearing, inner body feelings, and so on (Damasio 1999: 9). Mental images are the ways in which consciousness constructs the world as phenomena which are experienced by the self. They may be described as the experiential contents of consciousness. The self does not directly experience the object. Instead, its experience is mediated by the mental images which the brain constructs of the objects of experience, along with the conceptual categorizations of these. 6.2 The Interactive Relation ofSelf to the Object: Proto-interpersonal Meaning Without repeating the arguments discussed earlier, a second crucial aspect of Damasio's account consists of the self's interactive relationship with the object of consciousness. As I shall show here, this aspect of the self-object relationship may be seen as proto-interpersonal in character. In this perspective, self and object are both differentiated from each other at the same time as being placed in an interactive relationship with each other (see section 3, pp. 241-6). The phenomenological experience of core consciousness is an awareness (reentrant loop) of our doing/being-in-the-world as an undivided, pre-polarized whole. We have selfawareness not of the self per se, but of the self-in-interaction-with-nonself (e.g. other people and objects). Furthermore, this relationship may be modulated by different embodied feelings which the self has towards the object; it may also have implications for the specific courses of action which the self decides to undertake in relation to the object. In core consciousness, the distinction between self and object corresponds to a different kind of differentiation with respect to the experiential kind discussed in the previous section. In the present case, it is the interactional distinction between the self and the 'object' (the nonself, the Other, a second) with which the self interacts and to which the self orients. This may be explained in relation to mental images as follows. A mental image is a neural construal of some object or event either in the world outside the organism's body or within the organism's internal milieu (see above). With respect to this image, the self stands in a particular perspectival relationship to it. The self is not a part of the whole which is construed by the mental image. Rather, it is distinct from the mental image; it stands in some kind of interactive relationship with it. What is important here is the relationship which the self has with the mental image or the object which the image constructs. From this point of view, the self can orient to the object in terms of the course of action the self intends to take with respect to the object. The self can also experience subjective feelings and bodily sensations in relation to the object. Furthermore, the self can acquire knowledge of the object and, therefore, incorporate the object to a framework of beliefs and expectations which are stored in memory for activation on other occasions in extended consciousness.

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 259 Core consciousness yields an experiencing of self-in-world where there is not yet a typological-categorial separation of self from world. Instead, there is a first soft polarization (weak classification) whereby self and nonself are not sharply or totally distinguished from each other. Instead, there is a relationship between the two which is based on topological-continuous variation; self and nonself are continuous with each other. There is topological variation without typological distinction. This provides a first, most general basis for the subsequent emergence of the many more differentiated and specified relationships with increasingly more separate others on higher integrative levels, particularly the symbolic level. The interactive relationship between self and object in core consciousness can be seen as a less specified interactional-affective relational holism from which the semiotic resources of linguistically mediated interpersonalinteractional meaning ultimately derive. The interpersonal semiotic resources of language symbolically constitute further typological differentiations between, say, T, 'you', and 'she', 'it', and so on, though in ways which do not totally transcend their basis in more primitive, more holistic, more iconic forms of self-object relations in core consciousness. Damasio (1999: 36-7) points out that consciousness is rooted in and modulated by inwardly directed feeling states of the body which precede emotion and may not always be conscious. These feeling states of the body can, however, be displayed as socially recognizable emotions. This observation can help us to forge a link between the interpersonalinteractional semiotic resources and the feeling states of the experiencing body in semiosis. Damasio (1999: 79) argues that emotions are represented by a system of neural dispositions located in a number of brain regions, notably the 'subcortical nuclei of the brain stem, hypothalamus, basal forebrain, and amygdala'. The activation of these emotional dispositions, Damasio continues, leads to a number of consequences. Following Damasio closely, these may be summarized as follows. First, a given pattern of neural activation yields a neural representation of a particular emotion which can then become an object of consciousness awareness. Secondly, this same pattern of neural activation induces changes in body state as well as in the state of other brain regions. It is this which creates an emotional state which is palpable to an observer, who is able to interpret the individual as being emotionally engaged with some object. Thirdly, the internal state of the organism is altered inasmuch as it now has an emotion as a neural object of consciousness and at the same time senses a change in the body's feeling-state provided that the neural pattern of activation crosses the threshold of consciousness. Damasio further argues that the neural patterns which form the substrate of feeling-states of the body bring about 'two classes of biological changes' (1999: 79, viz. 'changes related to body state and changes related to cognitive state' (1999: 79). Damasio explains changes related to body state as follows: The changes related to body state are achieved by one of two mechanisms. One involves what I call the 'body loop'. It uses both humoral signals (chemical messages conveyed via the bloodstream) and neural signals (electrochemical messages conveyed via nerve pathways). As a result of both types

260

BRAIN, MIND, AND THE SIGNIFYING BODY

of signal, the body landscape is changed and is subsequently represented in somatosensory structures of the central nervous system, from the brain stem up. The change in the representations of the body landscape can be partly achieved by another mechanism, which I call the 'as if body loop'. In this alternate mechanism, the representation of body-related is created directly in sensory body maps, under the control of other neural sites, for instance, the prefrontal cortices. It is 'as if' the body had really been changed but it was not. (Damasio 1999: 79-80) Changes related to cognitive state are summarized by Damasio as follows: ... (1) the induction of specific behaviors such as those aimed at generating bonding, nurturing, exploration, and playing; (2) a change in the ongoing processing of body states such that the body signals filtered or allowed to pass, be selectively inhibited or enhanced, and their pleasant or unpleasant quality modified; and (3) a change in the mode of cognitive processing such that, for example, the rate of production of auditory or visual images can be changed (from slow to fast or vice versa) or the focus of visual images can be changed (from sharply focused to vaguely focused); changes in rate of production or focus are an integral part of emotions as disparate as those of sadness or elation. (Damasio 1999: 80) It is not difficult to see in the above proposals by Damasio that the interactionalaffective holism that I mentioned above constitutes the first level in a specification hierarchy whereby embodied feeling-states and affective orientations, along with their corresponding cognitive states, are integrated in the brain such that the neural images which form the objects of consciousness are simultaneously construed as having both experiential-cognitive content and interactionalaffective values and orientations. The patterns of neural activity associated with these dimensions are tightly cross-coupled as the self interactively relates to specific objects of consciousness and in ways which modulate this relationship. The three factors which Damasio singles out in his discussion of changes in cognitive state (see above citation) suggest that the self can (1) select from a repertoire of possible actions for relating to an object of consciousness; (2) modulate its affective orientation and feeling-state towards the object; and (3) modulate the object itself so as to alter the ways in which the self emotionally orients to the object. Thus, self, object or the interactional relations between them can all be modulated and re-shaped in ways which channel and direct the workings of consciousness and the action trajectories which flow into the environment as a consequence of this. 6.3 The Grounding ofSelf in its Internal Milieu: Proto-textual Meaning The most fundamental kind of grounding concerns the distinction between what is 'in' the structure of the organism and what is 'out' in its environment. The biological maintenance of the self is grounded in the relative structural invariance - the relative stability - of the body over time so that this invariance 'can dispense continuity of reference across long periods of time' (Damasio

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 261 1999: 135) to the self. At the level of core consciousness, the self may be said to be grounded in the relative stability afforded by the body, which functions as the immediate environment of the self's time-bound trajectory. Damasio points out that representations of the body 'come from the remarkable invariance of the structures and operations of the body' (1999: 141). Body representations signify this stability for the neurally realized self in the brain throughout the life of the organism. It is the somata-sensory system and somatic signalling which functions to send information to the brain about the state of the various parts of the body. Some of this signalling occurs via nerve fibres that convey signals from the body to the central nervous system; other forms of somato-sensory signalling use chemical substances carried by the bloodstream (Damasio 1999: 149). Damasio groups somato-sensory signalling into 'three fundamental divisions', viz. the internal milieu and visceral division; the vestibular and musculo-skeletar division; and the fine-touch division (1999: 149). Summarizing Damasio (1999: 150-3), the three divisions are as follows: •

•

•

The internal milieu and visceral division creates moment-by-moment multiple maps of the internal milieu by means of introceptive sensing operations which detect changes in the chemical environment of cells in different parts of the body. This division is also concerned with the detection of pain via nerve pathways. It also maps the state of the smooth muscles throughout the viscera which are under autonomic control; The musculoskeletal division 'conveys to the central nervous system the state of the muscles which join moving parts of the skeleton, that is, bones' (1999: 153). These muscles are under the control of the will and this division of the somato-sensory is largely concerned with proprioception or kinaesthesis. Again, this division forms multiple maps which are 'placed at multiple levels of the central nervous system, all the way from the spinal cord to the cerebral cortex. The vestibular system, which is responsible for mapping the body's spatial orientation, is the highest-order mapping of somato-sensory information; The fine-touch division 'describes the alterations which specialized sensors in the skin go through when we make contact with another object and investigate its texture, form, weight, temperature, and so on' (Damasio 1999: 153). In contrast to the first two divisions, which are concerned with internal states, this division is concerned with external objects as specified by the haptic information which is picked up by the body's surface.

We can see here how at this first level of specification, the self of core consciousness is shown to be a self-referential perspective which collects some of the effects of its own cascading (Salthe 1993: 158). In so doing, the self comes to reveal itself through the mapping of somato-sensory information from the bodily environment in which it is grounded and which constitutes a second with which the self interacts. Following the principle of self-as-firstness, this process of selforganization builds up patterned arrangements whereby the self is experienced as different from, yet grounded within, the milieu which is constituted by the body. Only in this way can the self come to recognize and interact with other

262

BRAIN, MIND, AND THE SIGNIFYING BODY

selves because self-organization entails consistency of behaviour relative to a viewpoint, as embedded in a system of interpretance. The stability of the bodyas-environment-for-the-self provides, by means of somato-sensory mappings, a basis on which the self can model itself. Self-organization thus preserves a textured wholeness or unity in the core self's experience of its own body and of the body's experience of the outside world. This grounding of the core self in the body is founded on a primordial sense of iconic wholeness: self grounded in, yet experienced as continuously varying from, though not categorically distinct from, the stable environment afforded by the body. Interestingly, the three divisions described by Damasio illustrate a progressive (topological-continuous) differentiation from a less differentiated whole, as seen in the gradual move from the introception of the first division to the proprioception of the second division and the exteroception of the third division. As we shall see, this sense of the experiential unity or wholeness of self-in-relation-toits-body, experienced as the ground of its stability and continuity in time, is a necessary forerunner of the semiotic wholeness and unity that is embodied in the notion of textuality. Again, we see from the proto-textual perspective of the self's iconic grounding in the body that core consciousness emerges when the reentrant connectivity of neural groups creates the me ta-loop that closes the loop on the 'self' pole of awareness. That is, the reentrant connectivity of neural groups created by mappings of somato-sensory signals leads, in core consciousness, to the closing of the loop on the self's experience of the embodied milieu in which it is irrevocably grounded. From the perspective of the specification hierarchy, the textuality of symbolic semiosis is integrated with a hierarchy of iconic and indexical levels of proto-semiotic grounding, beginning with the core self's grounding in the body. 6.4 The Recursive Nature of Consciousness: Proto-logical Meaning The ability to link 'scenes' comprising objects and events in primary consciousness to each other depends on the animal's ability to connect these objects and so on through memory to its previous value laden experience (Edelman and Tononi 2000: 108). Each such remembered scene qua mental image is available to conceptual categorization, which means that the given scene can be thought about. In this way, the perceptual experience is available to thinking in the form of pre-linguistic conceptual categorizations. It is important to remember that we are not dealing with language at this stage. Carruthers (1996: 194) points out that perceptual information about some object is held in short-term memory such that the information so stored is available to further reflexive thinking. That is, short-term memory is a form of record which enables each thought to be re cursively fed back to the same short-term memory store so that each thought is available so that it may be consciously thought about in its turn by further thoughts (Carruthers 1996: 194). To paraphrase Carruthers, consciousness is the reflexive making available to (conscious) thought of thoughts which, in their turn, are made available to further thought, and so on. The short-term memory store qua record of each thought's being fed back to the same memory store so that it is available to be thought about in turn may be seen as a precursor, at the lower integrative level in mental life I proposed by Edelman and Tononi

THE METAFUNCTlONAL CHARACTER OF CONSCIOUSNESS 263 (2000: 203-4), of the semiotic notion of text as the products and records of symbolic forms of meaning-making, including symbolic thinking, or inner speech, in the case of language. In this view, my having a conscious experience of the pen lying on the desk as I write these words is based on my ability to make this experience of the pen available to further thinking. So, I can see the pen lying on the desk, then have the thought, That pen is the wrong colour', then think about that thought with the further thought, 'I must buy a blue one', and think about that further thought with the still-further thought, 'I'll go to the stationery shop in the morning', and so on. According to Carruthers's (1996: 195) account, which I am following closely here, each of these thoughts is reflexively fed back into shortterm memory so that each thought is, in its turn, made available to further conscious thinking. The (invented) sequence just described may be schematized as follows: [CURRENT PERCEPTUAL EXPERIENCE OF PEN]/I[THAT PEN IS THE WRONG COLOUR]/I[I MUST BUY A BLUE ONEJ!I[I'LL GO TO THE STATIONERY SHOP IN THE MORNING] What emerges in this re-constructed sequence is the way in which each thought that is reflexively fed back into short-term memory so that it may, in turn, be thought about is linked to the preceding thought by principles of causal, temporal, and other modes of logical (temporal, causal, consequential, and so on) connectivity such that a series of thoughts are, for example, logically linked to each other in an overall process of reasoning about some phenomenon. In this way, short-term memory serves as a kind of proto-entextualization of conscious thinking in the sense that it (memory) enables thoughts to be linked to each other in meaningful ways. We can see here a pre-linguistic analogue of the notion of logico-semantic expansion in natural language. Each thought that is reflexively fed back into the memory store expands the meaning of the previous thought(s) in the overall series. Moreover, the fact that the whole series of thoughts is held in memory suggests the parallel (on a lower integrative level, as I said before) with the unfolding development of text in natural language. Memory is the means of ensuring that the whole series can be thought about as it unfolds in time, rather than each thought disappearing forever as soon as it is thought. This also means that the reflexive addition of further thoughts to the previously existing store has the potential to modify the significance of previously held thoughts as they are consciously thought about by new thoughts that are fed back into the memory store. What memory does is provide a means of integrating the temporally unfolding series of thoughts as a larger whole which itself, within limits, can be reflexively accessed and consciously thought about. Memory is not just a store against the ravages of time, whatever the timescale involved. Rather, it is a principle of integration whereby each thought that is reflexively (recursively) deployed to think about the prior thoughts and the relations between these becomes part of a larger whole which is consciously attended to and reflected on. As we shall see in section 12, the notion of text can be seen as a further

264

BRAIN, MIND, AND THE SIGNIFYING BODY

specification and reorganization of this property on the symbolic level. This further entails that the possibilities for the integration of diverse space-time scales are massively enhanced as compared to the small-timescale short-term memory store discussed by Carruthers.

7. Integrating the Interaction System and the Meaning System Perspectives on (Self)Consciousness In my view, Damasio's neurobiological proposals concerning the central role of bodily feeling-states and emotions in consciousness are on the threshold of a more encompassing ecosocial discourse which can show how embodied feelingstates, inner sensations, and primary emotions can be integrated with the social meaning-making practices of some community. Damasio's neurobiological theory of consciousness, like Edelman's, is based on neurological and other processes at the level of the individual biological organism. The self-object interactional relationship that is central to Damasio's theory of consciousness is concerned with the relations between neural events in the brain, electrochemical processes linking brain to the internal milieu of the body, and so on. That is, consciousness is grounded in a complex of physical, chemical, and biological events and relations within the hyper-complex system of the human body-brain. From this physical-material perspective, the body-brain participates in exchanges of matter, energy, and information with its internal and external environments. It was for this reason that I said earlier that consciousness is always founded on larger-scale supersystem transactions. Furthermore, these exchanges extend beyond the skin of the body out into the ecosystem which the organism inhabits. In Damasio's neurobiological perspective, consciousness is a physical process linked by physical, chemical and biological relations. This is the view which Lemke (1984b: 113-21; 1995c: 166-75) describes as the Interaction System view of a given ecosocial community. That is, the entire complex of physical, chemical, biological and ecological processes and relations which constitute the community and which maintain its physical integrity and survival in time. The individual qua biological organism is a relatively small-scale entity within this larger-scale complex of ecosocial relations. From the perspective of the individual biological organism, the neurobiological dynamics of consciousness described by Damasio constitute the ground of a higher-scalar Meaning System (see Lemke 1984b: 113-21; 1995c: 166-75) whereby the physical-material, chemical, and biological processes of the Interaction System have meaning for us. Meanings are immanent in and are constructed in and through the physical-material resources and matter-energy flows of the Interaction System. Thus, we use and entrain patterns of sound, body movement, information in the ambient optic array, and so on, in accordance with the recognized patterns of meaning-making in a given community, subcommunity, and so on. As Lemke explains, Interaction System processes and dynamics are selectively contextualized by the Meaning System, and at the same time the Meaning System is embodied in and constituted by Interaction System processes and dynamics. Interaction System processes and dynamics include the

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 265 individual body-brain system and the way this is cross-coupled to other bodybrain systems and to physical-material processes in the environment, including the larger-scale material ecosystems in which these physical-material processes are embedded (Lemke 1995b: 118-25; Salthe 1993: 19-20). I said earlier that current discussion of consciousness often places a lot of emphasis on the singularity and uniqueness of individual consciousness. From the Interaction System point of view where the neurobiological processes and dynamics of the individual body-brain are focal, this is legitimate - up to a point. Thus, the neurally realized mental images in the individual's brain, the self's particular perspective on these, the memory-based conceptualizations of these images and their re-elaboration in terms of past and future memories, the inner body feelings and sensations that cross-couple with and help to entrain the self's experiential knowledge and understanding of mental images and so on, are all grounded in the individual diversity of the self-organizing reentrant neural mappings whereby a sense of self and individual consciousness is individuated along a historical-biographical trajectory. Importantly, this lower-scale individual diversity also constitutes an important source of potential bifurcations at higherscalar levels of organization. The brain is constantly self-organizing its own neural connections as well as generating new neurons (neurogenesis). These brain processes are part of the Interaction System which grounds all the possible meanings of a community, where Meaning System processes are differentially distributed over the diverse body-brains in the Interaction System (see section 12, pp. 277-80). Both the constant reorganization of the brain's neural connections and the processes of neurogenesis mean that the enormous diversity of such connections possible in individual consciousness yields proto-meanings which are not necessarily typed or recognized by the contextualizing relations of the culture's meaning systems. In other words, not all of the 'experiences of experiencing' that enter into core and extended consciousness, not all of our subjective experiences of our internal milieu and our external environments, not all of our perceptions of our inner feeling-states and bodily sensations are necessarily recognized as meaningful in our culture's meaning system at any given moment. Many of these experiences remain as proto-meanings within individual consciousness. But because social meaning systems are themselves constantly changing, though on a slower timescale than those of the much faster timescale of individual neurological processes, there also exists the potential for protomeanings constructed in individual consciousness to be assimilated to the higher-scalar meaning system of the ecosocial system on the basis of individual and social innovations in meaning-making practices. That is, newly emergent Interactional System processes on the scale of the individual's neurobiological dynamics may be assimilated into the higher-scalar system of interpretance on the ecosocial scale. This is possible, I argue, because consciousness is not concerned with neurological processes per se, but with the contents of consciousness - the mental images and the memory-based relations among these - in the perspective of a self. These contents are proto-meanings, as mentioned earlier. Neurological processes realize the contents of consciousness, but are not themselves accessible to consciousness. Proto-meanings in

266

BRAIN, MIND, AND THE SIGNIFYING BODY

individual consciousness can, therefore, become objects of further elaboration and reflection, to the point where they can be connected to the symbolic meaning-making resources of higher-order consciousness and further entrained to its dynamics. From what we now know about the brain as a self-organizing producer and elaborator of (proto-)meanings in response to continually changing contextual factors both within the individual's body-brain and in the external environment, I would argue that the proto-meanings of individual core and extended consciousness belong to a specification hierarchy such that there is a continuum of possibilities from the least specified iconic meanings of core consciousness to the most specified symbolic meanings of higher-order consciousness. Proto-meanings are not constructed out of nowhere; they, too, are constructed in response to feedback loops with other systems both within the brain and the internal milieu of the body as well as in the external environment (context of situation and context of culture). That is, they, too, are contextually constrained. Positive feedback from all of these contextual factors may take the system far from equilibrium so that new forms of meaning and organization emerge. Each level of consciousness in the specification hierarchy entails increasing differentiation of the neural dynamics realizing mental images, higher-order conceptual structures and thoughts, and symbolic meanings. Each level in the hierarchy imposes its own constraints on the overall hierarchy such that proto-meanings may be entrained to and reorganized by the neural attractors which embody lexicogrammar in symbolic consciousness. It is important to point out here that the non-reducibility of proto-meanings to neurological processes indicates that proto-meanings are realized by neural activity in ways which are analogous to the realization of the content stratum of language (its semantics and lexicogrammar) by the expression stratum (phonology or graphology). Language is, of course, a full-fledged social semiotic system (Halliday 1978a); proto-meanings in the brain are not. However, the principle of realization applies in both cases, though not in exactly the same way. The difference lies in the fact that the same mental image (content) can be variably realized by different patterns of neurological processes (expression) whereas the semantics and lexicogrammar on the content stratum of natural language are constrained by the phonological or graphological patterns of organization on the expression stratum, and vice versa. Nevertheless, this does not amount to an all-or-nothing distinction. Phonemes, syllables, and other phonological units do not exist as discrete units in the acoustic stream of speech. There is, in fact, considerable merging of the acoustic energy for a given unit with the energy of other units (Handel 1989: 159). Moreover, the same phonemic segment can be realized by different acoustic cues in different contextsjust as the same acoustic cue can realize different phonemes in different contexts (HandelI989: 159). So, there is more variation here than what speakers and listeners are usually aware of, though this variation does not, for the most part, prevent speakers and listeners from picking up the relevant acoustic information from the articulatory gestures. Doubtless, this is so because the sensori-motor and neural activities involved in the individual's linguistic activity are entrained to the dynamics of higher-scale attractors on the ecosocial scale such as the phonological categorizations of a

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 267 particular language system. The study of neurobiological processes in the bodybrain may be able to explain the dynamics of proto-meaning in core and extended consciousness as emergent properties of neurological processes and neuroanatomy. However, it will remain unable to explain the emergence of symbolic consciousness in the individual. Any theory which seriously attempts to break with in-built language-acquisition devices, bio-programs or language instincts will also need to show that language and other social meaning-making resources exist as dynamically structured attractors at the ecosocial semiotic level. This cannot be explained as a mere bottom-up aggregation of individual bodybrains because it has its own principles of organization on the higher-scalar level. Whereas the multiple realizability of proto-meanings by neurological processes in the brain is entrained by lower-level neural dynamics, symbolic consciousness is also entrained by the collective dynamics of entire systems of meanings and their associated practices at the cultural level. Proto-meanings in the individual brain have the potential to become meaning in this semiotically more specified sense. Because meanings are higher-scalar and collective phenomena, their effects across many diverse space-time scales are much greater than are the proto-meanings that are confined to an individual's brain. This is so because symbolic meanings, which are cultural, are potentially accessible and sharable by large numbers of individuals such that they can affect large-scale ecosocial processes across very diverse space-time scales (Lemke 2000a: 191-3). As we shall see in the following section, recent research on mirror neurons suggests ways in which the individual's dialogically co-ordinated actions are entrained to the higher-scalar dynamics of the dyad which organizes self-nonself transactions.

8. Minding the Gap between Minds: Mirror Neurons and Interpersonal Meaning The dialogic basis of (proto)-semiosis receives support from neurological evidence concerning the existence of mirror neurons in both monkeys and humans (see Rizzolatti and Arbib 1998). Mirror neurons, which are located in the rostral part of the ventral premotor cortex of monkeys, discharge during active movement of the hand or mouth. Transcranial magnetic stimulation and positron emission tomography (PET) experiments also indicate the existence of mirror neurons in humans, including Broca's area. Rizzolatti and Arbib discovered that this neuronal discharge correlates with intentional action, 'rather than with the individual movements that form it' (1998: 188). On the basis of their findings, Rizzolatti and Arbib explain the existence and functioning of mirror neurons as follows: The proposal that we and others have advanced is that their actIVIty 'represents' actions. This representation can be used for imitating actions and for understanding them. By 'understanding' we mean the capacity that individuals have to recognize that another individual is performing an action, and to use this information to act appropriately. According to this view, mirror

268

BRAIN, MIND, AND THE SIGNIFYING BODY

neurons represent the link between sender and receiver that Liberman postulated in his motor theory of speech perception as the necessary prerequisite for any type of communication. (Rizzolatti and Arbib 1998: 189) and Our proposal is that the development of the human lateral speech circuit is a consequence of the fact that the precursor of Broca's area was endowed, before speech appearance, with a mechanism for recognizing actions made by others. This mechanism was the neural prerequisite for the development of interindividual communication and finally of speech. We thus view language in a more general setting than one that sees speech as its complete basis. There is obviously an enormous gap between recognizing actions and sending messages with communicative intent. We offer now a hypothesis ... on how this gap might have been bridged. Whether an individual is about to perform an action or observes another individual performing an action, premotor areas are activated. Normally, a series of mechanisms prevents the observer from emitting a motor behavior that mimics the observed one, and the 'actor' from initiating the action prematurely. In the case of action observation, for example, there is a strong spinal cord inhibition that selectively blocks the motor neurons involved in the observed action execution (L. Fadiga, pers. commun.). Sometimes, however, for example when the observed action is of particular interest, the premotor system will allow a brief orefix of the movement to be exhibited. This prefix will be recognized by the other individual. The fact will affect both the actor and the observer. The actor will recognize an intention in the observer, and the observer will notice that its involuntary response affects the behavior of the actor. The development of the capacity of the observer to control his or her mirror system is crucial in order to emit (voluntarily) a signal. When this occurs, a primitive dialogue between observer and actor is established. This dialogue forms the core of language. The capacity to notice that one has emitted a signal and associating it with changes of the behavior of others might or might not have developed simultaneously. However, there is no doubt that, once established, this new association should have yielded enormous benefits of adaptive value for the group of individuals that started to make use of it, providing the selective pressure for the extension of communicative capacities to larger groups. (Rizzolatti and Arbib 1998: 190-1) The existence of mirror neurons indicates that there is a neurological basis to the enacting of interpersonal relationships between addresser and addressee. The findings of Rizzolatti and Arbib show that dialogically co-ordinated intentional activity entrains neuromuscular activity. This also corresponds to the distinction made by Peng (1994) between motor functions, which are linked to linguistic processes, and motor activity, which is not. The point is that higherscalar semantically organized intentions entrain motor functions to their dynamics (see also chapter 1, section 1, pp. 3--8). Moreover, this entraining is not confined to the individual organism; instead, it results from the ways in which

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 269 individuals are integrated to the essentially dyadic supersystem structures which regulate interpersonal transactions between individuals. Rizzolatti and Arbib (1998: 192) somewhat confusedly apply Fillmore's case grammar in order to derive imperative and declarative case structures, which they qualify as 'action structures', rather than linguistic representations. Given that case structures in Fillmore's account are concerned with the experiential domain of (linguistic) meaning, it is more appropriate, from a linguistic point of view, to relate the imperative and declarative action structures to the mood system of the interpersonal grammar. In Halliday's account, mood is centrally concerned with the enacting of dialogic, interpersonal relationships between addresser and addressee. Mood is one of the interpersonal grammatical systems which indicate the interactional status of utterances; choice of mood integrates the clause into an interactive context of speakers' goals and intentions. Declarative and interrogative moods express propositions. These two moods hold the proposition in their scope and modify it so as to indicate whether the proposition is being asserted by the speaker or interrogated (McGregor 1997: 214). Oblative and imperative moods express proposals for action, rather than propositions. Proposals cannot be argued about, asserted, believed, or denied in the way propositions can. In proposals, the mood operator holds a proposal in its scope and modifies it accordingly so as to indicate how the proposal is to be interpreted or responded to interactively, i.e. as oblatives, in which the addresser is the intentional source of the proposed action, or imperatives, in which the addressee is selected (by the addresser) as the one who is required to carry out the action desired by the addresser. Different languages may express mood in different ways. English uses the syntagmatic combination of the functions Subject and Finite (declarative and interrogative) or their absence (imperative) to indicate the mood of the clause; other languages may use a morpheme of some kind, or intonation (McGregor 1997: 219). The mood marker, however it is expressed grammatically, acts as an interpersonal operator which holds the entire clause in its scope and modifies it according to the interactive meaning the speaker intends it to have. Mood is a grammatical category which has a specific interpersonal meaning, as discussed above. The grammatical meaning of the various mood categories in a specific language does not exhaust the meaning of the mood choices in specific contexts. In specific contexts of use, the abstract mood categories interact with other lexicogrammatical categories as well as with features of the context so as to yield a context-specific semantics. The neurological evidence provided by research on mirror neurons suggests that the interpersonal modification of clauses through the making of different selections from the mood system is the construction in the interpersonally organized neural space of an attractor which is placed just a little ahead of both the executed action by the addresser and its recognition by the addressee. In discourse, the formulating of an intentionally sourced and directed interactive move according to mood choice thus operates as a top-down control operator (Juarrero 1999: 192) which entrains the neural and motor activity of both addresser and addressee in the dyad of the temporally unfolding interpersonal exchange. Thus, neural, motor, linguistic, and physical-material factors may all

270

BRAIN, MIND, AND THE SIGNIFYING BODY

become entrained to the semantic organization of the intentionally directed interactive move. Rizzolatti and Arbib's discussion of imperative and declarative action structures, in spite of their confusion between experiential case grammar and the interpersonal-interactive nature of mood, shows how both addresser and addressee are entrained by the intentional character of the act. The intentional nature of this interpersonal semantic attractor enacts a trajectory through semantically organized neural space so as to ensure that the resulting actions of both parties to the interaction are entrained by the dynamic organization that is embodied in the semantics of the intentionally sourced interactive move. HaIIiday's research in child language development has clearly demonstrated that meaning is first enacted in interpersonal contexts before being construed experientially (1993: 103). In the emerging adult language system, the two kinds of meaning gradually come together such that interpersonal selections simultaneously configure with experiential ones in the grammar of the clause. It is the configuring of options in meaning from a number of different metafunctional regions that yields the semantics of a particular clause. This cannot be fully explained in terms of any single metafunction in isolation from the others. Intentionally sourced and directed interactive moves in discourse are, then, top-down boundary conditions which are assembled from self-organizing metafunctional constraints which act on lower-level neural and motor activity so as to produce a particular act-token. This semantic space functions as an attractor which entrains bodily (neural and motor) behaviour as well as selected aspects of the physical-material world along a particular goal-seeking pathway. It is not difficult to see at this point how the taking up and negotiating of interpersonal meaning in discourse by social agents is concerned with the enacting and construal of a dynamical system's operators whereby participants negotiate a trajectory from one point in semantic state space to another. The interpersonal enacting and negotiating of meaning thus embody top-down boundary conditions that lead participants through specific trajectories within this semantic space. The fact that interpersonal and experiential selections are brought together in lexicogrammatical form thus provides a way of explaining how the intentionally sourced dialogic act or move constitutes a strategy for bringing some selected aspect of the world into line with one's experiential construal of it. In imperatives, for example, it is the dynamical organization of context-dependent constraints which seeks to make the actional outcome conditional on the intentional source of the imperative clause. The uttering (by the addresser), negotiation (by both addresser and addressee), and consequent execution of the desired action (by the addressee) is a trajectory, whose finalization, to use Bakhtiri's expression, embodies the experiential content of the intention from which it (the trajectory) has its source, and which flows through the entire system of constraints to ensure the completion or resolution - both semiotic and material - of the action trajectory. Rizzolatti and Arbib have pointed out the reciprocally controlling nature of mirror neurons in the recognition and execution of intentional acts (see above). The stratified nature of language as a semiotic system means that voluntarily emitted vocal, gestural, and other motor signals on the expression stratum are

THE METAFUNCTlONAL CHARACTER OF CONSCIOUSNESS 271 constrained by the metafunctional dynamics of a self-organizing semantic space on the content stratum. It therefore follows that there is no need to see intentions as separate from and prior to action. Peng's (1994) important distinction between motor functions and motor activities (see chapter 4, section 15, pp. 217-23) means that semiotically significant bodily actions or motor functions, such as articulation in the act of speaking, as distinct from nonintentional or involuntary body movements, entail action trajectories that integrate the motor functions (not activities) to the contextual, semantic, lexicogrammatical, and expression stratum levels of neuronal organization of language in the brain. In this way, motor functions are entrained into the selforganizing dynamics of a metafunctionally organized semiotic space which directs and constrains the always embodied meaning-making trajectories of social agents and their actions along a specific trajectory (section 2). In the following two sections, I shall explore some of the ways in which inner speech is no exception to the considerations made in this and previous sections of this chapter.

9. Inner Speech as Linguistically Realized Higher-order Thinking Inner speech is linguistically realized thinking (see also Kinsbourne 2000). This does not mean that language translates prior, non-linguistic thoughts into language. Rather, inner speech is linguistically constituted thinking. It is a form of language deployment which has been specialized to the inner realm. Moreover, inner speech, as Carruthers (1996: 225) points out, is a form of conscious thinking through language. Inner speech is language which operates without the constraints of the external stimulus flux. This means that verbalization in the inner realm takes place without activating the motor routines that are required for the articulation of, for example, speech sounds. However, this does not mean that inner speech lacks an expression stratum. Both expression and content are constitutive of inner-speech speech just as they are of speech which is implemented by vocal-tract articulatory activity as speech sounds. Langacker is one modern linguist who recognizes the psycho-perceptual nature of internal language processes (see also Saussure 1971 [1915]: 98). From the perspectives of both articulation and audition, there are cognitive routines that direct both the production and the reception of speech sounds. These cognitive routines are directed by stored acoustic images which serve to categorize speech sounds as being instances of this or that phonological type. Langacker points out that the acoustic or auditory image, rather than the physical sound, is primary (1987: 79). In this way, listeners have two possible ways of perceiving speech sounds. The listener may actually hear the speech sound as an acoustic event, or he or she may imagine hearing it by the activation of an acoustic image of the sound in silent verbal thought (Langacker 1987: 78-9). Analogously, the speaker may actually implement an articulatory routine and utter a given speech sound, or the speaker may imagine doing so. In the latter case, this means that the speaker 'can mentally run through the motor routine without this mental activity being translated into muscular gestures' (Langacker

272

BRAIN, MIND, AND THE SIGNIFYING BODY

1987: 79). Acoustic images direct and control auditory and articulatory routines in both inner and outer speech. Acoustic and auditory images are invariants that have been extracted from the stimulus flux such that speakers and listeners are sensitized to these invariants and can operate these without the constraints of sensori-motor activity. Acoustic and auditory images are, then, phonological invariants which are de-coupled from the muscular activity involved in articulating and/or listening to speech sounds. In any case, inner speech, whether imaged in consciousness as spoken by one's self or heard as the voice of another, is, nevertheless, stratified: it has both expression and content. In the case of inner speech, it is the expression plane, comprising acoustic and/or auditory images which enable the motor routine to be activated mentally without, however, translating it into the muscular gestures responsible for the production of actual speech sounds. Inner speech is always heard as an imagined auditory sequence for this reason. This shows how, in both inner and outer speech, motor routines are governed by symbolic neural space (Peng 1994: 113-14). It is in this way that higher-order consciousness instantiates internalized linguistic activity (inner speech) as objects of consciousness which can be reflected on in the perspective of a self. Just as in outer speech, the meta-redundancy relations between the expression and content strata mean that sensori-motor routines and conceptual ones mutually recognize each other by virtue of the brain's functioning as a somatic recognition system (chapter 3, section 13, pp. 162-8). The stratified character of inner speech means that internalized linguistic tokens can be attended to as objects of (symbolic) consciousness which are both 'heard' and understood in the inner realm of imagination. Moreover, there is always a self which interacts with and relates to inner speech as object of consciousness. In other words, inner speech is always grounded in the perspective of the self who relates to inner speech as one's own linguistically mediated thoughts (section 11).

10. The Metafunctional Character of Inner Speech as Linguistically Constituted Thought The metafunctional character of inner speech makes it clear that inner speech, like outer speech, does not simply translate non-linguistic thoughts and mental images into the medium of language. Language is simultaneously both 'action' and 'reflection' in both its inner and outer manifestations (see chapter 4, section 4, pp. 184-9). The constitutive role oflanguage in symbolic thinking also means that it is not quite accurate to restrict the definition of thinking to the 'reflection' dimension of linguistic meaning. Typically, thinking is discussed in terms of its 'ideational', 'conceptual', or 'representational' characteristics. But if inner speech is a form of semiosis which has been specialized to the inner realm of the 'mind', then we shall need to reconsider the limited notion of thinking as ideation which has prevailed. As we shall see in the following paragraph, this requires us to take into account the full range of meanings that are characteristic of both inner and outer verbal activity. This also requires us to reconsider the status of less specified forms of thinking that occur on lower integrative levels

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 273 such as the conceptual thinking associated with primary consciousness (Edelman 1992: chapters 10 and 11). Rather than saying that 'thinking' on whichever integrative level is something which is private and non- or presemiotic, it is logically more appropriate to say that the symbolic thinking constituted by language in inner speech is a more specified subclass of internalized semiosis that includes less specified subclasses (of semiosis) on lower integrative levels. In this view, all forms of thinking are at least proto-semiotic in character. This means that the conceptual thinking characteristic of primary consciousness denotes a less specified form of inner semiosis with respect to the more highly specified forms of symbolic thinking characteristic of inner speech. The discoursal, rather than sentential, character of inner speech requires that the definition of semantics be extended to include all the dimensions of meaning which are present in any linguistically realized discursive event. In the systemic-functional framework, language is a semantic resource for (1) construing the phenomena of experience in and through the language's experiential categories; (2) enacting dialogically co-ordinated interpersonal relations and their associated axiological orientations; (3) construing relations of causality, temporality, consequence, result, and so on, between the figures which comprise a textually realized sequence; and (4) creating coherence both within the text and between the text and its context of situation. Linguistically realized thinking in inner speech is, then, fully metafunctional in character. Thus, linguistic utterances that are either heard as the voice of some other or articulated as one's own voice in the inner realm can play an executive function in planning and carrying out particular courses of action, in the way originally described by Vygotsky (1986 [1934], 1987 [1934]). In this view, there is no need to oppose the view that language serves above all for the 'exchange of information and the interpersonal coordination of action' to the view that language 'is itself the primary medium of . . . thought' (see Carruthers 1996: 2). In Carruthers's view, these two characterizations oflanguage correspond to what he designates as the 'communicative' and the 'cognitive' conceptions of language, respectively. According to Carruthers, the two views of language stand opposed to each other. However, the metafunctional character of language shows that both experiential reflection and interpersonal enactment are constitutively inseparable dimensions of linguistically realized thinking. In other words, inner speech is an internalized semiotic process which simultaneously allows for both conceptual (experiential) thinking and propositional (interpersonal) thinking. Insofar as conscious thinking is linguistic in nature, it can therefore be seen to satisfy both of these requirements: it makes use of conceptual categorizations and allows for the taking-up of modalized propositional attitudes. Moreover, it exhibits properties of textual unity and coherence in relation to both its internal (textual) organization and its contexts of utterance. Inner speech just is higher-order or symbolic thought. Moreover, the stratified nature of inner speech means that the utterances that occur in consciousness are heard and/or spoken in the perspective of the self who attends to them as objects of consciousness. That is, the expression stratum of the phonological invariants - the motor routines - that operates without the constraints of the external stimulus flux nevertheless provides an imagined perceptual basis to

274

BRAIN, MIND, AND THE SIGNIFYING BODY

inner-speech events. We imagine them as spoken or heard linguistic tokens. In both cases, they are grounded in the perspective of the self who attends to them as objects of conscious awareness which are immediately accessible to the self as such (see Carruthers 1996: 238). For this reason, tokens of inner speech are objects which are immediately given to consciousness as objects which can be reflected upon. The activation of the imaged phonological or other expression stratum routine in inner speech provides a perceptual basis to inner speech even though this is a form of inner perception of phonological invariants that have been detached from the stimulus flux. Nevertheless, the perceptual systems which detect and filter phonologically salient distinctions provide immediate access to the content stratum (lexicogrammar and discourse semantics) which is realized by imagined phonological routines in inner speech. My use of the term 'imagined' in this context may suggest that there is something less real or less concrete about the existence of these phonological routines in inner speech. Nothing could be further from the truth. Inner speech, like outer speech, is stratified. The notion of stratification is necessary for explaining the relationship between different orders of abstraction in the organization of language. Lexicogrammatical and discourse semantic units and relations on the content stratum are not comprised of phonological units on the expression stratum. Nor is the expression stratum simply a material means for conveying content. Rather, content stratum units and relations are realized by expression stratum ones. The realizational relationship links articulatory processes of the body to discoursal meanings by virtue of the fact that motor routines and conceptual ones mutually recognize each other. In other words, the two strata mutually redound with each other. Inner speech qua object of conscious reflection must be immediately accessible to SELF. This requires, in the first instance, a sensible dimension which is accessible to perception. The brain's functioning as a somatic recognition system thus provides a perceptual basis for inner speech without the constraints of the stimulus flux. Moreover, the phonological routines that are 'imagined' in inner speech show how the bodily basis of in ternalized semiosis is preserved. The point is that all forms of semiosis require some kind of expression stratum which is accessible to and/or produced by sensori-motor activity. There is no content floating free of its expression. Without an expression stratum, the brain would have no means of recognizing the content which is realized by the expression stratum. Once again, the stratified nature of inner verbalization highlights the intimate and internal connection between sensible bodily processes and meaning-making. Moreover, it lends further support to the view that inner speech is a specialized internalized deployment of the same semiotic resources that are used to engage in outer discursive activity. We can see the specification hierarchy at work here as well. We have already seen in chapter 3 (section 9, pp. 139-46) how the brain's functioning as a somatic recognition system relates sensori-motor routines to conceptual representations in primary consciousness. Similarly, perceptual categorization occurs on the basis of the organism's linking multimodal stimulus information which is picked up in its environment through sensori-motor activity to perceptual categorizations that are built up through reentry in the brain. Inner speech, we have seen, occurs on the basis of imaged

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 275 phonological (or other expression-stratum) routines redounding with the appropriate content-stratum ones. The basic principle is the same at all levels. What is different is the way in which each stage is more specified and more explicitly elaborated but, at the same time, with the prior stages not being transcended. Rather, later stages integrate prior ones. A symbolic being that routinely deploys inner speech in higher-order consciousness is still a being which makes use of conceptual and perceptual representations. Each level refers to the emergence of a level of complexity in the development and individuation of the organism along its trajectory. Prior integrative levels (e.g. perceptual and conceptual) refer to proto-semiotic orders of information and categorization that the organism has elaborated along its developmental trajectory on the basis of organism-environment transactions. By the same token, the transition to inner verbal activity means that the individual is cross-coupled to modes of thinking that can only occur through the semantic categories of natural language and the discourse genres in and through which linguistic meanings are organized as recognizable social situations. It makes no sense, then, to say that language simply 'translates' mental images into a publicly expressible and communicable form (cf. Damasio 1999: 107-8), for this is tantamount to saying that language plays no role in the constitution of the categories of higher-order or symbolic consciousness. Instead, the symbolic resources of language mean that consciousness is connected to space-time scales which go far beyond the organismic one. The further implication is that inner speech qua linguistically constituted higher-order thinking is most centrally a question of meaning. Furthermore, inner speech cannot be adequately characterized in terms of sentential forms which express linguistically constituted thoughts. The point is, rather, that linguistically mediated thought is a form of contextualized meaning-making that is specialized to the inner realm. Vygotsky (e.g. 1986 (1934]: 248) first recognized the semantic character of inner speech. The semantic character of inner verbal activity requires us to see that inner speech is a form of text or discourse, though not without its own specific characteristics, as first studied by Vygotsky. It is not comprised of sentences. Lexicogrammatical units realize their discourse semantics; they do not simply comprise them. The highly condensed and elliptical features of inner speech identified by Vygotsky as a high degree of semantic condensation and semantic opacity (Vygotsky 19S6 (1934]: 243-8; Thibault 1995c) may be explained by the fact that the meanings expressed in inner speech are only accessible to the consciousness of the SELF in whose perspective the given occurrence of inner speech is grounded; there is no functional requirement that these meanings be interpreted by others in the public realm. Instead, inner speech is a semiotically mediated means whereby the SELF is able to gain conscious access to its own thinking, in the process achieving self-monitoring, self-control, and self-awareness (see also Carruthers 1996: 247).

276

BRAIN, MIND, AND THE SIGNIFYING BODY

n. The Re-grounding of the Perspectives of Self and Other in Symbolic Consciousness Symbolic consciousness is freed from the kind of indexical tie to a given here-now situation that is characteristic of perceptual awareness (section 3). Linguistically construed objects of symbolic consciousness always entail a SELF who is aware that the object of consciousness is experienced by the SELF. The notion of SELF is defined as a self-referential perspective implicating observational perspectives or viewpoints. As we shall see below, a SELF is implicated in all occasions of semiosis, whether explicitly specified or not by linguistic or other means. The two poles of attention that Gibson identified in the act of perception apply here as well (section 3). Independently of whether the object of symbolic consciousness is spoken, heard, written, or read by the SELF, the linguistic object of consciousness is always grounded in the perspective of a SELF who experiences the object of consciousness. For the same reason, the SELF can 'hear' the voices (the utterances) of others in his or her inner speech. In the case of language, the difference with respect to perceptual awareness lies in the way in which the objects of symbolic consciousness can be re-grounded in the perspective of other selves who can experience the object of consciousness from their own perspective (Deacon 1997: 427; this volume chapter 4, section 2, pp. 173-6). The fact of my speaking or writing a particular instance of language on a given occasion does not mean that the resulting text is uniquely tied to or is uniquely grounded in perspective of the SELF who uttered it. Rather, my interlocutor (listener, reader) re-grounds it in the perspective of his or her self in the process of dialogically orienting to and taking up the particular instance of language. In this way, person deixis provides resources whereby first, second, and third persons and the meanings associated with them can be grounded in the perspective of a given SELF. This means that the experience of a given object of symbolic consciousness is not uniquely indexically tied to the first-person perspective of the person who uttered it. Furthermore, the resources of temporal proximity deixis and modality proximity deixis in the Finite element are not uniquely tied to the perspective of the self who originally produced the particular instance of language (see chapter 3, section 7, pp. 131-2). Again, the addressee re-grounds the temporal or modal orientation that is expressed in the Finite element in terms of his or her own perspective as the one who undergoes the particular symbolic experience. These interpersonal grammaticosemantic resources provide a dialogically co-ordinated frame of reference in which the temporal and modal (attitudinal, evaluative, and so on) orientations of diverse selves can be co-ordinated across different space-time scales. Both temporal and modal proximity deixis do not simply index the addresser's temporal perspective or modal evaluation of some proposition; they also work to organize the addressee's orientation as well. The same applies to the spatial proximity and other forms of deixis that operate in the nominal group. Here, too, the addressee re-grounds the given object of consciousness in the perspective of his own self rather than in that of the person who first uttered or otherwise produced the utterance.

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 277 Person deixis and finiteness ground the symbolic object of consciousness in a dialogically co-ordinated frame of reference. Person deixis enables meanings pertaining not only to the first-person perspective of SELF, but also meanings pertaining to second and third persons to be made relevant to and integrated to the first-person observational perspectives of SELF. The resources of temporal proximity deixis and attitudinal proximity deixis in the Finite element dialogically co-ordinate addresser and addressee in terms of the temporal or modal grounding of the proposition. This means that addressees can re-ground the temporal or modal perspective specified by the Finite in terms of their own perspective. Both of these resources show how, from the point of view of the interpersonal metafunction, there is a dialogic complementarity between the perspectives of SELF and NONSELF. This means that the SELF's awareness that it is the SELF who is consciousness of the given object can be re-grounded in the perspective of some other SELF. The dialogic complementarity of SELF and NONSELF in the exchange process refers to the way in which the interpersonal semantic structure of a given dialogic move, as spoken, written or signed by some addresser, results in a corresponding change in the addressee. Thus, the addressee is internally modified in ways which result in the dialogic coordination of deictic perspectives. The information which is supplied by person deixis and the Finite are principles of order whereby cross-individual dialogic processes integrate individuals to higher-order dialogically co-ordinated frames of reference. At the same time, they also provide individuals with resources for being conscious of being conscious of something in the perspective of a SELF as a locus of (inter)action and intentionality.

12. Text, Social Meaning-making Practices and Higher-order Consciousness The artefactual nature of text-as-material-object can be seen as a further development and specification of the function of short-term memory that I discussed in section 6.4 in connection with the work of Carruthers. The kind of reflexive thinking described by Carruthers as necessary for consciousness is confined to the organismic scale of short-term memory. On the other hand, linguistically mediated discursive events, along with their entextualized products and records, mean that consciousness itself is extended across diverse space-time scales beyond the organismic one in its immediate environment. We saw earlier that in Carruthers's account short-term memory enables each thought to be recursively fed back to the same short-term memory store. In this way, each thought is available so that it may be consciously thought about in its turn by further thoughts. The recursive nature of this process is homologous with the logical metafunction in language. The recursivity described by Carruthers (see section 6.4, pp. 262-4) is a less specified version of the same kinds of logico-semantic resources that construe relations of causality, temporal sequence, and so on, between clausally realized figures in a given sequence. The logico-semantic resources of expansion enable clausally realized figures to be related to each other as sequences of figures (chapter 4, section 6, pp. 195-8). The relations of logico-semantic interdependency that are created between figures provide a

278

BRAIN, MIND, AND THE SIGNIFYING BODY

means whereby each figure of the figures in an unfolding sequence or even the sequence as a whole can be consciously reflected on by further figures as they emerge. Unlike the perceptual information referred to by Carruthers, an entextualized sequence oflinguistic figures is not necessarily limited to the small temporal scale of short-term memory. Carruthers's notion of reflexive thinking shows how even perceptual information on the immediate here-now scale of the body's interactions with its material environment can be implicated in space-time scales that go beyond the here-now. The fact that thoughts can be consciously thought about by other thoughts means that the thoughts and the sequences of thoughts so thought about are embedded in larger space-time scales. The artefactual character of linguistic text represents a massive expansion and elaboration of this possibility. Text can be viewed as material object or artefact; it can also be viewed as a semantic process of making or operationalizing meanings in some context. The dual status of text as material and semiotic entity means that (1) texts can be used, manipulated, transported, stored, and so on, on the immediate here-now scale as objects; and (2) their embedding in specific meaning-making practices in which their semantic potential is activated connects the here-now object (the letter from a family member that I hold in my hand) to far larger semiotic scales which are condensed in the lexicogrammatical and discourse patterns of organization of the text as a unit of meaning. Textually mediated interactions between individuals expand the possibilities for more and more indirect kinds of interactions with other individuals and institutions on different space-time scales. The proto-interpersonal relations characteristic of the early socio-affective dyads between mother and infant are thus expanded to include an increasingly differentiated array of interactions between increasingly differentiated first and second persons. Subsequently, this includes third persons who are made the focus as the dyad expands outwards to integrate with the perspectives of such persons. It is not difficult to see here how this increased expansion outwards to include third persons of various kinds is, in time, extended to include non-human objects of varying degrees of abstraction as semiotically mediated objects of symbolic consciousness, including those constituted by purely linguistic means. We have considered how metafunctional constraints which are intrinsic to the organization of linguistic and other semiotic forms enable meanings to be enacted and construed in and through semiotic forms of all kinds. Moreover, I have considered how the metafunctional basis of linguistic and other semiotic forms has eo-evolved with human beings and the ecosocial forms of organization in which human activity is embedded and by which it is both constrained and enabled. It is this hook-up between individual human agents and the higherorder boundary conditions constituted by the ecosocial system which enable the emergence in individuals of complex, individual properties and viewpoints on the basis of the self-organizing neural networks which are formed along a timebound trajectory in the individual brain. Higher-order or symbolic consciousness is the result of the self-organization of symbolic space whereby human individuals qua agents can carry out intentional symbolic activity in their interactions with others. The systemic possibilities of lexicogrammar and other semiotic

THE METAFUNCTIONAL CHARACTER OF CONSCIOUSNESS 279 modalities constitute, as Halliday (1993) has pointed out, a multidimensional and highly elastic semantic space which enables human beings to construe experience, to enact social-interpersonal relations with others, and to construct often extremely complex acts of internally coherent discourse. The meaning potential of language, depiction, and so on, constitutes a phase space whereby individuals are provided with systems of choices - meaningful alternatives - for making meanings in specific contexts. The notions of scalar heterogeneity and complex dynamic open systems that interact with their ecosocial environments under conditions far from equilibrium show us how social agents build both history and context into their own internal dynamics. Our symbolic meaning-making resources allow us to extend ourselves beyond the here-now scale of our immediate surrounding milieu so as to selectively interact with diverse, often very different space-time scales, as well as to internalize these interactions so that they become part of the individual's Innenwelt. In this way, we interact with not just a series of immediate contexts of situations, but also the context of culture which the former mediate and in which they are embedded. Thus, symbolic - not iconic and indexical- semiosis enables us to interact with and internalize scalar heterogeneity. It is our being situated at this intersection of maximally diverse scales, mediated by our symbolic resources, which in turn acts on our internal dynamics and modifies them. The further question that arises here concerns the constraints on the particular forms of meaningful action that can be carried out by agents in particular forms of social organization. If the systemically organized meaningmaking resources of language and other semiotic modalities both enable and constrain the kinds of meanings that can be made, then we need to investigate how variation in the relationship between semiotic forms and the social meaning-making practices in which agents are positioned leads to variation in the ways in which consciousness is organized. The higher-order systems of meaning-making practices of a given culture constrain both the agent's trajectory and his or her internal dynamics to the global probability landscape e.g. the coding orientations - of the culture as a whole. Thus, individuals are entrained to the overall dynamical organization of a particular coding orientation (Bernstein 1971, 1990) or habitus (Bourdieu (1977 [1972]) such that, in logogenesis, the probabilities regulating the individual's access to, participation in, and understanding of specific texts, discourse genres, and ways of making meaning are pre-set. As Hasan (1992b: 520-1) points out, variability in the ways in which different categories of social agents are positioned in and orient to the social structure leads to socially significant differences in the ways in which different categories of individuals in a given community orient to and deploy in their daily life the meaning-making resources of the community. This further suggests that the semiotically mediated nature of consciousness will lead to variant forms of consciousness according to the variable ways in which individuals are positioned by the differential distribution of power and control and, hence, to the semiotic and material resources of the society. The slower rate of the higher-scalar attractor space (e.g. the coding orientation which an individual belonging to a particular social group habitually accesses) regulates lower-level individual dynamics by entraining its much faster

280

BRAIN, MIND, AND THE SIGNIFYING BODY

dynamics to the slower ones on the higher level. This suggests that the very fast dynamics of neural activity in the brain are entrained to the higher-scalar constraints of particular coding orientations through the creation of attractors that reorganize the patterns of firing of individual neurons such that these are constrained by the attractor space that the coding orientation constitutes. As Benjamin Lee Whorf long ago understood, though without the benefit of modern neural science, higher-order consciousness is shaped and entrained by what he referred to as 'a whole large order of experience' (1956a: 137). That is, it is shaped by higher-scalar dynamical processes on the ecosocial level. Moreover, Whorf also understood that language is a more specified integrative layer of something far more general, which 'may generalize down not to any such universal as "Language", but to something better - called "sublinguistic" or "superlinguistlc" - and not ALTOGETHER unlike, even if much unlike, what we now call "mental" (Whorf 1956b: 239).

6 Brain, Meaning, and Consciousness

1. Biological and Socio-cultural Factors Form a Single System of Complexly Related and Interacting Factors: Putting Time and Activity Back into the Picture The science of language has existed in both the Western and Eastern traditions for at least the past two or three millennia. Linguists have been able to make statements about the phonological, graphological, lexicogrammatical, semantic, and discourse aspects of language because there exist neurological and sensorimotor processes that make the level(s) of organization about which linguistic statements are made possible. The discourses of biology are, of course, concerned with the former level. It does not follow from this line of reasoning that statements about language qua semiotic system must necessarily be made with reference to or reduced to statements about biology. The fact that several millennia of thinking about and theorizing about language has taken place in the absence of our modern understanding of biology shows that such a nonreductive discourse about language is eminently possible. Nevertheless, the historical order in which various scientific discourses have appeared should not distract us from the specific logical problem I am addressing here. Much of our received wisdom about language emerged well before the onset of a credible biological discourse. This fact has tended to exacerbate the trend towards talking about language as a metaphysical object that is dissociated from both nature and our bodies. The evidence for this trend abounds in many of the theoretical discourses about language that have predominated in the twentieth century. The resulting dissociation has tended towards fragmented and arbitrary accounts of language which do not connect with the biological dimension of our being. In chapters 4 and 5, we saw that the biological and socio-cultural dimensions of the specificity of the human species need not be dichotomized as two separate causal domains. In recent years, neurobiology has made considerable progress in showing that it makes no sense to oppose the human capacity for linguistically mediated meaning-making and higher-order symbolic 'thought' to our biological being. This has often led to the tendency to view the latter as something lofty and 'spiritual' and therefore beyond biology whereas the latter is seen as belonging to the crassly physical-material realm. In reality, the former belong to our biological make-up, and at the same time our biology has evolved so that we are the socio-cultural and symbolic beings that we are. Language in all

282

BRAIN, MIND, AND THE SIGNIFYING BODY

of its facets is intrinsic to our biological make-up. There is continuity with other species, as the ape-language research of Savage-Rumbaugh et al. (1998) shows, though this in no way implies that human biological specificity is reducible to that of other species. The more fundamental point is that all forms of biological organization are founded on elementary forms of semiosis whereby organisms of all kinds selectively interpret and adjust to their specific version of reality in the process of constituting their Umwelten. This fact is evident in the infant-caretaker semiosis characteristic of primary intersubjectivity. The reciprocal contributions of both caretaker and infant to the dyads characteristic of primary intersubjectivity (Trevarthen 1987, 1992; Halliday 1975, 1993) are based on forms of matter, energy, and information exchange both between the two members of the dyad and between the dyad and the higherorder ecosocial system in which the dyad is embedded. Semiosis, including linguistically mediated semiosis, emerges as a consequence of these processes of exchange. The fact that language is emergent from such forms of exchange means that it is intrinsically dialogical and social. Therefore, the biological organization of the human species is inherently dialogic from the outset. The elementary forms of semiosis that characterize the caretaker-infant dyad in primary intersubjectivity are what make the emergence of language in individuals possible. Doubtless, the genetically inherited value wash predisposes the newborn to participate in the elementary forms of dialogic communication with caretakers that Trevarthen has identified as primary intersubjectivity. In turn, this participation makes possible the infant's accessing and selecting of more complex forms of communication along his or her trajectory. Phylogenetically, it is the emergence of language from prior forms of meaning-making which makes possible the genetic predisposition to language. Rather than an innate linguistic competence which grows into language in the individual in a pre-programmed way, it is newly emergent socio-cultural forms of meaning-making which act on and select the biological substrate so as to modify and entrain it along a specific cultural trajectory so that language and communicative ecosystem both eo-develop and co-individuate in the individual (Bateson 1980; Deacon 1998 [1997]; Lemke 2000a; Thibault 2000a; 2003b). As Wheeler (1988: 15) points out, 'a mind with no programming is no mind'. The mind is programmed by the networks of meaning-making in which it participates. It is the dialogic engagement of world by organism whereby self interrogates and receives response from the world in the exchange process that enables meaning to be exchanged between the two (see also Wertsch 1991). In the process of interpreting the world through exchange, we also affirm our continuity with others linked in the same networks of exchange. This is possible because of the systematic nature of our meaning-making resources. Chomsky's nativist hypothesis, by contrast, sees the programming as being built into the genetic inheritance of the individual from the outset in the form of hard-wired formal rules in the brain. Chomsky's (1965) theory attempts to make explicit the formal properties that define the range of human languages. It is in this sense a characterization of the universal properties of human language. Competence excludes meaning and the relationships among language users, their language, and context. The latter

BRAIN, MEANING, AND CONSCIOUSNESS

283

refers to performance. Linguistic competence is seen as an abstract and formal knowledge which sits above the general cognitive system comprising memory, attention, real-time perception, and so on. Chomsky claims that competence is biologically constrained and characterized by knowledge of abstract linguistic rules which are genetically determined. The brain is thus seen as a store for a large number of formal linguistic rules that are hard-wired in the brain along with the means for operating on these according to predetermined logical criteria. Two problems with this theory are the failure (l) to provide convincing biological criteria for showing how abstract linguistic rules exist in the brain and (2) to demonstrate how competence is manifested in real-time discourse activity. The arguments to be developed below present a very different view. In chapter 4, section 10, pp. 202-5, I argue for the need to refocus on language development as an epigenetic process along a developmental trajectory which is typical of members of a given species. The human organism - the body-brain complex - is adapted to eo-develop and co-individuate together with language by virtue of its - the organism's - being embedded in a specific ecosocial system. Moreover, the body-brain complex is itself constructed and modified by the matter, energy, and information-meaning exchanges it engages in along its trajectory. Language, rather than being innate in the child from the outset, has 'universal' properties and functions on account ofthe species-specific genotypic commonalities and type-specific developmental pathways such that some kinds of information and structure rather than others are selected from the developmental and cultural context as being relevant to the development of the child in ways which are typical of the species (see also Cowley 2001). The Pinker-type distinction between 'what an organism learns' and 'what a brain innately knows' has no place in this conception (Pinker 1994,1997). Moreover, an individual's language potential is stored in the brain not as hard-wired abstract rules, divorced from time and context, but as memory-governed networks of associations which are acquired and adaptively modified along the individual's time-bound trajectory. There are important developmental and individuating implications of this claim.

2. Re-interpreting Flohr's Brain-based Theory of Phenomenal Awareness: A Three-level Hierarchy View of the Emergence of Proto-meaning in the Brain Hans Flohrs research is dedicated to overcoming the prevailing dualism between physical versus phenomenal brain states. Flohr (1991: 248-9) outlines two classes of inputs which are received by the cerebral cortex as follows: (1) specific afferents from sense organs which are relayed through specific thalamic nuclei before reaching the primary cortical projection areas; and (2) nonspecific afferents which reach the cortex from the mesencephalic reticular formation, where the term reticular formation applies to 'a network of neurons and nerve fibers extending from the caudal medulla to the diencephalon' (1991: 249). Such neurons form 'a widely distributed network of synaptic contacts' (1991: 249). Moreover, it is now established that this formation 'is organized into different nuclei with specific afferent and efferent connections' (1991: 249).

284

BRAIN, MIND, AND THE SIGNIFYING BODY

Flohr explains the projection of these specific and non-specific afferents within the cerebral cortex as follows: Within the cerebral cortex, primary auditory, somatosensory and visual fields each project to adjacent unimodal association areas which, in turn, project to contiguous secondary unimodal association fields. It is assumed that these successive areas are concerned with progressively higher-order abstractions within a modality. The unimodal association areas project to a number of polymodal sensory areas lying in the cingulate gyrus, parietal, temporal and frontal lobes. Their function is vaguely described as crossmodal association and synthesis. The polymodal association areas project to the inferior parietal lobe which has been termed a 'supramodal' area. Polymodal and supramodal regions have connections to the limbic system; these connections provide the anatomical substrate by which motivational states influence cortical processing of sensory stimuli. (Flohr 1991: 249) Flohr points out that previously consciousness had been understood as depending on the convergence of specific and non-specific afferrent systems, 'both of which contain complex feedback loops and are interconnected at various levels' (1991: 250). However, Flohr also points out that this understanding remains 'vague' after several decades of research 'into the role of the brain stem in the generation and maintenance of consciousness' (1991: 250). Moreover, it remains unable to explain both 'the difference in the subsequent cortical processing of sensory data distinguishing conscious and unconscious brains' and 'the specific nature of those brain processes which result from this convergence and whose occurrence is a precondition for the occurrence of consciousness' (1991: 250). The central problem that remains for Flohr is a two-fold one, viz. (1) what happens beyond this point?; and (2) which specific physiological states and processes result from this convergence of (specific and non-specific) afferent signals such that these might be 'constitutive of phenomenal experiences'? (1991: 250). In order to formulate a hypothesis which may provide a solution to the problem just outlined, Flohr first distinguishes two categories of possible neural processes, which are the result of the interaction between specific and non-specific afferents (1991: 251):

1. 'conventional' synaptic transmission, i.e. signal propagation in neural nets with rigid structure and given synaptic weights; 2. activity-dependent self-organization of neural nets, i.e. changes in synaptic connections which are induced by synaptic activation and dependent on the degree of activation. (Flohr 1991: 251) According to Flohr's hypothesis, the occurrence of phenomenal states in the brain is dependent upon processes that pertain to the second group of factors. On this basis, he proposes that phenomenal states depend on the rate of 'activitydependent synaptic changes' and the rate of formation of 'neural assemblies' (1991: 251).

BRAIN, MEANING, AND CONSCIOUSNESS

285

The concept of neural cell assemblies, as Flohr (1991: 251) points out, was first developed by D. O. Hebb (1949). Hebb's conception is also central in the work of Edelman (1987). Hebb proposed that '... repeated stimulation of specific receptors will lead slowly to the formation of an 'assembly' of association area cells which can act briefly as a closed system after stimulation has ceased .. .' (quoted in Flohr 1991: 251). Hebb had proposed that such an assembly is the simplest form of 'representative process' of an image or idea. Flohr continues: The basis of this self-organization of neural nets is the presence of plastic synapses, now termed Hebb synapses. Hebb assumed that synapses on a neuron that are active while the neuron discharges will be strengthened, whereas inactive synapses will be weakened. Synapses from differing inputs that are active at the same time on the same neuron will be reinforced and selected over others. When applied to nets of spatially distributed groups of neurons with nonspecific, random interconnections, these so-called Hebb rules lead to a relative stabilization and association of neurons firing in a correlated fashion. An assembly of preferentially connected, coherently active cells is formed. If coincident activity is induced in some neurons of such a net by a patterned input, an assembly will be formed because the synchronous activation selectively modulates the pathways connecting these neurons. The assembly detects and encodes the coherent properties of the stimulus pattern such that a representation of that pattern is generated. Once the assemblies have been formed, they would function as detectors of the same or similar input patterns expressing the detection of coherent features by co-ordinating their activities. It is easy to envisage that the output of such assemblies could in turn be used as input to other modifiable nets which then would reorganize their structure as a function of this input. Iteration of such processes would generate more and more abstract metarepresentations ... These nets are able to create topographical representations in which the relationships between elements of the stimulus pattern become encoded into spatio-temporal patterns of activities. Moreover, in such nets abstractions from primary sensory data at increasingly higher levels of generalization are formed and stored. Synaptic plasticity is the means by which neural nets detect and represent coincident properties and, hence, the nomological relations existing between elements of the input pattern. (Flohr 1991: 251-2; italics in original) Flohr's proposals can readily be assimilated to the three-level hierarchy, as follows: L+1: encoding of the properties of the stimulus pattern such that a representation of that pattern is generated; L: application to nets of spatially distributed neurons leads to stabilization and association of neurons firing in a correlated fashion to form an assembly of globally coherent cells;

286

BRAIN, MIND, AND THE SIGNIFYING BODY

L-l: strengthening and/or weakening of synapses on different units while neuron discharges as induced by patterned stimulus pattern information (inputs) . This shows how the strengthening of neural discharges on level L-l constitutes information which is reorganized on level L as a global cortical effect in the form of an 'assembly' of 'coherently active cells' which are, in turn, interpreted at level L+1 as being a representation of a stimulus pattern. The fact that assemblies, once formed, can function as 'detectors' of the same or similar patterns over longer periods of time shows that they function as filters or buffers such that only the salient information from neuronal discharges on level L-l is recognized on level L. Moreover, the fact that such self-organizing nets create topological representations on level L+1 spatio-ternporal patterns of activity shows how the representations created on this level constitute a system of interpretance comprising representational categories. For this reason, globally coherent assemblies of cells on level L are appropriately interpreted as representations of this or that representational category. A critically important component of Flohr's argument is the postulated existence of rapid synaptic changes as against the previously dominant assumption that synaptic weights remain constant in the performing of neural nets. Flohr argues that the NMDA-receptor channel complex acts as the molecular agent which brings about rapid synaptic changes. I shall quote Flohr at some length on this rather technical question before venturing some remarks of my own: The essence of Hebb's postulate is that a synapse is strengthened if the presynaptic activation of the synapse is coincident with the firing of the postsynaptic neuron. This poses two difficulties: (a) some mechanism of the synapse must respond to coincident pre- and postsynaptic activate and initiate temporary changes in the weights of the synapse; (b) in the case of Malsburg-type synapses this mechanism must be fast enough to allow rapid changes [ ... ]

One (but probably not the only one) such mechanism that could fulfil both conditions has been described. Much recent data indicate that the NMDA receptor channel complex may serve as the molecular mechanism for Hebbtype synapses. This channel is voltage-dependent ... , i.e, it opens under two conditions: (a) presynaptic activity that releases a transmitter that binds to the receptor, and (b) sufficient postsynaptic depolarization. At membrane potentials around resting potential, NMDA channels are blocked by magnesium ions; if the membrane is depolarized, to say -30 to -20 mY, the block is removed and the channel is switched on. The NMDA reccptor-linked channel is permeable to Na', K' and Ca". Thus, NMDA receptors provide a mechanism to (a) amplify synaptic currents, and (b) increase Ca" influx ifa particular level of postsynaptic depolarization is present. Ca" is hypothesized to trigger a number of Ca"-dependent molecular changes inside the post-

BRAIN, MEANING, AND CONSCIOUSNESS

287

synaptic terminal that modify the efficacy of synapses (other than those gated by NMDA). (Flohr 1991: 253) Postsynaptic depolarization through the effects of the NMDA-receptor means that the afferent stimulus patterns which impinge upon the neurons belonging to a coherent population are indexical signs of inner and outer environmental events. However, the amplification of synaptic currents, including the firing properties of the postsynaptic neuron, on detection of correlated signals converging on a single cell, leads to the formation of positive feedback loops such that neurons near the threshold level will 'persist as a co-operative, coherently active subpopulation' (1991: 253). It is this depolarization which leads to the formation of coherent assemblies. Depolarization of the postsynapse is the translation via the coherent cell assembly of the indexical signs - the afferent inputs - related to some environmental event into symbolic signs which translate the indexical sign from the inner perspective of the coherent cell assemblies which are formed in the cerebral cortex. In this way, indexical signs of the Umweltare translated into subpopulations of neurons. The latter, following Salthe (1993: 176), are symbolic because they do not stand in any necessary relationship to the afferent inputs relating to environmental events. The fact that these same assemblies can then function as 'detectors' over longer timescales, as discussed above, constitutes the basis on which an inner perspective which constructs models of its Umwelt is built up. By contrast, the afferent input qua index of environmental event occurs on the shorter timescale of the here-now occurrence and pick-up of that event. It is on the basis of such models that a system of interpretance emerges. The phenomena of consciousness are attended to and experienced from the inner perspective of a self. Flohr (1991: 255) draws attention to the rate of activation of a cell population: unconscious brain structures exhibit a lower rate; conscious ones a much faster rate of assembly, which reaches and crosses a critical threshold. This emphasis on the rate of assembly activation follows an important principle of scalar hierarchy theory, viz. processes on lower levels in the hierarchy have faster temporal cycles while those on higher levels have slower ones (Salthe 1993: 46). The faster rate of assembly of cell populations in the formation of a conscious experience goes hand in hand with the fact that the conscious experience itself is experienced on a slower timescale which corresponds to that of the observer perspectives and categories of the person who undergoes the conscious (phenomenal) experience. In accordance with the logic of scalar-hierarchy theory (Salthe 1993: 36-52), we could say that a phenomenal experience is a higher-scalar phenomenon on level L+1 (with respect to the lower-scalar entities and processes on levels L-l and L) which has a 'cogent moment' that lasts longer than any of the lower-scalar processes that sustain it. The cogent moments of conscious experience therefore reflect timescales pertaining to the higher-scalar entities of the observer's perspective rather than the lower scale of, for example, the synaptic discharges on level L-l, which are not accessible to consciousness (section 9, pp. 310-13). Moreover, each level in the three-level hierarchy exhibits properties which are specific to that level such that relations and processes on different scalar levels are, to some extent, ontologically isolated from each other

288

BRAIN, M I N D , AND THE SIGNIFYING BODY

(Salthe 1993: 46). It is this factor of ontological isolation which enables us to understand how phenomenal consciousness can emerge from physical brain states and processes on different scalar levels without being reducible to the lower-scalar relations and processes. A further important aspect of Flohr's hypothesis concerns the possibility that 'iteration' of such processes can lead to the formation of increasingly abstract meta-representations (see above quote). Thus, phenomenal awareness of stimulus patterns deriving from both the external world, the body's surface, and the body's internal milieu can be integrated to and, therefore, reorganized as meaningful information for conceptual and symbolic consciousness, as outlined in Figure 6.1. Furthermore, awareness is 'the result of the system's capacity to actively generate representations and meta-representations' (1991: 255) such that self-referential and meta-cognitive activities leading to the formation of an Figure 6.1: Reentrant loop of conscious experience, showing relationship between self and object of experience Efferent outputs transduced as action ('attending to')

Pick-up of environmental information transduced as afferent inputs 'In here' = L+l: EXPERIENCER-IN-HERE attends to REPRESENTATION OF STIMULUS PATTERN L: GLOBAL CELL ASSEMBLY L-l: WEIGHTED SYNAPTIC DISCHARGES

'Out there' = ENVIRONMENTAL INFORMATION ABOUT INNER AND OUTER EVENTS

BRAIN, MEANING, AND CONSCIOUSNESS

289

inner perspective will emerge (1991: 258). It will be recalled at this point that phenomenal awareness ultimately derives from the patterned activation and modulation of receptor cells pertaining to the various perceptual systems. It is such patterns of stimulus information, derived from inner and outer environmental sources, which induce the strengthening or weakening of synaptic discharges on level L-l of the proposed three-level hierarchy in terms of which I interpreted above Flohr's proposals concerning phenomenal awareness. Flohr is not concerned, in the article I have discussed here, with conceptual and symbolic forms of consciousness. Phenomenal awareness is always awareness of an organized field of experience of objects, events, and so on, both within our bodies and in the world outside our bodies. This draws attention to the fact that the afferent stimulus inputs derive from sources outside the cerebral cortex. As I see it, Flohr's notion of phenomenal awareness can be described as 'conscious experience of our experiencing', i.e. 'before' meaning in the socially conventional sense necessarily enters the picture. Crudely, this can be interpreted as a reentrant loop whereby the conscious experiencer, in the form of a higher-scalar me ta-representation of itself as the one who has the phenomenal awareness or undergoes the conscious experience of the given phenomenon, is linked to the 'object' of this experience roughly along the lines shown in Figure 6.1. The purpose of Figure 6.1 is to show how conscious awareness of the kind theorized by Flohr is always embedded in a higher-scalar information-rich environment which constitutes the boundary conditions of phenomenal awareness itself.

3. Brain, Meaning, and Symbolic Consciousness Proto-meanings are made up of neural activities in the form of neurophysiological entities called impulses which enable communication to take place between neurons by means of bio-chemically based neurotransmitters. The protomeanings that are stored in the brain combine with brain waves from other brain regions where phonological systems and structures are stored. The brain waves corresponding to the latter should not be confused with the actual sounds that are produced by the vocal-tract activity in phonation. Peng (2000: 194) points out that some distance has to be travelled, 'because of the cortico-striato-pallidothalamo-cortical-loop', before the brain waves corresponding to acoustic images are transduced into physical sounds. Peng points out that neural impulses corresponding to proto-meanings are 'catalytically mapped onto, that is, bound to, an acoustic image' (2000: 195) such that 'in production, the impulse can no longer be amorphous' (Peng 2000: 195). It follows, from the way in which neural impulses that are analogous to the content and expression planes in the CNS are catalytically mapped onto each other only to be separated prior to articulation, that linguistic content as such is not transmitted from speaker to hearer. The only thing which is 'transmitted' is the physical speech sounds produced in vocal-tract gestural activity and propagated through the air as sound waves. Only the neural impulses corresponding to the expression stratum are transmitted via the loop to the vocal apparatus

290

BRAIN, MIND, AND THE SIGNIFYING BODY

for speech production. The impulses corresponding to the meaning - the content - remain, as Peng (2000: 195) shows, in the speaker's brain. Meanings do not travel through the air to the hearer. Instead, the hearer must re-construct in his or her own brain the relationship between the two strata by (1) recognizing the speech sounds which are picked up by the auditory system; and (2) combining Of binding in his or her brain the neural impulses deriving from the brain regions concerned with proto-meaning (content) and expression. The hearer performs this process of re-construction on the basis of the fact that he or she has a language system stored in long-term memory which is sufficiently similar to that of the speaker that the same kinds of patterned relations between neural impulses associated with expression and those associated with content are made. This presumes that the neural and physiological dynamics of the central and peripheral nervous systems of speaker and hearer have been entrained to the same higher-scalar ecosocial system and its meanings even if their individual trajectories through this can never be identical in every respect. In the previous section, I considered how Flohr's proposals represent an important statement concerning the way in which afferent stimulus patterns deriving from multimodal sources either within or outside the body are represented as conscious experience of phenomena by the brain. The reorganization across scales of lower-level physical brain processes led to their translation into higher-scalar representations of input stimulus patterns which have meaning for the self who is the conscious observer of these from an inner perspective, itself the result of higher-scalar meta-representations in the brain. The states of phenomenal awareness which form the basis of Flohr's discussion are largely perceptual in character, whether these derive from external environmental events or from within the internal milieu of the body. However, this distinction does not alter the fact that afferent inputs derive from physical stimulus information which is reorganized as conscious experience of phenomena in the way discussed in the previous section. Furthermore, the individual's experience of the given phenomenon is always mediated by a system of interpretance in terms of which the phenomena of which one is aware are experienced as phenomena corresponding to this or that category of experience. This shows that consciousness is semiotically mediated at all levels. It is about a meaningful relation between conscious experiencer and phenomenon experienced. In the case of phenomenal awareness, as discussed by Flohr, it is important to point out that 'meaning' should be taken to refer to something less specified and therefore more general than the more specified symbolic categories which are found in ecosocial semiotic systems. The question now arises as to how Flohr's discussion may help us to understand symbolic or higher-order consciousness and the role of symbolic modes of semiosis such as language in this.

4. The Embodiment of the Material and the Conscious Modes in Expression and Content Flohr's discussion of phenomenal awareness highlights the way in which stimulus patterns, derived from acoustic, photic, and other forms of energy, stimulate and

BRAIN, MEANING, AND CONSCIOUSNESS

291

modulate the receptor cells such that nerves are stimulated to fire and transmit sensory information to the brain in the form of nerve firings. In this way, the qualia of phenomenal awareness - i.e., the elementary conscious experiences of basic sensations such as the colour blue, a sweet taste, or low pitch - are registered on consciousness. This is sensory perception. But this is not, as Gibson (1983 [1966], 1986 [1979]) has shown, what perception is normally all about. Perception is of events which occur in an environmental context which the organism actively and adaptively explores and orients to. When we listen to speech sounds, we do not perceive raw acoustic sensations such as low pitch. Instead, we perceive a speech event. In other words, we perceive the syllables, rhythmic patterns, and intonation contours characteristic of speech events in a particular language. To be sure, our perception of this event is supported by acoustic energy in which many fine-grained distinctions not normally available to consciousness can be identified by physical measuring devices such as the spectrogram. The flow of energy measured by the spectrogram is continuously varying at the very short timescale of sensory perception. Many of these distinctions are not salient for the perception of the speech event and are filtered out by the particular phonological system in operation (see chapter 3, section 2, pp. 109-16, for a full discussion). Event perception is oriented to the pick-up of information about the meaning which the auditory event affords the listener. In the case of speech events, we perceive information about (1) the processes of vocal-tract articulation of the speaker who is the source and producer of the event and (2) the ways in which the phonological units and relations that are construed in the sound stream qua expression stratum may be cross-coupled with lexicogrammatical and semantic units and relations on the content stratum. It is important to remember here that content-stratum systems and structures are neither contained in nor transmitted by the expression stratum. I shall come back to this point. What I want to emphasize here is that all forms of semiosis, e.g. the act of speaking under consideration here, entail the perception of semiotic events and/or products (object-texts) qua expression stratum. The perception of the speech event as expression stratum means that listeners orient to and perceive phonological categories and relations on different scales of organization in the speech event (see chapter 2, section 9, pp. 83-6). This is not saying anything at all new. Nevertheless, it is important to reassert this fundamental point so that we can address the relevance of this to symbolic consciousness. The main point is that the expression stratum is itself a phenomenon of conscious awareness. Language users do not perceive raw acoustic sensations. Instead, speech sounds are phenomena of experience which are organized and perceived as instantiations of the phonological categories of a particular language system. The perception of a speech event therefore involves the introjection into the domain of the conscious of the vocal-auditory information which is picked up as the embodied experience of vocal-tract or other gestural activity. Halliday has discussed the overall semogenic, or meaning-creating, process in relation to the two strata of expression and content as follows: In the act of meaning, then, the two modes of experience [viz. the material and the conscious, ~JT] through the projection of the one by the other,

292

BRAIN, MIND, AND THE SIGNIFYING BODY

become fused and transformed into something that is new and different from either. We can think of this as creating a 'plane of content' in the Hjelmslevian sense. If we look at this process dynamically, it is meaning-creating, or semogenic. Ifwe look at it synoptically, as a relation construed by this process, it is semantic; and it appears as an interface (our original notion of semantics as 'interlevel' was relevant here), one 'face' being the phenomena of experience. We often refer to these phenomena collectively as 'the material', as if the only form of experience was what is 'out there'. But this is misleading. Our experience is at once both material and conscious; and it is the contradiction between the material and the conscious that gives these phenomena their semogenic potential ... But in order for meaning to be created there has also to be a second interface, a transformation back into the material, or (again, rather) into the phenomenal - this time in its manifestation in the meaning subject's own body: as physiological processes of articulation or gesture. This is the phonetic/kinetic interface; the 'expression plane', in Hjelmslev's terms. Since there can be no meaning without expression (meaning is intlnSubjective activity, not subjective), the act is 'doubly articulated', in Martinet's terminology: it is the transduction of the phenomenal back into the phenomenal via these two interfaces of content and expression. (Transduction not transformation, because as Lamb (1964) pointed out many years ago in transformation the original is lost, ceases to exist. And again I am suggesting that we should conceive of it as phenomenal rather than material, since both the 'outer' faces, that of the content substance on the one hand and that of the expression substance on the other, embody both the material and the conscious modes of being.) (Halliday 1992a: 21-2; italics in original) Both content and expression embody the material and the conscious modes. Many theories of meaning have lost sight of this material-phenomenal dimension. In doing so, they miss out on the fact that meaning-making requires an intersubjectively shared form of bodily expression. Meaning-making is not the transmission of an abstract content form in the form of coded ideas or thoughts from one 'mind' to another. Expression is not a mere vehicle or carrier of content, but is itself phenomenal. Meaning is not transmitted in this or any other way. The embodiment of both the material and the conscious modes may be explained as follows. Speech sounds qua physical and biological processes are based on topological-continuous variation. Phonological systems and structures, on the other hand, operate in terms of discrete typological-eategorial distinctions on various levels of organization. The physical and biological processes tend to be confined to the organismic scale and its material interactions with its immediate environment. However, the phonological systems and structures of a particular language system are ecosocial categories. Bodily processes of the first kind are epigenetically entrained to and selected by the dynamics of the higherscalar ecosocial level by virtue of the individual's participation in the meaningmaking practices of that community. From this point of view, the body qua phonetic/kinetic interface, to use Halliday's term, is a cultural tool which

BRAIN, MEANING, AND CONSCIOUSNESS

293

parUClpates in meaning-making practices that can extend across very many diverse space-time scales. This is the principle of scalar heterogeneity as developed by Lemke (2000a; see also Thibault 2000a). It is in this way that the categories of symbolic consciousness and its artefacts ramify across and integrate with many different scales beyond that of the physical-material processes of the body in interaction with its immediate environment. Lexicogrammatical and discourse systems and structures on the content stratum similarly operate in terms of typological-categorial distinctions for the most part. Again, the phenomena of experience in the material domain of the body's inner milieu and the body's environment are topological-continuous such that the contradiction between the material and the conscious manifests itself here as well. As I said above, these categories of content are not transmitted from one person's brain to another person's brain (e.g. speaker to listener), but are reconstructed in the individual's (the listener's) brain. The no-Iessembodied brain and its neural dynamics is also a tool which the individual activates and puts to use for the purposes of social meaning-making. As discussed in section 7 below, pp. 300-1, this is possible because of the meta-redundancy or contextualizing relations in operation. The cross-coupling of the two strata in the brain is possible because of the contextualizing relations that specify which expressions combine with which contents, and in which contexts. Rather than the transmission of information or thoughts from one mind to another, I would explain what happens in the following terms. It is the use in determinate social meaning-making practices of expression-stratum artefacts in the form of (1) bodily (gestural) activity which is projected into and/or introjected from the ecosocial environment or (2) material textual artefacts, that provides an intersubjectively shared material basis whereby the two strata can be cross-coupled in individual brains (see below) such that the transduction of the phenomena of experience back into the phenomenal can take place. Individuals make meanings by harnessing and entraining their body-brain dynamics to ecosocial ones. Intersubjective vectors of expression integrate them both materially and semiotically to this common purpose. Figure 6.2 provides a schematic representation of the relations between expression and content with respect to the central nervous system (CNS) and the peripheral nervous system (PNS). Phonological units and relations have both indexical and symbolic properties, which means that they are more typologically specified than are the topologicalcontinuous differentiations characteristic of the sensory perception of the acoustic information in the speech event (see above). Moreover, the expression stratum is an interface which connects the central nervous system to the external environment of the individual via the peripheral nervous system (see above). It is an interface in which sensori-motor routines have been entrained to and recategorized as the phonological routines of a particular language system. The more specified nature of phonological expression vis-a-vis less specified sensori-motor routines on lower integrative levels means that individuals are in contact with a greatly expanded Umwelt. Speech events which are picked up and attended to by a listener impinge on the listener as more than just indexes of necessary features of their correlated environmental events. In this way, the

294

BRAIN, M I N D , AND THE SIGNIFYING BODY

Figure 6.2: Content and expression and their relations to the central and peripheral nervous systems; adapted from Peng (1994: 105) Context of Culture Context of Situation CNS

Discourse

CONTENT STRATUM

Lexicogrammar Phonology, Graphology

EXPRESSION STRATUM PNS

Phonology, Graphology

speech event indexes features of the articulatory processes which produce it. To be sure, the logic of the specification hierarchy shows that speech events have iconic, indexical, and symbolic properties. This aspect is discussed in chapter 3, section 4, pp. 118-20. However, the main point to be emphasized for now is that the transduction of the speech sounds into nerve impulses which are transmitted to the central nervous system via the bulbo-cortical pathways results in their cross-coupling to lexicogrammatical and discourse levels of organization on the content stratum in the listener's brain. This cross-coupling is possible because the stimulus pattern which enters the cerebral cortex consists of a stable correlation of articulatory, perceptual, and visual inputs which are reentrantly mapped at progressively higher levels of abstraction to form a predictable language-specific phonological category. Coherent assemblies of cells are formed in the brain so as to represent a given phonological category. It is important to bear in mind here that expression-stratum categories such as phonological ones are also stored in the cerebral cortex. They do not belong to the peripheral nervous systems of sensori-motor activity per se. The presence of stable repertoires of language-specific phonological categories in the cerebral cortex means that stable assemblies of neuronal populations are epigenetically selected and strengthened in both logogenesis and ontogenesis as representations of phonological categories in the CNS. As we saw in section 2, neuronal nets with a high rate of assembly formation do not only produce complex, qualitatively different representations; they also produce progressively higher-level meta-representations. It is in this way, I suggest, that lower-level phonological repertoires are cross-coupled in the cerebral cortex with higher-level lexicogrammatical and discourse units and relations on the content stratum. Expression and content are cross-coupled in this way on the basis of the principle of semiotic alternation developed by Lemke (1999; see in the present volume chapter 1, section 4, pp. 18-22). Thus, the cell

BRAIN, MEANING, AND CONSCIOUSNESS

295

assembly (the neuronal repertoire) which represents a particular phonological category contains information which enables it to be translated into contentstratum units and relations by virtue of the cross-coupling dynamics involved. This is possible because of the brain's capacity to recategorize sensori-motor and phonological routines as conceptual and lexicogrammatical ones, respectively (see chapter 3, section 9, pp. 139-46). The term 'cross-coupling' serves to highlight the dynamic, time-bound nature of the processes involved. Expression and content are said to be cross-coupled to each other because their respective processes simultaneously unfold in time and act on and affect each other. (In the following section, I shall consider the selective and two-way nature of the crosscoupling dynamics involved in terms of Lernke's theory of meta-redundancy.)

5. Periodicity and the Intrinsic Temporal Organization of the Expression Stratum of Speech on Diverse Scalar Levels Speech events are characterized by periodicity patterns at different levels of hi erarchical organization on the expression stratum of phonological organization. Temporality is, therefore, intrinsic to the structural organization of such events. This means that time cannot be viewed as an abstract constant which is externally applied to such events. The timescale on which speech events are perceived as such is relative rather than absolute. Absolute or clock time only applies to the energy flows measured by the spectrogram. Relative time simply means that the duration of a given event is compared to that of other events rather than to some absolute external standard. Relative time is context-dependent and intrinsic to the rhythms of the organism in relation to his or her environment. Phonology is emergent from vocal-tract gestural activity, which is essentially neuromuscular in character. These (and other) gestures are comprised of cycles of periodic processes on various levels of organization. In systemic-functional phonological theory, for example, three vocal-tract cycles have been proposed (Halliday 1992b, 2000; Cleirigh 1998: 37-40). The three cycles correspond to the phonological ranks of syllable, foot, and tone group in systemic theory. The first cycle involves the processes of opening and closing the vocal tract through articulators so as dampen phonation in the production of syllables. This cycle can be modelled as the oscillation between different energy states of the neuromuscular systems involved (Cleirigh 1998: 38). In the syllable, the constituents of the Rhyme have the potential to be moraic, i.e. to function as a timing unit, whereas the constituents of the Onset are non-moraic. On this scale, the articulatory cycle consists of activating and de-activating syllable weight or moraicity. On the next-higher scale of activity, vocal-tract activity is associated with variations in loudness and duration so as to produce stress. This leads to the production of a rhythmic cycle, which is comprised of activating and de-activating stress. The third cycle is that concerned with major pitch shifts so as to create tonic prominence or tonicity. The processes involved on this scalar level produce intonation through a continual process of activating and de-activating tonicity. The temporal structure of the speech event is based on periodicities at

296

BRAIN, MIND, AND THE SIGNIFYING BODY

different scales of organization which provide internally predictable principles of time measurement based on the principle of regularity. Moreover, these periodicity patterns and their intrinsic temporal organization are inherent in phonological organization. On their respective scalar levels, both in- and out-ofphase properties of moraicity, rhythm, and tonicity are semiotically salient temporal events, Le. they are temporal differences which make a difference. It is my view that an important function of these events is to entrain the listener to the perception of the vocal-auditory event as meaningful (see above). It is on the basis of such entraining that the always time-bound cross-coupling processes between expression and content takes place. Periodicity patterning in speech and other semiotic modalities entrains the listener (and the speaker) to semiotically salient temporal structures in the overall temporal flux. This temporal structure consists of smaller units nested within larger units from the syllable, say, to a whole discourse event, or even to diverse discourse events on different occasions. Moreover, the fact that time-bound phonological periodicities also integrate with and demarcate lexicogrammatical and discourse levels of organization on the content stratum highlights their role in the cross-coupling dynamics referred to above. Consistent with Gibson's ecological theory of perception, the periodicities of semiosis and their pick-up qua semiotically salient environmental events are commensurate with the temporal structures which the human organism is adapted to perceive on the scale on which meaning-making takes place. In this way,bodily (neuromuscular) periodicities are entrained to higher-scalar ecosocial meanings. On the other hand, external clock time has scales from attoseconds to, say, 10100 years hence of the Dark Era which cosmologists predict as the ultimate fate of our expanding universe, when little more than waste photons, neutrinos, electrons, and positrons will remain. This refers to the external and absolute measurement of time by extrinsic measuring devices that I mentioned above. But that is not the temporal scale to which the human organism is biologically adapted in its ecosocial environment. The temporal scale of meaningmaking and perception is intrinsic to the organism in its environment; it is also relative, in that one event is measured through comparison with other such events rather than against some absolute standard external to these events.

6. The Creation of Symbolic Objects of Consciousness in Semantic Neural Space Flohr also draws attention to Hebb's notion that 'plastic neural nets can function as coincidence detectors, and that the eo-activation of certain neuron populations by stimulus patterns creates a representation of these patterns' (1991: 258). In this way, intentional contents can be the cause of physical brain processes, e.g. representations can activate stimulus patterns. This is coherent with the two-way and reciprocal nature of the cross-coupling dynamics whereby expression and content selectively contextualize one another. It is interesting in this regard to note that inner speech is associated with imagined auditory routines which are silent and which direct the temporal flow of inner dialogue. These imagined

BRAIN, MEANING, AND CONSCIOUSNESS

297

routines are cognitive routines realized by the activation of neuronal populations. They do not depend on the production of audible speech sounds, though they also direct these in the way discussed above. My reason for bringing this point up is two-fold. First, it demonstrates the independent existence of both phonological, graphological, sign, and other language-specific expression-stratum units and relations in the central nervous system. This does not change the fact that these have to be epigenetically selected in ontogenesis and experientially strengthened in logogenesis in order to get into the CNS in the first place. Secondly, the existence of both semiotic strata - expression and content - together with the reciprocal nature of the contextualizing relations which cross-link them in the CNS further suggests that the contents of symbolic neural space can 'cause' physical-brain and sensori-motor processes by entraining these lower-level relations along a specific action trajectory which loops from the organism out into its ecosocial environment (see chapter 1, section 1, pp. 3-8). The objects of conscious attention are capable of eliciting action trajectories with respect to that 'object', whether this is a material object in the immediate perceptual field of awareness of the individual or a symbolic object which has been generated by the cross-coupling of expression and content in symbolic neural space. Flohr also makes the point that objects selected for conscious attention 'are also different subjectively' and that 'this occurs in those domains in which the assembly turnover is enhanced by thalamo-cortical activation' (1991: 257). But it is time to rein in this discussion and come to some kind of statement as to what objects of symbolic consciousness are. Whether in the CNS of the individual in the form of silent inner speech, or in the form of the perceptual pick-up of stimulus information from speech events along with their transduction into nerve impulses which are transmitted to the cerebral cortex of the listener, the creation of objects of symbolic consciousness arises through the cross-coupling of neuronal populations corresponding to expression- and content-strata repertoires and routines in the brain of the individual. To take a simple example, the lexicogrammatical resources of clause grammar experientially construe a semantic figure comprising a configuration of a Process and Participant(s). The semantic figure so construed is just such an object of symbolic consciousness which can be attended to and focused on in symbolic neural space from the inner perspective of the self who so attends to it. This applies equally to all modalities of natural language - speech, writing, sign, Braille, and inner speech - irrespective of the specific means of transmission from peripheral to central nervous system. The example just given refers to the content stratum, though expression-stratum entities can also be attended to in the same way. A clause, say, is never just an experiential construal of some phenomenon of experience. It is also a way of acting on it and of intersubjectively orienting to it. This is the interpersonal dimension of meaning. Symbolic consciousness therefore means attending to its objects in both ways, simultaneously. Now, the objects so attended to - i.e. experientially construed and interpersonally enacted - are phenomena of experience which have been categorized by the symbolic resources of whatever semiotic system is in operation. These phenomena of experience may be actually perceived material objects, as I said before, or they

298

BRAIN, MIND, AND THE SIGNIFYING BODY

may be purely imaginary or hypothetical ones which can be imagined in symbolic neural space. They can also be recorded in the textual records and products whereby such objects can be re-constructed in different brains along the lines I sketched out above. In both cases, the individual's neural dynamics must cross-couple expression and content to produce the appropriate symbolic representation in his or brain. Consider the following example: (1) look at the mess Paul (2) you're gonna have to clean up in here (3) it's a pig sty The three clauses which comprise this speaker's utterance are addressed to her husband (Paul). In this example, the 'mess' referred to in clause 1 is taken to be given in the perceptual purview of both of the participants in the exchange, which occurs in the living room of their home. However, there is an important difference between the perception of some phenomenon and its linguistic construal as an object of symbolic consciousness, as this example makes clear. To illustrate this point, I shall consider in some detail the first clause in the above passage. The metafunctional analysis of this clause is presented in Table 6.1.

Table 6.1: Metafunctional analysis of the imperative clause (you) look at the mess

Metafunction

(you)

look at

the mess

Experiential

Behaver

Process: Behavioural

Range: affected

Interpersonal

Imperative: proposal for action

Textual

Theme/Given

Paul

Vocative

Rheme/New

The metafunctional analysis of the clause look at the mess reveals different facets of its functioning in simultaneously construing and enacting a symbolic act of consciousness. The experiential analysis shows that the speaker of the clause construes the given phenomenon of experience as a semantic figure. In this case, it is categorized as a behavioural process of the subtype concerned with conscious action (Halliday 1994 [1985]: 138-40; Halliday and Matthiessen 1999: 136). The clause analyses the overall semantic figure as comprising three semantic elements: Participant: Behaver"Process: Behavioural: Conscious Action 11 Participant: Range. The semantic category of behavioural process often refers to processes that are on the borderline between inner consciousness and outer actions and events in the material world. This is so in the sense that these processes do not occur in the external world of material actions and events, but are exterior actions of the body which, however, manifest or index conscious intent on the part of the person who carries out the behaviour.

BRAIN, MEANING, AND CONSCIOUSNESS

299

In this case, the process 'look at' construes a conscious act of visually attending to something. The Behaver is the participant who performs this process and the Range is that which is so attended to. In construing her interlocutor in this way, the speaker of the clause integrates him into a particular experiential frame of reference, as supplied by the experiential category which the clause instantiates. In this way, he is experientially construed as a conscious participant who is required to visually attend to the designated Range. Now, the nominal group the mess, which is used to construe a particular referent situation within the perceptual purview of the two participants, also functions to categorize some facet of the situation as an instance of the semantic category of Thing designated as 'mess' by the Thing element in the nominal group. The point is that 'mess' is a semantic category which belongs to the English language. It is not inherent in the referent situation as such. In using this particular selection, the speaker creates a symbolic object of consciousness which in itself can be attended to as a way of understanding the given referent situation. As I said before, the mess is Range in its clause such that the particular experiential frame chosen here integrates the implicit 'you' as the Behaver who attends to this symbolic object of consciousness in the way discussed above. The observations made in the preceding paragraph go hand in hand with the interpersonal semantics of the clause. Thus, the selection of imperative mood serves to modalize the addressee as the one who is required to carry out the action specified in the clause. The addressee is the implied grammatical Subject, and as such is selected as the entity in which the speaker invests a particular interpersonal take. In this example, it takes the form of a proposed action which she seeks to negotiate with her interlocutor (her husband, Paul) as the one who is designated as being responsible for carrying it out. In this way, the interpersonal meaning of the clause integrates or attempts to negotiate the integration of the addressee to the same evaluative stance on the proposed action. The selection of imperative mood therefore enacts a particular orientational stance towards the action. Moreover, the noun mess implicates a particular evaluative or attitudinal stance on the part of the speaker. Once again, this constitutes an attempt to integrate the addressee to the same stance. That is, she seeks to direct attention to the phenomenon of concern from a particular interested point of view in which a negative evaluation is implicit. Both of these interpersonal facets of the clause's overall meaning also implicate the speaker's ability, in proposing the course of action in question and in taking up the value stance that she does, to put herself in the position of the other (the addressee) who is being asked to adopt the course of action and therefore to accept its associated evaluative stance. This can only happen through the resources of symbolic meaning-making. The textual organization of the clause in terms of Theme-Rheme and Given-New shows (1) that 'you' as Theme is the local point of departure for the development of a message about the Theme in the Rheme and (2) 'you' is taken as the Given in relation to which the New is presented. In this particular example, this reflects the way in which the speaker presumes the 'you' as the contextually given item in relation to which the new contextual information is presented. The textual organization of the clause structures the message into items of consciousness of varying degrees of interest or significance. In the

300

BRAIN, MIND, AND THE SIGNIFYING BODY

present case, the 'you' whose consciousness is being acted on is being asked to focus on the New information, in particular the mess, which occurs in final position in this clause. I have chosen this particular example because it demonstrates a link between a perceptual purview at the same time as bringing out the radical difference between symbolic consciousness and consciousness which is tied to the perceptual awareness of phenomena. Perceptual awareness is tied to the pick-up of environmental information which impinges on the organism as indexes of necessary correlations with environmental events. As Gibson has shown, perception serves to orient and guide the behaviour of the organism in its ecological environment relative to the environmental events that are perceived. But the meaning-making resources of symbolic consciousness enable individuals to select from a symbolic system of possibilities which is not inexorably tied to the here-now perceptual purview by indexical necessities. Thus, speakers of a particular language can make choices from the resources of a linguistic system, for instance, such that they can create varying possibilities of experiential categorization, varying interpersonal and evaluative takes, and varying ways for presenting and organizing information in discourse such that a given item rather than some other is presented as, say, theme in the clause. Moreover, the metafunctional hypothesis which I have assumed here shows that all these diverse operations occur simultaneously on account of the way in which a diversity of semantic functions is mapped onto the one lexicogrammatical form.

7. Contextualizing Relations, the Principle of Meta-redundancy, and the Brain as ContextuaIizing Tool The cross-coupling of expression and content in the individual's neural dynamics depends on higher-scalar systems of contextualizing relations which function as a system of interpretance within the brain. One candidate for explaining the abstract and general principles which cross-link the two domains is that of meta-redundancy, developed by Lemke (1984c) on the basis of some earlier proposals by Bateson (1973b). Bateson developed the theory of redundancy in order to show how increasingly higher orders of contextualization can be modelled and related to each other as a hierarchy of logical types. All semiotic systems are based on the principle of meta-redundancy. This principle is, fundamentally, a contextual one. In a given semiotic system, we can say that the semantic contents (linguistic or non-linguistic, it does not matter for the principle is the same) a, b, c are realized by the expressions x, y, z. This means that there is a redundancy relation between the terms on either side of the content/expression divide. As Lemke (1984c: 35) points out, (meta-) redundancy relations are two-wayor symmetrical ones in the sense that two parts of the same message are redundant with each other in the sense that the information carried by one makes it possible to predict some or all of the information carried by the other. Halliday (1992a: 2) has aptly illustrated this by showing the two-way, or dialectical, nature of realization, i.e. the relationship which is expressible by the term 'redounds with' is implicated in the two-way nature of the real-

BRAIN, MEANING, AND CONSCIOUSNESS

301

izatory relationship between expression and content. For example, expression 'realizes' content and content 'is realized by' expression. Thus, given the meaning a, we can predict sound, gesture, graphological feature x, and given feature x, we can predict meaning a. (For further discussion of these principles see also Thibault 1991a: chapter 4.) The individual's ability to predict, with a more than even chance of being correct, the nature of the redundancy relations involved is always a probabilistic, or stochastic, process. Such first-order redundancy relations are, in turn, dependent on some still-higher-order contextual, or me ta-redundancy, relation(s). This can be explained as follows. The expressions x, y, z realize the meanings a, b, c, while the meanings a, b, c realize the still-higher-order contextual relations p, q, r. Now this relationship, as expressed by the concept of redundancy, is not a linear or causal one, as Halliday (1992a: 24) points out. Thus, it is not the case that 'a realizes p' or that 'p is realized by a' (see chapter 1, section 6, pp. 26-30). Rather, there is a meta-redundancy relation such that p, q, r does not redound with a, b, c, but with the redundancy of a, b, c with x, y, z. Following Bateson, the slash formalism, i.e. 'I', is used to indicate the orders of the redundancy relations involved. The single slash, the double-slash, and so on, indicate the various orders of contextualizing or meta-redundancy relations, at increasingly higher orders of abstraction, as in our simplified example. Thus: x, y, z I a, b, c I I p, q, r. Given that there is no directionality or causality implied, this may be read either way, according to the specific analytical or situational requirements and contingencies. In other words, p, q, r redounds with the redundancy of a, b, c with x, y, z; or, x, y, z redounds with the redundancy of a, b, c with P: q, r. Each of the three orders or levels of relations in our example may correspond to the different levels of organization (strata) in some semiotic system. One such example would be the tri-stratal system of language, comprising phonologyI graphology, lexicogrammar, and semantics. There is neither directionality nor causality because redundancy is a principle which implies no separation of the levels involved into separate ontological domains. Instead, it expresses how patterns of relations emerge and are construed in and through the hierarchies of contextualizing relations involved. In Flohr's description of 'activity-dependent self-organization of neural nets', the same general principle of redundancy operates. I have attempted to approximate the meta-redundancy relations as follows: stimulus pattern I assembly formation I I representation of spatio-temporal pattern The generation of increasingly higher orders of abstraction from the 'stimulus pattern' may have important consequences for theories of human development. The question then arises as to how such patterns of neural activity in the brain map onto emergent semiotic activity. This also raises the question as to how such higher-order social semiotic processes and dynamics act on and shape brain function and activity. This requires a theory of semiosis and the place of the brain in such a theory which is truly dialogic in character (see Thibault 2000a).

302

BRAIN, MIND, AND THE SIGNIFYING BODY

8. A Definition of Consciousness in Terms of the Semiosis that Occurs Within the Brain Consciousness of phenomenal states, or conscious experience of phenomena, does not require either mind-brain dualism or physical reductionism for its explanation. Both conscious experience and self-consciousness awareness or knowing that one's self is undergoing that experience are emergent properties of higher-scalar global patterns of neural organization that realize semantic properties. Furthermore, emergent semantic properties which are realized by global patterns of neural organization constrain both the possibilities for conscious experience of phenomena as well as the possibilities for action in response to these phenomena by entraining lower-scalar sensori-motor dynamics to their semantics (juarrerc 1999: 172). In this way, our conscious experiences of phenomena and our responses to them are semantically constrained by the selforganizing properties of the higher-scalar system of interpretance in the brain. Consciousness cannot be reduced to or viewed as identical with physical brain processes. It was Descartes who first understood that consciousness is not founded on the opposition between raw sensations and thoughts. Descartes's dualistic solution was to postulate that mind and brain are entirely separate entities. However, Seager (1999: chapter 1) has cogently argued that, without accepting Cartesian dualism, which is clearly false, there is a more important and positive lesson to be learned from Descartes's philosophy of mind. That is, consciousness is linked to 'ideas', which are basically representational. Therefore, every conscious experience is an experience of a representation of an idea in the mind. I shall argue below that this essential insight about the nature of consciousness can be entirely reconstituted within the ecosocial semiotic conceptual framework of the present study. In Descartes's view, neural firings, in modern terminology, realize what Descartes called ideas. Ideas can be about the outside world as perceived through our perceptual systems, our inner and outer bodies, as well as imagined ideas elaborated by the mind. Consciousness, in this view, is consciousness of the conceptual representations - the ideas - elaborated by the mind. We have conscious experiences of such representations and not the physical brain processes which realize these. Mind, in this view, is a system of meanings which are realized by, though not reducible to physical brain processes. I am using the term 'meaning' here in an extended sense to cover all semiotic processes that are encompassed by the semiotic hierarchy of iconic, indexical, and symbolic meanings. In this view, conscious experience encompasses meanings based on sensori-motor activity, pre-linguistic conceptual routines, and the higher-scalar symbolic resources typically associated with culture (e.g. language, depiction, and so on). Seager (1999: 188-9) points out the conceptual character of our consciousness of phenomena. That is, our consciousness of the perceived world, for instance, is always consciousness of bicycles, people, clouds, trees, and so on. Consciousness is perfused by the perceptual and conceptual categorizations characteristic of Edelman's primary consciousness. The interaction between memory and the neuronal groups that constitute the systems of perceptual and conceptual categorizations within the brain means that the categorizing activity

BRAIN, MEANING, AND CONSCIOUSNESS

303

of global mappings of neurons constitutes a system of interpretance whereby conscious experience is always consciousness of something as something. In Seager's (1999: 184) terms, we are always conscious of something in the sense that the intentional object of our consciousness is an object which is presented to our consciousness 'under a certain aspect'. This property of entering consciousness under a certain aspect is further extended, refined, and specified by the systems of linguistic categorization of a given language system such that we can, for example, experientially construe a given phenomenon of experience as an instance of a particular experiential category. These phenomena of experience do not have to be given to perception, but may be remembered, imagined, or hypothesized phenomena which are symbolically construed as intentional objects that can be made available to consciousness. Consciousness is semiotic interpretation of phenomena. Descartes's 'ideas' are meanings, however elementary, which cannot be reduced to physical interaction between organism and some physical stimulus such as the light waves which register on the retina of the eye or the sound waves propagated from a source which register on the auditory nerves. At the lowest-scalar level of perceptual activity, Gibson (1983 [1966]: 40-1) has identified the receptor cells which respond, for example, to mechanical, chemical or light energy. The interaction between the organism's receptor cells and the energy responded to is physical interaction. But Gibson identifies a scalar hierarchy, comprising receptors, organs, and perceptual systems, which is involved in the perception of some environmental event. Organs, Gibson (1983 [1966]: 41) writes, are 'functional, not anatomical, units' in a hierarchy of units and functions as summarized in Table 0.2. Gibson's point is that organs and perceptual systems are higher-scalar levels which mediate the lower-scalar level of the physical interaction between receptor cells and the energy which is applied to these. Organs and perceptual systems are adaptive; they are not passive mechanisms which simply receive a physical stimulus. Rather, they actively and adaptively respond to and modulate information about their environment. In this way, the organism as a whole, rather than the single receptor cell, responds to environmental information as signs of environmental events. That is, the organism is responding on a higher-scalar level than the physical stimulus which is applied to the receptor cells. Organs, perceptual systems and the cross-modal information from these which is reentrantly mapped in the brain function as a higher-scalar system of interpretance. Lower-scalar physical stimuli are reinterpreted at higher levels as (iconic and indexical) signs of events both in the external world and within our bodies which are available to conscious experience. Conscious experience is, then, intrinsically semiotic in nature because it involves interpreting information which is picked up by our perceptual systems as signs of experienced phenomena in relation to which the organism can organize a required response at a higher-scalar level. The processes involved here are physical processes involving interactions between receptors, organs, perceptual systems, the CNS, and stimulus energy in the environment. Yet, consciousness is not reducible to physical processes per se. After all, we have no conscious awareness or experience of the neural processes

304

BRAIN, MIND, AND THE SIGNIFYING BODY

Table 6.2: Scalar hierarchy of anatomical structures ofperception and relative functions; summarized from Gibson (1983 [1966J: 40-3) Anatomical Unit

FWlction

Receptors: cells and units

Microscopic structures, e.g. the photosensitive rods and cones of the retina, usually in groups of such cells. Receptors are connected by afferent neurons to the CNS. They register the sequence or arrangement of stimulation, that is, information, not the energy of stimulation.

Organs

An organ of sensitivity is a structure containing many

receptors that can adjust so as to modify the input from cells and units. An organ has muscles and these are supplied with efferent fibres from the CNS. Organs of sensitivityexist in a hierarchy: lower organs are subordinated to higher ones. Systems (of Organs)

A system has very many levels of function ranging from lower-level ones such as the single eye with adjustable lens for sharpening the retinal image and the pupil for adjusting intensity; the eye with attached muscles is a stillhigher-order system, stabilized in the head relative to the environment and with the help of the inner ear, it can scan the environment; the two eyes together make a dual system of still-higher-order, such that they converge for near objects and diverge for far ones; the two-eyes-withhead-and-body system, in cooperation with postural equilibrium and locomotion, for moving around in environment and looking at things.

which support, sustain, and realize conscious experiences in the brain. We have no conscious experience of the stimulus energy to which the receptor cells respond. Instead, conscious experience of phenomena involves a reinterpretation of information across scalar levels (Lemke 1999). Thus, the microscopic energy fluctuations to which the receptors respond are re-interpreted on higher-scalar levels as potentially meaningful information about (inner or outer) environmental events. Furthermore, such information constitutes the basis for a variety of responses or action repertoires in the sense that the same information can be variably interpreted and responded to according to differing contextual factors at higher scales. In the case of symbolic interpretation, which is mediated by a system of interpretance on the ecosocial scale where socio-cultural practices enter the picture, information from lower levels is constrained by social conventions and practices emanating from the ecosocial scale. Conscious experience of perceptual and other phenomena involves a semiotic process of interpretation by integrating contextual factors on different scales. There is nothing deterministic about the processes involved. Instead, there are complex inter-level relations among factors on many different scales which combine to produce emergent higher-order conscious experiences which can be used by the organism in adaptive ways.

BRAIN, MEANING, AND CONSCIOUSNESS

305

Descartes's view that nerve actions yield ideas in the mind and that it is these ideas which enter consciousness can be understood as meanings which are the emergent result of the inter-level interactions across many different scalar levels. For instance, firings in neural networks are reorganized (re-interpreted) as more global cortical effects. These effects are, in their turn, reorganized as higherscalar sensori-motor categories such as a perceptual experience of some phenomenon or the phonological categories of a given language. These categories - perceptual or phonological - in turn constitute an attractor space which guides and monitors motor action along a trajectory. The co-ordinated action produced by vocal-tract and other gestural activity in response to environmental events is an example of this. In turn, gestural activity is re-interpreted at the ecosocial scale as symbolic activity in and through the categories deriving from the paradigmatic contrast sets of a given system of interpretance. In Peircean terms, environmental information is a representamen, R. However, the R does not directly determine its object, X, which is a given environmental event. R is a sign of X according to the way it is mediated by a higherscalar SI. That is, R is a sign of X - an interpretant sign - according to the way in which the SI in operation interprets Rs. Is R food, therefore reason for attack, or a predator, therefore a reason for flight? Such interpretations will depend on the appropriate higher-scalar system of interpretance - the system of perceptionaction categories stored in the organism, though ultimately derived from its transactions with its higher-scalar ecosocial environment. The ecosocial environment (semiotically) mediates lower-scalar signs such that signs of environmental events are appropriately interpreted and responded to. Descartes's 'idea' is then a representamen, R, about some object, X, i.e. some event in the world or within the body of the perceiver. The higher-scalar SI assigns an appropriate interpretation to the relationship between Rand X. Conscious experience of phenomena is experience of grounded instances of categories of phenomena, rather than of pure sensations or a continuous flux of stimulus energy. When I look up from my computer while writing this to observe from my window, I see children cycling in the street below, trees in the garden, apartment buildings, a park on the other side of the street, the hills of Bologna in the distance, a couple of cranes towering above a nearby construction site, an aeroplane taking off in the distance, the planet Venus making its appearance in the early evening sky, and so on. I do not see an indistinct flux of visual sensations. Instead, the objects of my perceptual experience - the objects of consciousness - are interpreted by a higher-order system of interpretance as instantiations of categories which derive from and which are meaningful relative to this higher-scalar SI. In other words, meaning, though not necessarily symbolic or discursively organized (e.g. linguistic) meaning, constitutes the perceptual, the conceptual, and also the symbolic categories in and through which consciousness is itself constituted. This observation accords, in some ways, with Descartes's claim that perception is judgement. That is, seeing is judging in the sense that seeing is not a matter of raw (visual) sensations impacting on the eye. Rather, perception involves interpretative judgements about the perceived phenomena which are construed as objects of consciousness (Seager 1999: 5; see also this volume, chapter 4, section 16). In our terms, it involves higher-scalar processes of interpretation which

306

BRAIN, MIND, AND THE SIGNIFYING BODY

assign meaning to lower-scalar phenomena according to the categories of the SI which is in operation. The system of perceptual categories which is stored in the organism's brain is a higher-scale SI at level L+1 whereby the stimulus information which is picked up by the perceptual systems on the lower level L is as Lemke (1999) points out doubly construed. First, the stimulus information which is picked up and construed as a representamen is a given pattern of material (matter-energy) interactions at level L-l. Secondly, this pattern of material interactions is construed by the SI on level L+1 as having a meaning (an interpretant) which is an object of consciousness on the scalar level of the SI. That is, on the level where phenomena are construed as objects of conscious experience and awareness. The material interactions on level L-l do not in themselves count as phenomena of experience. To be a phenomenon of conscious experience, it has to be an interpreted phenomenon - one that is contextually closed by an irreversible act of amplification such as is made possible by the categories deriving from the higher-scalar SI on level L+l (see Wheeler 1988: 10). Only on this level does the object have relevance as a perceptual or other object of conscious experience. To be sure, when I look out of my window to view the scene before me, light energy registers on the receptors in the retina of my eyes. However, I am not conscious of this light energy or the effect it has on my receptors. Instead, I have a conscious experience of children riding their bicycles in the street below, and so on (see above). The light that reaches the receptors in the retina of my eyes is reinterpreted as environmental information that indexes an environmental event, viz. children riding their bikes, ete. The stimulus information about the environment which is picked up by my perceptual systems is meaningful for a given SI because it is a sign of some phenomenon of experience on the scale of the SI. This is the scale, for example, of the speciesspecific organism-environment interactions studied by Gibson (1986 [1979]: 8-12). Lemke (1999: 3) has re-interpreted Peirce's claim that the interpretant in its turn becomes a representamen that 'determines some still further interpretant', which in Peirce's work constitutes the basis for the notion ofa chain of indefinite semiosis, as the basis for an 'indefinite (open-ended) hierarchy of scale levels of systems of interpretance' (Lemke 1999: 3). Such an open-ended hierarchy can be used to conceptualize the relations between conscious experience of perceptual, conceptual, and symbolic categories of experience, respectively. Each of these three levels is describable in terms of the three-level hierarchy such that in each case the L+1 scalar level of the SI where the interpretant of perceptual categories is located becomes, in turn, the representamen of the next-highest triplicate of scalar levels. Figure 6.3 provides a schematization of the ways in which consciousness of perceptual phenomena is integrated to consciousness of conceptual phenomena, which is integrated to consciousness of symbolic phenomena in accordance with the logic of specification-hierarchy thinking. Seager (1999: 6) has pointed out that we are not conscious of the physical brain states and processes which support and in some sense generate conscious experience and awareness. Instead, we are conscious of the representations

BRAIN, MEANING, AND CONSCIOUSNESS

307

Figure 6.3: open-ended hierarchy of semiotic triplicates of levels showing the integration ofperceptual, conceptual, and symbolic phenomena of conscious experience Hierarchy of scalar levels, showing triplicates of levels comprising systems of perceptual, conceptual, and symbolic interpretance Symbolic-discursive L+l

SI comprising ecosocial semiotic systems of meaningmaking resources and conventions where symbolic (semantic) interpretants are located as objects of symbolic consciousness

L

Phonology/graphology and lexicogrammar

L-l

Conceptual and sensori-motor routines

Conceptual-mental L+l

SI comprising system of conceptual categories where object of conceptual experience in the form of mental images elaborated in the brain on the basis of the recategorization of sensori-motor routines

L

Sensori-motor routines

L-l

Object of perceptual experience

Perceptual L+l

SI comprising perceptual categories and values where object of perceptual experience is located

L

Stimulus information as sign of environmental event

L-l

Light, mechanical, etc. energy modulating nerve impulses in receptors

which physical brain processes produce. To take an example referred to by Seager, we are conscious of our experience of the colour red rather than the neural activity within our brain which generates this experience in our consciousness. In Seager's terms, which are adapted from Descartes's distinction between formal and objective reality, we are conscious of what is represented rather than the formal means of its representation. Seager (1999: 175-6) considers this aspect of consciousness puzzling in comparison with all other representational systems in which we are simultaneously aware of both the formal means of representation and the represented content. In my view, this distinc-

308

BRAIN, MIND, AND THE SIGNIFYING BODY

tion between formal means and represented content is itself unnecessary when seen in the light of the semiotic triplicates of levels discussed above. Conscious awareness therefore requires the connection of physical brain processes that are below the threshold of our awareness to meanings that are recognized and responded to on a higher-scalar system of interpretance within the overall system of relations involved. Consciousness means connecting to a meaning, whether this be iconic, indexical, or symbolic, or even all three simultaneously. Vague sensations, mental images, perceptual awareness, and language all entail the bringing into awareness of neural meanings that are themselves brought about on lower-scalar levels by the activations of neural maps and the communications that occur between these. The crossing of the threshold into consciousness awareness - i.e. from a prior state of neural activation below that threshold means that lower-scalar neural activities are semiotically reorganized on higherscalar levels such that they enter into contextual redundancies with emergent signs - iconic, indexical, symbolic. It is these signs which enable the lower-scalar activity to emerge in consciousness as an awareness of something, be this a sensation (iconic), or an elementary association between two phenomena (index), a mental image or a clause (symbolic), and so on (see also Beaugrande 1997: 114). In this view, discrete neural firings are reorganized as globally coherent phenomena at a higher-scalar level than the individual firings to produce a particular perceptual, conceptual, or symbolic (e.g. linguistic) categorization of some event, whether this occurs within the brain, in the body, or in the outside world. Meaningful patterns are emergent across global neural activity such that they are co-ordinated on higher scales of globally organized neural activity on larger timescales to produce the emergent phenomena of conscious experience. The process is a stochastic or probabilistic one insofar as particular configurations of neuronal groups that subtend a specific categorization are the most likely ones to be activated for various instances of a particular event. In terms of Lernke's (1999) Principle of Alternation, we can say that the transformation from discrete, local neural firings to global configurations on higher scales and the transformation of such global configurations to objects of conscious experience within the brain constitutes a semiotic transformation, as Lemke says, 'of the information content of lower levels as signs for higher levels' (1999; italics in original). Consciousness of the objects of experience is the result of dynamical processes across many different scales of organization within the brain such that consciousness is an emergent level of organization relative to the observational scales of the self. The objects of our conscious experience and awareness are objects which have information value for the higher-scalar system of interpretance, comprised of value-laden memory in interaction with globally organized neural groups that function to activate perceptual, conceptual, and symbolic categorizations. The systems of interpretance within the brain are themselves built up over time on the basis of the cross-modal sampling and reentrant mapping of the body-brain's transactions with its inner and outer environments. Each of these levels of categorization is characterized by its own systems of types and at the same time the 'lower' levels are integrated to and reorganized by the higher levels such that perception, for example, may be

BRAIN, MEANING, AND CONSCIOUSNESS

309

suffused with conceptual and symbolic categorizations. This does not alter the fact that different selection pressures act on different individuals differently along their individuating trajectories such that the neuronal configurations of different individuals exhibit considerable diversity. By the same token, such diversity is also entrained to the higher-scalar relations and dynamics of a given ecosocial semiotic system. We are conscious of the phenomena which the higher-scalar systems of interpretance in our brain recognize as differences which make a difference, i.e. which have informationalsemiotic value to that system. For this reason, many lower-scalar neural processes are no more than points in an overall phase space leading into the same higherscalar attractor. These lower-scalar processes include, of course, the configurations of neuronal activity which, to use Seager's term, 'represent' the contents of our conscious experience. It is the attractors and their basins on higher levels that specify, semiotically, what can be recognized as an object of consciousness at the same time as our interpretation of a given object of consciousness under a given aspect - perceptual, conceptual, or symbolic - leaves open the many degrees of topological freedom of its matter-energy flows. The view presented here obviates the need for a 'formal reality' which 'represents' the contents of conscious experience because it is now clear that 'what is represented' is a result of the semiotic closure that arises from the dynamics of the higher-scalar system(s) of interpretance within the brain. Lower-scalar neural firings, for example, do not formally express higher-scalar content; instead, they constitute it by being entrained to the dynamics of the more global levels of organization where conscious experience emerges in the perspective of the self. It is the expression stratum of semiosis which provides the means whereby the systems of categorization within the individual's brain may be communicated to other brains. This does not occur on the basis of any kind of transfer of coded thoughts or ideas from one brain to another. The dynamics of the body-brain complex as a whole is entrained to the dynamics of the higher-scalar ecosocial system. For this reason, both the systems of conceptual-semantic discriminations and sensori-motor ones are recategorized as the lexicogrammatical and semantic and phonological distinctions of the language system which is distributed across the participants in a particular social network on different space-time scales. In this way, the cross-eoupling of sensori-motor activity to lexicogrammatical and discourse patterns of organization in the production of meaning-making activities and the texts that may be derived from these activities allows for both embodied sensori-motor repertoires and conceptual repertoires stored in the brain to be re categorized as phonological and lexicogrammatical and discourse systems and structures which are semiotically salient in a particular ecosocial semiotic system or some part of this. These systems of distinctions on the expression and content levels of semiotic organization enter into the hierarchies of contextualization relations - Lernke's meta-redundancy relations - such that sensori-motor activity acquires meaning beyond that of mere body movement or vocalizing per se. The assigning of meaning to such activity occurs because a particular sensori-motor routine or body movement is construed as having a regular pattern which redounds with some higher-order cultural or discursive formation or genre convention.

310

BRAIN, MIND, AND THE SIGNIFYING BODY

The fact that such higher-order me ta-redundancy relations are distributed across and hence recognized as semiotically salient in a given community means that the participants in the meaning-making and text-creating activities of that community have access to or participate in meaning-making practices whereby they define and recognize the regular patterns in meaning-making activity and textual artefacts. Rather than encoding thoughts for transmission from one mind to another, these regular patterns are semiotic affordances which allow participants to re-interpret these patterns according to systems of interpretance within their brains. Such processes of interpretation conform, to degrees, to the relatively stable hierarchies of contextualizing relations that are operational within a given community. Therefore, perceivable patterns of embodied activity and their entextualizations can be re-contextualized in some other individual's brain as semantic or other semiotically salient categories which make a difference in both understanding and action. A semiotically salient difference is a value-laden difference in the sense that it makes a difference by virtue of its systematic relations with other differences within the overall meaning system. This suggests that Edelman's notion of value-category memory can be extended to the ecosocial scale of organization (Thibault 2000a). This is so in the sense that the value-categories or semiotically salient differences that operate on spacetime scales considerably greater than either the individuallifespan trajectories or the much smaller-scale logogenetic or meaning-making trajectories constitute a transindividual, cross-scalar ecosocial memory to which individuals are entrained.

9. The Three-level Hierarchy, Neuronal Activity, and the Emergence of the Self-perspective The self-perspective is not produced by neuronal activity. Instead, it is the selforganizing activity of the neuronal circuitry itself. Self-organizing systems are complex systems which have a history and individuality. This means that the many degrees of freedom of an individual's neuronal activity permit the construction of a continuity of individuality - a self-perspective - from one changeof-state in the system to another. Cerebral neurons are massively interconnected in ways which allow for very many degrees of freedom. It is these degrees of freedom which build time and individuality into the structure and organization of the cerebrum. Like all complex systems, an individual's neural networks can be in states which are different from the states of other individuals. This is possible because the individual's neural activity has a history of activity and interconnections which can never be exactly the same as another individual's. For this reason, the self-perspective described by Flohr is an emergent property of neural activity itself. In other words, an individual perspective on one's own neural activity arises as a consequence of being in a particular brain state. It is in this sense that we can say that consciousness is a result of the formation of highly specified neural systems of which an individual's self-organizing neural networks are a particular case. As Kinsbourne (1988: 246) points out, neurons are characterized by the property that when they interact they produce awareness.

BRAIN, MEANING, AND CONSCIOUSNESS

311

Consciousness is an emergent property of the interactions among neurons in a neural network. The same argument applies to the sense of a self who organizes and interacts with the self's own stream of consciousness. Patterns of neural activity undergo constant change. A given change in neural activity may correspond to a particular brain state. A brain state is a change in the overall neural network in response to a given input (Kinsbourne 1988: 242).The system of neural activity is able to constitute the 'before' and 'after' of a given brain state in ways which enable a continuity of individuality to be constructed. The individual character of self-organizing neural activity is a consequence of neural activity's network characteristics as well as the operation of excitation-inhibition balances which can either separate the cortical activity of a particular mental operation from the cortical activity of other mental operations or integrate it with other cortical activity. The self-organizing activity of the individual's neural activity has a history of the ways in which neurons are interconnected on the basis of their network characteristics. Furthermore, mental operations and the cortical activity that instantiates these can be separated from or integrated with other mental operations through the modulation of the excitation-inhibition balance. Both the network characteristics of cerebral activity and the excitationinhibition balance can be used to explain consciousness in terms of the threelevel hierarchy. This is a minimal assumption; it is not intended to explain the detailed neural architecture of the brain. Rather, it can help to sort out the relations between the various levels involved. According to Salthe's three-level paradigm, the focal level L is an emergent phenomenon which is constituted by the interactions among variables on some characteristic scale. Units on level L are constituted by the interactions among lower-scalar units on level L-1. For the sake of the present argument, let us say that level L-I refers to the interconnections among cerebral neurons whereby communication occurs between neurons. Level L units are mental operations which are constituted by the interactions among neurons on level L-l. Communications among neurons on level L-I take place on the basis of impulses that are carried by the fibre tracts that reciprocally link grey and white matter. The resulting cortico-cortical connections give rise to the cortical activity that instantiates a particular material operation on level L. We have seen that mental operations on level L are constituted by smaller-scale, faster interactions at level L-l among the cerebral neurons at that lower level. Now, mental operations on our putative level L can, as I pointed out above with reference to the work of Kinsbourne, be insulated from or integrated with other mental operations by virtue of the excitation-inhibition balances in operation. Kinsbourne argues that 'to the extent a mental operation is selectively insulated, it is automatic, and outside awareness, to the extent it is co-ordinated with other mental operations, it contributes to the "integrative field of awareness'''. By the same token, not all of the possible communications which might occur on level L-l do actually occur. The interactions which produce mental operations on level L are constrained by boundary conditions which are determined at the higher-scalar level L+1. Mental operations on level L are instantiated by areas of cortical activation. Moreover, 'each cortical area

312

BRAIN, MIND, AND THE SIGNIFYING BODY

represents input within the constraints of its neural specialization and current state of interrelations with other areas (including activating and inhibitory influences). Some representations are available as contributions to the field of conscious awareness and, if attended, enter it' (Kinsbourne 1988: 240). Level L+I imposes contextual constraints on level-L interactions. Further, only those interactions on level L will be meta-stable on the appropriate timescale if they conform to the constraints imposed by level L+ 1. In this perspective, the interrelations between different areas of neurally specialized cortical activity, including the modulation of the excitatory influences between areas, leads to the integration on level L+I of diverse mental operations to the integrative field of awareness identified by Kinsbourne. Rather than information processing, which would presuppose a one-to-one mapping between representations and outputs, the three-level hierarchy model shows that the realizatory relationship between neuronal activity on level L-I and mental operations on level L is a many-to-one relation. Representation implies redescription, which leaves no room for the kind of reorganization across scalar levels of a neuronal network that is fundamental in the perspective of the threelevel hierarchy. Re-description entails that discrete variants on one level are mapped to discrete variants on the next-higher level, or, alternatively, that continuous variation is mapped to continuous variation, again at the next-higher level. In each case, there is re-description, but not reorganization, across levels. However, Lemke points out that the significance of emergent new levels of intermediate organization lies in the ways in which each higher level has many possible realizations at lower levels. That is, there are many more degrees of freedom on the lower level than are salient on the higher level. In this way, a many-to-one mapping from level L-I to level L allows for the many degrees of freedom of the information on level L-I to be filtered or buffered. Neuronal interconnectivity on level L-I is activated as more global, larger-scale cortical activity realizing mental operations on level L. Furthermore, as we saw above, a mental operation on level L can be integrated to and interpreted as a more global neural network which embodies a conscious experience on level L+ 1. Level L+I is the higher-scalar, slower timescale of neuronal activity which integrates the cortical activity of diverse mental operations on the level below to its dynamics. Lernke's Principle of Alternation suggests that discrete neural firings on level L-I are reorganized as more global, continuous mental operations as one moves from lower to higher levels in the hierarchy. Information on level L-I is transformed as signs for higher levels such that both many-to-one classifications (level L-I to level L) and one-to-many context-dependent reinterpretations (level L to level L+I) are possible. Consciousness is an emergent level of organization whereby level L entities are re-interpreted on level L+I as signs in the perspective of the self of microstates of the system at level L-I. When neural activity on level L-I is a consequence of interaction with the environment at level L-I, higher levels of the system respond to them as signs at level L of phenomena in the environment at level L+I. Consciousness of phenomena in the environment of the self is a higher-scalar attractor state of the global neural network which may not, as Lemke points out, have any direct material-causal relationship to level-L-I interactions.

BRAIN, MEANING, AND CONSCIOUSNESS

313

The three-level hierarchy is incompatible with both dualism and reductionism. In the first instance, consciousness and the sense of a self, viewed as emergent properties of neuronal networks, do not transcend materialism in the way that mind-body dualism suggests. Instead, consciousness, mind, and self are higher, larger-scale meaningful patterns of organization of lower, smaller-scale neurophysiological processes. There is no suggestion that the higher-scale processes have an existence independent of the lower-scale ones. Moreover, the higher, larger-scale processes are emergent phenomena whose dynamical processes and principles of organization cannot be reduced to or causally explained in terms of the lower, smaller-scale ones. In the second instance, the three-level hierarchy view does not support the reductionist view that consciousness, mind, and self are reducible to and explainable in terms of lower-scale neuronal processes per se as being the more fundamental causal level. Reductionism precludes emergence because it entails that the lower level constitutes the more fundamental level and that everything else is epiphenomenal to this. The perspective afforded by the three-level hierarchy view as presented here does not, of course, resolve all of the problems of consciousness. For example, the present account does not provide a detailed account of the structure and organization of the cerebrum and its role in explaining the brain basis of consciousness. However, it does provide a perspective in which the tight linkages across the different scalar levels are seen as linking our material interactions with our inner and outer environments to our categories and ways of making meaning such that the body-brain complex is linked to the larger scales of ecosocial processes and their dynamics, including the always constructed trajectory of the self (chapter 4, section 3, pp. 176--84) (see Lemke 2000a: 193). More generally, it also helps us to see human consciousness as a highly specific case, at least from our own viewpoint, of more general types of complex selforganizing systems that exhibit properties of emergent organization, individuality, and developmental and individuating trajectories. Moreover, it shows that persons are embedded in and participate in ecosocial practices by virtue of the ways in which they construe meaningful pattern in their material interactions with their environment across diverse scalar levels. In this way, the distributed nature of the neural networks of consciousness in the individual's brain can be linked to its higher-scalar boundary conditions and its lower-scalar initiating conditions in a conceptually more unified way.

Epilogue

Towards a New Way of Thinking and Talking about Meaning-making This book isjust the beginning of an attempt to rethink meaning-making activity from the perspective of the body-brain system - the signifying body - embedded in its ecosocial semiotic environment. The key terms in this endeavour include activity, agency, body-brain system, contextualization, cross-eoupling, differentiation, dynamic open systems, ecosocial environment, embodiment, individuation, meaning-making, metafunctions, multimodality, process, scalar hierarchy, selforganization, specification hierarchy, system, timescales, topological, trajectory, typological, and value. As a result of the way of thinking that has been partially explored in this book, many previous habits of thinking in linguistics and semiotics now seem superseded and are no longer useful tools for thinking. These include the following dualisms: biology/society, behaviour/cognition, brain/mind, communication/cognition, form/meaning, mind/body, meaning/matter, nature/nurture, organism/environmen t, perception/semiosis, signifier/ signified, and structure/function. Some of the fundamental premises that I have presented in this book may be summarized as follows.

The Semiotic Activity of the Signifying Body Integrates Diverse Space-time Scales The notion of the 'signifying body' brings together the idea of a unitary bodybrain system, our socially constructed sense of an embodied identity-in-time, and the ecosocial semiotic environment. The body-brain system regulates matter, energy, and information flows between itself and its ecosocial semiotic environment, and at the same time that the latter reorganizes body-brain dynamics and functions. Semiotic activity requires no prior representation of signs within some preexisting system. The available resources - semiotic and material - which afford meaning-making are not all simply stored inside the individual's head. Instead, they exist in the ecosocial semiotic environment in the form of, for example,

EPILOGUE

315

other persons, artefacts, tools, texts, technologies, and so on, that can be integrated to and exploited in meaning-making activity. The structure of semiotic activity self-organizes in relation to its ecosocial environment. There is no single locus of control in the form of internal programs, competences, or plans. Rather, control is distributed along the entire trajectory that loops between organism and its ecosocial environment. Three-level hierarchy thinking shows that lower-scalar biological initiating conditions on level L-l, e.g. the many degrees of topological freedom of the vocal tract, or of hand-arm activity, provide both affordances and their own material constraints, without entailing a central program that controls the system's activity. A higher-scalar system of interpretance in the ecosocial environment (level Lt l ) imposes boundary conditions on the kinds of organismenvironment transactions that can occur on the intermediate level L. Meaning-making activity is a result of the dynamical interplay and integration of processes on many different levels. The structure of the activity is shaped by the way it interfaces with the body-brain system, on the one hand, and with the ecosocial semiotic environment, on the other. In this way, the signifying body is contextually integrated, through the mediating activity of semiosis, with diverse space-time scales that go far beyond the body's immediate material interactivity with its here-now environment.

Brain Activity Regulates Body-environment Relations and, at the same time, Body-environment Relations Organize and Shape Body-brain Systems and Functions Neural activity regulates body-environment relations by producing sensorimotor outputs whereby responses to environmental events are produced and projected back into the environment. The infant's high-pitched squeak in Halliday's (1993) example, as discussed in chapter 4, section 10, pp. 202-5, is a response in this sense. The infant's proto-linguistic squeak is a response to an event - the scattering of the pigeons - on the here-now scale. However, there is nothing which necessarily connects it to that event. In this sense, the infant's vocalization can be said to stand in a proto-symbolic relation to the environmental event to which it responds. It is only the connectedness of the infant's vocal-tract activity to the stimulus information (the sound) that is produced by his vocal-tract activity and which can be picked up by others that is, strictly speaking, indexical. On the other hand, the relation between the infant's vocalization and the environmental event - the pigeons - to which it constitutes a response, is protosymbolic from the point of view of observers. This is so because the transduction in an observer system (e.g. the mother) of the infant's sensori-motor activity as information about an event (the pigeons) that is not necessarily (indexically) tied to the infant's own body affords both dialogic engagement with - e.g. responding to and being responded to - the nonself and jointly attending to some phenomenon of experience. Thus, we see that neural activity regulates body-environment relations in ways that are not tied to external events in a fixed

316

BRAIN, MIND, AND THE SIGNIFYING BODY

relation of indexical necessity. This allows both for creativity and for the decreasing dependence of brain-regulated body-environment relations on the immediate here-now scale.

Body-environment Relations and Body-brain Systems and Functions are Shaped by Metafunctional Principles of Semiotic Organization At the same time, body-environment relations contribute to the reorganization of body-brain functions along progressively metafunctional lines as the infant's semiotic activity is increasingly hitched to and entrained by the models of discourse and society that are provided by more senior partners in meaningmaking. The vague outlines of the metafunctions are therefore evident in the infant's vocalization and can be related, as we saw in chapter 5, to the shape of consciousness itself.

Brain Activity is Contextually Integrated to and Directly Participates in Discourse, Rather Than Causing it Distributed patterns of neural activity within the networks of connections in the brain are contextually integrated to particular kinds of meanings and events in the ecosocial environment. A particular pattern of neural activity on its scalar level is integrated to patterns of activity on other scalar levels in the formation of stable categories of meaning and experience. In this view, all is activity and process on many different scalar levels of organization. There is no need for an inner program or homunculus which programs the activity. Nor is there any ultimate, lower-level substance on which the higher levels are supervenient (Bickhard and Campbe1l2000: 329-32).

Alterity is a Primitive Intrinsic Value that Motivates Self-nonself Relations and Meaning-making Activity The dialogical orientation of the self to the nonself is intrinsic to our biological inheritance from the outset. Alterity is therefore a fundamental organizing principle of the self right from the outset. In this sense, it is a biologically intrinsic value which selectively nudges the infant along some developmental pathways rather than others (chapter 1, section 11, pp. 49-54). It is a principle which is both 'inside' and 'outside' the individual by virtue of the fact that meaning-making is a distributed activity between body-brain systems and their ecosocial semiotic environments on diverse scalar levels of spatio-temporal and semiotic organization.

EPILOGUE

317

(Self)Consciousness is a Dialogical and Semiotically Mediated Relation between Self and Nonself (Self-)consciousness is necessarily and irreducibly a dialogical and semiotically mediated relation between self and nonself. The inner, self-referential perspective of the self is constructed in and through a complex chain of contextualizing relations which integrates processes on very many different space-time scales, e.g. from the assembly formation of nerve nets in the brain to the generating of meta-perspectives on the dialogical refraction of the self through the perspective of the other in discourse, as in Volosinov's (1976: 86) definition of 'selfawareness' or 'self-consciousness'. The assembling of nerve nets is based on patterns of neural activation. In the brain, as the theories of Edelman, Flohr and Kinsbourne show, patterns of neural activation are related to each other in complex hierarchies. To use the Batesonian parlance, there are patterns of patterns of patterns of patterns ... of neural activity. These patterns form complex hierarchies of contextualizing relations both within the brain and with the diverse scalar levels of organization that link brain, body, and ecosocial semiotic environment. Flohr (1991: 258) points out that the formation of a self-referential inner perspective is a consequence of the formation of such hierarchies of patterns of patterns ... of neural activity to very high orders of complexity. Furthermore, the resultant self-referential perspective integrates to its own perspective the brain's representation of the various parts of the body, including its internal milieu, along with the meaning-making activity of the body, such that the selfjust is this integration of the body, its meanings, and its experiences, on all levels of the specification hierarchy, to the self-referential perspective that is realized in symbolic neural space in the brain.

Towards a Process Ontology of Social Being-in-doing The 'arrow of time' of irreversible processes refers to concrete, lived historical time. This is the time of all complex hierarchically organized, dynamic, adaptive and open goal-seeking systems. This conception now makes it possible, historically speaking, for the first time, to construct what Lukacs (1990: 111-12) has designated 'a unitary ontology' of ecosocial semiotic processes and, hence, of our identities-in-time. The ontological bases of this ontology are the physicalmaterial and semiotic-discursive cross-couplings and the irreversible timedependent processes and activities on many scalar levels of organization which constitute them. The self-organization of self and nonself and, therefore, of meaning, embodiment, and consciousness, is an irreversible historical and ecosocial semiotic process in this most fundamental way.

References

Abercrombie, David 1967. Elements of General Phonetics. Edinburgh: Edinburgh University Press. Andersen, Peter Bagh 2000. 'Genres as self-organising systems'. In Peter Bogh Andersen, Claus Emmeche, Niels Ole Finnemann, and Peder Voetmann Christiansen (eds), Downward Causation: Minds, Bodies and Matter, 214--60. Aarhus: Aarhus University Press. Armstrong, David F., Stokoe, WiIliam C., and Wilcox, Sherman E. 1995. Gesture and the Nature of Language. Cambridge: Cambridge University Press. Astington, J. W, Harris, P., and Olson, D. R. 1988 (eds) , Developing Theories of Mind. Cambridge: Cambridge University Press. Bakhtin, Mikhail M. 1981 [1975]. 'Discourse in the novel'. In Michael Holquist (ed.), The Dialogic Imagination:FourEssays, 259-422. Austin: University of Texas Press. Bakhtin, Mikhail M. 1986. 'The problem of speech genres'. In Caryl Emerson and Michael Holquist (eds), Speech Genres & OtherLateEssays, 60-102. Trans. Vern W McGee. Austin: University of Texas Press. Bakhtin, Mikhail M. 1990 [1924]. 'The problem of content, material, and form in verbal art'. In Michael Holquist and Vadim Liapunov (eds), Art and Answerability. EarlyPhilosophical Essays by M. M. Bakhtin, Trans. Vadim Liapunov, 257-325. Austin: University of Texas Press. Baron-Cohen, S. 1989a. 'Perceptual role-taking and protodeclarative pointing in autism'. BritishJournal of Developmental Psychology 7: 113-27. Baron-Cohen, S. 1989b. 'The autistic child's theory of mind: a case of specific developmental delay'. Journal of ChildPsychology and Psychiatry 30: 285-98. Bates, Elizabeth 1979 (with the collaboration of Laura Benigni, Inge Bretherton, Luigia Camaioni, Vicki Carlson, Karlana Carpen, and Marcia Rossner). The Emergence of Symbols: Cognition and Communication in Infancy. New York and London: Academic Press. Bateson, Gregory 1973a. 'The cybernetics of "self": a theory of alcoholism'. In Gregory Bateson, Steps to an Ecology of Mind, 280-308. London and New York: Granada. Bateson, Gregory 1973b. 'Redundancy and coding'. In Gregory Bateson, Steps to an Ecology ofMind, 387-401. London and New York: Granada. Bateson, Gregory 1973c. 'Problems in cetacean and other mammalian communication'. In Gregory Bateson, Steps to an Ecology of Mind, 334-48. London and New York: Granada. Bateson, Gregory 1973d. 'Style, grace, and information in primitive art'. In Gregory Bateson, Steps to an Ecology of Mind, 101-25. London and New York: Granada. Bateson, Gregory 1973e. 'The role of somatic change in evolution'. In Gregory Bateson, Steps to an Ecology of Mind, 316-33. London and New York: Granada. Bateson, Gregory 1980 [1979]. Mind and Nature: A Necessary Unity. London: Fontana. Bateson, Gregory 1987 [1951]. 'Information and codification: a philosophical approach'.

REFERENCES

319

In Jurgen Ruesch and Gregory Bateson, Communication: The Social Matrix of Psychiatry, 168-211. London: Norton. Battacchi, Marco W. 1996. 'Conscience de soi et connaissance de soi dans l'ontogenese'. Enfance 2: 156---64. Battacchi, Marco W. 1998. 'Self-knowledge and self-consciousness'. Dipartimento di Psicologia, Universita degli Studi di Bologna: Mimeo. Battacchi, Marco W. 1999. 'Consciousness, awareness, and experience'. Teorie & Modelli IV(3): 55-63. Battacchi, Marco, Battistelli, Piergiorgio, and Celani, Giorgio 1998. Lo Sviluppo del Pensiero Metarappresentativo e della Coscienza: Milan: Franco Angeli. Beaugrande, Robert de 1997. New Foundations for a Science of Text and Discourse: Cognition, Communication and the Freedom of Access to Knowledge and Society. Norwood, NJ: Ablex. Benson,James, Fries, Peter, Greaves, William, Iwamoto, Kazuyoshi, Savage-Rumbaugh, Sue, and Taglialatela, Jared. 2002. 'Confrontation and support in bonobo-human discourse'. Functions ofLanguage 9 (1): 1-38. Benson, James, Fries, Peter, Greaves, William, Iwamoto, Kazuyoshi, Savage-Rumbaugh, Sue, and Taglialatela,Jared. 2004. 'Language, apes, and meaning-making'. In Geoffrey Williams and Annabelle Lukin (eds), Language Development: Functional Perspectives on Species and Individuals, 91-111, London and New York: Continuum. Bemstein, Basil 1971. Class, Codesand Control, Volume I: TheoreticalStudies Towards a Sociology of Language. London and Boston: Routiedge & Kegan Paul. Bernstein, Basil 1990. Class, Codes and Control, Volume JY.. The Structuring of Pedagogic Discourse. London and New York: Routiedge. Bernstein, N. 1967. Coordination and Regulation of Movements. New York: Pergamon Press. Berthoz, Alain 1997. Le Sens du Mouvement. Paris: Editions Odile Jacob. Bhaskar, Roy 1979. The Possibility of Naturalism: A Philosophical Critique of the Contemporary Human Sciences. Brighton, Sussex: Harvester. Bickhard, Mark H. and Campbell, Donald T. 2000. 'Emergence'. In Peter B0gh Andersen, Claus Emrneche, Niels Ole Finnemann, and Peder Voetmann Christiansen (eds), Downward Causation: Minds, Bodies and Matter, 322--48. Aarhus: Aarhus University Press. Bogdan, RaduJ. 1987. 'Mind, content and information'. Synthese 70: 205-77. Bogdan, Radu J. 2000. Minding Minds: Evolving a Reflexive Mind by Interpreting Others. Cambridge, MA and London: MIT Press. Bohr, Nielsen 1948. 'On the notions of causality and complementarity'. Dialectica 7(8): 313-14. Bolinger, Dwight 1985. 'The inherent iconism of intonation'. In John Haiman (ed.), Iconicity in Syntax, 97-108. Amsterdam and Philadelphia: John Benjamins. Bouissac, Paul 1998. 'Converging parallels: semiotics and psychology in evolutionary perspective'. Theory & Psychology 8(6): 731-53. Bourdieu, Pierre 1977 [1972]. Outline ofa Theory ofPractice. Trans. Richard Nice. Cambridge and Melbourne: Cambridge University Press. Braten, S. 1992. 'The virtual other in infants' minds and social feelings'. In A. Heen Wold (ed.), The Dialogical Alternative: Towards a Theory of Language and Mind, 77-97. Oslo: Scandinavian University Press. Browman, Catherine and Goldstein, Louis 1991. 'Gestural structures: distinctiveness, phonological processes, and historical change'. In Ignatius G. Mattingly and Michael Studdert-Kennedy (eds), Modularity and the Motor Theory of Speech Perception, 313-38. Hillsdale, NJ: Lawrence Erlbaum. Browman, Catherine P. and Goldstein, Louis 1995. 'Dynamics and articulatory phonology'. In Robert F. Port and Timothy van Gelder (eds), Mind as Motion: Explorations in the Dynamics of Cognition, 175-93. Cambridge, MA and London: MIT Press. Buhler, Karl 1990 [1934]. Theory of Language: The Representational Function of Language. Trans. Donald Fraser Goodwin. Amsterdam/Philadelphia: Benjamins.

320

REFERENCES

Carruthers, Peter 1996. Language, Thought and Consciousness: An Essay in Philosophical Psychology. Cambridge and New York: Cambridge University Press. Catford, J. C. 1998. A Practical Introduction to Phonetics. Oxford: Clarendon. Chomsky, Noam 1965. Aspects of the Theory of Syntax. Cambridge, MA: MIT Press. Chomsky, Noam 1976. Reflections on Language. London: Fontana Clark, E. and Clark, H. 1977. Psychology and Language. New York: Harcourt Brace. Cleirigh, Chris 1998. A Selectionist Modelof The Genesis of PhonicTexture: Systemic Phonology & Universal Darwinism. Department of Linguistics, University of Sydney: Unpublished Ph.D. thesis. Cole, Michael 1996. Cultural Psychology: A Once and FutureDiscipline. Cambridge, MA and London: Harvard University Press. Coleridge, Samuel Taylor 1967. Biographia Literaria. London: Dent. Corina, David P., Vaid, ]yotsna, and Bellugi, Ursula 1992. 'The linguistic basis of left hemisphere specialization'. Science 255: 1258-60. Coulthard, Malcolm 1992. 'The significance of intonation in discourse'. In Malcolm Coulthard (ed.), Advances in Spoken Discourse Analysis, 35-49. London and New York: Routledge. Cowley, Stephen J. 1998. 'Of timing, turn-taking, and conversations'. Journal of PsycholinguisticResearch 27(5): 541-71. Cowley, Stephen J. 2001. 'The baby, the bathwater, and the "language instinct" debate'. LanguageSciences 23: 69-91. Cowley, Stephen J. 2002. 'Why brains matter: an integrational perspective on The Symbolic Species'. LanguageSciences 24: 73-95. Cowley, Stephen J., Moodley, S. and Fiori-Cowley, Agnese. In press. 'Grounding signs of culture: primary intersubjectivity in social semiosis'. Mind, Culture, and Activity 11.2. Cremonini, Giorgio and Frasnedi, Fabrizio 1986. Nell'Universo delSenso. Bologna: Il Mulino. Damasio, Antonio 1999. The Feeling of What Happens: Body, Emotion and the Making of Consciousness. London: William Heinemann. Davidse, Kristin 1991. 'Categories of Experiential Grammar'. Ph.D. thesis. Leuven: Department of Linguistics, Katholieke Universiteit. Davidse, Kristin 1992. 'A semiotic approach to relational clauses'. open Papers in Systemic Linguistics (OPSL) 6: 99-131. Davidse, Kristin 1997. 'The Subject-Object versus the Agent-Patient asymmetry'. Paper presented at the congress 'Objects, grammatical relations and semantics', University of Ghent, 23-24 May 1997. Davidson, RJ. 1984. 'Hemispheric asymmetry and emotion'. In K. Scherer and P. Ekman (eds), Approaches to Emotion, 39-57. Hillsdale, NJ: Erlbaum. Davidson, R J. 1992. 'Anterior cerebral asymmetry and the nature of emotion'. Brain and Cognition 20: 125-51. Deacon, Terrence 1998 [1997]. The Symbolic Species: The Co-evolution of Language and the Human Brain. London and New York: Penguin. Descartes, Rene 1960 (1641]. 'Meditations'. In Discourse on Method and Other Writings, 101-69. Trans. Arthur Wollaston. Harmondsworth: Penguin. Doty, R W. 1984. 'Some thoughts and some experiments on memory'. In L. R Squire and N. Butters (eds), Neuropsychology of Memory. New York: Guilford. Dretske, F. 1972. 'Contrastive statements'. Philosophical Review 81: 411-37. Edelman, Gerald M. 1987. Neural Darwinism: The Theory of Neuronal Group Selection. New York: Basic Books. Edelman, Gerald M. 1989. The Remembered Present: A Biological Theory of Consciousness. New York: Basic Books. Edelman, Gerald M. 1992. Bright Air, Brilliant Fire: On the Matter of the Mind. London and New York: Penguin. Edelman, Gerald M. and Tononi, Giulio 2000. A Universe of Consciousness: How Matter Becomes Imagination. New York: Basic Books. Finnemann, Niels Ole 2000. 'Rule-based and rule-generating systems'. In Peter B0gh

REFERENCES

321

Andersen, Claus Emmeche, Niels Ole Finnemann, and Peder Voetmann Christiansen (eds), Downward Causation: Minds, Bodies and Matter, 278-302. Aarhus: Aarhus University Press. Firth 1957a [1948]. 'The semantics oflinguistic science'. In Papers in Linguistics 1934-1951, 139-47. London and New York: Oxford University Press. Firth,]. R. 1957b [1948]. 'Sounds and prosodies'. In Papers in Linguistics 1934-1951, 121-38. London and New York: Oxford University Press. Firth,]. R. 1957c [1951]. 'General linguistics and descriptive grammar'. In Papers in Linguistics 1934-1951, 216-28. London and New York: Oxford University Press. Flohr, Hans 1991. 'Brain processes and phenomenal consciousness: a new and specific hypothesis'. Theory & Psychology 1(2): 45-262. Fowler, Carol A. 1986. 'An event approach to the study of speech perception from a directrealist perspective'. Journal ofPhonetics 14: 3-28. Fowler, Carol A. and Rosenblum, Lawrence D. 1991. 'The perception of phonetic gestures'. In Ignatius G. Mattingly and Michael Studdert-Kennedy (eds) , Modularity and the Motor Theory of Speech Perception, 33-59. Hillsdale, NJ: Lawrence Erlbaum. Gee, James Paul 1992. The Social Mind: Language, Ideology, and Social Practice. New York and London: Bergin & Garvey. Gendlin, Eugene 1962. Experiencing and the Creation of Meaning: A Philosophical and Psychological Approach to the Subjective. Evanston, IL: Northwestern University Press. Gibson, James.J. 1983 [1966]. The Senses Considered as Perceptual Systems. Westport, CT: Greenwood. Gibson, James]. 1986 [1979]. The Ecological Approach to Visual Perception. Hillsdale, NJ and London: Lawrence Erlbaum. Goldfield, Eugene C. 1993. 'Dynamic systems in development: action systems'. In Linda B. Smith and Esther Thelen (eds), A Dynamic Systems Approach to Development: Applications, 51-70. Cambridge, MA and London: MIT Press. Goodwin, Charles 1996a. 'Practices of color classification'. Ninchi Kagaku (Cognitive Studies: Bulletin of the Japanese Cognitive Science Society). 3(2): 62-82. [Also published as Goodwin, Charles 2000. 'Practices of color classification'. Mind Culture and Activity 7(1&2): 19-36.] Goodwin, Charles 1996b. 'Transparent vision'. In Elinor Ochs, Emanuel A. Schegloff, and Sandra Thompson (eds), Interaction and Grammar, 370-404. Cambridge: Cambridge University Press. Grene, Marjorie. 1990. 'Perception and human reality'. In Roy Bhaskar (ed.), Hard and His Critics: Essass in Honour ofRom Harre with his Commentary on Them, 17-22. Oxford: Basil Blackwell. Crusser, 0.]. 1991. 'Cortical representation of head movement in space and some psychophysical considerations'. In A. Berthoz, P. P. Vidal, and W. Graf (eds) , The Head-Neck Sensory-Motor System, 497-509. Oxford and New York: Oxford University Press. Haken, H. 1977. Synergetics: An Introduction. Heidelberg: Springer-Verlag. Halliday, M. A. K. 1967a. A Course in Spoken English: Intonation. Oxford: Oxford University Press. Halliday, M. A. K. 1967b. 'Notes on transitivity and theme in English 1'. Journal of Linguistics 3(1): 37-81. Halliday, M. A. K. 1967c. 'Notes on transitivity and theme in English II'.Journal ofLinguistics 3(2): 199-244. Halliday, M. A. K. 1968. 'Notes on transitivity and theme in English III'. Journal of Linguistics 4(2): 179-215. Halliday, M. A. K. 1975. Learning How to Mean: Explorations in the Development of Language. London: Edward Arnold. Halliday, M. A. K. 1978a. Language as Social Semiotic: The Social Interpretation ofLanguage and Meaning. London: Edward Arnold. Halliday, M. A. K. 1978b. 'Meaning and the construction of reality in early childhood'. In

322

REFERENCES

Herbert L. Pick and Elliot Saltzman (eds), Modes ofPerceiving and ProcessingInformation. 67-96. Hillsdale, NJ: Erlbaum. Halliday, M. A. K. 1979. 'Modes of meaning and modes of expression: types of grammatical structure and their determination by different semantic functions'. In D. J. Allerton, Edward Carney, and David Holdcroft (eds), Function and Context in Linguistic Analysis: A Festschriftfor William Haas, 57-79. Cambridge: Cambridge University Press. Halliday, M. A. K. 1984. 'Listening to Nigel: Conversations of a very small child'. Unpublished manuscript, University of Sydney, Linguistics Department, Sydney, Australia. Halliday, M. A. K. 1988. 'On the ineffability of grammatical categories'. InJames D. Benson, Michael J. Cummings, and William S. Greaves (eds), Linguistics in a Systemic Perspective, 27-51. Amsterdam/Philadelphia: Benjamins. Halliday, M. A. K. 1991. 'Towards probabilistic interpretations'. In Eija Ventola (ed.), Functional and SystemicLinguistics: ApjmJachesand uses, 39-61. Berlin: Mouton de Gruyter. Halliday, M. A. K. 1992a. 'How do you mean?'. In Martin Davies and Louise Ravelli (eds), Advances in Systemic Linguistics: Recent Theory and Practice, 20-35. London and New York: Pinter. Halliday, M. A. K. 1992b. 'A systemic interpretation of Peking syllable finals'. In P. Tench (ed.), Studies in Systemic Phonology, 98-121. London and New York: Pinter. Halliday, M. A. K. 1993. 'Towards a language-based theory of learning'. Linguistics and Education 5: 93-116. Halliday, M. A. K. 1994 [1985]. Introduction to Functional Grammar. 2nd edn. London and Melbourne: Arnold. Halliday, M. A. K. 1995. 'On language in relation to the evolution of human consciousness'. In S. AlIen (ed.), Of Thoughts and Words: Proceedings of Nobel Symposium 92: 'The Relation Between Language and Mind'. Stockholm, 8-12 August 1994, 45-84. River Edge, NJ: Imperial College Press. Halliday, M. A. K. 2000. 'Phonology past and present: a personal retrospect'. Folia Linguistica XXXN(I-2): 101-11. Halliday, M. A. K. and Hasan, Ruqaiya 1976. Cohesion in English. London: Longman. Halliday, M. A. K. and Matthiessen, Christian 1999. Construing Experience through Meaning: A Language-based Approach to Cognition. London and New York: Cassell. Handel, Stephen 1989. Listening: An Introduction to the Perception of Auditory Events. Cambridge, MA and London: MIT Press. Harre, Rom 1979. Social Being: A Theory for Social Psychology. Oxford: Blackwell. Harre, Rom 1983. Personal Being: A Theory for Individual Psychology. Oxford: Blackwell. Harre, Rom 1990. 'Exploring the human Umwelt'. In Roy Bhaskar (ed.), Harre and His Critics: Essays in Honour of Rom Harre with his Commentary on Them, 297-364. Oxford: Basil Blackwell. Harre, Rom and Gillett, Grant 1994. The Discursive Mind. Thousand Oaks, CA and New Delhi: Sage. Harris, Roy 1995a. Language, Signs, and Communication. London and New York: Routledge. Harris, Roy 1995b. Signs of Writing. London and New York: Routledge. Hasan, Ruqaiya 1980. 'The texture ofa text'. In M. A. K. Halliday and Ruqaiya Hasan, Text and Context: Aspects of Language in a Social-semiotic Perspective, 43-59. Sophia University, Tokyo, The Graduate School of Languages and Linguistics, Linguistic Institute for International Communication: Working Papers in Linguistics, No. 6. Hasan, Ruqaiya 1992a. 'Rationality in everyday talk: from process to system'. InJan Svartik (ed.), Directions in Corpus Linguistics: Proceedings of Nobel Symposium 82 Stockholm, 4-8 August 1991, 257-307. Berlin and New York: Mouton de Gruyter. Hasan, Ruqaiya 1992b. 'Speech genre, semiotic mediation and the development of the higher mental functions'. Language Sciences 14(4): 489-528. Hasan, Ruqaiya 1996. 'Semantic networks: a tool for the analysis of meaning'. In Carmel Cloran, David Butt, and GeoffWilliams (eds), Ways ofSaying: Ways of Meaning, 104-31. London and New York: Cassell.

REFERENCES

323

Hasan, Ruqaiya 1999. 'Speaking with reference to context'. In Mohsen Ghadessy (ed.), Text and Context in Functional Linguistics, 219-328. Amsterdam/Philadelphia: John Benjamins. Hauser, Marc D., Chomsky, Noam, and Fitch, W. Tecumseh 2002. 'The faculty of language: what is it, who has it, and how did it evolve?'. Science Vol. 298, 22 November 2002: 1569-78. Hebb, D. O. 1949. The Organization of Behavior. New York: Wiley. Hjelmslev, Louis 1954. 'La stratification du langage'. Word 10(2-3): 163-88. Hjelmslev, Louis 1961 [1943]. Prolegomena to a Theory of Language. Trans. Francis J. Whitfield. Rev. English edn. Madison, Milwaukee and London: University of Wisconsin Press. 1edema, Roderick A. M. 1997. 'Interactional dynamics and social change: planning as morphogenesis'. University of Sydney, Department of Linguistics: Unpublished Ph.D. thesis. Jakobson, Roman and Santilli, Kathy 1980. Brain and Language: Cerebral Hemispheres and Linguistic Structures in Mutual Light. Columbus, OH: Slavica. Johnson, M. H. and Morton,J. 1991. Biology and Cognitive Development: The Case ofFace Recognition. Oxford: Blackwell. Johnston, Trevor 1992. 'The realization of the linguistic metafunctions in a sign language'. SocialSemiotics 2 (1): 1-43. Juarrero, Alicia 1999. Dynamicsin Action:Intentional Behavioras a Complex System. Cambridge, MA and London: M1T Press. Kauffman, S. 1993. The Originsof Order: Self-organization and Selection in Evolution. New York and Oxford: Oxford University Press. Kaye, Kenneth. 1984 [1982]. The Mental and SocialLife of Babies: How Parents Create Persons. London: Methuen. Kelso,J. A. S., Saltzman, E. L., and Tuller, B. 1986. 'The dynamical perspective on speech production: data and theory'. Journal of Phonetics 14: 29-59. Kinsbourne, Marcel 1988. 'Integrated field theory of consciousness'. In A. J. Marcel and E. Bisiach (eds), Consciousness in Contemporary Science, 239-56. Oxford: Clarendon. Kinsbourne, Marcel 2000. 'Inner speech and the inner life'. Brain and Language 71: 120-3. Kinsbourne, Marcel 2001. 'Unity and diversity in the human brain: evidence from injury'. In Gerald M. Edelman and Jean-Pierre Changeux (eds), The Brain, 233-56. New Brunswick, NJ and London: Transaction Publishers. Kosslyn, S. M., Chabris, C. F., Marsolek, C. J., Jacobs, R. A., and Koenig, O. 1995. 'On computational evidence for different types of spatial relations encoding: reply to Cook'. Journal of Experimental Psychology 21 (2): 423-31. Kress, Gunther and Van Leeuwen, Theo 1996. ReadingImages: The Grammarof VisualDesign. London and New York: Routledge. Lakoff, George andJohnson, Mark 1999. Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. New York: Basic Books. Lamb, Sydney M. 1966. 'Epilegomena to a theory of language'. Romance Philology 19(4): 53]-73. Langacker, Ronald W. 1987. Foundations of Cognitive Grammar. Vol. I. Theoretical Prerequisites. Stanford, CA: Stanford University Press. Langacker, Ronald W. 1991. Foundations of Cognitive Grammar. Volume Il. Descriptive Application. Stanford, CA: Stanford University Press. Latour, Bruno 1986. 'The powers of association'. In John Law (ed.), Power, Action and Beliej, 264-80. London and Boston: Routledge & Kegan Paul. Latour, Bruno 1996a. 'On interobjectivity'. Mind, Culture, and Activity 3( 4): 228-45. Latour, Bruno ]996b. 'Pursuing the discussion of interobjectivity'. Mind, Culture, and Activity 3(4): 266-9. Lave,Jean 1997. 'The culture of acquisition and the practice of understanding'. In David Kirshner and James A. Whitson (eds), Situated Cognition: Social, Semiotic, and Psychological Perspectives, ] 7-35. Mahwah, NJ: Lawrence Erlbaum.

324

REFERENCES

Laver, John 1970. 'The production of speech'. In John Lyons (ed.), New Horizons in Linguistics, 53-75. Harmondsworth: Penguin. Lemke, Jay L. 1983. 'Thematic analysis: systems, structures, and strategies'. Recherches Sbniotiques/Semiotic Inquiry (RSSI) 3(2): 159-87. Lemke, Jay L. 1984a. 'Action, context, and meaning'. In J. L. Lernke, Semiotics and Education, 63-93. Victoria University, Toronto: Toronto Semiotic Circle Monographs, Working Papers and Prepublications, No. 2. Lemke, Jay L. 1984b. 'Making trouble'. In J. L. Lemke, Semiotics and Education, 94-149. Victoria University, Toronto: Toronto Semiotic Circle Monographs, Working Papers and Prepublications, No. 2. Lemke, Jay L. 1984c. 'The formal analysis of instruction'. In J. L. Lemke, Semiotics and Education, 23-62. Victoria University, Toronto: Toronto Semiotic Circle Monographs, Working Papers and Prepublications, No. 2. Lernke, Jay L. 1984d. 'Towards a model of the instructional process'. In J. L. Lemke, Semiotics and Education, 6-22. Victoria University, Toronto: Toronto Semiotic Circle Monographs, Working Papers and Prepublications, No. 2. Lemke, Jay L. 1985. 'Ideology, intertextuality, and the notion of register'. In James D. Benson and William S. Greaves (eds) , Systemic Perspectives on Discourse, Volume I, 275-94. Norwood, NJ: Ablex. Lemke, Jay L. 1990. TalkingScience: Language, Learning, and Values. Norwood, NJ: Ablex. Lemke, Jay L. 1995a. 'The social construction of the material subject'. In Jay L. Lemke, TextualPolitics: Discourse and Social Dynamics, 80-99. London and Bristol, PA: Taylor & Francis. Lemke, Jay L. 1995b [1993]. 'Discourse, dynamics, and social change'. In Jay L. Lemke, TextualPolitics: Discourse and Social Dynamics, 100-29. London and Bristol, PA: Taylor & Francis. Lemke, Jay L. 1995c. 'Retrospective Postscript'. In Jay L. Lemke, TextualPolitics: Discourse and Social Dynamics, 154-84. London and Bristol, PA: Taylor & Francis. Lemke, Jay L. 1997. 'Cognition, context, and learning: a social semiotic perspective'. In David Kirshner and James A. Whitson (eds), Situated Cognition: Social, Semiotic, and Psychological Perspectives, 37-55. Mahwah, NJ: Lawrence Erlbaum. Lemke, Jay L. 1998. 'Multiplying meaning: visual and verbal semiotics in scientific texts'. In J. R. Martin and R. Veel (eds), Reading Science: Critical and Functional Perspectives on Discourses of Science, 87-113. London and New York: Routledge. Lemke,Jay L. 1999. 'Opening up closure: semiotics across scales'. Paper presented at the conference, 'Closure: Emergent Organizations and their Dynamics', University of Ghent, Belgium; May 1999; website: http://academic.brooklyn.cuny.edu/ education/ jlemke/index.htm Lemke, Jay L. 2000a. 'Material sign processes and emergent ecosocial organization'. In Peter Bogh Andersen, Claus Emmeche, Niels Ole Finnemann, and Peder Voetmann Christiansen (eds), Downward Causation: Minds, Bodies and Matter, 181-213. Aarhus: Aarhus University Press. Lemke, Jay L. 2000b. Across the scales of time: artefacts, activities, and meanings in ecosocial systems'. Mind, Culture, and Activity 7(4): 273-90. Liberman, A. and Mattingly, 1. 1985. 'The motor theory of speech perception revised'. Cognition 21: 1-36. Libet, Benjamin 1996. 'Neural processes in the production of conscious experience'. In Max Velmans (ed.), The Science of Consciousness: Psychological, Neuropsychological and ClinicalReviews, 96-117. London and New York: Routledge. Lieberman, Philip 1984. The Biology and Evolution of Language. Cambridge, MA and London: Harvard University Press. Lindblom, Bjorn 1991. 'The status of phonetic gestures'. In 19natius G. Mattingly and Michael Studdert-Kennedy (eds), Modularity and the Motor Theory of Speech Perception, 7-24. Hillsdale, NJ: Lawrence Erlbaum.

REFERENCES

325

Longacre, Robert E. 1983. The Grammar ofDiscourse. New York and London: Plenum Press. Lorenz, Konrad. 1969 [1963]. On Aggression. Trans. Marjorie Latzke. London: Methuen. Lorenzana,Juan Alvarez de 1993. 'The constructive universe and the evolutionary systems framework'. Appendix in Stanley N. Salthe, Development and Evolution: Complexity and Change in Biology, 291-308. Cambridge, MA and London: MlT Press. Lukacs, Gyorgy 1980 [1978]. The Ontology of Social Being 3: Labour. Trans. David Fernbach. London: Merlin. Lukacs, Gyorgy 1981. Per l'ontologia dell'essere sociale II: i complessiproblematici piu importanti. Italian trans. Alberto Scarponi. Rome: Editori Riuniti. Lukacs, Gyorgy 1990. Prolegomeni all'ontologia dell'essere sociali: questioni di principio di un ontologia oggi divenuta possibile. Italian trans. Alberto Scarponi. Milan: Guerini e Associati. Luria, A. R. 1976. Cognitive Development: Its Cultural and Social Foundations. Trans. Martin Lopez-Moril1as and Lynn Solotaroff, Michael Cole (ed.). Cambridge, MA: Harvard University Press. Manghi, Sergio 1988. '11 soggetto ecosistemico: Identita e complessitii. biosociale'. In Ferruccio Andolfi (ed.), Figure d'identita: ricerche sul soggetto moderno, 173-227. Milan: Franco Angeli. Malinowski, Bronislaw 1923. 'The problem of meaning in primitive languages'. Supplement I in C. K. Ogden and I. A. Richards, The Meaning of Meaning. London: Kegan Paul Trench. Malinowski, Bronislaw 1935. 'An ethnographic theory of language. Part IV'. In Coral Gardens and their Magic, Vo\. 2. London: Alien & Unwin. Marshal1, J. C. 1980. 'Clues from neurological deficits'. In U. Bel1ugi and M. StuddertKennedy (eds), Signed and Spoken Language: Biological Constraints on Linguistic Form, 275-90. Weinheim and Deerfield Beach, Florida: Verlag Chemie. Martin,James R. 1991. 'Intrinsic functionality: implications for contextual theory'. Social Semiotics 1(1): 99-162. Martin, James R. 1992a. English Text: System and structure. Philadelphia/Amsterdam: John Benjamins. Martin, James R. 1992b. 'Macro-proposals: meaning by degree'. In W. C. Mann and S. A. Thompson (eds), Discourse Description: Diverse Analyses of a Fundraising Text, 359-96. Amsterdam and Philadelphia: Benjamins. Martinec, Radan 2000. 'Rhythm in multimodal texts'. Leonardo 4,3: 289-97. Martinec, Radan 1998. 'Cohesion in action'. Semiotica 120(1/2): 161-80. Matthiessen, Christian 1990. 'Metafunctional complementarity and resonance in syntagmatic organization'. Department of Linguistics, University of Sydney: Mimeo. Matthiessen, Christian 1993. 'The object of study in cognitive science in relation to its construal and enactment in language'. In M. A. K. Halliday (guest ed.), Language as Cultural Dynamic. Cultural Dynamics 6 (1) .

Matthiessen, Christian 1995. Lexicogrammatical Cartography: English systems. Tokyo: International Language Sciences Publishers. Maze, John R. 1991. 'Representationism, realism and the redundancy of "mentalese'", Theory & Psychology 1(2): 163-85. McGregor, William B. 1997. Semiotic Grammar. Oxford: Clarendon. McKel1ar, Gordon Bruce 1987. 'The place of socio-semiotics in contemporary thought'. In Ross Steele and Terry Threadgold (eds), Language Topics: Essays in Honour of Michael Halliday, Vo\. 11, 523-48. Amsterdam and Philadelphia: John Benjamins. McKellar, Gordon Bruce 1990. 'The language of neurolinguistics: principles and perspectives in the application of linguistic theory to the neuropsychology of language'. In M. A. K. Halliday, John Gibbons, and Howard Nicholas (eds), Learning, Keeping and Using Language, Vol. 1. Selected Papers from the 8th World Congress of Applied Linguistics, Sydney, 16-21 August 1987, 331-51. Amsterdam and Philadelphia: John Benjamins.

326

REFERENCES

McNeill, David 1992. Hand and Mind: What Gestures Reveal about Thought. Chicago and London: University of Chicago Press. Merleau-Ponty, Maurice 1983 [1942]. The Structure of Behavior. Trans. Alden L Fisher. Pittsburgh: Duquesne University Press. Natsoulas, Thomas 1998. 'The case for intrinsic theory: Ill. Intrinsic inner awareness and the problem of straightforward objectivation'. TheJoumal ofMind and Behavior 19(1): 1-19. Natsoulas, Thomas 2000. 'On the intrinsic nature of states of consciousness'. Consciousness & Emotion I (I): 139-66. Ninio, A. and Snow, C. E. 1996. Pragmatic Development. Boulder: Westview. Norton, Alec 1995. 'Dynamics: an introduction'. In Robert F. Port and Timothy van Gelder (eds), Mind as Motion: Explorations in the Dynamics of Cognition, 45-68. Cambridge, MA and London: MIT Press. Noth, Winfried 1990. Handbook of Semiotics. Bloomington and Indianapolis: Indiana University Press. Oldenburg, Jane 1987. 'From Child Tongue to Mother Tongue: a case study of language development in the first two and a half years'. Ph.D. thesis. University of Sydney, Department of Linguistics. O'Toole, Michaell994. The Language ofDisplayed Art. London: Leicester University Press. Painter, Clare 1984. Into the Mother Tongue: A Case Study in Early Language Development. Open Linguistics Series. London: Pinter. Peng, Fred C. C. 1994. 'Language disorders and brain function'. Acta Neurologica Sinica 3(3): 103-30. Peng, Fred C. C. 2000. 'Review article: "Re-reading Saussure: the dynamics of signs in social life" by Paul J. Thibault'. Lingua Posnaniensis XLII: 175-98. Perinat, Adolfo and Sadurni, Marta 1999. 'The ontogenesis of meaning: an interactional approach'. Mind, Culture, and Activity 6(1): 53-76. Petitot-Cocorda,Jean 1985. Les catastrophes de la parole: de RnmanJakobson it Rene Thom. Paris: Maloine. Piaget,Jean 1946. La formation du symbolechez l'enfant. Neuchatel: Delachaux et Niestlee. Pike, Kenneth L. 1967. Language in Relation to a Unified Theory of the Structure of Human Behaviour. 2nd, rev. edn. The Hague and Paris: Mouton. Pike, Kenneth L. 1982. Linguistic Concepts: An Introduction to Tagmemics. Lincoln: University of Nebraska Press. Pinker, Stephen 1994. The Language Instinct: The New Science of Language and Mind. Harmondsworth: Penguin. Pinker, Stephen 1997. How the Mind Works. London: Norton. Pi tt, V. H. (ed.) 1977. The Penguin Dictionary ofPhysics. Harmondsworth: Penguin. Popper, K. R. and Eccles, J. C. 1977. The Self and its Brain. Heidelberg and New York: Springer. Premack, D. and Woodruff, G. 1978. 'Does the chimpanzee have a theory of mind?' Behavioural and Brain Sciences 1: 515-26. Prevignano, Carlo L. and Di Luzio, Aldo 2003. 'A Discussion with John J. Gumperz'. In Susan L. Eerdmans, Carlo L. Prevignano, and Paul J. Thibault (eds), Language and Interaction: Discussions with John J Gumperz; 7-29. Amsterdam/Philadelphia: John Benjamins. Prigogine, lIya and Stengers, Isabelle 1985 [1984]. Order out of Chaos: Man's New Dialogue with Nature. London: Fontana. Prodi, Giorgio 1977. Le basi materiali della significazione. Milan: Bompiani. Prodi, Giorgio 1987. Alia radice del comportamento morale. Genoa: Marietti. Prodi, Giorgio 1989. L'individuo e la sua firma. Bologna: 11 Mulino. Riegel, Klaus F. 1979. Foundations ofDialectical Psychology. New York and London: Academic Press. Rizzolatti, G. and Arbib, M. A. 1998. 'Language within our grasp'. Trends in Neuroscience 21(5): 188-94.

REFERENCES

327

Robins, RH. 1957. 'Aspects of prosodic analysis'. Proceedings of University of Durham Philosophical Society, series B, Vo!. 1 (Arts): 1-12. Salthe, Stanley N. 1990. 'Sketch of a logical demonstration that the global information capacity of a macroscopic system must behave entropically when viewed internally'. Journal ofIdeas 1:51-6.

Salthe, Stanley N. 1993. Deoelopment and Evolution: Complexity and Change in Biology. Cambridge, MA and London: MIT Press. Sartre,Jean-Paul 1969 [1943]. Being and Nothingness: An Essay on Phenomenological Ontology. Trans. Hazel E. Barnes. London: Methuen. Saussure, Ferdinand de 1971 [1915]. Cours de linguistique generate. Paris: Payot. Saussure, Ferdinand de 1993. Eisuke Komatsu (ed.), Cours de linguistique generate:premier et troisiemecours d 'apresles notes de Reidlinger et Constantin. Collection Recherches Universite Gaskushuin no. 24. Tokyo: Universite Gakushuin. Savage-Rumbaugh, Sue, Shanker, Stuart G., and Taylor, TalbotJ. 1998. Apes, Language and the Human Mind. Oxford: Oxford University Press. Schank, RC. and Abelson, RP. 1977. Scripts, Plans, Goals, and Understanding. Hillsdale, NJ: Erlbaum. Seager, William 1999. Theories of Consciousness: An Introduction and Assessment. London and New York: Routledge. Shanker, Stuart G. 1996. 'Autism and the theory of mind'. Department of Philosophy and Psychology, York University, Toronto: Mimeo. Shanker, Stuart G. 1998. 'Philosophical preconceptions'. In Savage-Rumbaugh, Sue, Shanker, Stuart G., and Taylor, Talbot J. 1998. Apes, Language and the Human Mind, 77-138. Oxford: Oxford University Press. Shannon, Claude E. and Weaver, Warren 1949. The Mathematical Theory of Communication. Urbana, IL: University of Illinois Press. Shepard, R N. 1984. 'Ecological constraints on internal representation: resonant kinematics of perceiving, imagining, thinking, and dreaming'. Psychological Review 91: 417-47. Sherrington, C. S. 1940. Man on his Nature. Cambridge: Cambridge University Press. Silverstein, Michael1976. 'Shifters, linguistic categories, and cultural description'. In K. H. Basso and H. A. Selby (eds), Meaning in Anthropology, 11-55. Albuquerque: University of New Mexico Press. Silverstein, Michael 1985. 'The functional stratification of language and ontogenesis'. In James V. Wertsch (ed.), Culture, Communication, and Cognition: Vygotskian Perspectives, 205-35. Cambridge and Melbourne: Cambridge University Press. Silverstein, Michael 1987a. 'Cognitive implications of a referential hierarchy'. In Maya Hickmann (ed.), Social and functional Approaches to Language and Thought, 125-64. New York and London: Academic Press. Silverstein, Michael 1987b. 'The three faces of "function": preliminaries to a psychology of language'. In Maya Hickmann (ed.), Social and Functional Approaches to Language and Thought, 17-38. New York and London: Academic Press. Silverstein, Michael 1997. 'The improvisational performance of culture in realtime discursive practice'. In R Keith Sawyer (ed.), Creativity in Performance, 265-312. Greenwich, CT: Ablex. Sperry, R W. 1952. 'Neurology and the mind-brain problem'. American Scientist 40: 291-312. Sperry, R W. 1980. 'Mind-brain interaction: mentalism yes; dualism no'. Neuroscience 5: 195-206. Stevens, K. N. 1989. 'On the quantal nature of speech'. Journal ofPhonetics 17: 3-45. Studdert-Kennedy, M. and Lane, H. 1980. 'Clues from the differences between signed and spoken language'. In U. Bellugi and M. Studdert-Kennedy (eds), Signed and Spoken Language: Biological Constraints on Linguistic Form, 29-39. Weinheim and Deerfield Beach, Florida: Verlag Chemie. Thelen, Esther 1995. 'Timescale dynamics and the development of an embodied

328

REFERENCES

cognition'. In Robert F. Port and Timothy van Gelder (eds), Mind as Motion: Explorationsin theDynamics of Cognition, 69-100. Cambridge, MA and London: MIT Press. Thelen, Esther and Smith, Linda B. 1994. A Dynamic Systems Approach to theDevelopment of Cognition and Action. Cambridge, MA and London: MIT Press. Thibault, Paul]. 1986a. 'The cognitive hypothesis: a critical comment'. In Text, Discourse, and Context: A Social Semiotic Perspective, 26-45. Victoria University, Toronto: Monographs, Working Papers and Prepublications of the Toronto Semiotic Circle, Vo!' 3. Thibault, Paul]. 1986b. 'Thematic system analysis and the construction of knowledge and belief in discourse: the headlines in two Italian newspaper texts'. In Text, Discourse, and Context: A SocialSemiotic Perspective, 44-91. Victoria University, Toronto: Monographs, Working Papers and Prepublications of the Toronto Semiotic Circle, Vo!' 3. Thibault, Paul]. 1987. 'Interview with Michael Halliday'. In Ross Steele and Terry Threadgold (eds), Language Topics: Essays in Honour of MichaelHalliday, Vol. II,601-27. Amsterdam and Philadelphia: Benjamins. Thibault, Paul]. 1989. 'Semantic variation, social heteroglossia and intertextuality: thematic and axiological meanings in spoken discourse'. Critical Studies 1(2): 181-209. Thibault, Paul J. 1990. 'Questions of genre and intertextuality in some Australian television advertisements'. In R. Rossini Favretti (ed.), The Televised Text, 89-131. Bologna: Patron. Thibault, Paul]. 1991a. SocialSemiotics as Praxis: Text, Social Meaning Making and Nabokov's 'Ada: Theory and History of Literature series, Vo!' 74, Wlad Godzich and Jochen Schulte-Sasse (eds), Minneapolis and Oxford: University of Minnesota Press. Thibault, Paul]. 1991b. 'Grammar, technocracy and the noun: technocratic values and cognitive linguistics'. In Eija Ventola (ed.), Recent Systemic and OtherFunctionalViews on Language, 281-305. Trends in Linguistics Studies and Monographs. Berlin: Mouton de Gruyter. Thibault, Paul]. 1992a. 'Minor clauses and interpersonal meaning'. Dipartimento di Scienze del Linguaggio, Universita degli Studi di Venezia: Mimeo. Thibault, Paul J. 1992b. 'Grammar. ethics, and understanding: functionalist reason and clause as exchange'. SocialSemiotics 2(1): 135-75. Thibault, Paul]. 1993. 'Using language to think interpersonally: experiential meaning and the cryptogrammar of subjectivity and agency in English'. In M. A. K. Halliday (guest ed.), Language as CulturalDynamic. CulturalDynamics 6 (1-2): 131-86. Thibault, Paul]. 1994. 'Text and/or context?: an open question'. State-of-the-Art article. In The Semiotic Reviewof Books (Toronto) 5(2), May 1994. Thibault, Paul]. 1995. 'The interpersonal grammar of Mood and the ecosocial dynamics of the semiotic exchange process'. In Ruqaiya Hasan and Peter Fries (eds), On Subject and Theme: From thePerspective of Functions in Discourse, 51-89. Amsterdam and Philadelphia: Benjamins. Thibault, Paul]. 1997a. Re-reading Saussure: The Dynamics of Signs in Social Life. London and New York: Routledge. Thibault, Paul]. 1997b. 'The sounds of language'. Saussure and Beyond, Lecture No. 4. Cyber Semiotics Institute (University of Toronto), website: http://www.chass. utoronto.ca/epc/srb. Thibault, Paul]. 1998a. 'Embodiment, perception, consciousness, personhood: The cascading/collecting dialectic of langue and parole in the individual'. Cyber Semiotics Institute, University of Toronto (Canada); website: http://www.chass.utoronto.ca/ epc/srb. Thibault, Paul]. 1998b. 'Metasemiosis'. SRB Insights. The Semiotic Reviewof Books 9(2), May 1998: 8-12. Thibault, Paul]. 1998c. 'Inner speech'. In Paul Bouissac (ed.), The Encyclopedia of Semiotics, 312-14. Oxford and New York: Oxford University Press. Thibault, Paul]. 1999a. 'Communicating and interpreting relevance through discourse

REFERENCES

329

negotiation: an alternative to relevance theory'. Journal ofPragmatics 31: 557-94. Thibault, Paul J. 1999b. 'Putting Humpty Dumpty's theory of meaning back together again: can Saussure help?' Belgian Essays on Language and Literature 9: 7-34. Liege: Belgian Association of Anglicists in Higher Education. Thibault, Paul]. 2000a. 'The dialogical integration of the brain in social semiosis: Edelman and the case for downward causation'. Mind, Culture, and Activity 7(4): 291-311. Thibault, Paul]. 2000b. 'The multimodal transcription of a television advertisement: theory and practice'. In Anthony Baldry (ed.), Multimodality and Multimediality in the Distance Learning Age, 311-85. Campo Basso: Lampo. Thibault, Paul]. 2001. 'Multimodality and the school science textbook'. In Carol Taylor TorseIlo, Giuseppe Brunetti, and Nicoletta Penello (eds), Corpora testuali per ricerca, traduzione e apprendimento linguistico, 293--335. Padua: Unipress. Thibault, Paul]. 2002. 'Interpersonal meaning and the discursive construction of action, attitudes and values: the Global Modal Program of one text'. In Peter Fries, Michael Cummings, David Lockwood, and William Spruiell (eds), Relations and Functions within and around Language, 56-116. London and New York: Continuum. Thibault, Paul]. 2003a. 'Body dynamics, social meaning-making, and scale heterogeneity: reconsidering contextualization cues and language as mixed-mode serniosis'. In Susan Eerdmans, Carlo Prevignano, and Paul]. Thibault (eds), Discussing John J Gumperz, 127-47. Amsterdam/Philadelphia: Benjamins. Thibault, Paul]. 2003b. 'Contextualization and social meaning-making practices'. In Susan Eerdmans, Carlo Prevignano, and Paul]. Thibault (eds), Discussing John J Gumperz; 41-61. Amsterdam/Philadelphia: Benjamins. Thibault, Paul]. 2004. 'Agency, individuation, and meaning-making: reflections on an episode of bonobo-human interaction'. In Geoffrey Williams and AnnabeIle Lukin (eds), Language Development: Functional Perspectives on Species and Individuals 112-36. London and New York: Continuum. Thibault, Paul.J. In press a [2004]. 'Semiology'. In Jeff Stepnisky and Andrew Wernick (eds), The Encyclopaedia of Social Theory, London and Newbury Park, CA: Sage. Thibault, Paul.J. In press b. Agency and Consciousness in Discourse: Self-other Dynamics as a Complex System, London and New York: Continuum. Thibault, Paul J. and Van Leeuwen, Theo 1996. 'Grammar, society, and the speech act: renewing the connections'. Journal ofPragmatics 25: 561-85. Threadgold, Terry 1997. Feminist Poetics. London and New York: Routledge. Togeby, Ole 2000. 'Anticipated downward causation and the arch structure of texts'. In Peter Begh Andersen, Claus Emmeche, Niels Ole Finnemann, and Peder Voetmann Christiansen (eds) , Downward Causation: Minds, Bodies and Matter, 261-77. Aarhus: Aarhus University Press. Trager, G. 1.. and Smith, H. 1.. 1951. Outline ofEnglish Structure. Norman, OK: Battenburg. Trevarthen, Colwyn 1978. 'Modes of perceiving and modes of acting'. In Herbert 1.. Pick and E!liot Saltzman (eds), Modes of Perceiving and Processing Information, 99-136. Hillsdale, ~: Erlbaum. Trevarthen, Colwyn 1980. 'The foundations of intersubjectivity: Development of interpersonal and cooperative understanding in infants'. In D. Olsen (ed.), The Social Foundations ofLanguage and Thought, 316-42. New York: Norton. Trevarthen, Colwyn 1987. 'Sharing makes sense: intersubjectivity and the making of an infant's meaning'. In Ross Steele and Terry Threadgold (eds), Language Topics: Essays in Honour of Michael Halliday, Vo!. I, 177-99. Amsterdam/Philadelphia: Benjamins. Trevarthen, Colwyn 1992. 'An infant's motives for speaking and thinking in the culture'. In Astri Heen Wold (ed.), The Dialogical Alternative: Towards a Theory ofLanguage and Mind, 99-137. Oslo: Scandinavian University Press. Van Leeuwen, Theo 1999. Speech, Music, Sound. London: Macmillan. Varela, F. and Maturana, R. 1980. Autopoiesis and Cognition. Dordrecht: ReideJ. Vihman, Marilyn May 1991. 'Ontogeny of phonetic gestures: speech production'. In

330

REFERENCES

Ignatius C. Mattingly and Michael Studdert-Kennedy (eds), Modularity and the Motor Theory of Speech Perception, 69--84. Hillsdale, !'U: Erlbaum. Vihman, Marilyn and de Boysson-Bardies, Benedicte 1994. 'The nature and origins of ambient language influence on infant vocal production and early words'. Phonetica 51: 159-69. Vihman, Marilyn May, Mackek, Marlys A., Miller, Ruth, Simmons, Hazel, and Miller,jim 1985. 'From babbling to speech: a re-assessment of the continuity issue'. Language 61(2): 397-445. Volosinov, V.N. 1973 [1930]. Marxism and thePhilosophy of Language.Trans. Ladislav Matejka and I. R. Titunik. New York and London: Seminar Press. Volosinov, V. N. 1976 [1927]. Freudianism: A Marxist critique. Trans. and ed. I. R. Titunik and Neal H. Bruss. New York and London: Academic Press. Volosinov, V. N. 1983. 'The construction of the utterance'. Trans. Noel Owen. In Ann Shukman (ed.), Bakhtin School Papers: Russian Poetics in Translation, No. 10, 114-37. Oxford: RPT Publications (in association with Department of Literature, University of Essex, Colchester). Vygotsky,Lev 1986 [1934]. Thought and Language.Trans. Alex Kozulin. Cambridge, MA and London: MIT Press. Vygotsky, L. S. 1987 [1934]. 'Thinking and speech'. In R. W. Rieber and A. S. Carton (eds) , The Collected Works of L. S. Vygotsky, Vot. 1: Problems of General Psychology, 37-285. New York: Plenum Press. Walkerdine, Valerie 1997. 'Redefining the subject in situated cognition theory'. In David Kirshner and j ames A. Whitson (eds) , Situated Cognition: Social, Semiotic, and Psychological Perspectives, 57-70. Mahwah, NJ: Lawrence Erlbaum. Wall, Patrick D. 1996. The placebo effect'. In Max Velmans (ed.), The Science of Consciousness: Psychological, Neuropsychological and ClinicalReviews, 162-80. London and New York: Rout/edge. Wells, Cordon 1999. Dialogic Inquiry: Towards a Sociocultural Practice and Theory of Education. Cambridge and New York: Cambridge University Press. Wertsch,James V. 1991. Voices of theMind: A Sociocultural Approach to Mediated Action.London and Singapore: Harvester Wheatsheaf. Wheeler, John Archibald 1988. 'World as system self-synthesized by quantum networking'. IBMJoumal of Research Development 32(1): 1-12. Whitson, James A. 1997. 'Cognition as a semiosic process: from situated mediation to critical reflective transcendence'. In David Kirshner and James A. Whitson (eds), Situated Cognition: Social, Semiotic, and Psychological Perspectives, 97-149. Mahwah, !'U and London: Lawrence Erlbaum. Whorf, Benjamin Lee 1956a [1939]. 'The relation of habitual thought and behaviour to language'. Injohn B. Carroll (ed.), Language, Thought, & Reality, 134-59. Cambridge, MA: MIT Press. Whorf, Benjamin Lee 1956b [1941]. 'Languages and logic'. In John B. Carroll (ed.), Language, Thought, & Reality, 233-45. Cambridge, MA: MIT Press. Wilden, Anthony 1980 [1972]. System and Structure: Essays in Communication and Exchange. 2nd edn. London: Tavistock. Wilden, Tony 1981. 'Semiotics as praxis: strategy and tactics'. Recherches Semiotiques/Semiotic Inquiry 1(1): 1-34. Wilson, Elizabeth A. 1998. Neural Geographies: Feminism and the Microstructure of Cognition. London and New York: Rout/edge. Zelazo, Philip D. 1999. 'Language, levels of consciousness, and the development of intentional action'. In Philip D. Zelazo, Janet W. Astington, and David R. Olson (eds) , Developing Theories of Intention: Social Understandingand Self-control, 95-117. Mahwah, NJ: Lawrence Erlbaum.

Name Index

Abelson, R P. 7 Abercrombie, David 121, 123, 124, 125 Andersen, Peter B0gh 5 Arbib, M. A. 267,268,269,270 Aristotle 183 Armstrong, David F. 72, 80,90, 91 Astington,j. W. 174 Bakhtin, Mikhail M. 97, 183,211, 215,270 Baron-Cohen, S. 174 Bates, Elizabeth 37 Bateson, Gregory 8, 13, 14, 23, 24, 26, 27, 34,87,148,172,173,192,282,300,301 Battacchi, Marco W. 174,213,246,247 Battistelli, Piergiorgio 174 Beaugrande, Robert de 48, 308 Bellugi, Ursula 190 Benson, James 38 Bernstein, Basil 279 Bernstein, N. 79,82 Berthoz, Alain 74,75,77,79,81,83,105 Bhaskar, Roy 31,32 Bickhard, Mark H. 41,121,136 Bogdan, Radul. 146,205,206,207,209,247 Bohr, Nielsen 33, 35 Bolinger, Dwight 92, 98, 99 Bouissac, Paul 34, 40 Bourdieu, Pierre 279 Boysson-Bardies, Benedicte de 85, III Braten, Stein 52 Browman, Catherine 90,91,92,145,162 Biihler, Kad 194, 197, 200, 221 Campbell, Donald T. 41, 121, 136 Carruthers, Peter 262,263,264,271,273, 274, 275, 277, 278 Catford, j. C. 122 Celani, Giorgio 174 Chabris, C. F. 105

Chomsky, Noam 7, 80, 226, 282, 283 Clark, E. 205 Clark, H. 205 Cleirigh, Chris uz, ll3, ll5, ll9, 120, 121, 122, 123, 124, 145, 154, 160,295 Cole, Michael 207,216 Coleridge, Samuel Taylor 81 Corina, David P. 190 Coulthard, Malcolm 84, 85 Cowley, Stephenj. 40,44,45,124,190,283 Cremonini, Giorgio 97 Damasio, Antonio 55,147,148,177,210, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 264, 275 Davidse, Kristin 70,132,167,198,199 Davidson, Rj. 189 Deacon, Terrence 16,55, 126, 173, 174, 175,190,191,219,276,282 Descartes, Rene 224, 225, 226, 302, 303,

305, 307 Di Luzio, Aldo 115 Dory, R 172 Dre~ke,E

205,224

Eccles, j. C. 172 Edelman, Gerald M. 19,20,51,52,55,62,

71,84,88,92,101,139,140,141,144, 146, 160, 163, 164, 176, 179, 184, 185, 209,210,214,215,220,221,228,231, 238,242,246,255,256,258,262,264, 273,285,302,310,317 Fadiga, L. 268 Fillmore, Charles 269 Finnemann, Niels Ole 20, 24, 29 Firth,j. R 108, ll5, ll7, ll9 Fitch, W. Tecumseh 7 Flohr, Hans 56, 177, 283, 284, 285, 286,

332

NAME INDEX

287,288,289,290,296,297,301,310, 317 Fowle~ Carol 86,89,93,101,102,103 Frasnedi, Fabrizio 97 Fries, Peter 38 Gee,james Paul 3 Gendlin, Eugene 23 Gibson,jamesJ. 8,11,12,13,14,60,86, 87,88,102,121,152,185,186,202,218, 219,220,222,228,229,230,234,246, 247,276,291,296,303,304,306 Gillett, Grant 177 Goldfield, Eugene C. 141 Goldstein, Louis 90,91,92,145,162 Goodwin, Charles 3 Creaves, William 38 Grene, Marjorie 11, 12 Griisser,O.J. 106 Gumperz,johnJ. 115 Haken, H. 245 Halliday, M. A. K 5,23,31, 37,40,42,46, 47,48,50,55,63,84,94,95,99,107, 117, 127, 130, 134, 154, 156, 162, 163, 164,166,167,176,194,195,196,197, 198,199,200,202,203,205,206,207, 212,214,226,239,240,241,242,243, 244,245,257,266,269,270,278,282, 291, 292, 295, 298, 300, 301, 315 Handel, Stephen 120, 124, 266 Harre, Rom 6,12,38,176,177,178,213 Harris, P. 174 Harris, Roy 44, 89 Hasan, Ruqaiya 6,156,200,214,279 Hauser, Marc D. 7 Hebb, D. O. 285, 286, 296 Heisenberg, W. 35 Hjelmslev, Louis 40,54,59,60,61,65, 66,67,69,70,104,116,117,126,236, 292 Iedema, Roderiek A. M. 77 Iwamoto, Kazuyoshi 38 jacobs, R. A. 105 jakobson, Roman 190 jespersen,Otto 66 johnson, Mark 165, 181 johnson, M. H. 52 johnston, Trevor 47,186,192 juarrero, Alicia 4, 13, 16, 17, 19,20,63, 138, 254, 269, 302

Kanzi 38 Kauffman, Stuart A. 103, 104 Kaye, Kenneth 37 Kelso,J. A. S. 84,90,141 Kinsbourne, Mareel 4, 17, 18,21,22,52, 271,310,311,312,317 Koenig, O. 105 Kosslyn, S. M. 105 Kress, Gunther 47, 192 Lakoff, George 165, 181 Lamb, Sydney M. 40, 292 Lane, H. 91, 102, 105 Langaeker, Ronald W. 75,76,156,198, 271, 272 Latour, Bruno 77, 186 Lave,jean 3 Laver,john 84,86 Lernke, jay L. 3,8,9, 16,20,22,23,25,26, 27,30,34,39,40,41,43,44,47,48,54, 65,70,72,82,109,118,137,141,146, 151,178,181,182,186,187,192,193, 207,216,237,251,252,255,256,264, 265,267,282,293,294,295,300,304, 306,308,309,312,313 Liberman, A. 100,268 Libet, Benjamin 171, 172, 173, 174, 180 Lieberman, Philip 84, 85, 86 Lindblom, Bjorn 93, 94 Longaere, Robert E. 95, 97 Lorenz, Konrad 213 Lorenzana, juan Alvarez 142 Lukaes, Gyorgy 99,216,236,237,317 Luria, A. R. 188,216 Malinowski, Bronislaw 188, 194 Manghi, Sergio 3 Marshall,J. C. 73,74 Marsolek, C.J. 105 Martin,jamesR. 40,117,142,215 Martinec, Radan 78,79,93,151,192 Martinet, Andre 292 Matthiessen, Christian 6, 46,94, 95, 97, 196, 197, 239, 298 Mattingly, I. 100 Maze.john R. 11 MeGregor, William B. 127,269 MeKellar, Gordon Bruee 33, 38, 48 MeNeill, David 186 Merleau-Ponty, Mauriee 78, 100 Morton, J. 52

NAME INDEX Natsoulas, Thomas 233 Ninio, A. 205 Norton, AJec 145 Noth, Winfried 34 Oldenburg,jane 240 Olson, D. R. 174 O'Toole, Michael 192 Painter, Clare 239 Pan banisha 38 Peirce, Charles Sanders 34,35, 116, 126, 249, 305 Peng, Fred C. C. 190, 217, 218, 254, 268, 271,272,289,290,294 Perinat, Adolfo 213 Petitot-Cocorda,jean 87 Piaget,jean 213 Pike, Kenneth L. 94, 125, 134 Pinker, Stephen 283 Pitt, V. H. 129 Popper, K. R. 172 Premack, D. 174 Prevignano, Carlo 115 Prigogine, Ilya 10, 178, 182 Prodi, Giorgio 16,35,38,87, 186 Riegel, Klaus F. 176,183 Rizzolatti, G. 267, 268, 269, 270 Robins, R. H. 94 Rosenblum, Lawrence D. 101, 102, 103 Sadumi, Marta 213 Salthe, S. N. 3,4,8, 10, 14, 16,20,25, 39, 40,43,54,55,61,62,63,70,74,75,77, 79,80,82,83,86,88,106,112,114,118, 126,145,147,148,159,161,176,177, 181, 182, 183, 187, 190, 206, 216, 224, 226, 227, 230, 232, 237, 239, 250, 255, 257,261,265,287,288,311 Saltzman, E. L. 84,90, 141 Santilli, Kathy 190 Sartre, jean-Paul 176 Saussure, Ferdinand de 45,47,49,52,59, 60,61,65,66,69, 133, 184, 189, 236, 271 Savage-Rumbaugh, Sue 38, 282 Schank, R. C. 7 Seager, William 225, 302, 303, 305, 306, 307, 309 Shanker, Stuart G. 55, 74, 224, 282 Shannon, Claude E. 34

333

Shepard, R. N. 81 Silverstein, Michael 42,194,201,215,255 Smith, H. L. 84 Smith,LindaB. 10,19,51,52,55,172, 185,210,211,241,242,243,245 Snow, C. E. 205 Sperry, R. W. 172 Stengers,Isabelle 10, 178, 182 Stevens, K. N. 141 Stokoe, William C. 72, 80, 90, 91 Studdert-Kennedy, M. 91, 102, 105 Tagliatela,jared 38 Talbot, Taylor]. 282 Thelen, Esther 10, 19,51,52,55, 162, 163, 166,172,185,210,211,241,242,243, 245 Thibault, Paul]. 3,6,8,19,29,34,38,39, 40,47,48,49,52,60,62,72,74,87,90, 118126,141,151,154,156,173,181, 184,189,192,200,213,214,215,217, 237,241,256,275,282,293,301,310 Threadgold, Terry 83 Togeby, Ole 17, 144 Tononi, Giulio 176, 262 Trager, G. L. 84 Trevarthen, Colwyn 36,37, 179, 202, 212, 214,240,282 Tuller, B. 84,90,141 Vaid,jyotsna 190 Van Leeuwen, Theo 47, 192,215 Vihman, Marilyn May 85, III Volosinov, V. N. 74,241,317 Vygotsky, Lev S. 49,216,273,275 Walkerdine, Valerie 3 Wall, Patrick D. 203 Weaver, Warren 34 Wells, Gordon 48 Wertsch, lames V. 282 Wheeler, John Archibald 282, 306 Whitson,james A. 70 Whorf, Benjamin Lee 280 Wilcox, Sherman E. 72, 80, 90, 91 Wilden, Anthony 23, 34, 148, 178, 179, 215, 222 Wilson, Elizabeth A. 48 Woodruff, G. 174 Zelazo, Philip D. 6

Subject Index

ability, symbolic referential 174 abstraction, symbolic 214 abstractness, conceptual 70 account, hypostatic local 31 acoustic cue 266 act articulatory 88 dialogic 37, 270 indexical 162 intentional 270 action 250, 251 artefact mediated 216 intentional 267 action schema 75, 76 actional semiotic 79, 83 activity 8, 14, 18 biomechanical 45 contextualizing 164 distributed 316 exploratory 12, 200, 220 gestural 105 linguistic motor dimension of 75 meaning-making 3, 4, 6, 10, 13, 14, 17, 47, 48, 49, 50, 72, 108 motor 185, 217, 218, 268, 271 multimodal social 208 neural (neuronal) 19,76,171,174,246, 260, 315 neuromuscular 14,41,42,43,45,150 perceptual 160,303 performatory 185, 186 reading 81 ritualistic 213 self-organizing 52 semiotically mediated 138 sensori-motor 13, 15, 18,81,82, 100, 144,181,203,294,302,309 internalized 82

symbolic 213 vocal tract (articulatory) 53,65,71,83, 86,88,91,94,100,101,117,125, 136, 137, 146, 157, 159, 187 activity-structure 98, 140 act-token 270 affect 189 affordance 4,11,12,13,14,15,16,19,20, 24,36,41,43,60,86,162,164,310 agency 15,16,32,33,63,95,97,182 agent 4,5,15,45,63,64,65,77,147,183, 217,233,279 alterity 316 alternative, paradigmatic 98 amplification, irreversible act of 306 analysis, metalinguistic 44 ape-language research 282 arrow of time 317 artefact 15,16,44,45,51,53 textual 310 array ambient optic 264 acoustic 100, 101 visual 234 articulation 40,43,83,89,93,120,121, 123, 159, 163, 218, 271, 289, 291, 292 phonetic gestural theory of 90 articulator 87,89,92,94,101,103,105 Artificial Intelligence (AI) 72 association 11 attention 204, 283 poles of 246, 276 attraction, basin of 104 attractor 4,19,43,65,103,104,187,209, 239, 245, 266, 269, 270, 279, 280, 305, 312 attractor basin 148, 167

SUBJECT INDEX attractor landscape 167,245 audition 271 awareness 20, 288 integrative field of 311, 312 intrinsic 223, 233 meta-semiotic 214 non-perceptual 178, 229 perceptual 276, 300, 308 phenomenal 56, 289 babbling 162, 163 behaviour, motor 4 bifurcation 255 binding 172 body, social 77 bonding, sentimental 205,207 bonobo 38 boundary condition 14,29,121,167,186, 187,231,232,239,278,289,313, 315 Braille 78, 297 brain, hemispherical organization of 17, 55 brain lesion 22 breath group 84, 85, 86, 92, 93, 104 Broca's area 140, 267, 268 cascading 261 case structure 269 category 5,7 digital 222, 225 experiential 167,197,212,299,300 lexicogrammatical 21 observer's 227, 230 perceptual-motor 243, 306 phonological 221,222, 291, 294, 295 schematic 22, 165, 197 semantic 298 semiotic 82 symbolic 225, 226, 228 value-producing 189 categorization (category formation) 220, 225, 228, 240, 242 conceptual 105, 167, 238, 258, 302 linguistic 168 perceptual-motor 178,216,274,275, 302 phonological 266 sensori-motor 21 symbolic 231 systems of 184, 309 causality Aristotelian 137 efficient 24,31, 166

335

final 145, 183 transcategorial 32, 33 cause efficient 49,61,72, 138, 183 final 138 formal 72, 138 material 61, 138 clause 196, 198, 298 minor 156 closure dialogic 52 functional 236 informational 224 semiotic 309 eo-adaptation 15 eo-articulation 89,91, 104 codification 44 meta-linguistic 45 coding analogic 23 digital 23 coding orientation 279, 280 eo-evolution 15, 67, 177 cogent moment 41,112,115,145,182,287 cognition 3,7,8,13,14,49,150 computationalist model of 150 information-processing model of 238 cognitive science paradigm (cognitive science) 225, 240 coherence, interactional 215 cohesion lexico-semantic 47 phonological 119 communication, analogic 148 competence, linguistic 282,283,315 complementarity 12,31,33,36,68,82 dialogic 277 wave/particle/field 136,148 complexity, organizational 12,244 concatenation (phonological) 119 condensation, semantic 275 consciousness 55 core 147,250, 253, 262 dialogic basis of 183 extended 147,253,255 object of 55 primary 184 stream of 311 trajectory of 182, 183 condition boundary 78, 79, 84, 100 enabling 4,5, 16, 177, 183 initial 246 initiating 5,19,83,120,220,313

336

SUBJECT INDEX

neural architecture as 106 constituency, hierarchical 39 constraint 32 context-sensitive 84 contextual 4,14,30,63,312 ecosocial 24 formal 84 informational-semiotic 145 integrative 105 higher-order 16 metafunctional 278 semiotic 6 time-bound 83 construal, symbolic 62 content 60, 61 content form 64, 67, 69 content purport 64 content stratum (plane) 46, 61, 63,80, 107 content substance 64, 66, 67 context 14,165 context of situation 155 contextualization 27,33,81, 251,252 contextualization cue 115 contrast, discrete/typological 25 control, locus of 315 control parameter 159 control structure 100, 101 cortex cerebral 283,284,289,294,297 frontal 240 pre-frontal 185, 260 crypotype 95 cycle articulatory 84, 295 respiratory (respiration) 84,93 rhythmic 295 vocal-tract 160 Darwinist view 24 deixis 134 evaluative/modal proximity 132,198, 199, 276,277 person 132,199,276,277 temporal proximity 132,198,199,276, 277 demarcation (phonological) 119 depolarization, postsynaptic 287 development, language 241 developmental dynamic 163 developmentallifespan psychology 176 dialogue 183,231,232 difference 9,20,34,45,51,52,80,132, 185,186,193,195,228 system of 49,193

typological-categorial 135, 222 value-laden 223, 310 differentiation 12,25,46,52,61,142,187, 189, 206, 228 analogue 23 categorial 256 functional 50 of self and other 210 sensori-motor 140 symbolic 211 topological-continuous 168,262 typological-categorial 142 discourse 48, 78, 117, 140, 157, 181, 200 meta-semiotic 45 multimodal 192 discourse semantics 274 discourse voice 50 discrimination conceptual 164 conceptual-semantic 163 distinction categorial 237, 243 phonological 53 semiotically salient 184 typological-categorial 51, 52, 178, 193, 225, 292 diversity, metafunctional 86,136,145, 164 dualism 313 between physical vs. phenomenal brain states 283 Cartesian 74, 302 mind-body 89 mind-brain 302 nco-Cartesian 89 dyad 12,19,37,38,49,102,176,179,231, 244, 270, 278, 282 mother-infant 35, 161 dynamics ecosocial 9 material-physical 9 neural 16 egoreception 219 emergence 313 developmental 80,206 historical 65, 80, 182 synchronic 32 emotion 98, 99, 259, 260, 264 empathy, phonetic 124, 125 enactment, interpersonal 273 entextualization 147 entrainment 15,53 entropy 10, 159 epigenetic process 283

SUBJECT INDEX event distal 100, 101, 102 ecological 87,88, 102 environmental 28, 86, 204, 207, 220, 230 mental 222 neural 251 perceptual 93 environment 8, 14 ecosocial 3 physical-material 6 evolution cortical 185 semiotic 146 exchange 35,37,38,213,228 dialogic 178, 207 dyadic 211,245 energy 227 information 231 interpersonal 214,215,270 of matter, energy and informationmeaning 8, 71 socio-affective 243 experience contextual nature of 32 epigenetic 240 perceptual 248 value-laden 262 explanation causal 102 causal-reductive model of 34 exploration abstract symbolic 185 haptic 165 interpersonal 217 movement based 242 sensori-motor 7,212,241 expression 60,61,63, 71 expression form 64, 69, 87, 88 expression purport 64, 66, 104 expression stratum (plane) 54,63, 70, 80, 87, 89, 234 expression-substance 62, 64, 66, 68, 87, 88, 91 exteroception 219,220,221,246,262 face 185,187,189,210,211 feature distinctive 122 paradigmatic 120 feedback, positive 244, 266, 287 feedback loop 266,284 feeling-state 259, 260, 264 field articulatory 122

337

deictic 200 interpersonal 129, 130, 135 phonogenetic 120 symbolic 197 figure 128, 196, 197, 198,277,278,297, 298 finalization 183, 270 Finite 131,132,198,199,216,269,276, 277 finiteness 156 firstness 34, 35, 93, 249 foot 122, 123, 124, 125, 154, 157, 295 form 71 graphological/phonological 59 frequency, fundamental 103 freedom degrees of 10,74,75,79,83,84, 139, 141, 165,228,244, 312 semiotic 215 topological 309 friction 74, 75, 77, 85, 94, 217 function brainstem 218 "etc. '

higher cognitive 74 higher mental 21 indexical 115, 119, 120 mental 209 motor 61,217,218,268,271 neural 209 gaze 37,45,185,192,195,201,202,247, 248 genre 5,44,45,140,142,188,194,213, 239, 279 gesture 26,27, 29, 30, 51, 62, 65, 68, 86, 89,91,92,100,101,102,186,192, 193,246,266 perceptual (phonological) salience of 94 vocal 164 Given 128, 129,299 goal-seeking 215 goal-state 215 grammar 46, 47 formal 46 interpersonal 269 universal 48 grapheme 118, 234 graphology 70 ground 201, 202 non-symbolic 226 grounding 135,156,158,177,199,211, 247, 260, 262, 277 group, neuronal 220

338

SUBJECT INDEX

habitus 5, 74, 279 hand 184, 185, 187, 189 hemisphere cortical 189 left and right 18,189,190,191,212 heterogeneity, scalar (or scale) 43,54,65, 70,97,116,145,157,181,279,293 heteroglossia, social 50, 214 hierarchy combinatorial 142,143 constituent 125 integration/implication (of iconic, indexical, and symbolic modes of semiosis) 221,223 integrative 177,237 presupposition-cum-implication 211, 219,222,223,225,230,240 referen tial 126 scalar 181, 184, 216,287 specification (implication) 126, 195 three-level 19, 34, 35, 36, 40, 109, 110, 143,158,181,250,289,311,313 holism interactional-affective 260 relational 259 homunculus 20,316 homogeneity, scalar (or scale) 43,70 icon 126,135,195,211,222,230,240 iconicity 255 Ictus 113, 121, 122, 124, 125, 154 idea (Descartes) 303, 305 image acoustic 271, 272 auditory 76,271,272 body 181 mental 249,250,251,257,258,265,266, 275, 308 unified field of 249 neural 260 motor 75,76 imagery mental 76 proto-indexical mental 255 visual 81 implication, scale of 46 impulse afferent 219 efferent 219 neural 289, 290 index 126,135,195,202,211,222,230, 240 individuation 16,38,140,146 infodynamics 55

information 6, 11, 12, 13, 22, 36, 42, 87, 89,93,114,115,141,147,152,202, 220,221,226,227,245,246,249, 264,286,289,291 Dretske's definition of 224 environmental 303 haptic 261 indexical 88 macroscopic 226 perceptual 60,61,162,262,278 stimulus 25, 27, 60, 121, 164, 203, 204, 218,219,220,222,225,228,230, 231,233,243,274,306,315 multimodal association of 243 information potential 20 information-processing 8 information theory 34 InnenweU 25,61,69,78,175,176,203,217, 227,230,236,251,255,256,279 instantiation 133, 156 integration contextual 49, 158, 232 phonological 119 space-time 81 intent, teleological 100 intention 4,17,31,33,36,48,74,100,101, 102 communicative 115 proximate 138, 139 semantically organized 268 intentional content 296 intentionality 68 interact (semantic/dialogic) 38,215,216, 217 interaction dialogic 36, 68 forceful (between body and world) 165, 166, 168 non-linear 17 Interaction System 9, 264, 265 interdependence, semiotic-material 23 interface, bio-kinematic 83 internal milieu 261,290,293 interpersonal moral order 6, 214 interpretance, system of 15, 16,20,25,28, 35,63,71,83,175,178,180,181, 223, 224, 226, 231, 232, 233, 242, 250,251,255,257,262,287,300, 303,304,305,306,307,308,309, 315 self as 174 interpretant sign 305, 306 in terpreter 35 intersubjectivity, primary 19, 36, 179, 282

SUBJECT INDEX intonation 85,92,94,98,99,101,113,119, 161,295 introception 262 invariant perceptual 230 phonological 272, 274 visual 234 isolation, ontological 43 judgement 305 faculty of 224 kinaesthesis 81,82,186,187,261 kinematics 82 kinesics 24 knowledge declarative 212 procedural 212,213 representational 213 sensori-motor 213 labour, social division of 99,142,237 language Cartesian view of 74 formalist models of 7 formal theories (models) of 48, 137 metafunctional organization of 164-165 metafunctional theory of 195 mixed-mode character of 153 multifunctional character of 194 physical manifestation of 78 stratal organization of! stratified nature of 136, 271 stress-timed 123 syllable-timed 123 language faculty 7, 73, 74 language gene 24 langue 49, 60 langue interieure 61,69,76 lateralization (of brain functions) 190, 191 level focal 14,15, 16 integrative 39,127,146,147,148,161, 173,177,201,206,221,248,254, 259,262,273,293 lexicogrammar 42, 46, 55, 62, 69, 70, 77, 78,83,99,107,126,127,140,205, 211,225, 274 as state space 245 metafunctional organization of 152, 153 symbolic possibilities of 77 limit cycle 103 linguist, integrational 44 linguistics, systemic-functional 70, 79

339

logic, propositional 194 logogenesis 145,149,150,294,297 loop (looping) Batesonian 250 reentrant 207,256,258,288,289 magnetic resonsance imaging (MRI) 171 map first-order neural 252, 253 sensori-motor 252 mapping neural 265 reentrant 71,209,210,220,240 somato-sensory 262 materialism 313 eliminative 223 meaning experiential 47,86,128,196 iconic 266 interpersonal 47,56,86,212 linguistic 47 logical 47, 86 referential theory of 229 symbolic 256 textual 47,86, 128, 129 meaning-making multimodal 19 multiplicative character of 192 meaning-potential 13,239,244 Meaning System 264, 265 mechanism causal 10, 73 central 73, 74, 78 internal 72 lower-scalar 72 mediation, semiotic 44, 138, 139, 194, 221 memory 214,234,235,253,258,263, 283 ecosocial 310 long-term 111, 290 short-term 262, 263, 264, 277, 278 value-category 310 value-laden 308 message unit 156, 157 meta-discourse 46 metafunction 46, 103, 107, 126, 127, 149, 150,191,192,193 experiential 124, 125, 196 interpersonal 94,124,199,200 logical 93, 130, 131,277 textual 123, 200, 201 meta-redundancy 25,26,27, 29, 30, 31, 32, 56,89,164,176,193,227,230,244, 272,293,295,300,309

340

SUBJECT INDEX

meta-redundancy hierarchy 28 me ta-representation 287, 288, 294 micro-function 42, 147, 166, 168, 245 mind 6, 7, 8, 175, 188, 225, 229, 282, 302, 305, 313 theory of 55, 174, 176, 184 mirror neurons 56,267,268,269,270 modality 129, 130, 198,217 semiotic 49 modalization 216 model Cartesian-Euclidean-Newtonian 73 cybernetic 72 modulation 216 mood 129, 130, 132, 201, 205, 215, 269 moraicity 114,121,295 morpheme 70, 155, 158 morphosyntax 43 movement, theory of 78 multimodality 46 mutuality, of animal and environment 12 nativist hypothesis 282 naturalism 223 necessity, indexical 61,77,85, 134, 230, 300, 316 negotiation, discursive 215 neo-cortex 185, 187, 218 pre-frontal 215 net, neural/neuronal 294, 296 self-organization of 284, 285, 301 network neural 22, 305, 312 neuronal 41 neural cell assembly 284, 285, 294 neuron 171 neuronal group selection 241 New 128,129,299,300 nominal group 133, 134, 155, 156, 157, 276 notation, written 89 observer 77,78,105,222,223,232,233, 248, 290 categorial reach of 127 observer system 249 Onset 112 ontogenesis 48, 139, 245, 246, 294 ontogeny 230 organization cerebral 21 emergent 313 heterarchical 21 hierarchical 21

metafunctional 18, 19,91, 103, 104 phonological 161,295,296 oscillator, nonlinear 84 paradigmatic contrast set 305 paralanguage 24 pathway corti eo-bulbar 218, 294 developmental 19,316 goal-seeking 270 pattern afferent stimulus 287 mental 249 paradigmatic 30 syntagmatic 30 thematic 116 percept, analogue 225 perception 7,11,12,29 allocentric 107 ecological theory of 11, 296 event 291 periodicity, rhythmical (rhythmic) 84, 85, 92,93,104,295,296 periodicity wave 128 perspective (view) field, particle, and wave 121 first-person 173 externalist 224, 229 inner 287,289,297 internalist 224 internalized 255 intra-organism and inter-organism (on language) 48 observational 175 observer 41, 137, 287 particle 128 self-referential 177, 250,255,261,276, 317 third-person 173, 257 phase space 42, 121, 151 semantic 241 phonation 86,112,160,218,289 phoneme 70,91,119,121,222,266 phonetics 72 phonology 43, 70, 72, 87, 117 metafunctional organization of 152, 153 quantal theory of 141 phylogenesis 48, 139 pick-up of information 12, 13, 28 perceptual 27, 54, 118 pitch 94,95,97,111,113,119,295 pitch fluctuation 124 polarization, of self and world 210, 259

SUBJECT INDEX positing, teleological 237 positron emission-tomography (PET) 171, 267 possibility, symbolic 77 potential, bio-kinematic 84 power, causal 41 practices, meaning-making 9 precursor world 213 primate 80, 247 principle, contextualizing 37 Principle of Alternation 20,22,54,56, 109, 137, 294, 312 program, central 82 prominence, tonic 113, 295 proprioception 220,221,246,261,262 process bio-physical topological-categorial properties of 82 neurological 19,20 context-sensitive nature of 252 neuronal 43 self-organizing 49 project 77 reductionist 39 proposal 269 proposition 130,208,269 prosody 95,97, 107, 115, 116, 119 articulatory 86, 92 field-like nature of 107 gestural 101 interpersonal 94,97, 107 semantic 95 proto-imperative 211 proto-indicative 211 proto-language 38,42,55,139,147,156, 161, 163, 167, 168,239,240,243, 244 proto-meaning 212,254,265,266,267, 289, 290 proto-modalization 38 proto-proposition 205, 206, 208 proto-self 147,252,254,255 proto-semiosis 220,245,251,267 multiple realizability of by neurological processes 267 proto-Subject 205, 208 quantification 156 quantum mechanics 33,35, 121 rank scale 154, 157, 158 lexicogrammatical 151 phonological 94,114,151,161,295 reactance 95

341

reading 81 realization 70,116,144,182,254,266,312 reasoning, syllogistic 188 recategorization 141,144,146,163 receptor 105 kinaesthetic 74 peripheral 203 receptor cell 11, 289, 291, 303, 304 receptor organ 25, 61 recursion 7, 93 reduction, downwards 31, 73 reductionism, physical 302, 313 redundancy 26,27,28,29,251,301 contextual 15, 36, 165, 187,308 reentry 275 reference 126 hierarchy of 219 symbolic 175 reference scale 62 reflection experiential 273 meta-semiotic 233 register 142 regulation, sensori-motor 21 relation contextualizing 23,30,34, 176,229,247, 265,293,300,310,317 heteroglossic 50 relationship indexical 61,116 realizatory/realizational 40,78,274,301 Remiss 121,122,124,154 repertoire conceptual 139, 140, 146 phonetic 139 sensori-motor 146 representamen 305 representation 12, 173, 175,240,307,312, 314 mental 7,8 resource somatic 77 tactic 40 value-creating 47 retina 235 Rheme 128, 129,299 rhetoric 97 Rhyme 112,295 rhythm 93,94,112,116,119,123,124, 154, 160, 296 robustness 64, 65, 233, 257 routine articulatory 271 auditory 296

342

SUBJECT INDEX

cogmtive 271,297 conceptual 55,138,143,144,160,163, 295 lexicogrammatical 295 motor 75, 272, 274 imagined 187 phonological 274, 295 sensori-motor 55,140,141,143,144, 163, 293, 295 rule deterministic 72 innate 239 linguistic 283

salience, motivational 184, 189 scale, space-time 10, 17, 65 schemata (schema) action 165 Body-Forceful-Interaction-Environment 165, 166 corporeal 82 interpersonal action 167 movement 83 sensori-motor 167 score, gestural 91,92 secondness 34,35,75,77,93,94,249 R~I~~l~l~l~lnl~l~

183,198,206,220,223,229,247, 249,252,254,258,261,265,274, 276,288,312,313,317 autobiographical 253 experiential unity of 262 grounded perspective of 248 neurally realized 261 qua self-organizing system 34 self-awareness 250, 275, 317 selfhood 182 self-organization 38, 55, 56, 141, 176, 177, 242,261,262,317 self-reference 246 sememe 70 semiosis 6, 29, 31, 38, 43 iconic, indexical, and symbolic modalities of indefinite 306 infant 42, 213 internalized (inner) 177, 238 linguistic mixed-mode character of 151 stratified nature of 233 symbolic 262 topological 23 typological 23, 43 semiotic action formation 44

sensation 223,225,229,291,302,305 acoustic 291 bodily 258 empiricist theories of 222 haptic 106 sharing, mental 205, 207 sign 34,39,40,52,67,202,303,314 iconic 119 indexical 115, 116,243,287 indexically creative or entailing 255 indexically presupposing 255 symbolic 287 system of 80, 203 signified 107 signifier 59, 60, 62 simulation, internal 76, 107 solipsism, Cartesian 236,257 sound (Saussure) 60,61 space 105, 106 allocen tric 106 attractor 50, 108 distant 106 egocentric 106 extra-corporeal 106 extra-personal 106 interpersonal neural 269 personal 106 n-dimensional semantic/semiotic 50, 245 multidimensional physiological 142 multidimensional semantic 55, 194, 239, 245, 278, 279 self-organizing 271 semantic neural 17 symbolic neural 6, 144, 159, 160, 187, 193,272,278,297,298,317 space-time, Newtonian 38 specialization, hemispheric 191 specification hierarchy 39, 54, 55,99, 254, 255,260,262,266,294,317 specification, levels of 38 speech, inner 10,81, 187, 189,263,271, 272,273,274,275,297 speech function 215 state emotional 98 mental 5, 19,22,49,247 metastable 182 neurophysiological 32, 33 stimulus perceptual 152 physical 303 sensory 76 stimulus-response dichotomy 230

SUBJECT INDEX stratification 217,274 linguistic 39,40,63, 70, 118, 142 ontological 33, 34 semiotic 43, 233 structuralism 49 structure constituent 122 coordinative 141 dissipative 10,25,112,146,147,159, 161, 182, 244 phonological 85, 90 Subject 132,199,269,299 substance, phonetic 59 substrate neural 18 neuromuscular 115 topological 82, 103, 104 supersystem 160,227,229,238,255,264 supervenience 80 syllable 84,90,94,112,113,119,120,154, 157,159,222,266,295,296 syllable boundary 120 syllable weight 160 symbol 51,126,135,195,211,222,230, 240 internal 240 symbol string 51, 74 symmetry-breaking 78 symesthesia 81 synapse 285,286 syntagm 98, 124 system abiotic 24 central nervous 13, 15,28,51,227,234, 260,261,290,293,294,297 closed 10 dynamic open 8,10,11,40,48,49,138, 178,179,207,244,279 ecosocial 43 evolutionary 142 far-from-equilibrium 244 goal-seeking 24,216,317 higher-scalar 35 infant 45 language 49,50, 51, 52, 60 multimodal 45 organism-plus-environment 36 perceptual 186 peripheral nervous 227, 290, 293, 294 phonological 63,111,291,292 scalar hierarchical 20, 43 selective recognition (brain as) 55, 139, 165

343

self-organizing 11, 182, 236, 244, 245, 310, 313 semiotic resource 237 skelotomuscular 75 somatic recognition (brain as) 164, 187, 222,234,272,274 somato-sensory 261 symbolic 300 transitivity 79 three-level 14 three-level hierarchical 15 vestibular 261 text 3,16,72,117,154,263,276,277,278, 279, 309 texture, phonological 120 Theme 128, 129,299 theory breath-group 84 homeokinetic 84 motor 100, 101, 102 movement 90 systemic-functional 47,48, 117, 154,238, 273 Thermodynamics, First and Second Law of 226 Thing 133, 134, 155, 157 thinking 188 conceptual 7 higher-order 216 linguistically realized 271 reflexive 277, 278 symbolic 189,263 verbal 188 thirdness 71,80,93 thought (Saussure) 60,61 semiotically mediated 215 time absolute (clock) 295,296 evolutionary 240 historical-biographical 75 real 151 relative 295 timescale(s), non-adjacent 43 tone group (see also breath group) 84,85, 295 tonicity 113, 121, 161,295,296 tool body as 106 brain as 293 topic-comment parsing 205 topic predication 206, 207, 209 trajectory 3,4,5,6,10, 13, 14, 16, 18,20,

344

SUBJECT INDEX

21,50,83,138,141,147,150,159, 160, 161, 167, 177, 181, 217, 228, 230,242,253,261,270,275,278, 313 action 260, 297 agentive determinability of 114 developmental 275, 283 historical-biographical 4, 44, 50, 55, 111, 140, 227, 265 individuating 102,147,313 lifespan 180 logogenetic 146, 159, 310 ontogenetic 3,54,182,226 rhythmic 112 semiogenetic 4,5,25,51 social (moral) 176 syllabic 115 temporal 151 vocal-tract 111,112,139 transcranial magnetic stimulation 267 transduction 26, 63, 203, 219, 233, 292, 297, 315 symbolic 88, 234 transformation 93 transitivity role 97, 156, 197, 198 type-category 197 type-specification 156 Umwelt 11, 12,37, 60, 69, 71, 79, 83, 85, 176, 178, 189, 211, 217, 236, 282, 287, 293

vagueness 45,46,50,176,180,184,190, 207,230,234 valence,affective 204,212 value 47,49,51,52,71,88,89, 101, 104, 132,133,184,189,207,209,210, 211,212,214,217,220,242,244, 245,282,309,316 conceptual 223 interpersonal 195 social 200 symbolic 203 value bias 51,52,53,54 value stance 299 value system 160 variability 11, 29, 50 variation (variety), topological-eontinuous 23,25,26,42,52,109,118,125,127, 135,161,167,178,190,193, 200, 216,218,222,230,234,243,259, 292 vocalization 6 vocal tract 41,67, 103, 152, 189 vowel 40 wave, rhythmic 151 Wernicke's area 140