Distributed Language (Benjamins Current Topics)

Distributed Language

Benjamins Current Topics Special issues of established journals tend to circulate within the orbit of the subscribers of those journals. For the Benjamins Current Topics series a number of special issues of various journals have been selected containing salient topics of research with the aim of finding new audiences for topically interesting material, bringing such material to a wider readership in book format. For an overview of all books published in this series, please see http://benjamins.com/catalog/bct

Volume 34 Distributed Language Edited by Stephen J. Cowley These materials were previously published in Pragmatics & Cognition 17:3 (2009)

Distributed Language Edited by

Stephen J. Cowley University of Hertfordshire, UK

John Benjamins Publishing Company Amsterdamâ•›/â•›Philadelphia

8

TM

The paper used in this publication meets the minimum requirements of American National Standard for Information Sciences – Permanence of Paper for Printed Library Materials, ansi z39.48-1984.

Library of Congress Cataloging-in-Publication Data Distributed Language / Edited by Stephen J. Cowley. p. cm. (Benjamins Current Topics, issn 1874-0081 ; v. 34) Includes bibliographical references and index. 1. Psycholinguistics 2. Language acquisition. 3. Language and culture. I. Cowley, Stephen J. P37.4.D51284 2011 401’.9--dc23 2011029013 isbn 978 90 272 0253 6 (Hb ; alk. paper) isbn 978 90 272 8415 0 (Eb)

© 2011 – John Benjamins B.V. No part of this book may be reproduced in any form, by print, photoprint, microfilm, or any other means, without written permission from the publisher. John Benjamins Publishing Co. · P.O. Box 36224 · 1020 me Amsterdam · The Netherlands John Benjamins North America · P.O. Box 27519 · Philadelphia pa 19118-0519 · usa

Table of contents About the Authors Distributed language Stephen J. Cowley The role of anticipation in reading Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela The experiential basis of speech and writing as different cognitive domains Alexander V. Kravchenko

vii 1 15

33

Insightful thinking: Cognitive dynamics and material artifacts Evridiki Fioratou and Stephen J. Cowley

57

Actualizing semiotic affordances in a material world Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

81

Languaging in Shakespeare’s theatre Evelyn Tribble

101

Semiotic cognition and the logic of culture Barend van Heusden

117

Ecological pragmatics: Values, dialogical arrays, complexity, and caring Bert Hodges

135

Symbols as constraints: The structuring role of dynamics and selforganization in natural language Joanna Rączaszek-Leonardi

161

Beyond mind: An extended ecology of languaging Sune Vork Steffensen

185

Subject Index

211

Name Index

219

About the Authors

Stephen Cowley is a Senior Lecturer in Developmental Psychology at the University of Hertfordshire, UK. Although his PhD and early career were in Linguistics, since 2000 he has lectured in Psychology and Cognitive Science. Recent empirical work has focused on interactions between mothers and infants, children and robots, and simulations of health emergencies. He founded and co-ordinates the Distributed Language Group, a growing community who aim to transform the language sciences by focusing on how directed, dialogical activity imbues human intelligence with a collective dimension. Timo Järvilehto is Professor of Psychology in the University of Oulu since 1986. His basic field of study is brain research and psychophysiology, starting in the 1960’s with EEG problems (CNV, evoked potentials), extending to problems of sensory physiology and psychophysics (human microneurography), and to unit studies in behaving animals. Since 1990 he has been developing psychological theory under the heading “systemic psychology” based on the Theory of the Organism-environment System. Veli-Matti Nurkkala has a Master’s degree in Sport and Health science and is currently doctoral student in the University of Oulu (SkilLab). His doctoral thesis investigates the time course of reading process in different reading situations in order to develop a model of reading process. His other interests include emotional dynamics, well-being and stress levels of athletes and pupils. Kyösti Koskela has a Bachelor of Engineering degree and is currently studying for a Master of Sport Science at the University of Jyväskylä. He is currently working as a laboratory engineer in SkilLab, specialising in eye movement research. His other fields of interest include motion analysis, heart rate variability, and electromyography in skilled actions. Alex Kravchenko is Chair of the Department of Foreign Languages at the Baikal National University of Economics and Law. Since the late 1980s he has worked in cognitive linguistics and, specifically, the perceptual groundedness of grammatical categories. He developed a unified cognitive theory of aspect in English and Russian, and more recently, he has focused on the biological theory of language and cognition, the phenomenology of semiotics, and epistemological issues in the philosophy of language. Evridiki Fioratou is currently a Research Fellow at the University of Aberdeen, working on anaesthetists’ intraoperative problem solving. She has previously been a Research Fellow at the University of Hertfordshire on problem solving projects, including individual differences and verbalization and self-explanation effects, on the student population. Her research focus is on the cognitive dynamics affecting problem solving in the lab and “in the wild” and she is an advocate of the distributed cognition approach.

viii

About the Authors

Kristian Tylén has an MA in Cognitive Semiotics from University of Aarhus and a PhD in Linguistics from the University of Southern Denmark. He currently holds a postdoctoral position at the Center for Semiotics, University of Aarhus, and the Center for Functionally Integrative Neuroscience, Aarhus University Hospital. His main interests reside in the interface between semiotics, aesthetics, and the cognitive sciences and the application of various experimental approaches (including brain imaging) to the study of human meaning construction. Johanne Stege Philipsen has an MA in Linguistics and Cognitive Semiotics from University of Aarhus, Denmark. She has worked as a research assistant at the Institute of Language and Cognition, University of Southern Denmark and Center for Functionally Integrative Neuroscience, Aarhus University Hospital. Her main interests are within social neuroscience, cognitive semiotics, and neurolinguistics. Ethan Weed has an MA in Cognitive Semiotics and is currently a Doctoral Candidate at the Department of Linguistics, University of Aarhus. His research interests include language, cognition, and pragmatics. His work combines lesion studies with ERP techniques to investigate communication impairments following right hemisphere damage. Evelyn Tribble is Donald Collie Professor of English at the University of Otago. She is the author of Cognition in the Globe: Attention and Memory in Shakespeare’s Theatre (New York: Palgrave, 2011) and (with Nicholas Keene) Cognitive Ecologies and the History of Remembering: Religion, Education, and Memory in Early Modern England (Basingstoke, Hampshire: Palgrave, 2011). Her interests include memory studies, performance and memory, theatre history, and literature and cognition. Barend van Heusden is senior lecturer in the Department of Arts, Culture, and Media Studies of the University of Groningen (NL). His research focuses on the semiotics of culture and the arts. Since 2008 he has been leading the Dutch national research project ‘Culture in the Mirror’ (2008–2012), which aims to develop a theoretical framework for an integrated culture education curriculum for children and youngsters from 4 to 18 years old. Bert H. Hodges received his PhD in Psychology from Vanderbilt University and is Professor of Psychology, Gordon College, Wenham, MA, and Senior Research Scientist, University of Connecticut, Storrs, CT. His work is focused on developing and applying an ecological account of values in perception-action, social psychology, and language. He has co-edited with Reuben Baron a special issue of Ecological Psychology, Making social psychology more ecological and ecological psychology more social and another with Carol Fowler, Distributed, dynamical, and dialogical: New coordinations for language. Joanna Rączaszek-Leonardi received her MA at the University of Warsaw, and her PhD at the Center for Complex Systems and Brain Sciences at Florida Atlantic University. She is presently Marie Curie Fellow at the University of Bologna and an assistant professor at the University of Warsaw. She is a psycholinguist interested in language as a dynamical system, and is engaged in identifying important dynamics and timescales in language functioning, learning, and evolution. Her experimental interests concern methods for studying dynamical processes of on-line communication.



About the Authors

Sune Vork Steffensen is associate professor at the Institute of Language and Communication at the University of Southern Denmark. He leads the Research Group: Holism, Dialogue, Organization which investigates cognition, language and interactivity in organizational settings. His research interests include holistic, ecological and distributed theories in (interactional) linguistics as well as health interaction. He has co-edited the books Language, Ecology and Society: A Dialectical Approach (London: Continuum, 2007) and Signifying Bodies: Biosemiosis, Interaction and Health (Braga: Portuguese Catholic University Press, 2010), and authored papers and chapters on language and interaction. He is also co-ordinator of the international Distributed Health Interaction programme.

ix

Distributed language Stephen J. Cowley

University of Hertfordshire, England

In viewing language as multi-scale co-ordination, the distributed perspective challenges two dominant orthodoxies. First, it denies that language is essentially ‘symbolic’ and, second, that verbal patterns are represented inside minds (or brains). Rather, language is, at once, collective, individual and constitutive of the feeling of thinking. It is distributed between us. In illustration, the opening Chapters report empirical work on the anticipatory dynamics of reading, its cognitive consequences, Shakespearean theatre, what images evoke and solving insight problems. Having given reason to consider this challenge to linguistic autonomy, the collection concludes with theoretical papers. First, it is argued that language depends on a species specific form of semiotic cognition. Second, it is suggested that realizing values is a central function of language. Third, as with all social activity, this is traced to how cultural and biological symbols coregulate human dynamics. Finally, Steffensen (this volume) argues, far from being organism-centred, language gives us access to an extended ecology in which, through co-ordination, we enact our own history.

1. Beyond symbol processing Computational views of mind invoke a system that functions syntactically and, for that reason, without reference to human life. Today, however, the study of cognition is moving away from such models. Living human beings rely on, not just symbols, but also interactions that sensitise us to each other and our cultural practices. This commonplace idea took on new life when Hutchins (1995a, 1995b) used cognitive models to examine how we navigate ships and land planes. He showed that culturally-specific artefacts and narratives serve in propagating representations in a public domain. Cognition is cultural and embodied: while much happens in the brain, events arise as people interact both with each other and the world. While the view is now mainstream in cognitive science, Distributed Language shows its radical implications for language. Since humans do not need artefacts to embody thoughts, language is fundamentally dynamic. Verbal patterns constrain bodily

2

Stephen J. Cowley

movements and the feeling of thinking as people co-ordinate the flow of activity. The perspective thus challenges theories that privilege linguistic form and/or function. Co-ordination becomes a means of embodying thoughts: language is, at once, ecological, dialogical and non-local. 2. The distributed perspective Language can be traced to how living bodies co-ordinate with the world. On this perspective, far from being a synchronic ‘system’, language is a mode of organization that functions by linking people with each other, external resources and cultural traditions. We concert speech, thinking, gesture and action in speciesspecific ways. Language arises as we give voice to wordings, make gestures, imagine and deal with objects and institutions. It is whole-bodied activity that shapes sense-making and, once skills develop, allows texts and institutions to enrich what we think and do. Rather than view language as an object, we live in a social meshwork (Steffensen, Thibault and Cowley, 2010; Thibault, 2011) whose dynamics fuse events that draw on many time-scales. Linguistic experience alters who we become as we orient to others (who orient to us). Just as I co-ordinate with my imagined reader, you draw on your expectations, scan what is before your eyes, evoke memories and, perhaps, see future prospects. Even in reading, language-activity connects eye and head movements with inscriptions and wordings. For those concerned with the results, we can ask what happens as we create and construe language and, generally, manage human action. Language links the here-and-now with what has been and, crucially, what is to come. It is thus beyond dispute that, in this sense, language is a distributed phenomenon. Though languages and their parts constrain sense-making, humans also rely on intertwining gestures, voices and artifacts. When the language sciences focus on these multi-scale dynamics, they discover an alternative to positing a priori linguistic signifiers (or language-systems). Before turning to dynamics, I briefly sketch difficulties that arise from putting symbols (or words) first. Above all, these are abstractions that unzip language from embodied activity. They conflate acts of utterance (movements) with descriptions of results (as verbal patterns). Not only does this mask the interdependency of voices, gestures and artifacts, but languages become disembodied ‘systems’. They come to be reified in terms of letterlike constituents, larger units and linguistic forms. While of value to characterise meanings, words, and grammars, abstractions cannot explain human behaviour. Quite simply, what we do and say – how we embody thoughts – is cultural activity. Symbol-first or disembodied approaches overlook activity and, in its place, offer



Distributed language

explanations about the organism (or its parts). Instead of tracing skills to experience, these are ascribed to the functioning or minds, brains, discourse or, perhaps, knowledge of social conventions. Appeal to language-systems excludes real-time dynamics by invoking brains or minds that make, construe and manage utterancetypes said to be generated or produced by a single organism. For Saussure, regularities are ‘imposed rather than freely chosen’; forms and/or functions draw on an inheritance that we have ‘no choice but to accept’ (Saussure, 1983: 71). On a disembodied view, linguistic signs are given in advance and human languaging reduces to how ‘systems’ manipulate verbal patterns. In the distributed language movement, by contrast, we reject symbol-first views of language. 3. The Distributed Language Movement: Prehistory Language can be traced to multi-scalar dynamics that spread across groups, artifacts and time-scales. The perspective emerged from linking integrational critique of linguistics (Harris, 1981; Spurrett, 2004), with distributed cognition (see, Hutchins, 1995a; Hollan, Hutchins and Kirsh, 2000; Giere, 2004). The move made clear that challenges to code views of language parallel those mounted on the symbolic view of mind (see, Love, 2004; Kravchenko, 2007; Cowley, 2007a). Like human cognition, language is embodied, embedded and intrinsic to a cultural world. In Love’s (2004) terms, first-order activity (e.g. speaking and hearing) can be perceived, described and interpreted as verbal patterns or second-order cultural constructs. On this view, learning to talk depends on human bodies that enact intersubjective behaviour (Trevarthen, 1979). Contextualizing bodies prompt us to vocalize, engage with others and, eventually, act in line with constraints that are perceived as verbal patterns (Cowley, 2004). We learn to act and, when it suits us, to frame explicit messages. This, however, depends on a a form of control based on hearing utterances as examplars of verbal patterns. By coming to do this, or taking a language stance (Cowley, 2011), we develop skills based on careful use of wordings. This contrasts with spontaneous language that arises as we engage with people, things and even our own thoughts. At all times, however, digital signalling (Ross, 2004) is co-ordinated with bodily expression and prosody. In the resulting utterance-activity, vocal and non-verbal expression are integrated by bodies that adjust to events in a cultural world. As Ross (2007) argues, following Dennett (1991), we narrate selves into being. Biological agents reorganize as persons who integrate events (of various kinds) with structures based on ontogenesis, learning, history, and human phenotypes. To pursue reciprocal links between language and humanity, a group of scholars set the goal of transforming the language sciences.

3

4

Stephen J. Cowley

We founded the Distributed Language Group (DLG) whose first meeting at Sidney Sussex College gave rise to papers (see, Cowley, 2007b) that inform the current focus on dynamics. However, before turning to their multi-scalar complexity, I sketch how language is ecological, dialogical and non-local. 4. Language: ecological, dialogical and non-local Whereas disembodied views place language in either the mind or in society, the distributed perspective treats language as part of the ecology. It arises as social events link bodies with the physical environment and cultural traditions. Language is therefore neither localized within a person (or a body) nor a property of the environment. This ecological perspective challenges all organism-centred models. It asserts that, “in any functional sense organism and environment are inseparable and form only one unitary system” (Järvilehto, 1998:329). Rather than separate language from artifacts and actions, a history of bodily co-ordination gives us the necessary skills. As we go about our lives we encounter selves and others whose lives are deeply affected by linguistic resources. Language is activity in which wordings play a part. The umbrella definition permits us to connect up concepts that include ‘languaging’ (Maturana, 1988; Kravchenko, 2006), ‘utterance-activity’ (Cowley, 1994, 2009b; Thibault, 2011), ‘first-order language’ (Love, 2004), ‘dialogue’ (Linell, 2009), ‘colloquy’ (Jennings and Thompson, in press) and ‘embodied, embedded language use’ (Fowler, 2010). In denying that individuals produce and process utterance-types, co-ordination is traced to use of ecological resources. As we engage with language, we dream, think, talk and use texts, telephones, computers and so on. In recognising this diversity, the study of language becomes ecological (see, Hodges, 2007; Hodges and Fowler, 2010; Thibault, 2011; Cowley, 2011). More specifically, emphasis falls on what results from continuous activity by Organism-Environment Systems (see, Jarvilehto, 1998; 2009). On this view, the concepts of language, action and perception can all describe the same events. To read, for example, is to perceive and, necessarily, to actively construe what one sees. In dialogue, as we speak, voice dynamics shape hearing, feeling and thinking. Even writing depends on monitoring the results of movements both in real time and by means of editing. As part of action and, given imagination (and consciousness), language becomes part of silent thought: it is gradually insinuated into perception as reiterated phonetic gestures are connected with both wordings and our modes of life. Utterance-activity arises as we make and track phonetic gestures (Fowler, 2010) that prompt us to hear utterance-types. Using different time-scales, rich linguistic memory evokes



Distributed language

experience (Port, 2010) that gives wordings a particular sense. Utterance-types that we report (or transcribe) are also co-ordinated voice dynamics. Dialogical activity is constrained by phonetic gestures that prompt us to hear wordings and, at times, to attend to verbal structures: languaging arises as phonetic gesture is coordinated with other neuromuscular activity. We talk, experience selves, encounter others and enrich the feeling of thinking (Harnad, 2006). From an ecological perspective, this is neither internal nor organism-centred: feelings contribute to a common world. Language is also fundamentally dialogical. While its ecological side can be traced through phonetic science, its ‘digital’ or verbal aspects profoundly influence our sense of self and experience of others. In part, we are how we speak with each other. As Bakhtin (1981) and Mead (1932) saw, experience of sense-making insinuates a self into how we feel, act and think. As this emerges, we discover the criss-crossing or overlapping senses that are evoked by wordings. Semantics is dialogical and rich in connotations: if this seems odd, it is because written language bias (Linell, 2005) masks the other orientation (Linell, 2009) of human communication. To speak is to anticipate the response of the other: vocalising and moving drives flexible, adaptive behaviour. Though inseparable from cognition, language unites social action, verbal pattern, meaning and, crucially, real-time understanding. Further, the brain self-organizes as social co-ordination prompts us to individuate. In learning to talk, we speak, monitor the saying, the said and displays of expectations. Contingencies of our lives lead to the flowering of language. To make sense of its complexities, we entwine dialogue with actions and, thus, set off expressions of power and experience of relationships. This may seem puzzling: how can a focus on organism-environment relations be linked to the wordings and dialogical events of social life? How can language enact expressive control, feeling, thinking, and prompt us to hear people saying something? Taking a phenomenological view, Linell (2009) emphasises situations while recognising that expression evokes traditions, voices and ever changing circumstances. Thus, while situated, language exemplifies double dialogicality by linking lived events�����������������������������������尓����������������������������� with material and, inseparably, silent or ‘third-party’ phenomena (2009: 21). Linell echoes Bakhtin’s, “the world is a drama in which three characters participate (it is not a duet but a trio)” (1986: xviii). Far from appealing to linguistic signification/meaning or a Peircean triad of sign/object/effect, a world of social norms prompts us to orient to absent others. We integrate activity, what we hear, and bundles of social expectations. This contributes to what Goffman (1959) calls the ‘public presentation of self ’ by means of, for example, use of interactional regularities, genres, register and language varieties. Language is thus grounded in neither bodies nor society but the play of dialogue. The challenge to the distrib-

5

6

Stephen J. Cowley

uted movement is that of reconciling our dialogical propensities with, first, our cultural nature and, second, our co-evolutionary history. The challenge of integrating time-scales demands a non-local ontology (Steffensen and Cowley, 2010). By linking phonetics, phenomenology and its products, language becomes measurable activity that, oddly, is perceived around historicallybased patterns. Its symbiotic character undermines any simple division between subject (the observer) and object (the observed). We are bound to be sceptical that social events, including linguistic events, can be traced to a localised ‘cause’ or, in Whitehead’s (1926) terms, that they can be explained around the assumption of simple location. Indeed, even objects like stones exist in a state of change – however slow the change may be. While much could be said, a cautious view highlights biological function. Living systems do not ‘occupy’ space-time because, among other things, their genetic structures outlast phenotypes. Even simple human observations and actions link evolutionary, developmental and collective history. Explanations of real-time language cannot ignore non-local “regions of space-time” (Whitehead 1926: 62). Given that linguistic states or processes do not ‘occupy’ a determinate space-time zone, Steffensen and Cowley (2010) propose a principle of non-locality. Cognitive dynamics evoke (non-local) wordings: language is both measurable first-order activity and organization that sustains traditions. It is both dynamical and symbolic or, in short, a symbiotic mode of communication. While some trace this duality to our use of virtual structures (Cowley, 2007, Love, 2007), others stress that biological dynamics result from physical constraints (Carr, 2007; Rączaszek-Leonardi and Kelso, 2008).1 In enacting utterance-activity, they argue, we draw on physical structures (in some to-be-specified sense). While the future of this debate is unknown, the underlying premise is clear. However non-locality is construed, language links people, phonetic activity, wordings and history. Though occurring ‘in’ time, it is not wholly situated. Activity is ‘mediated’ by patterns that make language, at once, a phenomenological construct and a biological product of evolutionary change (including natural selection). 5. Investigating the glue of cognition The DLG views language as ecological, dialogical and non-local. While specifying this perspective is a step towards transforming the language sciences, it is just a beginning. Making a further move, this volume shifts the emphasis from symbols to bodily dynamics. Five derive from a workshop on the Dynamics of Linguistic Material where several authors (Cowley, Kravchenko, Fioratou, Tylén, Van Heusden, and Rączaszek-Leonardi) scrutinised the view that material symbols extend the



Distributed language

mind (Clark, 1998; 2008).2 Though differences abound, all concur that dynamics are at least as important as ‘symbols’ (or slow dynamics) and, in terms offered by David Kirsh, that co-ordination is ‘the glue of cognition’ (Kirsh, 2006). In scrutinising co-ordination, several papers focus on how historically derived resources affect language-activity. First, Järvilehto and colleagues (this volume) show the crucial role played by anticipatory dynamics in reading aloud. Then, turning to the theatre, Tribble (this volume) explores the languaging that occurred in Shakespeare’s historical and material context. Next, using ecological psychology, Hodges (this volume) shows how human values realizing draws on linguistic dynamics. Finally, having traced material symbols to grammatical tradition, Steffensen (this volume) suggests that a history of linguistic co-ordination has transformed the human ecology.3 Verbal patterns allow living bodies to use co-ordination to connect us with each other, artifacts, thoughts and actions. Given that language is multi-scalar, human voices locate utterance-activity in history. Language draws on collective resources that give meaning to individual actions (and lives). Authors give quite different emphasis to individual, interactional and collective factors. While some focus on second-order constructs (verbal patterns) others highlight first-order activity. All concur, moreover, that human experience arises as we co-ordinate with artifacts and each other. Literal meaning is often secondary even in making and construing written texts. Järvilehto et al. (this volume) demonstrate that, in reading aloud, people generate meaning and, using gaze, test expectations against inscriptions. Textual patterns are imbued with sense as we couple dynamics with the feeling of thinking. Given that co-ordination allows us to project meaning onto the text while monitoring its physical features, this exemplifies how we act as Organism-Environment Systems (Järvilehto, 1998). In this same spirit of body-world interdependence, Kravchenko (this volume) turns to social change in Russia. Rejecting symbol-first (or ‘code’) models of language, he argues that, with changing reading (and teaching) habits, the educated are losing inferential skills. With the abandonment of intensive study of (above all) literary texts, inference-making is in decline. While conjectural, the paper emphasizes differences between reading and dialogue: social reality depends on interaction between these cognitive domains. Reading, for example, gives us cognitive powers based on orienting to our selves as, among other things, observers of what we read. Other papers focus on how artifacts influence cognitive dynamics. Using an insight experiment, Fioratou and Cowley (this volume) contrast solutions between concrete and abstract versions of a task. They find, first, that using artifacts makes the task easier. Second, they argue, this is because they contribute to the functioning of the distributed cognitive system (see, Hollan et al. 2000) without being intrinsically cognitive (c.f., Giere, 2004). It is sufficient that human parts of

7

8

Stephen J. Cowley

the functional system co-ordinate with objects such that, on occasion, insight arises. By analogy, it is through active engagement with documents, programs, books and carvings that they come ‘alive’. As action human sense-making is constrained by objects and/or wordings. Remarkably, Tylén et al. (this volume) apply a similar logic to visual objects. Building on fMRI studies of how brains activate when experimental participants look at images that show various arrangements of everyday items (e.g. chairs), they map their findings onto verbal reports. In comparing descriptions of ‘signal’ images with ones depicting everyday scenes, they find that, in the former case, people offer other-oriented descriptions. Their intersubjective reports evoke collective values. While the examples are aesthetic, the same logic may well apply to axiological questions. Continuing the cultural theme, Tribble (this volume) reconsiders Elizabethan/ Jacobean performances of Shakespeare’s plays. Enacting a performance was more important than reiterating verbal (or coded) content. Company sharers, together with hired men, used material resources to improvise. Unlike actors who remain true to a text, they recreated what they imagined. Performing as distributed cognitive systems, they linked verbal patterns, gestures, metrical patterns in a public space. Given how Shakespeare’s work was performed, his writing shows collective influence. Certainly, collectivity illuminates feats like performing 6 plays in the same period. Next, Van Heusden (this volume) turns to intra-cranial resources. On his double processing hypothesis we develop internal signs based on perceiving differences. Though human memories never correspond to events, they stabilize reality and release us from the flowing present by introducing doubt. This shapes the mimetic semiosis which, for Donald (1991), underpins language and technology. In regarding language as a form of semiotic cognition, van Heusden’s view contrasts with that of the other contributors.4 The final papers develop ecological themes. Rejecting rules or value-free norms, Hodges (this volume) presents language as a caring system. It augments our perceptual and actional powers by realizing values that connect an individual with a collective domain. This is ecologically special and shapes human modes of being. Next, Rączaszek-Leonardi (this volume) addresses how language can be both symbolic and dynamic. Using biosemiotics, she suggests that, just as in living systems, no symbol can be abstracted from the meshwork in which it has evolved. As a result of constant renewal the meshwork’s dynamics function to enable and constrain social activity. Symbols measure dynamics, exert control and, crucially, prompt human measuring. Since the ‘same’ symbols allow many interpretations, language integrates events across real-time, development and evolution. Finally, Steffensen (this volume) spells out why it matters that language is non-local: he emphasises how, as individuals, we contribute to linguistic cognition. As a result,



Distributed language

we learn to act, not autonomously, but as responsible beings whose values realizing is crucial to the functioning of the extended ecology. 6. Imbumba: doing things together The symbiotic nature of language ensures that while its dynamics prompt us to action, what is said constrains what we do together. For biocultural agents like ourselves, while genes and brains matter, much also depends on wordings, artifacts and other non-local phenomena (e.g. the exchange of ‘turns’, face, money, education). Language combines skilled activity with how verbal and other recurrent patterns stabilize ways of living and working. While English lacks a term for this co-activity, in isiZulu, it is called Imbumba. As Donald (1991; 2007) suggests, skilled practices are likely to have co-evolved with language and culture. However, while van Heusden (this volume) traces the results to semiotic cognition, others stress how we anticipate what we (and others) are likely to perceive. Even reading is creative. Though more empirical work is needed, this sets up a debate. Whilst van Heusden emphasises brain-bound processes, Järvilehto and Steffensen focus on non-neural Organism-Environment relations. For Kravchenko (this volume), the resulting language meshwork (or overlapping ‘consensual domains’) give us the skills that connect up our powers. In reading, for example, we can make much use of how we orient to our changing selves. For Hodges, seeking out and grasping the affordances of words, in Evans and Levinson’s terms, enable them to “reflect cultural preoccupations and ecological interests” (2009: 436; cited, Hodges, this vol.: 148). In human cognition, biological constraints prompt us to engage with each other in a world of cultural norms and institutions. Verbal patterns, and hearing, prompt us to individuate as members of social groups. Semiotic cognition is compatible with human heterogeneity. For van Heusden (this volume), this is because semiotics indexes absence. There are no determinate linguistic entities and, equally, no sign possesses a (fully) specifiable meaning. Cultural phenomena are fuzzy or, in Hodges’ (this volume) terms, first-order languaging realizes values. This is compatible with seeing theatre as the re-enactment of a shared vision. History may have ensured that Shakespeare’s theatre was poised “between performance and poem.” Tribble (this volume) suggests that this shows in not only textual details but, equally in the performers’ practices and resources. Their heterogeneity exploits sensitivity to flow and, perhaps, human alienation (the inability to get signs quite right). Indeed, regardless of whether due to brains or a cultural ecology, there is convergence. Several contributors invoke of the inherent doubleness that appears when language is seen from a distributed perspective. While most explicit in semiotic cognition, a duality of symbols and dynamics

9

10

Stephen J. Cowley

reappears in Rączaszek-Leonardi’s view of biology. It also arises in Tylén et al.’s (this volume) contrast between what we report in images that vary between, on the one hand, personal response and, on the other, descriptions that reflect on inter-subjective attitudes. Deeper analysis calls for explanation of how we can hear utterances, see texts and perform plays as we do. Echoing Ross (2007), Tribble suggests that distributed resources give us shared meaning spaces. In Imbumba moments, people engage in flow and, at other times, experience alienation. We often have to deal with nonrecognition of signs and, on many occasions, come to see differently. Meaning arises as, together, we use dynamics to restructure our thinking. While appeal to material symbols permits only endless reshuffling, innovation is necessary if living beings are to rely on non-local resources. Precisely because these do not have the same ‘meaning’ for each person, we innovate as we co-ordinate (see, Hollan et al., 2000).5 Given non-local patterns, wordings prompt novel thoughts. In Imbumba moments, perturbances arise as linguistic resources trigger thoughts, feelings and habits (and vice versa). For Rączaszek-Leonardi (this volume), because we cognize the world, symbols become part of who we are. This happens because they constrain biodynamics as we speak, listen, think and, indeed, read/ create texts. Far from extending an inner mind, a history of co-ordination ensures that they come to be appropriated for realizing values. This applies during talk, reading aloud or silently solving insight problems. In reading aloud, languaging occurs when we do not inhibit. As Järvilehto et al. (this volume) show, Fixation Speech Intervals throw light on what we expect to see. The measures show that readers seek out what they expect to articulate. In dialogue too, we anticipate what other people will say (and do). Just as Fioratou and Cowley (this volume) find in solving the insight problem, we rely on monitoring opportunities. Agency uses material structures that invoke norms as, in problem solving, we seek out solutions. While insight can depend on conscious processes, it can also use serendipity. Given a language stance, events can generate insight. As Tylén et al. (this volume) report signal-like images also set off improvisation. Since brains ready us for interpretation, it is likely that a history of languaging leads to neural redeployment. When we anticipate, Tylén et al. (this volume) argue, Broca’s area contributes to interactive sense-making and meaning constructing. Indeed, the distributed view gives new weight to how, during co-ordinated activity, situations prompt us to both use routines and, where these fail, to come up with novel modes of acting and/or speaking. In Steffensen’s terms, a plenitude of other peoples’ voices, or ‘airborne synapses’ (this volume, p. 188) sustain language. To understand linguistic function, we need to ‘forget about language’, as it is represented by its surrogates. As a form of naïve realism, written language bias masks the importance of co-ordinating with artifacts and people. Having evolved



Distributed language

in a changing social meshwork, our extended phenotypes use wordings, affect and other cultural phenomena as indices of social norms. Experience gives us skills, ways of talking, thinking, feeling and acting, that constrain how others are likely to act. We all depend on artifacts and traditions that have been created, sustained and transformed by a history of human co-ordination. Human modes of life depend on an extended ecology where people co-ordinate as they realize values. 7. Future Prospects In viewing language as co-ordination, new meaning spaces arise. While the volume invites readers to change their view of language, this requires a double shift of perspective. It is not enough to acknowledge that brains control action and that language is distributed. It also needs to be seen that, because language is constantly renewed in the social meshwork, there is no need for verbal patterns to be represented ‘in the head’. Indeed, it is this insight that allows us to overthrow symbol-first models and, in their place, highlight the unfinalizability of language. The importance of this phenomenon is clearest in Imbumba moments or those in which we strive to understand how a task can be accomplished. For example, in reading this paper, many will have picked up allusions to Heidegger, Wittgenstein, Davidson and others. While none of these names have appeared in the text, acts of writing create a fluid surround that can be evoked (vaguely) by a reader (a potential ‘consensual domain’). Those with relevant skills and experience can use un-named sources to enrich their own acts of reading. Intertextuality arises, in cognitive terms, as we connect dynamical and symbolic aspects of language. Rączaszek-Leonardi puts it thus: The two perspectives do look at a single phenomenon. The most fascinating challenge is coming to understand how they relate to one another. Out of dynamical languaging, we come to discern patterns of sound that yield to formal (albeit ‘leaky’) descriptions. In this way an ordered system of sounds connects with a rich dynamics (this volume, p. 162)

We trace social reality to how, during first-order activity, voices set off sensemaking. To echo Wittgenstein (1978) concepts force themselves on us or, in Heidegger’s (1971) terms, language speaks. By opening up such ideas, the DLG perspective offers new challenges to the naïve realism that grounds post-Saussurian linguistics. To the extent that we succeed, this will show that dialogue shapes the cognitive dynamics from which language emerges. For now, one hypothesis is that, far from needing to re-use linguistic signifiers, we rely on anticipating results. Coordinated sense-making embodies thoughts: we depend on dynamics first and symbols afterwards.

11

12

Stephen J. Cowley

Notes 1. Figures such as Peirce (1940), Gibson (1979), Dennett (1991) and Ross (2000) claim that ‘virtual’ organization sustains both minded behaviour and language. On the other side, RączaszekLeonardi (see, 2008; this volume) offers a view that invokes physical symbols based on the work of Howard Pattee (e.g. 2008). 2. At the first meeting of the Swedish Association for Language and Cognition in Lund in November, 2008. 3. Most of the papers are revised versions of Cowley’s (2009a) Special Issue of Pragmatics & Cognition on Distributed Language (Järvilehto, Nurkkala, & Koskela, Kravchenko, Fioratou & Cowley, Tribble, van Heusden, Hodges and Rączaszek-Leonardi ). However, those by Cowley, Tylén, Phillipsen, & Weed and Steffensen were rewritten for this volume. 4. In Distributed language and dynamics, Cowley, (2009a) suggests that van Heusden separates language from behaviour. He denies this: “I wouldn’t say that I separate language from behavior. On the contrary – I see cognition, and within cognition, language, as forms of behavior…. I also stressed that language is social through and through. The double processing hypothesis is a hypothesis about human cognitive behavior” (van Heusden, personal communication). 5. A defining feature of distributed cognition is that cognitive processes are “distributed through time in such a way that the products of earlier events can transform the nature of later events” (Hollan et al. 2000).

References Bakhtin, M. M. 1981. The Dialogic Imagination: Four Essays. (Trans. C. Emerson & M. Holquist, ed. M. Holquist). Austin: University of Texas Press. Bakhtin, M. M. 1986. Speech Genres and Other Late Essays (Trans.V.W. McGee & C. Emerson, M. Holquist (eds.). Austin: University of texas Press. Carr, P. 2007. “Internalism, externalism and coding”. Language Sciences, 29/5: 672–689. Clark, A. 1998. “Magic words: how language augments human computation”. In P. Carruthers and J. Boucher (eds) Language and Thought: Interdisciplinary Themes. Cambridge: Cambridge University Press, 162–183. Clark, A. 2008. Supersizing the Mind: Embodiment, Action and Cognitive Extension. Oxford: Oxford University Press. Cowley, S.J. 1994. The Place of Prosody in Italian Conversations. Unpublished PhD, University of Cambridge. Cowley, S.J. 2004. “Contextualizing bodies: how human responsiveness constrains distributed cognition”. Language Sciences, 26/6, 565–591. Cowley, S.J. 2007a. “The codes of Language: turtles all the way up?” In M. Barbieri (ed) The Codes of Life. Berlin: Springer, 319–345. Cowley, S.J. 2007. “Cognitive dynamics and distributed language”. Language Sciences, 29/5: 575–583.



Distributed language

Cowley, S.J. 2009a. “Distributed language and dynamics”. Pragmatics & Cognition, 17/3: 495–507. Cowley, S.J. 2009b. “Language flow: opening the subject”. Cognitive Semiotics, 4: 64–92. Cowley, S.J. 2011. “Taking a language stance”. Ecological Psychology, 23/3: 185–209. Dennett, D.C. 1991. “Real patterns”. The Journal of Philosophy, 88 (1), 27–51. Donald, M. 1991. The Origins of the Modern Mind. Cambridge MA: Harvard University Press. Donald, M. 2007. “The slow process: a hypothetical cognitive adaptation for distributed cognitive networks”. Journal of Physiology – Paris, 101: 214–222. Evans, N. and Levinson, S. 2009. “The myth of language universals. Language diversity and its importance for cognitive science”. Behavioral and Brain Sciences, 32: 429–492. Fioratou, E. and Cowley, S.J. This volume. “Insightful thinking: Cognitive dynamics and material artifacts”. In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 57–80. Fowler, C. 2010. “Embodied, embedded language use”. Ecological Psychology, 22: 286–303. Gibson, J.J. 1979. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin. Giere, R.N. 2004. “The problem of agency in scientific distributed cognitive systems.” Journal of Cognition and Culture. 4(3/4): 759–774. Goffman, E. 1959. The Presentation of Self in Everyday Life. New York: Anchor Books. Harnad, S. 2005. “Distributed processes, distributed cognizers and collaborative cognition.” Pragmatics & Cognition, 13 (3): 501–514. Harris, R. 1981. The Language Myth. London: Duckworth. Heidegger, M. 1971. “Language”. In: Poetry, Language, Thought. London: Harper Collins, 185–208. Hodges, B.H. 2007. “Good prospects: Ecological and social perspectives on conforming”, Language Sciences, 19/1 584–604. Hodges, B.H. & Fowler, C. 2010. “New affordances for language: Distributed, dialogical and dynamical resources”. Ecological Psychology, 22: 239–253. Hodges, B.H. This volume. “Ecological pragmatics: values, dialogical arrays, complexity and caring”. In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 135–160. Hollan, J., Hutchins, E., & Kirsh, D. 2000. “Distributed Cognition: Toward a new foundation for human-computer interaction research”. ACM Transactions on Computer-Human Interaction, 7, 174–196. Hutchins, E. 1995a. Cognition in the Wild. Cambridge, MA: MIT Press. Hutchins, E. L. 1995b. “How a cockpit remembers its speed”. Cognitive Science 19, 265–288. Järvilehto, T. 1998. �����������������������������������尓������������������������������������尓��� “The theory of the organism-environment system: I. Description of the theory”. Integrative Physiological and Behavioral Science, 33, 321–334. Järvilehto, T. 2009. “The theory of the Organism-Environment System as a basis of experimental work in psychology”. Ecological Psychology, 21:112–120. Järvilehto, T. Nurkkala, V.M and Koskela, K. This volume. “The role of anticipation in reading”. In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 15–32. Jennings, R.E. & Thompson, J. J. In press. “The biology of language and the epigenesis of recursive embedding” To appear, Interaction Studies. Kirsh, D. 2006. ”Distributed cognition: a methodological note”. Pragmatics & Cognition, 14 (2): 249–262. Kravchenko, A. 2006. “Cognitive linguistics, biology of cognition and biosemiotics: Bridging the gaps”. Language Sciences 28 (1): 51–75. Kravchenko, A. 2007. “Essential properties of language: why language is not a digital code”. Language Sciences, 29 (5): 650–621. Kravchenko, A. This volume. “The experiential basis of speech and writing”. In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 33–55.

13

14

Stephen J. Cowley Linell, P. 2005. The Written Language Bias in Linguistics: Its Nature, Origins and Transformations. London: Routledge. Linell, P. 2009. Rethinking Language, Mind and World Dialogically: Interactional and Contextual Theories of Sense Making. Charlotte, NC: Information Age Publishing. Love, N. 2004. “Cognition and the language myth”. Language Sciences, 26: 525–544. Love, N. 2007. “Are languages digital codes?” Language Sciences, 29 (5): 690–709. Maturana, H.R. 1988. “Reality: The search for objectivity or the quest for a compelling argument”. Irish Journal of Psychology, 9,(1): 25–82. Mead, G.H. 1932. The Philosophy of the Present. Chicago: Open Court. Pattee, H.H. 2008. “Physical and functional conditions for symbols, codes and languages”. Biosemiotics, 1: 147–168. Peirce, C.S. 1940. “Philosophy and the sciences: a classification”: In J. Buchler (ed.) The Philosophy of Peirce. London: Routledge, 60–73. Port, R. 2010. “Rich memory and distributed phonology”. Language Sciences, 32/1: 43–55. Rączaszek-Leonardi, J. This volume. “Symbols as constraints: the structuring role of dynamics and self-organization in natural language”. In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 161–184. Rączaszek-Leonardi, J. & Kelso, J.A.S. 2008. “Reconciling symbolic and dynamic aspects of language: toward a dynamic psycholinguistics”. New Ideas in Psychology, 26:193–207. Ross, D. 2000. “Rainforest realism”. In D. Ross, A. Brook and D. Thompson (eds.) Dennett’s Philosophy: a Comprehensive Assessment. Cambridge MA: MIT Press, 147–168. Ross, D., 2004. “Metalinguistic signalling for coordination amongst social agents”. Language Sciences 26(6), 621–642. Ross, D. 2007. “H. sapiens as ecologically special: what does language contribute?” Language Sciences, 16 (1): 710–731. Saussure, F. de [1916] 1983: Course in General Linguistics (trans. Roy Harris). London: Duckworth. Spurrett, D. 2004. “Distributed cognition and integrational linguistics”. Language Sciences, 26/6: 497–501. Steffensen, S.V. This volume. “Beyond mind: an extended ecology of languaging.” In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 185–209. Steffensen, S.V. & Cowley, S.J. 2010. “Signifying bodies, health and non-locality: the aftermath”. In Cowley, S.J., Major, J.M., Dinis, A. and Steffensen, S. (eds.), Signifying Bodies: Biosemiosis, Interaction and Health. Braga: Portuguese Catholic University Press, 331–355. Thibault, P.J. 2011 “First-order languaging dynamics and second-order language: the distributed language view”. Ecological Psychiology, 23/3: 210–245. Trevarthen, C. 1979. “Communication and co-operation in early infancy: A description of primary intersubjectivity”. In M. Bullowa (ed.), Before Speech. Cambridge: Cambridge University Press, 321–347. Tribble, E. This volume. “Languaging in Shakespeare’s theatre. In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 101–115. Tylén, K. Phillipsen, J.S. and Weed, E. This volume. “Actualizing semiotic affordances in a material world.” In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 81–100. Van Heusden, B. This volume. “Semiotic cognition and the logic of culture”. In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 117–133. Whitehead, A. 1926. Science and the Modern World. Cambridge: Cambridge University Press. Wittgenstein, L.W. 1958. Philosophical Investigations. (2nd ed.). Blackwell: Oxford.

The role of anticipation in reading* Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela University of Oulu, Finland

The paper introduces measurement of fixation-speech intervals (FSI) as an important tool for the study of the reading process. Using the theory of the organism-environment system (Järvilehto 1998a), we developed experiments to investigate the time course of reading. By combining eye tracking with synchronous recording of speech during reading in a single measure, we issue a fundamental challenge to information processing models. Not only is FSI an authentic measure of the reading process, but it shows that we exploit verbal patterns, textual features and, less directly, reading experience. Reading, we conclude, is not a matter of decoding linguistic information. Far from being a text-driven process, it depends on integrating both sensory and motor processes in anticipatory meaning generation based on the history of experience and cultural context of the reader. Finally, we conclude with remarks on the social character and cognitive history of reading. Keywords: anticipation, eye tracking, fixation-speech interval, neural basis, organism-environment system, reading

1. Introduction Experimental research on reading has long been dominated by the view that language is based in text-like entities or words that are processed by the brain. Thus, the first experiments on reading used single words in the study of the speed of naming (Cattell 1886). During the last century the experimental possibilities were enlarged by the development of methods for tracking eye movements and for recording electrophysiological and hemodynamic measures of brain activity. However, there has been a surprising lack of interest in methods for determining the time course of the events in the reading process. In study of oral reading, most work has used different kinds of naming experiments where participants make lexical decisions or rapidly articulate single words, pseudowords, or short separate sentences (for a review, see e.g., Altmann 2001). However, since context matters

16

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela

to reading (e.g., Stanovich 2000:â•›3ff), this is open to criticism. No comprehensive theory of reading can rely on studies based on measures of how participants name single words or read short sentences. The easiest way to determine the time course of reading is by continuous measurement of the interval between the instant of the fixation to a word by the eyes and the start of word articulation. In fact, such a measure was developed over a hundred years ago. Pioneering quantitative measures, Quantz (1897) recognised that, before articulation, the eye identifies the word. Further, since the eye led the voice, Quantz called the spatial distance between the two eye-voice span (EVS). In the beginning of the 20th century EVS was usually specified by the number of characters or words whereby the eye led the voice (Buswell 1920). However, probably for technical reasons, temporal measurements were seldom made. After the middle of the century the measurement of EVS continued to be used (see Levin 1979), as an indicator of the reading process and reading skills. In more recent decades, however, interest in such measurements has fallen away. This may be because, in the information processing framework, reading is seen as a predominantly visual decoding process. Accordingly, there is little interest in oral reading. Further, as Rayner (1998:â•›384–385) suggests, many believe that eye movement dynamics and cognitive processes differ (at least in respect to the time scale). 2. Information processing or anticipation? Reading research is still dominated by the information processing model. On this view, reading is seen in terms of how the brain processes information that is stored in the written text. While theorists debate how the processing is realised (e.g., Rayner 1998:â•›388), models generally trace the process to when eyes fixate on the word. They assume that this leads to the transfer and further processing of visual information. It is posited that visual or phonological codes (or both) serve in identifying word and sentence meaning. When reading aloud, the perceptual part of the process is said to be followed by selection of motor programs and realisation of speech sounds. Such a conceptualisation of the reading process follows the general cognitive model of perception and motor action as linear/parallel information processing in the nervous system. While willing to debate whether reading is based on transmission of single letters, syllables or whole words, or whether word recognition is based on dual route or interactive models (see, e.g., Altmann 2001), these theories share a single assumption. Their cornerstone is the claim that word recognition occurs only after the presentation of a stimulus or fixation on a textual item. Recognition is based on comparing incoming information with representations that



The role of anticipation in reading

are stored in a putative inner lexicon. It is often said that these psychological models gain support from the use of EEG or MEG brain imaging that traces sequential activations of the brain (e.g., Sereno and Rayner 2003; Parviainen et al. 2006). Even 19th century work challenged linear information processing models of perception (Bain 1855; Dewey 1896). More recent traditions have pursued this critique (e.g., v. Uexküll and Kriszat 1932; Gibson 1979). Using the theory of the organism-environment system (Järvilehto 1998a), experimental work on animals and humans has been used to defend an alternative. Accordingly, Järvilehto (1999:â•›97) suggests that: Perception is not a linear process proceeding from the stimulus to the percept, but, rather, a circle involving both the sensory and motor organs as well as the events in the environment. A perceptual process does not start with the stimulus, rather the stimulus is an end of this process. The stimulus is like the last piece in a jig-saw puzzle. The last piece of the puzzle fits in its place only because all other pieces of the puzzle have been placed in a particular way. It is just this joining of the other pieces, their coordinated organization, which leaves a certain kind of hole into which this last piece can be fitted. Thus, it is just the organization of the other pieces which defines a possible last piece with which we may finish the puzzle. Exactly in the same way, a stimulus is present only if there is an organisation into which this stimulus may be fitted. Thus, the stimulus is as little in a causal relation to the percept as the last piece of the puzzle to the constructed picture. The stimulus is a part of the process of reorganization of the structure of the organism-environment system, which forms the basis of new knowledge.

Applied to reading, the model denies that the brain analyses and interprets marks or inscriptions in a linear fashion. Rather, the organism-environment system integrates what can be seen with current organisation. Far from processing ‘word or sentence stimuli’, inscriptions serve to create words and meanings. This is because, like single pieces of a jigsaw puzzle, inscriptions take on significance only as they become constituents of the organism-environment system. Since they lack intrinsic meaning, it is a mistake to say that words and sentences exist on paper or that they are decoded by a brain. Rather, reorganisation of the whole organismenvironment system gives rise to these results. To look1 at inscriptions on paper enables written marks to be included in anticipatory organisation that leads to the formation of the personal meaning and/or an articulatory act (in reading aloud). This process is not determined by the stimulus. Rather, it is to be traced to how the reader’s cultural and experiential history set the structural conditions for the dynamics of the organism-environment system. Although fixation is often viewed as the starting point of information transfer from the written word, there are competing explanations. On the basis of ‘miscue’ analysis of oral reading, Goodman (1969:â•›9) likened reading to a “guessing game”

17

18

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela

and, quite explicitly, denied that it depended on veridical information transfer. More recently, McDonald and Shillock argue that “the remarkable efficiency of reading is due, at least in part, to the on-line formation of predictions about upcoming words. The statistical properties of the linguistic environment offer a viable source for these predictions” (2003:â•›651). In recent work on comprehension, Levy (2007:â•›11) stresses expectations in his theory of resource-allocation processing difficulty. In parallel, Federmeier (2007) used recording of event related potentials (N400) associated with expected and unexpected words to suggest that “the brain seems to deal with the speed and complexity of language processing by ‘‘thinking ahead’’, by generating information about likely upcoming stimuli and preparing ahead of time, at multiple levels, to process them” (Ibid.: 502). Reading is a special skill that is carried out at a remarkable speed. This is why anticipation is likely to have a major role in reading just as it does on other temporally demanding skills (see, e.g., Abernethy et al. 2001). 3. Fixation-speech interval (FSI) as a measure of temporal dynamics in reading The relative ease with which words with transposed letters can be read also suggests the likely importance of anticipation (Rayner et al. 2006). However, Rayner et al. (2006:â•›192–193) also found that there is a cost when the order of letters in the word is changed: reading rate decreases and the number of fixations increases. This was interpreted by Rayner et al. (2006:â•›192–193) in terms of the mainstream information processing model. 3.1 FSI and type of text Järvilehto et al. (2008) examined the effect of letter transposition in more detail. In so doing, we introduced two kinds of controls alongside ‘scrambled’ text: normal ‘discrete’ text and ‘continuous’ text without spaces between words. Further, in addition to recording the usual eye movement parameters, we developed a measure of reading efficiency. This consists in fixation-speech interval (FSI) or the timeinterval between initial fixation to a word and the start of articulation (in reading aloud). The FSI measure allows ecologically valid investigation of parameters that impact on time-intervals between fixating an inscription and articulating its meaning. In this way, the whole reading situation opens up to temporal investigation. Eye tracking has an important part in this general process in that it serves to capture the use made of separate parts as defined by a series of fixations and saccades.



The role of anticipation in reading

Were the linear processing hypothesis correct, we would expect striking differences in time-intervals between ‘normal’ and ‘scrambled’ text. This is because, on the decoding model, text is analysed and processed in a linear fashion. Thus, analysis of scrambled words should take longer than that of the normal words and, it would be predicted, this would apply even where these were continuous (viz. writtenwithoutspaces). By contrast, on the anticipatory model, the scrambling of letters is expected to make less difference than whether they are discrete (normal) or continuous. This is because to generate meaning, an anticipatory model requires only that we use fragments of an inscription. Since the most conspicuous changes were associated with ‘continuous’ text, this supports the anticipatory model. In fact, participants had little difficulty in reading the scrambled text. It was the ‘continuous’ condition that slowed reading down and significantly disrupted the normal reading rhythm. Such findings are contrary to the linear model. They suggest that appeal to linear decoding of inscriptions provides little insight into the reading process. The type of text has a significant effect on fixation-speech interval. Thus, as noted, the longest intervals were associated with the continuous text. Other significant differences were found in how fixation-speech intervals correlated with an item’s location on the text line. While ‘words’ at the beginning of the text line were associated with short intervals, those in the middle of the text line elicited much longer fixation-speech intervals. This indicates that the duration of FSI is not correlated with parafoveal preview (for the possible significance of parafoveal preview in reading, see Vitu 1991).2 3.2 FSI in relation to articulation, sentence structure, and reading experience Recent work has used FSI measures that are accurate to within 4 milliseconds.3 This allows closer investigation of relation between reading and speech (especially at the role of articulation), sentence structure, and reading experience. In the experiments we presented for oral reading three types of Finnish texts (a fairy tale about sheep and mountain goats, not known earlier to the subjects). As above this was presented in scrambled, discrete, and continuous versions. Each participant was asked to use normal reading speed in dealing with screens of text (with approximately the same number of letters in each trial). Each screen consisted in a trial: the first consisted in discrete text. In the second, letters were scrambled (for about 50% of the words only the beginning and the end was correct: e.g., the sujbect had to raed this knid of wodrs) and, in the third continuous trial, words were written without spaces (e.g., thesubjecthadtoreadthiskindoftext). Eye movements were recorded with an EyeLinkII (SR Research) tracker using pupil and corneal reflection tracking (accuracy of sampling 4 ms; noiseâ•›<â•›0.1

19

20

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela

degrees of visual angle). Before the start of the experiment the recording system was calibrated. Drift correction was carried out in the beginning of each trial. The voice signal was recorded synchronously by the display program (programmed by Experiment Builder, SR Research) with the help of miniature microphone attached to the tracker close to the mouth of the S. The program produced two separate data files for storing of eye movement measurements and voice signals with synchronised timing information. The results confirm the findings of the previous study (Järvilehto et al. 2008) with respect to FSI measures of discrete, scrambled and continuous text. Detailed

Figure╯1.╇ Example of fixation-speech relations for reading of one sentence by one subject. Abscissa shows time in ms and ordinate the sequence of the words in the sentence (starting from the bottom of the figure). The symbols at the line corresponding to each word show the duration of fixation and articulation of the word (Fixation start-End, and Articulation start-End). Eye movements were recorded with an EyeLinkII tracker using pupil and corneal reflection tracking (accuracy of sampling 4 ms; noise╛<╛0.1 degrees of visual angle). The voice signal was recorded synchronously with the help of miniature microphone attached to the tracker close to the mouth.



The role of anticipation in reading

analysis of how FSI varies (Figures 1 and 2) shows that articulation of one word often starts before fixation to the next. Most typically, fixation leads speech by one or, perhaps, two words.4 Variability in FSI is also evident in the distribution of intervals. These may be less than 200 ms (the means of different Ss (Nâ•›=â•›33; age range 14–59 yrs.) and range from 443 to 824 ms; grand meanâ•›=â•›625 ms, standard deviationâ•›=â•›217 ms). Intervals as short as these cast doubt on the view that nervous systems carry out the many ’inner‘ operations posited in standard reading models. This is especially so when account is given to the approximately 100 ms interval between activating laryngeal EMG and the vocal signal (Gallena et al. 2001). It would, of course, be possible to interpret this as indicating that subjects use text context to ‘guess’. However, on such a view, one faces the task of explaining how ‘guessing’ can be accomplished by a brain. It is thus preferable to posit that the word is formulated in an anticipatory process in the nervous system. If this is so, the process will start before the subject generates the exact content of the word. As to the role of articulation, FSI may be influenced by speech performance. In other words, the interval may be ‘contaminated’ by articulatory factors. For

Figure╯2.╇ Pooled distribution of fixation-speech intervals of all subjects (n╛=╛33).

21

22

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela

Figure╯3.╇ Average FSI as a function of the place of the word in the sentence for more (n╛=╛17) and less (n╛=╛16) reading Ss. The bars show standard error of the mean.

example, it could be that the longer the articulation, the longer the FSI. We therefore investigated the relation between FSI, the previous item’s articulation duration and pause between articulations. While there is a positive correlation (râ•›=â•›.167, pâ•›<â•›.01, nâ•›=â•›1287, Pearson) between articulation-duration and current FSI, other findings are more striking. There are also significant positive correlations in the pause between articulations (râ•›=â•›.332, pâ•›<â•›.01, nâ•›=â•›1184; Pearson), and (at higher levels of significance) between the pause-duration after preceding and current items FSI d (râ•›=â•›.479, pâ•›<â•›.01, nâ•›=â•›1164; Pearson). Rather than emphasise how the length of a textual item contaminates FSI, we take this to show that FSI captures the rhythm of reading. Such interpretation gains further support when we consider how FSI maps onto where a textual item occurs. For each text type FSI consistently depends on whether these are sentence initial or occur after a comma (Figure╯3; place in sentence effect significant at pâ•›<â•›.001; F(4, 1440)â•›=â•›32.14; one-way ANOVA). This modulation of FSI correlates better with the rhythm of reading (producing meaningful thoughts) than sentence marks. Indeed, even where there are no such marks, continuous texts show the same pattern. Finally, correlation between articulation duration of the preceding word and the current FSI depends on the place of the



The role of anticipation in reading

word in the sentence. Thus the sentence initial position gives a high positive correlation (râ•›=â•›.256, pâ•›<â•›.05, nâ•›=â•›91; Pearson), and, in sentence final position, correlation is negative (râ•›=â•›−.166, pâ•›<â•›.05, nâ•›=â•›161; Pearson). Eye tracking data give related results. While average duration of a first fixation to an item varies between 186 and 354 ms (meanâ•›=â•›254, standard deviationâ•›=â•›119), saccade amplitudes vary between 1.3 to 2.3 degrees of visual angle (meanâ•›=â•›1.7; sdâ•›=â•›1.37 degrees of visual angle). Thus the place of a word in the sentence influences gaze duration (sum of fixation durations/word) in a way that is inversely related to FSI. Specifically, on FSI measures, duration was longest before a comma, and at the end of a sentence.5 In contrast to FSI, gaze duration was longest at the start of a text line and shortest at the end. We also examined the relation between reading performance and FSI. This was done by examining the relation between the FSI, reading duration, and the subject’s reports of reading experience (measured by the number of books claimed to be read per year). We found marked positive correlation between FSI and reading duration (râ•›=â•›.519, pâ•›<â•›.01, nâ•›=â•›33, Pearson). Strikingly, subjects with more reading experience (nâ•›=â•›17) had significantly shorter FSIs than those who read less (nâ•›=â•›16); on average, the difference was 46 ms (difference significant at pâ•›<â•›.001; tâ•›=â•›3.789, dfâ•›=â•›1443). Differences were most marked in the middle of sentences (Figure╯3). In relating reading to eye movement parameters, we found that people with more reading experience used fewer fixations and larger saccades. These differences too were more pronounced within sentences and with mid-line items. 3.3 FSI: Implications for reading It is misguided to idealise reading as a linear analysis. There is no evidence for the view that it is a process of neural comparison driven by fixations on data strings. Far from being descriptions of data decoding, information-processing models are flawed by unacknowledged theoretical assumptions. This establishes the importance of the FSI measure. It provides a novel means of obtaining data about normal reading that opens up the study of contextual aspects of the reading cycle. It thus contrasts with approaches based on measuring separate parameters of reading activity (such as duration or number of fixations).6 The fixation-speech interval is an authentic measure of a person’s reading performance. Our results show that FSI captures, not mechanical (speech) characteristics of the reading process, but the rhythm of reading (sentence structure) and reading experience. By connecting FSI measures with analysis of eye tracking, we can throw new light on the time course of events in the reading process. In the future, moreover, the approach might be combined with electrophysiological or hemodynamic measures of brain activity.

23

24

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela

In information processing models, decoding is data driven. For theoretical reasons, we are assumed to approach texts with grammars and/or mental lexicons in the brain. By investigating the time-course of reading, we have shown this view to be implausible. Measurement of FSI, however, enables stronger claims. Hitherto work on distributed language has emphasised the role of dynamics, i.e., language is grounded not just in word-forms, but in “the embodied, situated and cultural dynamics of talk” (Cowley 2007a:â•›106, 2007b). Given the focus on dialogical and directed speech, little or no experimental work has examined the dynamics of reading. It is thus important that FSI shows reading to be anticipatory. The subject integrates inscriptions with previous experience by making sense of what is before the eyes. 4. Anticipatory dynamics in reading Information processing models use a weak notion of anticipation (or ‘readiness’, ‘preparation’). The term is used to evoke waiting for the activation of an inner model with which a stimulus can be compared (see e.g., Neisser 1976; Rosen 1985), or, in other contexts, as a general process said to advance the processing of the incoming stimuli. 4.1 The concept of anticipation On the view taken here, the organism-environment system organises by means of anticipation (Järvilehto 1998a:â•›331). This shapes both prospective acts and how features of the environment influence the processes that lead to action results (e.g., reading aloud). The nervous system does not wait to compare stimuli with inner models but rather shapes events in a process of continuous transformation. The organism-environment systems’ anticipatory organisation determines what environmental constituents will serve to realise a person’s actions. Long before accomplishing a task, continuous transformation readies a system to use environmental constituents (e.g., inscriptions on paper) while configuring prospective movements (e.g., control of laryngeal activity when uttering the word). Action is thus almost simultaneous in any task that makes critical use of environmental features. In reading, many processes that are often posited to follow stimulus presentation occur even before measures are made. This is because in an experimental setting there is much that a subject knows in advance. It comes as no surprise that a text is presented in visual form or that it has to be read aloud. Anticipatory organisation is thus completed as the eye fixates an inscription and, by so doing, comes up with the action result (e.g., articulation of a specific word).



The role of anticipation in reading

4.2 Neural models of anticipation In contrast to neural information-processing models, the theory of the organismenvironment system posits that the nervous system is organised together with bodily elements and environmental constituents to obtain action results (Järvilehto 1998b). According to this theory, there is nothing for which neurons could decode information. Rather, like other cells, neurons seek to maintain their metabolism. When this is disturbed they fire and, by so doing, influence neurons that form complex neural nets. By using neurotransmitters, the net can influence the muscles and, as a result, restore its metabolic state by cessation of firing. If what results is of no value, the neuron dies. In the nervous system, this happens all the time. Neuron firing is therefore not ‘information processing’. Neurons are living cells organised in metabolic systems that connect with other neurons. As cells, they cannot perform psychological ‘functions’ (like perception or reading). No neural network can analyse features of a written text, undertake mental operations, or build models and representations. Rather, they enable the organism and environment to use dynamics in a single functioning system. While reading does not happen in the brain, the brain is, of course, important for the reading process. However, reading does not happen in the brain, but in a larger system consisting of the organism and environment as well as of the whole cultural context where the reading occurs. The agent of the reading process is not a neuron or the brain, but a person who cannot be defined by physiological concepts. The organism-environment system features an anticipatory neural system as is sketched in Figure╯4. On this view, it is important that sensory receptors or receptor matrices are not literally ‘receptive’. Rather, they create the environmental connections that support the formation of the whole organism-environment system. Efferent influences on receptors are of special importance. This is because, as Järvilehto (1999) shows in detail, they condition receptor matrices for the selective use of environmental constituents (such as inscriptions of the text on paper). Without this conditioning, the system could not achieve its action results. Similarly, muscles are efferent organs that also contain afferents (‘muscle afferents’ in Figure╯4) whose significance appears in the interplay with receptors. It is because muscles and receptors have a similar structure (afferent and efferent innervation), that, together, they define parts of the environment (e.g., certain inscriptions on the paper) which enable the whole organism-environment system to achieve useful action results (e.g., understanding of the text). Perception and action are not separate sensory and motor processes, but simply two aspects of the same coin, of an integrated whole. Thus reading is a sensory as well as a motor process. We cannot afford to ignore such claims when we use changes in brain activity as the basis for measuring the time course of reading (Sereno and Rayner 2003;

25

26

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela

Figure╯4.╇ Anticipatory neural net as part of the organism-environment system. In the anticipatory neural net muscles and receptors act together and define the environmental parts that can be fitted to the organism-environment system. Of special importance are the efferent connections to receptors that condition the receptors for specific environmental connections. Small dots around neurons depict transmitters.

Parviainen et al. 2006). Indeed, if an inscription on the page completes anticipatory organisation, event-related potentials following the stimulus presentation (or fixation of the eyes to an inscription) cannot be indicators of sensory processing of the stimulus. Rather, they reflect completion of integrated anticipatory organisation and thus the transition from one act to another. Much of what is conceptualised as stimulus related ‘processing’ occurs before looking at the page. The ‘processing’ has been carried out before using the page as a ‘trigger’ to completion of the action or, in oral reading, to articulating what is seen. Linear and hierarchical models of reading conflate psychological and neurophysiological concepts. Since they begin by describing operations deemed ‘necessary’ to a reading task, they are bound to ignore the temporal dynamics of the nervous system. Indeed, it is traditional to present the task components in boxes whose contents are fixed by how peripheral processing enables us to ‘decode’ the stimulus. For theoretical reasons, the nervous system is said to follow this logic. There is, however, no reason for the nervous activity to follow the temporal order of hypothetical psychological operations. The posited stimulus ‘input’ may differ from neural processes following the stimulation of the receptors. As we read, we do not have to wait for events in the environment. Indeed, the task structure itself sets off anticipatory neural processes before actual performance. Not surprisingly, then, systems form for dealing with the action results. The reader is not an empty



The role of anticipation in reading

bag to be filled by information from the text. Rather people actively use the inscriptions encountered in the situation to achieve useful action results or, specifically, articulation of meaning. 5. Reading as generation of meaning Meaning is based on relations within the organism-environment system. In other words, far from being contained in the inscriptions of a text, any factor that leads to an action result has meaning. Where we regard text perception as a result, then all perceived inscriptions carry meaning and, crucially, this is necessary to reading both silently and aloud. Thus, even ‘mechanical reading’ generates meaning. When attributing sound to letters (as in reading phonetic nonsense words), we engage in meaningful activity. It thus follows that meaning is not like understanding. It can be correctly attributed to even single words, letters, or even parts of letters. Its basis lies in the reader’s ability to formulate a word (or sound-pattern) on the basis of inscriptions on the paper (cf. for a related interpretation, Cowley 2007c:â•›83–104). 5.1 The unit of meaning in reading In reading for understanding we are not aware of the meaning of words or their graphic counterparts. Rather, as we read, we generate thoughts and ideas because the anticipatory process is directed to just such action results. Where reading for understanding, therefore, the unit is larger (e.g., sentence or thought) than in ‘mechanical reading’. This interpretation is confirmed by variations in length of FSI that occur in reading continuous texts (oneswithoutwordbreaks). Since no information is transferred, eye fixations serve in modulating the process of meaning generation or, in other terms, confirming the results of anticipatory organisation. Fixations neither start with feature analysis (cf. Paulson 2002 for a related interpretation) nor do inner representations feed the verbal content into a ‘speech processor’. Rather, they serve in managing the production of speech. Since fixations modulate the ongoing speech process, it is not surprising that measures of FSI can be as short as reaction times. Further, far from needing to fixate on a particular inscription, it is enough to identify a fragment for meaningful articulation. An inscription exists as a word or meaningful unit only as it is integrated into the organism-environment system. There is no need for feature analysis or lexical search. Since inscriptions contain no information, they are informative only in the context of the organism-environment system’s anticipatory organisation. This arises from the subject’s history of social relations in a cultural context that, as a result, determines the unfolding of the anticipatory process.

27

28

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela

5.2 Silent and oral reading Where reading is viewed as a visual process, little interest is shown in reading aloud. Accordingly, on information-processing models, fixation-speech intervals are seen as indexing processes unlike those of silent reading. In this “snap-shot” theory, it is posited that fixations should “wait” for word articulation (Rayner 1998:â•›375). This leads to a prediction. In principle, there should be differences between the timing of eye movement control and cognitive processing when we read aloud and silently. In fact, our research provides no support for this view which has been questioned also in comparisons of oral and silent reading by Kondo and Mazuka (1996:â•›358). Where text perception is regarded as meaning generation that integrates sensory and motor constituents in social action, the time course of oral and silent reading is expected to be similar. Indeed, both processes are posited to draw on the same anticipatory processes. This is confirmed by the fact that our fixation durations and saccade amplitudes are of the same order of magnitude as those of Rayner’s (1998) work on silent reading. Not only do our eye tracking results7 confirm the organism-environment view, but they suggest a general conclusion. Reading silently or ‘out loud’ are, it seems, different aspects of a single process. In both cases, the text acts as a means to speaking even if, in silent reading, this is inhibited. In silent reading, we articulate subvocally as is found in studies of the EMG activation of laryngeal muscles (Sokolov 1972; Abramson and Goldinger 1997). Reading used to be entirely oral. While occasional instances of silent reading occur before the tenth century (Manguel 1996:â•›40–53), our findings reflect on our history. Texts are written to be spoken and, in many languages, no lexical contrast marks acts of reading from speaking (ibid.). Since language is social activity that creates cooperation this is, perhaps, not especially surprising (Järvilehto 2000:â•›48–49). Indeed, like dialogue, reading may be regarded as a process directed towards the expression of the ideas conveyed by the writer. Given the importance of semantic aspects of the text, there is some dispute whether phonological processing is necessary (see e.g., Perfetti and McCutchen 1982; Wagner and Torgerson 1987). In learning to read however, phonology plays an important role (Wagner and Torgerson 1987; see also Lukatela et al. 2004). Accordingly, silent reading is best seen as s special case of oral reading where one component, overt speech, does not occur. Reading is a social process which encompasses both the reader and the writer of the text who engage in a cultural context. It is a dialogue with the writer, a kind of cooperation which does not, in principle, differ from dialogue between two people (for a different view, see Kravchenko this volume). Far from being ‘interaction’ between people, dialogue can be seen as a process of generating shared mean-



The role of anticipation in reading

ing as we concurrently orient to each other. In reading the author is not physically present but, in spite of this, participates in the generation of meaning. Just as if the other was present, the reader can pose questions to a text and, strangely, find answers. The text permits cooperation between the writer and the reader which can result, among other things, in the creation of new thoughts and ideas.

Notes *╇ We would like to thank Stephen J. Cowley for many helpful suggestions and corrections during the preparation of the manuscript. We also thank three anonymous reviewers for their useful comments. 1.╇ It should be pointed out that, according to the theory of the organism-environment system, behaviour (such as looking) is reorganisation of the whole organism-environment system. Psychological concepts (such as perception) describe different aspects of the dynamics of organisation in the organism-environment system (see Järvilehto 2000). 2.╇ Parafoveal preview means the possibility of using textual information that is projected to the parafoveal region of the retina. 3.╇ For technical reasons, in the results reported in the study above (Järvilehto et al. 2008) the accuracy of measurement of FSI was 40 milliseconds. 4.╇ It is not known how much our results depend on specific features of the Finnish language (e.g., complexity of morphosyntax), but comparable eye-voice span was earlier reported by e.g., Levin (1979) for English. Levin, however, did not measure temporal parameters, such as FSI. 5.╇ Taken together these results could be indications of ’prosody of looking’, i.e., rhythm of action related to the generation of meaning. 6.╇ As pointed out in the Introduction, a related measure is the EVS, but it has been applied only for spatial measurements, and it thus offers no possibility for determination of the time course of the reading process. Another related method is the presentation of single words or short sentences and measuring the voice reaction time, but this measure has the disadvantage of bad ecological validity. 7.╇ There are, however, no systematic comparisons for different languages.

References Abernethy, B., Gill, D.P., Parks, S.L., and Packer, S.T. 2001. “Expertise and the perception of kinematic and situational probability information”. Perception 30: 233–252. Abramson, M. and Goldinger, S.D. 1997. “What the reader’s eye tells the mind’s ear: Silent reading activates inner speech”. Perception & Psychophysics 59: 1059–1068.

29

30

Timo Järvilehto, Veli-Matti Nurkkala, and Kyösti Koskela Altmann, G.T.M. 2001. “The language machine: Psycholinguistics in review”. British Journal of Psychology 92: 129–170. Bain, A. 1855. The Senses and the Intellect. London: Longmans, Green. Buswell, G.T. 1920. An experimental study of the eye-voice span in reading. Chicago: Supplementary Educational Monographs: 17. Cattell, J. McK. 1886. “The time it takes to see and name objects”. Mind 11: 63–65. Cowley, S.J. 2007a. “Distributed language: Biomechanics, functions, and the origins of talk”. In C. Lyon, C. Nahaniv, and A. Cangelosi (eds), Emergence of Communication and Language. London: Springer, 105–128. Cowley, S.J. 2007b. “The cognitive dynamics of distributed language”. Language Sciences 29: 575–583. Cowley, S.J. 2007c. “How human infants deal with symbol grounding”. Interaction Studies 8: 83–104. Dewey, J. 1896. “The reflex arc concept in psychology”. Psychological Review 3: 357–370. Federmeier, K.D. 2007. “Thinking ahead: The role and roots of prediction in language comprehension”. Psychophysiology 44: 491–505. Gallena, S., Smith, P.J., Zeffiro, T., and Ludlow, C.L. 2001. “Effects of Levodopa on laryngeal muscle activity for voice onset and offset in Parkinson disease”. Journal of Speech, Language and Hearing Research 44: 1284–1299. Gibson, J.J. 1979. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin. Goodman, K.S. 1969. “Analysis of oral reading miscues: Applied psycholinguistics”. Reading Research Quarterly 5: 9–30. Järvilehto, T. 1998a. “The theory of the organism-environment system: I. Description of the theory”. Integrative Physiological and Behavioral Science 33: 321–334. Järvilehto, T. 1998b. “The theory of the organism-environment system: II. Significance of nervous activity in the organism-environment system”. Integrative Physiological and Behavioral Science 33: 335–343. Järvilehto, T. 1999. “The theory of the organism-environment system: III. Role of efferent influences on receptors in the formation of knowledge”. Integrative Physiological and Behavioral Science 34: 90–100. Järvilehto, T. 2000. “The theory of the organism-environment system: IV. The problem of mental activity and consciousness”. Integrative Physiological and Behavioral Science 35: 35–57. Järvilehto, T., Nurkkala, V.-M., Koskela, K., Holappa, E., and Vierelä, H. 2008. “Reading as anticipatory formation of meaning: Eye-movement characteristics and fixation-speech intervals when articulating different types of text”. Journal of Transfigural Mathematics 1: 73–81. Kondo, T. and Mazuka, R. 1996. “Prosodic planning while reading aloud: on-line examination of Japanese sentences”. Journal of Psyholinguistic Research 25: 357–381. Kravchenko, A. This volume. “The experiential basis of speech and writing” In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 33–55. Levin, H. 1979. The Eye-voice Span. Cambridge, MA: The MIT Press. Levy, R. 2007. “Expectation-based syntactic comprehension”. Cognition 106: 1126–1177. Lukatela, G., Eaton, T., Sabadini, L., and Turvey, M.T. 2004. “Vowel duration affects visual word identification: Evidence that the mediating phonology is phonetically informed”. Journal of Experimental Psychology: Human Perception and Performance 30: 151–162. Manguel, A. 1996. A History of Reading. New York: Viking. McDonald, S.A. and Shillock, R.C. 2003. “Eye movements reveal the on-line computation of lexical probabilities during reading”. Psychological Science 14: 648–651.



The role of anticipation in reading

Neisser, U. 1976. Cognition and Reality: Principles and Implications of Cognitive Psychology. San Francisco: Freeman. Parviainen, T., Helenius, P., Poskiparta, E., Niemi, P., and Salmelin, R. 2006. “Cortical sequence of word perception in beginning readers”. Journal of Neuroscience 26: 6052–6061. Paulson, E.J. 2002. “Are oral reading word omissions substitutions caused by careless eye movements?”. Reading Psychology 23: 45–66. Perfetti, C.A. and McCutchen, D. 1982. “Speech processes in reading”. Speech and Language: Advances in Basic Research and Practice 7: 237–269. Quantz, J.O. 1897. “Problems in psychology of reading”. Psychological Monographs 2 (1, Whole No. 5). Rayner, K. 1998. “Eye movements in reading and information processing: 20 years of research”. Psychological Bulletin 124: 372–422. Rayner, K., White, S.J., Johnson, R.L., and Liversedge, S.P. 2006. “Reading wrods with jubmled lettres: there is a cost”. Psychological Science 17: 192–193. Rosen, R. 1985. Anticipatory Systems. Oxford: Pergamon Press. Sereno, S.C. and Rayner, K. 2003. “Measuring word recognition in reading: eye movements and event-related potentials”. Trends in Cognitive Sciences 7: 489–493. Sokolov, A.N. 1972. Inner Speech and Thought. New York: Plenum Press. Stanovich, K. 2000. Progress in Understanding Reading: Scientific Foundations and New Frontiers. New York: Guilford Press. von Uexküll, J. and Kriszat, G. 1932. Streifzüge durch die Umwelten von Tieren und Menschen. Frankfurt am Main: Fischer. Vitu, F. 1991. “The infuence of parafoveal preprocessing and linguistic context on the optimal landing position effect”. Perception and Psychophysics 50: 58–75. Wagner, R.K. and Torgerson J.K. 1987. “The nature of phonological processing and its causal role in the acquisition of reading skills”. Psychological Bulletin 101: 192–212.

31

The experiential basis of speech and writing as different cognitive domains Alexander V. Kravchenko

Baikal National University of Economics and Law, Russia

Traditionally, writing is viewed as a code that stands in one-to-one correspondence to spoken language, which is therefore also viewed as a code. However, this is a delusion, which is shared by educators and has serious consequences for cognition, both on individual and on social levels. Natural linguistic signs characteristic for the activity of languaging and their symbolizations (graphic markings) are ontologically different phenomena; speech and writing belong to experiential domains of different dynamics. These dynamics impact differently on the linguistic/behavioral strategies of individuals and communities, viewed as second- and third-order living systems operating in a consensual domain as structure-determined systems. Failure to acknowledge this contributes to the spread of functional illiteracy in modern societies, which may lead to cognitive/ communicative dysfunction. Technology-enhanced new literacies challenge the value of traditional written culture, raising questions about the relationship between speech and writing and their roles in human evolution Keywords: consensual domain, phenomenology, speech, structural determinism, understanding, writing

1. Speech, text, and dynamics The legacy of first-generation (objectivist) cognitive science, with its emphasis on language as abstract disembodied symbols processed by a syntactic engine found in the head, weighs heavily on linguistics (Kravchenko 2009a). Linguistic signs are treated as well-defined binary structures analyzable in terms of form (expression) and content. It is posited that these structures convey information by making it possible for both the sender (speaker) and receiver (listener) to analyze them. Using a mutually shared procedure, we are said to identify quanta of meaning that display the semantic values of signs. Such analysis, according to the computational

34

Alexander V. Kravchenko

theory of cognition and language, consists in a series of decisions made as we process ‘encoded’ meanings. The classic approach to Artificial Intelligence aims to develop an algorithmic script that analyzes all possible signs into a priori sets of values. No matter how appealing this logical approach may seem, it overlooks the experiential character of (linguistic) semiosis. For an international, interdisciplinary academic community known as the Distributed Language Group, however, experience comes to the fore (Cowley 2007a). The distributed language view focuses on language as a key aspect of social (dialogical) activity distributed over different time scales. In the non-objectivist paradigm, the ‘object’ of communication is not a referential state of things in an objective external reality, but “the co-ordination of actions between the interacting cognitive agents” (Stewart 1996:â•›318); hence, other-orientation, contexts, interaction, and semiotic mediation become key concepts (Linell 2007). As Cowley emphasizes, “language activity is tightly constrained by both our sensitivity to circumstances and our skills in using many second-order cultural constructs” (2007a:â•›576). Among many cultural artifacts we use, writing is by far the most intriguing: its symbiotic relationship with natural linguistic activity, and the degree to which it impacts — both cognitively and evolutionarily — on humans, is yet to be appraised (Menary 2007). Speech and writing have different ontologies, and to avoid a grave conceptual fallacy (Love 2007; Kravchenko 2007a), they should not be conflated (Linell 2005; Love 2004). Yet, the written language bias characteristic of the classical approach (Linell 2005), and the code model of language have masked profound differences between speech and writing. This paper undertakes to explore the most important difference, the experiential one, caused by different cognitive dynamics of linguistic interactions in the domains of speech and writing. Linguistic behavior, or languaging, is a specifically human kind of interactional behavior which establishes, in Maturana’s terms (1970, 1978a), a cognitive domain.1 Dialogical interactions occur in space-time as communicating parties monitor, and respond to, what happens (Linell 2005). By contrast, written texts are typically not read by others as they are produced. For a normal reader, the focus must fall on textual products. Since the production process itself is inaccessible, analysis of written language necessarily focuses on texts and their component parts. Letters, words, sentences, paragraphs, etc. resemble objects that can be manipulated (especially when typesetting and editing a text on a computer). Given the interactional nature of speech, this sets up an ontologically distinct semiotic system. While, in Maturana’s terms (1978b), languaging is orientational, the primary cognitive function of writing is analogous to internal, or biological, memory. Unlike languaging, which extends the human sensorium (Morris 1938), writing is “a storage and retrieval system that allows humans to accumulate experi-



The experiential basis of speech and writing

ence and knowledge” (Donald 1991:â•›309). Once we are skilled in writing, we can ‘dump’ the cognitive load onto material artifacts and, later, use them to interact with our cognizing selves. While languaging is embodied and distributed, writing (sentences, texts) establishes cultural artifacts. Unlike speech, these impact on individuals and communities across historical and evolutionary scales. In spite of tradition, therefore, speech and writing are not two manifestations of a single phenomenon. As separate cognitive domains, they set up contrasting body-world relations through contrasting cognitive dynamics. The role of writing in human society can hardly be exaggerated. Indeed, the development of industrial and post-industrial societies has shown that writing is one of our greatest inventions (Hagège 1996). Today, text-making is increasingly seen as an important part of recent human co-evolution (cf., e.g., Donald 1991). Texts — and their computational extensions — increasingly determine important aspects of human life. Plainly, human cognition exploits the world beyond the body (Hutchins 1995) and, in some sense, extends the human mind (Clark and Chalmers 1998). For Clark (1997), we live in a cognitive niche based on language-enhanced cognition wherein material symbols augment individual intelligence. Others treat this as an overstatement: for Harnad (2005, 2007), even if cognition is language-enhanced, what matters is thinking. Vygotsky (1994) argued that thinking and languaging cannot be separated; a biological theory of cognition makes a stronger claim — that “the human brain thinks in language” (Maturana, Mpodozis, and Letelier 1995:â•›24) . Once material aspects of language are taken seriously, speech and writing have to be rethought. They are materially different phenomena with quite different temporal and spatial distribution. From an evolutionary standpoint, writing helps overcome the spatio-temporal limitations of languaging. It gives us an atemporal (as contrasted with local) existential/ informational environmental medium. Thus, whereas humans who lack writing systems live in a domain based on how they use experience to language together, this consensual domain is transformed by the invention of writing. In Maturana’s terms, if individual humans are secondorder living-systems, then communities are third-order living systems. Since these increasingly depend on texts in their organization, the rise of writing leads to the emergence of a new ecology (cf. Bang et al. 2007). A third-order living system is sustained as a unity of interactions2 through the relational domain of linguistic behavior (schematized in Figure╯1). In such a system, human individuals, each in his specific physical environment (solid circle), establish their consensual domains of interactions with others (overlapping circles). Since these include linguistic interactions (overlapping dotted ovals), the events create a relational domain which surpasses the physical boundaries of any given individual’s environmental niche.

35

36

Alexander V. Kravchenko

Figure╯1.╇ A community of individuals as a unity of linguistic interactions

The linguistic behavior of a third-order living system exploits a relational domain that depends on uninterrupted space-time continuity. Conversely, when links between individuals in domains of communicative interactions are severed for an extended period, the community’s unity ceases to exist (see Figure 2). In this way one living system disintegrates into two or more smaller systems. Human history gives many examples of such disintegration. In standard terms, this leads to

Community 1

Community 2

Figure╯2.╇ Disintegration of a third-order living system as a unity of interactions



The experiential basis of speech and writing

the emergence of dialects which may later become new languages with associated communities/cultures/nations. The cognitive potential of natural linguistic signs (words and utterances) is greatly extended with the invention of graphic signs. This is because they can exert orientational influence across different time scales and, by so doing, become a major historical factor. For example, the cognitive potential of phonetic signs is limited by durational and other thresholds of auditory perception. As a result, the distance between languaging humans and their respective bodily orientations in real time-space become crucial. This, indeed, gave reason for the use of visual symbol systems like semaphores. But as soon as phonetic features of the sign are replaced by artifacts symbolically used in communication — such as graphic images — the products cease to be signs of natural language. In contemporary scientific discussions the terms sign and symbol are often used as synonyms. Thus, in first-generation cognitive science, written marks become ‘symbols’ that are devoid of meaning. Basically, this leads to the symbolgrounding problem (Harnad 1990; Cowley 2007b; Belpaeme and Cowley 2007) or, roughly, that of how reference can be attributed to purely abstract forms. This arises when abstract symbols (graphic artifacts) are identified with signs of natural language or acoustic phenomena that are integrated in dynamically complex behavior. From a human perspective, since the signs of natural language are never abstract, no such problem arises. It is thus useful to use sign and symbol in discriminating between concepts (Kravchenko 2003). In etymology, the meaning of the Latin signum is ‘mark’, while that of symbol (from symbolum, a borrowing from Greek: συµβολον), is ‘put together’, ‘thrown in’. Thus, while signs evoke perception and experience, symbols presuppose something more elaborate. Physical spoken words are signs — perceived empirical phenomena (dynamic attractors) which function as ‘marks’ of previous experience. As for corresponding graphic images we call ‘words’, they are merely symbolizations of phonetic signs. Graphic images, such as letters of which written words are made, are abstractions that have been ‘put together’ and ‘thrown in’ to the relational domain of human linguistic behavior. Of course, once written words begin to play a crucial role in human life, they transform our cognitive abilities (cf. Menary 2007). A system of graphic signs, such as alphabetic writing, enhances human developmental continuity. This is possible because texts mediate relevant experience between generations and communities. As Donald (1991) notes, it also has evolutionary implications. However, new linguistic functions compete with older ones. Texts no longer perform the indexical functions of natural languaging or, indeed, permit real-time integration with concurrent activity. Whereas speech is usually coordinated between people, in dealing with texts, we orient to — not living bod-

37

38

Alexander V. Kravchenko

ies — but lifeless artifacts. As a result, the dynamics and skills that contribute to speech are radically different from those used to manipulate artifacts. This contrast is especially marked in the domain of hypertext and texting (Jenkins et al. 2006; Chrystal 2008). While some believe that technology can enhance effective learning (cf. Dror 2008), others blame new technologies for the decline in aspects of literacy valued in the modernist era. Quoting Mark S. Schneider, commissioner of education statistics, The Washington Post suggests: “It may be that institutions have not yet figured out how to teach a whole generation of students who learned to read on the computer and who watch more TV. It’s a different kind of literacy” (Romano 2005). Indeed, some see the degradation of reading as symptomatic of social degradation. As emphasized by Dana Gioia, Chairman of the National Endowment for the Arts (USA), in the preface to the NEA 2004 report entitled “Reading At Risk: A Survey of Literary Reading in America”, [W]hile oral culture has a rich immediacy that is not to be dismissed, and electronic media offer the considerable advantages of diversity and access, print culture affords irreplaceable forms of focused attention and contemplation that make complex communications and insights possible. To lose such intellectual capability — and the many sorts of human continuity it allows — would constitute a vast cultural impoverishment (Bradshaw and Nichols 2004:â•›7).

Moreover, as the NEA survey shows, the decline in reading “parallels a larger retreat from participation in civic and cultural life. The long-term implications of this study not only affect literature but all the arts — as well as social activities such as volunteerism, philanthropy, and even political engagement” (ibid.). So, is there a relationship between declining reading skills and lack of participation in civic and cultural life? To answer this question, we must abandon the view that language is a code (Kravchenko 2007a). Speech and writing have different histories and, as a result, contrasting cognitive dynamics. While both can be analyzed as material symbols, we interact differently with people and lifeless graphic artifacts. In pursuing this contrast, I argue that a new view of literacy (and social literacies) is needed. Indeed, unless we understand how they constrain events in their respective cognitive domains, we will understand neither social life nor how they impact on our cognizing selves. 2. The linguist’s view of literacy Donald’s (1991) influential work stresses that mimesis led to the cultural evolution — and language — that transformed the primate mind. In turn, this underpins Clark’s (1997) views on how tools and material symbols extend the mind. But how



The experiential basis of speech and writing

literally are we to take this idea? Does it support the view that ‘language’ is an underlying ability? Or must we consider differences between speech and writing? While Clark’s appeal to material symbols is an attempt to avoid this, it is surely mistaken to overlook cognitive dynamics. Indeed, if we each had our own inner language, we would expect skills in social conversation to map onto our use of graphic signs. Yet, skills in speaking and listenting do not correlate with literacy. More dramatically, on the autistic spectrum high performers may be literate and yet show lacunae in their languaging. There is a cognitive discrepancy between the two domains. Skills in social interaction and literacy show no close association. Literacy is often seen as a matter of encoding/decoding meanings that are associated with written marks. As a result, the process is ‘explained’ in a rather simplistic way. Language acquisition is seen as mastering an inventory of linguistic signs and rules which permit meanings to emerge from sign sequences. Taking stock of what is included in such an inventory is accomplished by viewing written marks as resulting from the use of a special code. Hence literacy is seen as an ability to perform a three-step operation of matching and identification: i. a particular letter (graphic encoding, or symbol) is matched with a particular acoustic phenomenon (sound type, or phoneme), ii. a particular concatenation of letters (the graphic word) is matched with a particular concatenation of sound types (the spoken, or physical word), iii. the graphic word is identified with the spoken word. These cognitive abilities are posited to underpin the acquisition of reading and writing skills. By implication, to be literate is attributed to the possession of such skills. This narrow view is quite inadequate. No one would think of a child who can use a pencil to draw unsophisticated lines and patterns on a piece of paper as an artist. Likewise, an older child who can produce an octave on the piano while matching the sounds to the graphic symbols of music notation is no musician or composer. Although he can read the music score (is musically ‘literate’), such a child cannot play the piano. But if a child can vocalize, more or less correctly, graphic representations of words, we may say, “He can read”. The analogy with paintings and music matters: readers too need to do more than decode meanings. To understand what we read, we need to do more than decipher other people’s thoughts. Any code-based view of literacy is, at best, radically incomplete. Education provides young people with tools and techniques for cognitive development. Literacy thus plays an important part in developing powers of reasoning and judgment based on the ‘world on paper’ (Olson 1994). It is, however, an indisputable fact that the ability to read and write of itself does not make one socially mature. What the UNESCO projects of the 1950s and 1960s attributed to

39

40

Alexander V. Kravchenko

literacy was essentially imaginary (Olson and Torrance 2001). The development of reasoning skills turned out to depend not on literacy per se but on establishing meaningful relationships between knowledge acquired from education and life experiences. In other words, literacy is also “a set of prescriptions about using knowledge” (Cook-Gumperz 2006:â•›1). As Koïchiro Matsuura, Director-General of UNESCO, emphasized on International Literacy Day, literacy helps in the acquisition of learning and life skills that, when applied across the lifespan, permit individuals — and communities — to develop in ways that are sustainable (Matsuura 2008). Literacy, then, is not a cognitive ability based on reading and writing but, rather, the basis for skills that can sustain our cognitive development. This depends on using the languaging and the environmental dynamics that sustain human existence to become skills in assigning meanings to phenomena that exert orientation effects on others. In an experiment conducted by the author in Russian universities, undergraduate students (linguistics majors) were asked to read and interpret a sentence purporting to be from a 19th century Russian novel. The sentence contained 19 words and was an account of scanty belongings (totaling four items) of a person named Triffon:

(1) A dampened roll of flannelette, a folding wooden arshin, an old cast-iron block, and an open bottle of Lafite — that was all that was left after Triffon.

Podmochennaja shtuka bumazei, skladnoj derevjannyj arshin, dampened roll flannellette GEN folding wooden arshin3 staryj chugunnyj bolvan i pochatyj shtoff lafita — vot vsjo, old cast-iron block and beginPPa bottle Lafite GEN — that all chto ostalos’ posle Triffona. what remain P after Triffon GEN

Unsurprisingly, few subjects understood what was left after Triffon’s death. This was because the items mentioned in the sentence (bumazeja, arshin, chugunnyj bolvan, shtoff) have long gone out of use and are not part of the younger generation’s experience. However, partly because of this initial failure, they also failed to ‘extract’ the larger chunks of knowledge ‘contained’ in the sentence: i. Triffon died in poverty ii. He was a cheap garment maker iii. He had known better times. Remarkably, some subjects (about 7% of the sample) failed to infer that Triffon was dead. Like many others, they failed to use how his belongings are described



The experiential basis of speech and writing

to infer (ii). Having overlooked background knowledge, they found inference (iii) especially difficult. They failed to use one item (‘an open bottle of Lafite’) to draw specific inferences. Since vodka in old Russia was cheaper than wine, it is striking that a poor garment maker spends his money on more expensive alcohol. In all probability, therefore, the habit developed when Triffon could either afford French wine or, more likely, had access to it as a servant to a Russian noble. Although other inferences can be drawn, the point is simple. Students were unable to make head or tail of a sentence in their native tongue even as they approached the end of higher education. While able to vocalize (‘decode’) the sentence, they did not possess literacy skills necessary for warranting inferences like those described. Similar observations are often made in countries with established educational traditions. For example, a Washington Post report (Dec 25, 2005) on recent adult literacy assessment, claims that “only 31 percent of college graduates can read a complex book and extrapolate from it. That’s not saying much for the remainder” (Romano 2005). How can we explain this anomaly? Why has modern education failed to enhance the inferential powers of these individuals? This anomaly may arise from the fact that educators misleadingly view writing as a code which represents spoken language. Much is hidden by the metaphor. It blinds us to how speech and writing serve as scaffolding which we use to orient ourselves and others in the cognitive domain of interactions. By invoking the code metaphor, we ascribe these linguistic interactions to the same cognitive domains. Given that quite different dynamics sustain oral speech and dealing with disembodied written texts, neither can ‘represent’ the other. Experience of natural linguistic behavior is not necessarily commensurate with experience of making or interpreting written texts. The students of the experiment talk and think interactively. Difficulty with inference-making shows lack of experience with texts compounded, perhaps, by fear of failure. As a result, they lack interpretative abilities appropriate to the domain of texts. Instead of using texts to interact with their own ‘cognizing selves’, they seek out the literal meaning prescribed by the code model. They overlook what Menary (2007:â•›622) identifies as one of the most striking properties of writing: while I can listen to myself during talk and, occasionally, discover aspects of my thinking, this is central to acts of writing. The reflective nature of writing is encouraged because — in contrast to the meshing dynamics of conversation — we can cut ourselves off from other displays of affect. Learning to exploit written marks is a gradual process. Given similar histories of interactions in the consensual domain of languaging, individuals manifest varying degrees of proficiency in engaging with the text-world. To become skilled at drawing the appropriate inferences, we depend on socially located experience of working with written texts. In other words, mak-

41

42

Alexander V. Kravchenko

ing sense of written marks draws on accumulated experience of world, speech, and writing. 3. Languaging and experience In the cognitive philosophy of language (Kravchenko 2003, 2008, 2009b) based on the epistemology of autopoiesis (Maturana 1978a; Maturana and Varela 1980; Di Paolo 2005), meaning is a cognitive function whose argument is experience. As such, meaning is enacted as a dynamic relationship between an organism and its environment. At every moment, it is determined by the value which environmental aspects hold for an organism (Zlatev 2003). The cognitive effort of the interpreter of linguistic signs (the observer of communicative verbal behavior) yields results which derive from how physical, biological, social, historical, cultural, etc., parameters bear on communicative experience (Kravchenko 2010). An organism’s current state characterizes it as a structure determined system — “a system such that all that happens in it or with it arises as a consequence of its structural dynamics, and in which nothing external to it can specify what happens in it, but only triggers a change in its structure determined by its structure” (Maturana 2000:â•›461). Living systems are unities of interactions which exist in an environment in structural coupling. They depend on dynamic congruence with the medium. The organization of a cognitive system determines the domain of interactions in which it can act meaningfully to sustain itself, and cognition is an actual behavior in this domain. In neuroscientific terms, experience exploits relative states of neuronal activity caused by past interactions with the environment. For functional reasons, these biosemiotic phenomena (Kravchenko 2003) serve as representations of interactions. Cognizing individuals draw on these representations to enhance their adaptive power. In so doing, they avoid direct physical interaction with potentially hazardous aspects of the environment. As experience emerges from interactions with the world, its structure and content are determined by the cognizer’s interactional history. In Clark’s (1997) metaphor, the ‘niche’ includes linguistic phenomena. Since no two individuals can have identical experience, structurally and content-wise it implies non-identity of meanings even where both deal with the same object or phenomenon. As argued by Russell (1948:â•›4), “the meaning that the child comes to attach to the word is a product of his personal experience, and varies according to his circumstances and his sensorium”. But how, then, does mutual understanding become part of communication? The definition of linguistic sign as a unit of language used to denote things in the world does not clarify how language relates to other phenomena. Although



The experiential basis of speech and writing

languaging is human biological activity (which is evanescent), members of a community treat linguistic phenomena as natural constituents of a physical and cultural medium that shapes their world. Typical linguistic signs or words can only exist as physical substance. They stand in spatio-temporal relations to various physical parameters that shape the context of every word’s being, or its environmental medium. Changes in the physical context alter the nature and number of a particular word’s relationships with other phenomena constitutive of its environment. Accordingly, the emergence of what can be called ‘experience of sign’ draws on highly variable associations. A sign’s so-called ‘content’ varies. Understanding is an interpretative process which crucially depends on the degree to which our experience as language users is congruent with that of others (Kravchenko 2007a). To the extent that this occurs, we exploit a consensual domain or an ensemble of mutually defined behaviors. Meaning-making thus gives rise to a complex cognitive dynamics that characterizes what we are. Like other organisms, we exploit unique histories of ontogenetic structural coupling. Uniquely, however, this also draws on how behaving in a consensual domain shapes the (linguistic) environment. In humans, therefore, the cognitive domain of orientational interactions takes on an additional dimension. While established by interactions of the orienting organism and the organism’s states of relative neuronal activity, it is also influenced by how each orients to the other’s orienting. Counter to centralist views of cognition, therefore, language is not a mental capacity.4 Rather, as a biological phenomenon, language is a mode of operating in consensual coordinations of behavior (Maturana 1978b). For Emmeche, while rooted in the organic world, humans “transcend our biological form of existence by producing, through culture, language and social institutions, specific dynamic modes or patterns in which our organic and animate existence is realized” (Emmeche 2007:â•›456). To understand linguistic signs is a cognitive process: in making sense, the cognizer looks for common experiential ground to interpret the behavior of others. It is only because our brains are inherently dialogical that we can enter the relational domains of others (Linell 1998; Thibault 2000). Thus, in the biological theory of cognition (Maturana 1970) natural language is connotational. Since it is not based on denotations (or reference), it cannot be defined as a code (Love 2004, 2007; Kravchenko 2007a). Moreover, when graphic symbolizations constituted by code symbols (letters) are viewed as standing for the structural elements of spoken words (sounds), an insurmountable problem arises. To explain how letters can stand for spoken words and, by extension, for their meanings, demands some kind of vitalism. To identify written marks with a living word is thus to make a category mistake. Sentences are structural units of disembodied written texts (Linell 2005). They are symbolic artifacts or second-order cultural constructs. As historical entities,

43

44

Alexander V. Kravchenko

they are not grounded in first-order dialogical activity. Languaging occurs in a cognitive domain which has no need for graphic images that symbolize acoustic phenomena. In human interactions with texts, written ‘words’ become orienting signals. Mistakenly, we may think that these graphic marks possess meanings. However, in languaging meanings are constructed on the fly as we assign orientational values to the physical words as components of human behavior. Sensemaking (understanding) is neither subjective nor objective. In Linell’s terms, it lives in the relations between the ‘inner’ and the ‘outer’, in the ‘inter-world’ of an organism and its environment. It is based in experience of previous interactions in our cognitive domain. At the same time, it also involves online ‘processing’ of all the perceptual data provided by the physical context in which linguistic interactions take place. As argued by Linell (2005:â•›610), [I]f languaging and thinking are other-oriented, it means that actions and cognitions are both responsive to prior events and actions, and anticipatory to possible next actions (on the part of oneself and the other). This in turn means that there is a close relationship between understanding and responding. The distinction between (utterance) production and comprehension is not as radical as has traditionally been assumed in models of senders (speakers, actors) and receivers (listeners).

Because of an intricate scaffolding provided by the physical context of a particular discourse occurring in real time, a behavioral response which we interpret as understanding is often obtained before the speaker has completed a particular linguistic interaction aimed at orienting the other in his consensual domain.

4. The cognitive domain of written marks Whereas speaking humans interact with each other, written communication occurs peripherally to engagement. The dynamics of writing depends on the use of writing implements and materials on which markings remain. When lifeless objects are subsequently read, the results can be seen as reflecting a writer’s interaction with his own thoughts. While there is a sense in which this is ‘interactive’, its inherently reflexive nature contrasts with dialogue. Since the reader’s object of interaction is a static artifact (e.g., a text), the results differ from what happens when we engage with a living person. Nor should we suppose that the writer’s thoughts can be accessed through the written symbolizations of linguistic signs. Rather, the reader’s changing interpretations of the symbols depend on a cognizing self. As for natural languaging, this draws on the reader’s life-time experience of linguistic and non-linguistic interactions. This is why those with minimal experience in the domain of writing can be rapidly picked out. In Russian folk wisdom, it is said of



The experiential basis of speech and writing

people with code-level literacy, Smotriat v knigu, a vidiat figu. Roughly translated, this means that they ‘look in the book and see nothing but a fig’. Such readers lack the skills needed to ‘extract’ knowledge from texts. In the terms of this paper, they lack experience in the relevant cognitive domain Environments can be divided into distinct domains of interaction that include personal space, home territory, and public territory (Sommer 1969; Slane et al. 1981). Viewed as a structured domain of interactions, public territory can be further partitioned into educational, intra-cultural, and cross-cultural domains. In the first, crucial stage of language acquisition, the young human organism interacts in personal space and on home territory. As the child actively in-forms himself in linguistic environment, he develops initial standards of linguistic behavior. During interactions, these are continuously tested through direct or indirect appeal to authority (parents, caretakers, members of the family, etc.). The adequacy of a child’s linguistic behavior is assessed by relating it to the family members’ personal values, priorities, and idiosyncrasies. These will, of course, vary with respect to nature, character, and public relevance. In other words, in the process of language acquisition children learn to conform (Tomasello 2006; Hodges 2007), thus ensuring their ‘functional fit’. In the intra-cultural domain of interactions, the concept of functional fit is controversial. Dialectical contradictions often oppose an individual’s goals to those of the community. Over time, therefore, behavioral patterns emerge as adaptive solutions to dealing with events in the social world. Each individual must come up with behavioral strategies that produce actions whose functions ‘fit’ the intracultural domain. This leads to tension. A child may seek to maximize autonomy as a cognizer by minimizing the restrictive influence of the (social) environment. At the same time, other interests exert counter influences. The living human community (society) will act to enhance the group’s adaptive potential by minimizing individual cognitive autonomy. Thus, the intra-cultural domain of interactions (as well as the larger cross-cultural domain of interactions) gives rise to oscillating patterns of cognitive activity. In evolutionary time, attractors will seek to establish dynamic equilibrium between the parts and the whole. For social psychology, one consequence of this cognitive flux is the inevitability of conflicts — often categorized as criminal behavior — at the interactional interface between second- and third-order living systems. As a young individual grows, his domain of interactions with the environment expands into the domain of public relations. However, before integrating into society and its culture, the child’s development is socialized by educational interactions. In other words, the child is in-formed in the human habitat as he draws on the norms and standards of the consensual domain by attempting to

45

46

Alexander V. Kravchenko

elicit encouragement and approval from purveyors of socially approved norms. Professional educators too have their own personal values and preferences which play out in multiple consensual domains giving rise to similar, though contending goals. A human individual, whose linguistic behavior is influenced by the educational domain of interactions, comes under pressure to build specific interpretative strategies based on numerous contextual parameters. It is necessary to judge, in relation to his consensual domain, whether his own linguistic behavior has been ‘approved’ by those who have respective authority. Equally, he must decide whether or not to use the same model as in the domain of family/home interactions. As a result, when interacting in the domain of written texts he is apt to structure his linguistic behavior (interpretation of texts) with reference to how he interprets the current physical context — which has nothing to do with the text to be interpreted by definition. Graphic markings, which resemble code symbols, make interpretation possible across different uses in time and space. Since this is the most socially relevant function of writing, graphic images are often (mistakenly) seen as natural linguistic signs. This failure to draw an ontological/experiential distinction between speech as local and temporal, and written text as non-local and atemporal, lays the ground for functional failure in linguistic behavior (both reading and writing). In dealing with written texts, it leads to communicative dysfunction (Kravchenko 2009b). A striking example of communicative dysfunction on the part of a writer would be a notice attached to an office door, such as:

(2) Be back in 20 minutes.

By creating this text its author enters the domain of specific linguistic interactions by identifying graphic markings with signs of natural language that might serve in the consensual domain of interactions. Thus the author of the notice strips his linguistic behavior of its ability to exert orientational influence for the simple reason that, in these circumstances, there is no consensual domain. Since no reader can establish a common point of reference, it is necessary to guess whether one will have to wait 5, 10, or all of the 20 minutes (granted that the author of the notice is true to his word). Of course, this interpretation of the situation assumes that the writer is not cognitively/linguistically sophisticated. It is also possible that whoever wrote it realized fully well what he was doing. The sign may also be used, for example, to provide cover for absence from the workplace. On the part of the reader, failure to functionally interact with the domain of written texts must arise from treating written (codified) signs as if they were natural linguistic signs. For example, consider how the Russian expression udostoverenije lichnosti (‘proof of identity’, ‘ID’ — such as a passport, military service



The experiential basis of speech and writing

card, driver license, work pass, etc.) is interpreted in the domain of public interactions. To establish one’s identity is to verify, by the authority delegated to a particular office or institution, that the person whose photograph is on the ID is the one with the name stated on the ID. In real life, however, the situation is somewhat peculiar as, counter to the codified meaning of udostoverenije lichnosti, government officials and others, characteristically, accept only two kinds of ID: national passports and military service cards. Indeed, even a Russian passport issued for foreign travel (known as ‘foreign passport’) is often refused as a valid identity document. Although the official regulations issued by the Central Bank of Russia stipulate that an acceptable ID is a valid passport — without specifying whether it should be national, foreign, or diplomatic — this is often interpreted as referring only to the national Russian passport. This surprising state of affairs is readily explained. A typical rank-and-file interpreter of udostoverenije lichnosti is driven by his desire to use, as cognitive scaffolding, the authority of another interpreter in the consensual domain whose linguistic behavior, because of pragmatic considerations, produces maximal orientational effect on the bureaucrat’s linguistic behavior. In other words, the bureaucrat is guided by his supervisor or boss. With regard to the cognitive domain of texts, the interpretative abilities of the authorities often leave much to be desired. Those with power lack experience of interactions in the relevant domain. In short, their developmental history did not feature the extensive reading and writing appropriate to the information era. This is unsurprising in individuals aspiring to ‘make it to the top’, for interactions in the domain of languaging (being oriented by others, such as the higher-ups, and orienting others by return) is paramount in adapting to the immediate environment. They underrate the cognitive importance of the domain of written texts. For many people, those in command mold their immediate environment. This, too, is captured in folk wisdom: (3) a. ‘If you’re the boss, I’m the fool; if I’m the boss, you’re the fool’ b. Ty nachal’nik — ja durak, ja nachal’nik — ty durak you boss I fool I boss you fool

5. Beyond code models of language Far from being a code, language is an array of potentialities. These, it is argued, can be realized in differing ways in different cognitive domains and, most strikingly, those of written texts and languaging. By focusing on connotational behavior in a cognitive domain of interactions, we gain skills in using an array of linguistic resources. These are used both in producing and actively orienting to utterances:

47

48

Alexander V. Kravchenko

These resources […] are designed to be completed only in situated meaning-making. They do not ‘encode’ or ‘contain’ their meanings; rather, they index, cue or prompt understandings in terms of reference, conceptualization and intervention (Linell 2007:â•›611).

By contrast, in written texts understanding is not intrinsically dialogical. We are not cued by the ever changing circumstances in which communicating individuals find themselves. Given their ontology, written texts are not intrinsically dialogical. So while languaging functions to extend the human sensorium, texts act as scaffolding that enhances human cognitive powers. Second-order symbolizations serve us as ‘reminders’ of past experiences and/or reference-points for currently experienced cognitive states. There is no common code. To repeat, it is the ontological difference in cognitive dynamics that makes speech and writing into different cognitive domains. Languaging is behavior that occurs in a consensual domain. Adequate interpretation depends on taking into account the physical context in which linguistic behavior is enacted. In canonical situations, we make extensive use of dynamical features that include vocalizations, gestures, facial expressions, particular ways of orienting to others and the world (Cowley 2004). In languaging — and learning to talk — these are crucial in nudging children to hear and interpret linguistic signs. Texts lack equivalents to these. Over historical time, however, this has been turned to our advantage. For, once we cut off the ‘extralinguistic’ part of natural linguistic signs, we construct the domain of (written) texts. As emphasized by Clark (2007:â•›427), written language “is not simply a poor cousin of face-to-face vocal exchange. Instead, it provides a new medium for both the exchange of ideas and (more importantly) for the active construction of thoughts”. While ‘exchange of ideas’ is nothing but a metaphor, writing has a role in the ‘active construction of thought.’ Moreover, as argued above, its products enhance sustainability of thirdorder living systems (societies with written cultures). This became especially striking when the printing press provided a new level of organization to the cognitive domain of interactions. Metaphorically, the domain of written texts became a kind of Petri dish for culturing individual humans. There is, however a difference: by thriving in the cognitively nourishing medium of written texts, humans, far from depleting it, grow and cultivate it by adding new texts. Yet, we pay a price. Individuals who cannot thrive in the culture, or, indeed, who eschew it — put their potential for social adaptation in double jeopardy. Cognitive dysfunction can arise from failure to interact effectively with written texts (or the second-order relational domain). While able to participate effectively in languaging, many people have diminished capacity for adaptation. Through being a boon and a revolutionary advance, literacy does not produce cognitive development. Even if it is a kind of tool,



The experiential basis of speech and writing

its value emerges only as that tool is put to effective use. This can only be achieved experientially. The highest level or developmental stage of literacy is characterized by Harris (2000:â•›xi) as follows: Full literacy which arguably no society has yet quite reached, is one in which writing is no longer regarded just as a ‘profitable invention for continuing the memory of time past, and the conjunction of mankind,’ but as a particular mode of operation of the human mind and the key to a new concept of language.

To my eyes, this ‘new concept of language’ will recognize that written culture is an ecological medium. Human society can live as a third-order system with sustained historical continuity. The orientational effect written texts can have on an interpreter’s behavior depend, to a large extent, on how the interpreter links this domain to life-time experience. In other words, our interpretative skills are a function of the structural dynamics of a structure determined system. The lay term for these structural dynamics is ‘background knowledge’. Increasingly, these derive from experience in the textual domain. Texts mediate knowledge as categorized experience of species specific interactions with the world. Thus, for interpretative activity with texts to be successful, we depend heavily on shared background knowledge. In turn, this derives from a language user’s text-mediated interactions and contexts in which graphic words are used — or, rather, the combined experience of such (highly variable) contexts that an interpreter possesses as a result of his interactions in the domain of written marks. This is a very important point. In interpreting (‘understanding’) written texts, structure determined systems neither process information nor ‘extract’ knowledge as a reaction to text (or spoken words, for that matter). Rather, they rely on their intrinsic organization which results from ontogenetic structural coupling that determines the significance of the parts of the environment (including the dynamics of the other) acting as “stimuli”. So how do individual linguistic interactions sustain the conditions for the normal functioning of modern society? In order to understand the relations between language and society, we need to highlight phenomenological difference between spoken and written language. If human society resembles an organism, spoken language is the ‘nervous system’ whose central part consists in written signs. These give rise to an ontological duality that serves to coordinate the functioning of the organism as a whole. Signals from the central nervous system are thus neurological results that arise from interpreting the states of the organism’s parts as a functional unity. Such signals possess greater adaptive relevance than do ones generated at the nervous system’s periphery. Further, the greater the complexity of an organism as a functional system, the greater the complexity of its brain. Indeed, in the case of higher vertebrates, this becomes the ‘soft spot’ for the organism’s normal func-

49

50

Alexander V. Kravchenko

tioning. Consequences of damage to the brain outweigh comparable (in terms of body mass) damage to other organs. Finally, functional ‘glitches’ and disturbances at the vegetative level may interfere with, or interrupt, the adequate (cognitively ‘correct’) interpretation of signals from the central nervous system. As a result, a particular organ may not be able to perform its function within the system and the organism’s overall adaptive ability may decline. Similar considerations apply to how spoken and written language regulate the life of the third-order system we call human society. Every speaker of a common tongue contributes to how individuals and community are mutually co-determined by and through language. Linguistic communication and a cognitive domain of linguistic interactions are thus necessary for the formation and development of both communities and individuals. On this view, the primary function of language is socio-generative. Accordingly, it is necessary to distinguish different hierarchical levels of linguistic interactions. The textual domain is ontologically secondary when compared with the individual’s (natural) linguistic or consensual domain. However, it is primary when society is recognized as a structure determined system. The reason is simple. Given a history of structural coupling, individuals and society are mutually co-determined. Consequently, normal individual functioning which, in turn, pre-conditions the normal functioning of society itself, presupposes the primacy of orientational effects of the domain of written texts over the orientational effects of (natural) linguistic consensual domains. Different phenomenologies of the domain of written text and spoken language call for different systems of interpretations based on different points of reference. The optimal functioning of an individual in society thus depends on the development of skills for interpretation in these different coordinate grids. Undeveloped or underdeveloped skills for interpretation of orientational effects generated by interactions in the domain of written texts (as well as lack of skills for projecting the results of such interpretation onto the consensual domain of natural linguistic interactions) underpin functional illiteracy or an individual’s failure to modify linguistic behavior recognizing the orientational effects of interactions in the domain of written texts. Degradation of reading culture thus leads to degradation of society itself. Once we view society as a unity of interactions, we see the crucial sustaining role of a linguistic ecology. That, however, is an issue for future research. 6. Conclusion Culture, understood in a broad sense as “an integrated pattern of human knowledge, belief, and behavior that depends upon the capacity for symbolic thought and



The experiential basis of speech and writing

social learning” (wikipedia.org), may be viewed as a continuous evolutionary space sustained by the relational domain of interactions of a third-order system (human society). Crucially, this depends on languaging as integrated dynamically complex interactional behavior in a consensual domain involving various artifacts and practices. By contrast, in the literate era humans in their adaptive orientational activity began to depend more and more on the ‘world on paper’. Writing, as an artifact of human culture, began to mediate interactions in relational domains of third-order systems as unities of interactions, thus acquiring paramount importance in providing for cultural continuity and becoming an indispensable tool in enhancing human cognitive powers. Accumulated experience of interactions in the domain of written texts accounts for an individual’s level of ‘functional fit’ in a third-order system. The goal of education as a social institution is to facilitate cognitive development of young individuals as second-order living systems to be functionally integrated in third-order systems. Such effective integration is impossible without individuals being able to use writing as ‘a particular mode of operation of the human mind’ which enhances their adaptability in the relational domain. However, the ‘linguist’s view’ of literacy mistakenly conflates speech and writing. Drawing on the code model of language, it obscures crucial differences between how we interact with other living systems and with lifeless cultural artifacts (texts). Consequently, educators often fail to see how lack of relevant experience in the domain of written texts over the developmental history of an individual leads to functional illiteracy and — not surprisingly — lack of participation in cultural life. Are text-based inferencing skills declining? Once construed as concerning events in a cognitive domain, this becomes an empirical question. By contrast, when we picture reading as akin to interpreting Morse code, the question is intractable. This is because the issue is conceptualized at a (putative) level of encoding. Given deeply rooted belief in inner languages, linguists fail to ascribe speech and writing to different (if overlapping) cognitive domains. It is time to recognize that, in principle, this may result from changes in how we work with texts. Second, we must not lose sight of the fact that this is bound up with new technologies. It is quite possible that, while we are losing inferencing skills, we may be creating new ways of exploiting what texts afford: together with new — technology enhanced — ways of interacting, which impact on structural dynamics of both second- and third-order living systems, new ways of thinking may be emerging. We need new ways of investigating how we engage with an evolving (computer enhanced) text-world. We need to ask how, during an individual’s life time, text-based thinking develops and, at the same time, uncover individual differences. On the bio-cognitive view this depends, in the first instance, on changing

51

52

Alexander V. Kravchenko

cognitive dynamics and how we integrate our use of inscriptions with other skills. The material nature of inscriptions — and the surfaces and screens on which they appear — is secondary to thinking. We need models of their dynamics and, above all, how our cognitive capacities draw on experience of making and construing inscriptional entities. Once these matters are clearer we may begin to develop modes of interacting that encourage people to develop their skills. This, however, will require new views of literacy — ones that focus not only on their social functions (on the diversity of literacies), but also on how it is that these change our ways of engaging with others and, most strikingly, our own thinking selves.

Notes 1.╇ Maturana writes, “We as observers exist in a domain of descriptions, and this domain as a consensual domain is a cognitive domain. In fact we operate in many different cognitive domains which constitute many different ways of realizing our autopoiesis. Furthermore, each cognitive domain constitutes a closed domain of relations or interactions defined by features of internal consistency specified in the structural coupling that determines it” (1978a:â•›46). 2.╇ In the autopoietic explanation of observed phenomena a ‘unity’ is “an entity, concrete or conceptual, dynamic or static, specified by operations of distinction that delimit it from a background and characterized by the properties that the operations of distinction assign to it” (Maturana 1978:â•›31). 3.╇ Arshin – old Russian unit of measure (~ 71 cm); chugunnyj bolvan – an iron heated on a stove; shtoff – a square short-necked bottle (1.2 l). 4.╇ This is debated in a Pragmatics and Cognition special issue (Kravchenko 2007b; Harnad 2007).

References Belpaeme, T. and Cowley, S.J. 2007. “Extending symbol grounding”. Interaction Studies 8: 2–6. Bang, J.C., Døør, J., Steffensen, S.V., and Nash, J. 2007. Language, Ecology and Society: A Dialectical Approach. London: Continuum. Bradshaw, T. and Nichols, B. 2004. “Reading at risk: A survey of literary reading in America”. National Endowment for the Arts: Research Division Report 46. Washington. DC. Crystal, D. 2008. Txtng: The Gr8 Db8. Oxford: Oxford University Press. Clark, A. 1997. Being There: Putting Brain, Body and World Together Again. Cambridge, MA: The MIT Press. Clark, A. 2007. “A sense of presence”. Pragmatics & Cognition 15(3): 413–433. Clark, A. and Chalmers, D. 1998. “The extended mind”. Analysis 58: 7–19.



The experiential basis of speech and writing

Cook-Gumperz, J. 2006. “The social construction of literacy”. In J. Cook-Gumperz (ed), The Social Construction of Literacy. Cambridge: Cambridge University Press, 1–18. Cowley, S.J. 2004. “Contextualizing bodies: Human infants and distributed cognition”. Language Sciences 26: 565–591. Cowley, S.J. 2007a. “The cognitive dynamics of distributed language”. Language Sciences 29: 575–583. Cowley, S.J. 2007b. “How human infants deal with symbol grounding”. Interaction Studies 8: 83–104. Di Paolo, E. 2005. “Autopoiesis, adaptivity, teleology, agency”. Phenomenology and the Cognitive Sciences 4: 429–452. Donald, M. 1991. Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition. Cambridge, MA: Harvard University Press. Dror, I.E. (ed) 2008. Learning Technologies and Cognition: Special issue of Pragmatics & Cognition 16(2). Emmeche, C. 2007. “A biosemiotic note on organisms, animals, machines, cyborgs, and the quasi-autonomy of robots”. Pragmatics & Cognition 15(3): 455–483. Hagège, C. 1996. L’Homme de Paroles: Contribution Linguistique aux Sciences Humaines. Paris: Fayard. Harnad, S. 1990. “The symbol grounding problem”. Physica D 42: 335–346. Harnad, S. 2005. “Distributed processes, distributed cognizers, and collaborative cognition”. Pragmatics & Cognition 13(3): 501–514. Harnad, S. 2007. “Maturana’s autopoietic hermeneutics versus Turing’s causal methodology for explaining cognition”. Pragmatics & Cognition 15(3). 599–603. Harris, R. 2000. Rethinking Writing. London: Athlone Press. Hodges, B. 2007. “Good prospects: Ecological and social perspectives on conforming, creating, and caring in conversation”. Language Sciences 29(5): 584–604. Hutchins, E. 1995. Cognition in the Wild. Cambridge, MA: The MIT Press. Järvilehto, T. 2000. “The theory of the organism-environment system: IV. The problem of mental activity and consciousness”. Integrative Physiological and Behavioral Science 35: 35–57. Jenkins, H., Clinton, K., Purushotma, R., Robinson, A.J., and Weigel, M. 2006. “Confronting the challenges of participatory culture: Media education for the 21st century”. MacArthur Foundation. Available at Kravchenko, A.V. 2003. Sign, Meaning, Knowledge: An Essay in the Cognitive Philosophy of Language. Frankfurt/Main: Peter Lang. Kravchenko, A.V. 2007a. “Essential properties of language, or, why language is not a code”. Language Sciences 26: 650–671. Kravchenko, A.V. 2007b. “Whence the autonomy? A response to Harnad and Dror”. Pragmatics & Cognition 15(3): 587–597. Kravchenko, A.V. 2008. Biology of Cognition and Linguistic Analysis: From Non-realist linguistics to a Realistic Language Science. Frankfurt/Main: Peter Lang. Kravchenko, A.V. 2009a. “Reassessing the project of linguistics”. In J. Zlatev, M. Andrén, M.J. Falck, and C. Lundmark (eds), Studies in Language and Cognition. Newcastle upon Tyne: Cambridge Scholars Publishing, 27-42. Kravchenko, A.V. 2009b. “Speech, writing, and cognition: the rise of communicative dysfunction”. In W. Oleksy and P. Stalmaszczyk (eds), Cognitive Approaches to Language and Linguistic Data. Studies in honor of Barbara Lewandowska-Tomaszczyk. Frankfurt: Peter Lang, 225–240.

53

54

Alexander V. Kravchenko

Kravchenko, A.V. 2009c. Language and mind: A bio-cognitive view. In H. Götzsche (ed.), Memory, Mind and Language. Newcastle upon Tyne: Cambridge Scholars Publishing, 103–124. Kravchenko, A. V. 2010. Native speakers, mother tongues, and other objects of wonder. Language Sciences 32(6): 677–785. Linell, P. 1998. Approaching Dialogue: Talk, Interaction and Contexts in Dialogical Perspectives. Amsterdam: John Benjamins. Linell, P. 2005. The Written Language Bias in Linguistics: Its Nature, Origins and Transformations. London: Routledge. Linell, P. 2007. “Dialogicality in languages, minds, and brains: Is there a convergence between dialogism and neuron-biology?”. Language Sciences 29: 605–620. Love, N. 2004. “Cognition and the language myth”. Language Sciences 26: 525–544. Love, N. 2007. “Are languages digital codes?”. Language Sciences 29: 690–709. Maturana, H.R. 1970. Biology of Cognition. BCL Report # 9.0. University of Illinois, Urbana. Maturana, H.R. 1978a. “Biology of language: The epistemology of reality”. In G. Miller and E. Lenneberg (eds), Psychology and Biology of Language and Thought. New York: Academic Press, 28–62. Maturana, H.R. 1978b. “Cognition”. In P.M. Hejl, W.K. Köck, and G. Roth (eds), Wahrnehmung und Kommunikation. Frankfurt: Peter Lang, 29–49. Maturana, H.R. 2000. “The nature of the laws of nature”. Systems Research and Behavioral Science 17(5): 459–468. Maturana, H.R and Varela, F. 1980. Autopoiesis and Cognition: The Realization of the Living. Boston: D. Reidel. Maturana, H.R, Mpodozis, J., and Letelier, J.C. 1995. “Brain, language, and the origin of human mental functions”. Biological Research 28: 15–26. Matsuura, K, 2008. “Message from Mr. Koïchiro Matsuura, Director-General of UNESCO”, International Literacy Day, 8 Sept 2008. Available at http://unesdoc.unesco.org/ images/0016/001621/162186e.pdf Menary, R. 2007. “Writing as thinking”. Language Sciences 29(5): 621–632. Morris, C.W. 1938. “Foundations of the theory of signs”. In O. Neurath, R. Carnap, and C.W. Morris (eds), International Encyclopedia of Unified Science, Volume 1, Number 2. Chicago: The University of Chicago Press. [Reprinted in one cloth-bound volume containing the 10 numbers of Volume 1; Chicago: The University of Chicago Press, 1955, 77–137]. Olson, D.R. 1994. The World on Paper: The Conceptual and Cognitive Implications of Writing and Reading. Cambridge: Cambridge University Press. Olson, D.R. and Torrance, N. (eds). 2001. The Making of Literate Socies. Oxford: Blackwell Publishers. Romano, L. 2005. “Literacy of college graduates is on decline: Survey’s finding of a drop in reading proficiency is inexplicable, experts say”. Washington Post, December 25; A12. Russell, B. 1948. Human Knowledge: Its Scope and Limits. New York: Simon and Schuster. Slane, S., Petruska, R., and Cheyfitz, S. 1981. “Personal space measurement: A validational comparison”. Psychological Record 31(2): 145–151. Sommer, R. 1969. Personal space: The Behavioral Basis of Design. Englewood Cliffs, N.J.: Prentice-Hall. Stewart, J. 1996. “Cognition = life: Implications for higher-level cognition”. Behavioural Process 35: 311–326.



The experiential basis of speech and writing

Thibault, P.J. 2000. “The dialogical integration of the brain in social semiosis: Edelman and the case for downward causation”. Mind, Culture and Activity 7: 291–311. Tomasello, M. 2006. “Acquiring linguistic constructions”. In D. Kuhn, R.S. Siegler, W. Damon, and R.M. Lerner (eds), Handbook of Child Psychology, sixth ed., Cognition, Perception, and Language, vol. 2. New York: Wiley, 255–298. Vygotsky, L.S. 1994. Thought and Language. Edited by A. Kozulin. Cambridge, MA: The MIT Press. Zlatev, J. 2003. “Meaning = life (+ culture): An outline of a unified biocultural theory of meaning. Evolution of Communication 4: 253–296.

55

Insightful thinking Cognitive dynamics and material artifacts* Evridiki Fioratou and Stephen J. Cowley

University of Aberdeen, Denmarl / University of Hertfordshire, England

We trace how cognition arises beyond the skin. Experimental work on insight problem solving is used to examine how external artifacts can be used to reach the goal of assembling a ‘cheap necklace’. Instead of asking how insight occurs ‘in the head’, our participants in Experiment 1 can either draw solution attempts or manipulate real objects (specifically, chain links that make up a necklace). Even though performance with real chain links is significantly more successful than on paper, access to objects does not make this insight problem simple: objects themselves do not shape cognition. This challenges extended mind views. While failure often results from the inappropriate (to the current insight problem) application of hill-climbing, material artifacts can trigger solutions. In Experiment 2, we used ‘open link’ conditions of the concretized problem to prompt participants to act (or think). Solutions arrived via insight, serendipity, or trial-anderror. By investigating how objects are used, we show that they do more than supplement neural events. Rather, participants monitor and anticipate the effects of action (and thinking) within an organism-environment system. By analogy, language too draws on experience of monitoring real-time effects as bodily dynamics play out in a normative and cultural world. In engaging with public language, it is likely that verbal patterns function by constraining anticipatory (action-based) cognitive processes. Keywords: artifacts, distributed cognition, distributed language, extended mind, insight, problem solving

1. Prelude Occasionally, we come up with something surprising, new, or effective. In everyday life, this may be a practical action and, in thinking, it may strike home as important or even true. We deem this insightful or, if verbalization occurs, as showing insight. Traditionally, the process is traced to what happens ‘in the head’. Indeed,

58

Evridiki Fioratou and Stephen J. Cowley

insight entered psychology on the basis of Köhler’s (1925) work with chimpanzees. In one example, Sultan, a captive chimpanzee ‘insightfully’ joined two sticks to form a pole for reaching bananas (Köhler 1925:â•›188). Although tool based, this was viewed as a change in (neural or mental) representation. Whether or not an ‘aha’feeling occurred, the ape restructured its environment to reach its goal (the hanging bananas). The use of sticks as a pole is seen as peripheral to the ape’s insight. In humans too, material artifacts are usually regarded as marginal to problem solving. Material artifacts lack any standing in cognitive psychology. Where acknowledged, they are treated as constraining and guiding behavior by reducing the load on working memory (Gilhooly and Fioratou 2007; Zhang 1997; Zhang and Norman 1994). Seeking to redress the balance, the paper examines how material artifacts contribute to insightful thinking. To do this we contrast how different artifacts contribute to solving the same problem (viz. pen and paper and actual objects). This permits us to contrast two externalist cognitive models. First, for Clark and Chalmers (1998) artifacts scaffold cognition. On their extended mind (EM) view, it is predicted that, in some circumstances, material objects augment cognition. Second, following Hutchins (1995), it is posited that distributed cognitive systems can connect people with objects (see Zhang and Norman 1994; Zhang 1997). Rather than regard artifacts as themselves part of cognition, they exert causal effects on body-world experience. On the distributed cognition (DCog) view, material objects are more likely to augment cognition than other aids. This mirrors debate on whether language is part of (inter) action or, rather, “the ultimate artefact” (Clark 1997:â•›218). On the DCog view, verbal patterns are second order constructs (’words’) that constrain anticipatory processing and first-order languaging (see Love 2004; Thibault 2004; Cowley 2007; Järvilehto 2009). By contrast, on the EM view language is a cognitive tool. For Clark, the material symbols of language alter “the nature of the computational tasks involved in various kinds of problem solving” (1997:â•›193). 2. Insight in internalist tradition Highlighting change in representation or restructuring, Gestalt theory focused on putative neural events. In Koffka’s terms, “The situation forces the animal to act in a certain way, although the animal possesses no pre-established special devices for the act”; there remains a mystery about the “exact cause of these organizational processes” (Koffka 1935:â•›135). Even today, most accept that parts become a whole in what Newell and Simon (1972) call a problem space. Comprehension can lead to a striking experience and, with hindsight, solutions can seem obvious. For Duncker (1945:â•›29), “The decisive points in thought processes, the moments



Insightful thinking: Cognitive dynamics and material artifacts

of sudden comprehension, of Aha!, of the new, are always at the same moments in which a sudden restructuring of the thought material takes place”. Representational restructuring parallels perceptual interpretation of optical illusions (Ellen 1982; Smith 1995). It is dynamical, active, and event-directed. Since tradition ignores the world beyond the skin, Wertheimer (1959) contrasts two kinds of (inner) thinking. In productive or insightful thinking, we grasp structural relations in a problem or situation and then restructure the parts as a whole. In reproductive thinking, by contrast, we use ‘blind’ repetition of learned responses to individual subparts.1 While situationally appropriate, insight problem solving alone is therefore inner restructuring. Radical change arises as a problem solver (or brain) sees a problem. Gestalt theory emphasizes that dynamics are discontinuous. For example, Maier’s (1931) two-string problem required participants to pick up strings hanging from the ceiling at different ends of a wide room. To achieve insight, one participant needs to use one of the objects (e.g., a pair of pliers) by swinging it on a pendulum towards the other person. In one case, it was reported, a participant solved only when the experimenter ‘accidentally’ moved the string. Although the participants did not report observing the cue, solution rates increased. Like Köhler, Maier (1931) ignores bodily reaction. Overlooking ‘external resources’, he traces insight to restructuring that organized previous experiences. The combination occurred under what was called an influence. “The parts… must be combined in a certain manner, and a direction or way the problem is attacked seems to be a factor which determines the nature of the combination” (Maier 1931:â•›143). Once given ‘direction’, participants were thus likely to come up with solutions. For Duncker (1945), direction is (inner) reformulation of a problem that initiates a solution process. It does not direct action but, rather, contributes at an early phase of a solution. Other ‘aids’ contain the solution’s later properties. However, Duncker’s view that direction leads to reformulation of a problem can also be construed as claiming that it sets up a goal. 2.1 Information-processing models Like much of cognitive psychology, the field was transformed by the rise of information-processing. Newell and Simon (1972) proposed viewing problem solving as a journey from an initial state to a goal state. In an age of symbol processing, computations were ascribed to an inner space. Incremental problem solving came to be described in terms of applied operators and heuristics. In such cases, participants seek new representations (Kaplan and Simon 1990). Solutions arise from changing, not representational processes but, rather, search procedures. Insight problems thus contrast with problems like the Tower of Hanoi where reaching the goal state depends on a set of steps (Weisberg 1995). For the same reason,

59

60

Evridiki Fioratou and Stephen J. Cowley

once discovered, insightful solutions can seem simple. In single or few-step problems, new situations and limitations affect problem solving. Specifically, one can examine the qualitative differences that separate insight and incremental problems (Chronicle, MacGregor, and Ormerod 2004). From Newell and Simon’s (1972) perspective, insight uses ordinary processes (e.g., Bowers, Farfolden, and Mermigis 1995; Weisberg and Alba 1981a). This nothing special view regards insight problems as variations on general problem solving. This fits Weisberg and Alba’s (1981a, 1981b, 1982; Weisberg and Suls 1973) retrieval framework. Emphasizing prior knowledge, they challenge the Gestalt view that restructuration is spontaneous. Rather, “restructuring of a problem comes about as a result of further searches of memory, cued by new information accrued as the subject works through the problem” (Weisberg and Alba 1982:â•›328). Ohlsson’s (1992) representational change theory (RCT) falls between information-processing and Gestalt theories. He traces insight to “the context of an impasse, which is unmerited in the sense that the thinker is, in fact, competent to solve the problem” (Ohlsson 1992:â•›4). We reach impasse when, not knowing what to do, we halt problem solving activity. If there is no impasse, there can be no insight: solutions are reached without deterrence. Finally, no insightful solution is possible unless a solver has necessary competence. Accordingly, RCT focuses how impasse happens. A participant’s initial mental representations set up blockage when it evokes ‘knowledge’ (i.e., analogies, concepts, ideas, operators, principles, rules, schemas, skills, etc.) irrelevant to a solution. This inhibits or suppresses what is necessary. Impasse arises from both activation of unhelpful elements and inhibition of helpful knowledge. Therefore, solving an insight problem depends on revising initial representations. Change alters how memory is activated by giving access to knowledge elements that have been ignored. The appearance of unheeded knowledge in working memory (and hence in subjective experience) generates the aha-experience. Problem representations can be revised in by, for example, constraint relaxation or deactivating a blocking knowledge element. Knoblich, Ohlsson, Haider, and Rhenius (1999) tested RCT with matchstick arithmetic problems. While not having access to actual matches, participants corrected false arithmetical statements formatted by arithmetical operations (+, −, and =) and Roman numerals (I, II, III, etc.). A false statement can be transformed into a true one by ‘moving’ (not discarding) matchsticks. To make the false statement IV = III + III true, the left stick of IV must be placed on the right of V (giving VI = III + III). Since each problem has a unique solution, the difficulty cannot be traced to the number of moves. Rather, for Knoblich et al. (1999:â•›1534), this depends on necessary representational change. Difficulty arises because prior knowledge inhibits the search for the insightful solutions. It depends on the structure of arithmetic statements. Thus experience tells us that statements often have the form



Insightful thinking: Cognitive dynamics and material artifacts

of X = f (Y) or pair equivalent left and right-hand values. The statement III = III + III can only be successfully revised by relaxing constraint that a statement must have left and right-hand equivalents. It is thus difficult. The required representational change consists in placing the operator’s vertical bar horizontally to express a tautology (i.e. + becomes = to give III = III = III). Since this is the hardest case, Knoblich et al. (1999:â•›1552) propose hierarchical categorization of constraint relaxation. Constraints requiring the relaxation of values are easier than those which require the relaxation of operators. In turn, these are easier than those that require statement of a tautology. Knoblich et al. (1999:â•›1535) also demonstrate that the scope of the constraint affects the ease of a matchstick arithmetic problem. It thus is in parallel to a solver’s general experience. While concentration on a participant’s perceptual representation is not common to all problem solving, it connects the Gestalt school with information-processing (Ohlsson 1992). First, the solver reaches an impasse in trying to solve an insight problem. Second, the problem solver has the ability and necessary knowledge to solve the problem from the outset. Building on this impasse-based framework, MacGregor, Ormerod, and Chronicle (2001) and Ormerod, MacGregor, and Chronicle (2002) proposed the criterion for satisfactory progress (CSP) theory. To maximize progress towards a goal state, we use something like the hill-climbing heuristic of general problem solving (Ormerod et al. 2002:â•›792). In both classes of problem (i.e., insight and non-insight), a solver seeks to maximize closure between where they are and the journey’s end. People monitor each move to see if the progress made equals or surpasses expectations. In fact, however, the necessary move already surpasses the criterion. Specifically, MacGregor et al. (2001) offer an account of performance on the nine dot problem which recalls that of the non-insight Tower of Hanoi problem. In the nine dot problem, participants are told that, without removing the pen from the page, they can use four straight lines to cancel the nine dots of a 3x3 array. MacGregor et al. (2001:â•›177) find that apparently unfavourable next moves are rejected for ones seeming to offer progress. Rather than being planned, selection reflects the apparent potential for reaching the goal state. It is locally-rational in that, at each move, people cancel as many dots as possible. What MacGregor et al. (2001:â•›192) call the maximisation heuristic states that moves are evaluated by a criterion of satisfactory progress (CSP). In the nine dot problem, the CSP is that a move will cancel the maximum number of dots given by the ratio of remaining dots to be cancelled divided by the remaining lines. MacGregor et al. (ibid.) argue that insight draws on experience of not finding apparently successful moves. Until and unless criterion failure occurs, individuals make moves within the initial square. Non-maximizing moves must therefore be incentivized. Early criterion failure will increase the number of solutions. In otherwise identical settings, it acts as an incentive. CSP theory generalizes to, for

61

62

Evridiki Fioratou and Stephen J. Cowley

example, the eight coin problem (Ormerod et al. 2002). Given a starting array of eight coins, a participant makes two moves. Following these, each and every coin must touch exactly three others. In reaching a solution, use must be made of three dimensions. Two coins from the center of the array must be stacked the other coin triads. According to the CSP theory, the many two-dimensional moves that satisfy a locally rational criterion (moves where coins touch three others) make this difficult. Ormerod et al. (2002:â•›793) showed that, when made unavailable, more participants solve the problem. Even a visual hint to the relevance of three dimensions (placing one coin on top of another in the initial state) has little effect in solution rates unless criterion-satisfying moves are unavailable. 3. Externalist extensions While throwing light on problem solving, neither RCT nor CSP theory fully explain insight. Attention to hypothetical processes ‘in the head’ ensures that the theories overlook how material artifacts contribute to insight. In internalist tradition, the non-visible properties of material objects are generally ignored. However, as resources for action, these can contribute to insight. Indeed, this is why problems use a range of artifacts. Whilst the nine dot problem, for example, uses paper and pen, material tokens serve in the eight coin problem. Causal processes beyond the skin can thus contribute to insight. This also applies to classic cases. As reported above, Sultan used material sticks to get the bananas and Maier’s (1931) ingenious experiments required participants to co-ordinate thinking with action. Next therefore, we link this work to the externalist views that increasingly dominate contemporary cognitive science. There is new emphasis on how human powers arise as bodies connect with each other and the world (Varela, Thompson, and Rosch 1991; van Gelder 1998; Clark 1997). In many fields weight is given to the embodied and embedded nature of intelligent activity. While all animals depend on adapting their bodies to the environment, others (including humans) rely on niche construction (Laland and Sterelny 2006). Humans use material culture (Donald 1991; Hutchins 1995; Clark 2006). Artifacts and language change brains in evolutionary (Deacon 1997; Ross 2007), developmental (Tomasello 1999; Cowley 2007a), and experiential time (Goodwin 2000; Glenberg 2008). For Hutchins (1995), much flexible, adaptive behavior arises as we participate in distributed cognitive systems. In intelligent action, including problem solving, we often engage simultaneously with external artifacts and/or other people. Given that human information processing is historical, cognition is culturally distributed. We now show how this work can clarify the mechanisms that contribute to insight.



Insightful thinking: Cognitive dynamics and material artifacts

Alongside events ‘in the head’, material artifacts can influence action and perception. Indeed, Noë (2004) shows that this applies even to vision. Given the two visual streams (and integrated neural processes) seeing depends on acts of looking. For Noë “What we perceive is determined by what we do (or what we know how to do); it is determined by what we are ready to do” (2004:â•›1 [original italics]). Anticipation matters. While visual perception uses superficial properties, insight can draw on more than meets the eye. Remarkably, the non-visible is best known in Clark’s (1997, 2006) view of language. The “forms and structures of a language” (2006:â•›1) are said to connect neural resources with the material symbols that extend the mind: Language (and material symbols more generally) play a double role. On the one hand they do (crucially always) activate other kinds of cognitive resource, bits of mentalese or neuralese as you prefer. But they also play an irreducible role as the material symbols that they are (Clark 2006:â•›2).

Clark’s emphasis on material symbols can be challenged. For Love (2004:â•›539), the approach reifies code-like units (a priori ‘forms and structures’). While not singling out Clark’s view, Harnad (2006) demands that the feeling of thinking be made part of cognitive models. To recognize affect (and avoid code-models), others stress cognitive dynamics or linguistic co-ordination (Cowley 2007a; Thibault 2004; Love 2007; Ross 2007; Kravchenko 2007; Port 2007). Experience shapes skills in hearing verbal patterns (Cowley 2007b) and, later, using these in speaking. On the distributed view, dynamical and normative contingencies shape our cognitive powers (Cowley and MacDorman 2006) or, in another idiom, we act to realize values (Hodges 2009). Turning to insight problems, we can examine if artifacts affect solution rates. Once we have shown that this is the case, we can ask whether this depends on artifacts themselves or the workings of a distributed system. By looking beyond the skin, it is none of our intention to diminish the brain’s role in the events. Indeed, as we shall later argue, our findings seem largely compatible with work based in both the theory of representational change and that stressing criteria for satisfactory progress. Using Silveira’s (1971) cheap necklace problem, as modified in Fioratou (2006), we ask how (if at all) artifacts contribute to insight. This allows us to use standard instructions, formats and artifacts to explore how participants act and perceive. While recognizing the importance of neural activity (and conscious thoughts), we treat problem solving as action. By so doing, we can use behavioral evidence about the emergence of insightful solutions. This enables us to ask whether artifacts prompt solutions or if participants rely on body-world dynamics. For until we specify how problem solvers alter their environment we cannot evaluate the role of anticipation or sensorimotor experience.

63

64

Evridiki Fioratou and Stephen J. Cowley

4. The cheap necklace problem The cheap necklace problem (henceforth CNP) requires a participant to construct a necklace. In the experimental setting, he or she is presented with four chains, each consisting of three links (Figure╯1). The goal is to connect all four chains to form a complete closed necklace. In each step in the process, there are costs. To complete the task successfully, one must spend no more than 15 cents where it costs two cents to open a link and three cents to close it back up again. The problem demands insight because participants must do something that is not intuitively obvious. Specifically, to make a bigger chain, one of the smaller chains must be broken. Thus, one should open the three links of one chain, say chain A, at a cost of 6 cents in total and then use these three links to connect the remaining three chains, B to C, C to D, and D to B, at a cost of 9 cents in total. Putting these together, 6 cents to open the three links plus 9 cents to close these links, costs 15 cents exactly. The choice of the CNP as a paradigm for exploring insight problem solving offers advantages. Not only is it an exemplary, and difficult, insight problem (Silveira 1971) but it has been used extensively by researchers such as Wickelgren (1974), Metcalfe (1986a, 1986b), Metcalfe and Wiebe (1987), Gilhooly and Murphy (2005), and Gilhooly, Fioratou, and Henretty (2010). Unlike many insight problems, it has a clearly defined initial state and a visualisable goal state. Further, it is a transformation problem (Greeno 1978) whose solution requires a sequence of operations. Crucially, as Simon (1978) noted, most of what we know about problem solving and learning is due to analysis of behavior by people who solve transformation problems.

You are given four separate sets of chain links (A, B, C, and D), as shown in the Initial State. Each set is composed of 3 links which are closed at the beginning of the problem. It costs 2 cents to open a link and 3 cents to close a link. Your goal is to form a complete closed necklace, as shown in the Goal State, using sets A, B, C, and D, at a cost of no more than 15 cents.

Figure╯1.╇ The cheap necklace problem in its original form (adapted from Fioratou 2006).



Insightful thinking: Cognitive dynamics and material artifacts

Taking an embodied, embedded view of cognition, we use the CNP in two experiments. In the first, we test hypotheses about how external resources facilitate insight. Accordingly, we present two versions of the problem. On the EM view, solutions depend literally on artifacts. What matters is not experience of resources but, rather, how artifacts extend cognitive powers. Conversely, on the DCog view, what matters is how resources are incorporated into distributed cognitive systems. Insightful solutions will depend on how we cope with embedded cultural events. Insight is less likely to occur in imagining and drawing movements than in manipulating the beads and chains of a real necklace. 4.1 Experiment 1: Paper and pen versus concretized versions To test this approach, the task was presented in both an abstract (paper and pen version) and a concrete (physical) counterpart. Both versions carry information relevant to the solution. The instructions are identical and people can do things with either the pen or by handling the chain links. On the EM view, external resources scaffold problem solving and so, if this happens, experience makes little difference. By contrast, if we rely on processes ‘in the head’, solution rates would fall just as if parts of the brain were missing (cf. Clark and Chalmers 1998:â•›9). On the DCog view, distributed cognitive systems are inseparable from opportunities associated with their components. In the CNP experiment, therefore, sensorimotor experience associated with manipulating objects is expected to affect solution rates. The necklace will give better performance than paper and pen because, while writing gives feedback, manipulating links can set off events that show a path towards the goal. 4.1.1 Method Participants. Seventy-two undergraduate and postgraduate students at Lancaster University acted as participants. Of these 39 were placed in the paper and pen condition and 33 in that dealing with a necklace material. Materials. Paper and pen and concretized versions of the CNP were used. The concretized version used physical links that formed the chains as in the paper and pen version. The links could be screwed open and close. The same instructions (see Figure╯1) were used for both the paper and pen and concretized versions of the CNP. Design. The experiment used a between-participants design with two groups. The groups differed in their external resources. The control group received paper and pen and the experimental group received the physical chains. The dependent variable was whether the problem was solved or not.

65

66

Evridiki Fioratou and Stephen J. Cowley

Procedure. Participating students were randomly assigned to the conditions. They were given 10 minutes to work on the problem. While those in the paper and pen condition drew their solutions, those manipulating necklace-links made use of material chains. Attempts were scored as successful or unsuccessful. 4.1.2 Results and discussion Only 3% (1/39 participants) solved the original CNP in the paper and pen version whereas 30% (10/33 participants) solved it when using the necklace material. The difference between the two being highly significant, χ2(1, Nâ•›=â•›72)â•›=â•›10.626, pâ•›<â•›.001. There is a significantly higher success rate with a material necklace than pictures of the same. Nonetheless, given a 23/33 failure rate, access to actual links does not make the problem easy. Since more participants solved the problem by acting with a physical necklace, the results compare with the general problem solving described by Zhang (1997). In assembling the cheap necklace, material objects impact significantly on success. 4.2 Experiment 2: The open links experiment Insightful solutions arise more often in manipulating with necklace material. Given the need to ‘restructure’ — or realize that to create one must destroy– we now turn to how these are achieved. Experiment 2 is based on Ormerod et al.’s (2002) view that people attempt to maximize progress by moving towards a hypothesized goal state (Fioratou 2006). Since the goal state consists of a set of connected chains, they are expected to maximize gap closure between the chains. It is possible to predict next moves by respecting a formula (viz. each move must equal or surpass the number of chains to be joined, divided by the number of opening and closing moves remaining). When these do not occur, participants are likely to make the next best choice. It is

Figure╯2.╇ The concretized CNP variant.



Insightful thinking: Cognitive dynamics and material artifacts

because people begin by joining chains that the problem proves difficult. Indeed, this move surpasses the criterion for success (for a first move, this is equal to 4 chains/3 movesâ•›=â•›a minimum of 1 and 1/3 chain-links). Only at the final permitted move do participants realize that they lack the money needed to complete the necklace. Accordingly, to highlight the option of ‘destroying’ the chain, the original CNP was varied. This involved presenting the top chain as featuring two open links on either side of a closed middle link (see Figure╯2). Since the problem was otherwise unchanged, a solution could be reached by, first, using the middle link to break the open chain. Second, each of the three links can be used to construct a whole necklace. Although we expected this variant concretized condition to be simpler than the original, we aimed to test whether success depends on the materials themselves. In order to see if experience and incentives also played a role, we set up conditions by manipulating the costs of opening and closing links. We hypothesized that a suitable wording might prompt them to open the appropriate link (i.e., the middle link of the chain with the already opened links). Costs were therefore changed to 5 cents for opening a link and 0 cents for closing one (with a maximum cost of 5 cents). In Ormerod et al.’s (2002) terms, this led to experience of early criterion failure. A similar cost manipulation with the CNP variant was created that would be similar to the behavior exhibited in the original CNP (i.e., the experience of late criterion failure). This was done to ensure direct comparison with our early criterion failure manipulation. Table╯1 contrasts early and late criterion failure under hill-climbing. Table╯1.╇ Early vs. late criterion failure (CF) under a hill-climbing heuristic. Order of Moves under Hill-Climbing

Early Criterion Failure 5 ¢ Open — 0 ¢ Close

Late Criterion Failure 0 ¢ Open — 5 ¢ Close

Close

â•⁄ 0

â•⁄ 5

Close

â•⁄ 0 – CF (5 ¢)

10

Open

â•⁄ 5

10

Close

â•⁄ 5

15

Open

10

15 – CF (19¢)

Close

10

20

4.2.1 Method Participants. Thirty-two undergraduates and postgraduates at Lancaster University were recruited; 16 serving in each condition. They were paid for their participation.

67

68

Evridiki Fioratou and Stephen J. Cowley

Materials. The same concretized variant version of the CNP, as shown in Figure╯2, was used in this experiment for both conditions. The instructions for the early criterion failure condition were as follows: You are given four separate sets of chain links (A, B, C, and D). Each set is composed of 3 links which are joined together at the beginning of the problem, except set A which is composed of 2 already opened links and one closed link. It costs 5 cents to open a link and 0 cents to close it back up again, that is, it is free to close a link. Your goal is to form a complete closed necklace, as shown below, using sets A, B, C, and D, at a cost of no more than 5 cents.

The instructions for the late criterion failure condition were as follows: You are given four separate sets of chain links (A, B, C, and D). Each set is composed of 3 links which are joined together at the beginning of the problem, except set A which is composed of 2 already opened links and one closed link. It costs 0 cents to open a link, that is, it is free to open a link, and 5 cents to close it back up again. Your goal is to form a complete closed necklace, as shown below, using sets A, B, C, and D, at a cost of no more than 19 cents.

Design. The experiment used a between-participants design with two groups. The groups differed in the assumed point of criterion failure. One group received the instructions pertaining to an early criterion failure and the other group received the instructions pertaining to a late criterion failure. The dependent variable was whether the problem was solved or not. Procedure. Participants were tested individually in recorded sessions. They were randomly assigned to one of the two conditions of the experiment and were given 10 minutes to work on the problem. 4.2.2 Results and discussion Eleven out of 16 participants (69%) solved in the early criterion failure condition and 10 out of 16 (62%) in the late criterion failure condition. There was no significant difference between the two conditions in terms of solution rates, χ2(1, Nâ•›=â•›32)â•›=â•›0.139, pâ•›>â•›.05. Experience of criterion failure was not critical in solving a concretized variant of the CNP. However, taking the results from both these conditions (see Table╯2) and contrasting them with the concretized original (CNP of Experiment 1) gives another picture. We find a significant difference between the original CNP and the open links variant CNP (whether pertaining to early or late criterion failure), χ2(1, Nâ•›=â•›65)â•›=â•›6.778, pâ•›<â•›.01. In the open links condition, few participants destroyed the open link chain. Often, having discovered that the naïve criterion failed, they nonetheless repeated the move. They found themselves at an impasse. However, in both early and late



Insightful thinking: Cognitive dynamics and material artifacts

Table╯2.╇ Number of Solvers and Non-Solvers for the Original and Open Links Conditions Conditions Solution Rates Total

Total

Original

Open Links

Not Solved

23

11

33

Solved

10

21

32

33

32

65

criterion failure conditions, the success rate increased. Dealing with a physical necklace was a little easier. Caution should thus be shown in ascribing success to the necklace. Given interaction between manipulations and criteria, solutions cannot depend on the material artifact. Rather, manipulating the chains may well relax problem solving constraints. Next, therefore, we ask how solutions were obtained. Three ways of solving the problem influence success rate: insight, accident (in the form of serendipity), and trial-and-error. From the 21 overall solvers, only 5 (3 in the early and 2 in the late criterion failure conditions) had the ‘insight’ before making the appropriate sequence of moves. Six (4 in the early and 2 in the late criterion failure conditions) solved the problem by accident. In other words, while manipulating the links (and having two already opened), they capitalized on the fact that one or both fell from the chain. Ten (4 in the early and 6 in the late criterion failure conditions) solved by trial-and-error — they tried different combinations, without explicitly putting aside a 3-link chain. Trial-and-error moves led to solutions as follows: Problem solvers produced the expected solution attempt under hill-climbing or by joining two chains to each end of the chain with the two already opened links. Then they proceeded by separating the resulting big chain of 9 links into two 4-link chains with one already opened end link each and the single closed link. Finally they connected each end of the two 4-link chains to the remaining 3-link chain and then opened the remaining single closed link to create the necklace. Problem solvers adopt Ormerod et al.’s (2002) heuristic for the maximization of gap closure. In the open links variant, manipulation of necklace material and cost manipulations produce more solutions. Next, having placed our results in theoretical context, we compare the predictions of extended mind and distributed cognition. 5. The experiments in theoretical context Success rate with the cheap necklace is greater with a rich external resource (i.e., a physical necklace). More participants reach insightful solutions when they

69

70

Evridiki Fioratou and Stephen J. Cowley

manipulate the necklace than in using pen and paper. In the ‘already open’ condition, we find further improvement. Solutions to the CNP thus depend on neither the person alone nor the artifacts in themselves. Rather, participants act to extend the problem space beyond the skin. While we must be cautious in generalizing from the CNP, we challenge, for example, Gick and Lockhart’s (1995) view that difficulties depend on (inner) cognitive dimensions. Equally, our findings show that insight does not always arise from execution difficulties (Schooler, Fallshore, and Fiore 1995).2 For the CNP, use of artifacts favors solutions. While thinking matters, information processing also uses real-time action and perception. Given the opportunity, participants seek solutions by linking experience with ongoing events. Since action (and accidents) contribute to what happens, insight can exploit practical and cognitive skills. The findings are consistent with Ormerod et al.’s (2002) maximization heuristic. Just as with hill-climbing, CNP moves are predictable. However, they are also in need of (momentary) deviation. Although connecting chains leads to apparent progress, this misleads. Conversely, while opening a link seems to lead off course, this must be overlooked. To use the maximization heuristic, the task must be reorganized. This is most easily achieved when already-open links fall off the chain. If noted, such events give new criteria for action. Deviant events prompt solvers to opportunistic solutions. This account of progress maximization (and failure to solve), parallels work on the Tower of Hanoi. O’Hara and Payne (1998) show that, even if they take longer to solve, participants like to avoid planning. They prefer the less cognitively demanding strategy of acting and evaluating. In parallel to the CNP most prefer to evaluate moves only after having joined chains together. Few plan or predict consequences. Indeed, even after reaching impasse, many perseverate with the same ineffective moves. 5.1 Beyond internalism Solutions to the CNP remain elusive. Unless chains fall apart, participants often use the maximization heuristic and thus fall into impasse. They will ‘see’ a solution only if a sense of failure (or similar) prompts a change in the initial strategy. Of course, this does not explain insightful solutions. One needs to connect internalist models with how co-ordination impacts on solution rates. Rupert (2002) is thus representative of a broad tradition in thinking that our cognitive powers make ‘unexpectedly rich’ use of external resources. Conceding that (inner) computation is supplemented by material entities is, he believes, sufficient to overcome the inflated claims for extended mind. In closed and open link conditions, the same props and resources sustain different solution rates. Since differences do not depend on the initial state, solutions



Insightful thinking: Cognitive dynamics and material artifacts

must draw on how participants act with links and chains. Yet, at the same time, changed conditions merely increase solution rates. Even where an ‘open chain’ is used to discourage moves to impasse, participants tend to ignore instructions, manipulations, reward values, and what they see. Many end up perseverating. Clearly, the rich environment cannot explain performance. Impasse is overcome only by using this in action. First, looking at an open chain can bring ‘undoing’ to mind. In Zhang’s (1997) terms, an object’s external representation leads to success.3 More often, however, participants ignore open links. They also overlook wordings which tell them that they are ‘free to’ open or close a link (designed to influence criterion failure). In spite of the experimenter, few use destructive action. As in the original condition, most seek to solve by acting. Making poor use of what they see, they monitor moves, what changes, and what happens. Overlooking cues, they seek a prompt to either a solution or a change of strategy. Typically, they are ‘seeded’ by movements and, perhaps, feelings. Solutions co-occur with changes in participant experience. Unlike machines that use external representations (pattern recognition), people use unexpected events and negative feelings. In some versions of the CNP, this invites opportunistic solutions or, indeed, a switch to trial and error. 5.2 Extended mind or distributed cognition? In the CNP, solutions depend on neither artifacts themselves nor what we see. While environmental factors contribute to solutions, external representations are less important than monitoring the possible effects of actions. Where events in a problem space set off insights, solutions arise from perceiving possibilities and noting the effects of current actions. We depend on experience. Problem solvers link sensorimotor routines with causal processes as they monitor perception. They anticipate. It is likely that much the same occurred when Sultan joined two sticks into a pole to reach bananas. In reaching for the bananas, we might say, experience prompted the ape to constitute a distributed cognitive system. This involves the pole insofar as material structures are necessary to assembling it out of two sticks. While not part of the mind, they prompted the chimpanzee to anticipate that the pole could alter causal process. Rupert (2004) is thus mistaken to say that artifacts ‘supplement’ computation. Rather, they become valued resources as they are integrated into action. Equally, however, we find no clear cut support for this view of extended mind: human cognitive processing literally extends into the environment surrounding the organism, and human cognitive states literally comprise — as wholes do their proper parts — elements in that environment (Rupert 2004:â•›293; cited in Clark 2008:â•›112).

71

72

Evridiki Fioratou and Stephen J. Cowley

In the CNP, artifacts do not function as elements of cognitive states. Rather participants use the causal spread of a distributed cognitive system. They act to spread agency across both the person and the chains and, as they do so, provoke opportunities for new kinds of action. Object manipulation sets off experience such as frustration and disappointment which, in synchrony with (inner) computation, enable us to anticipate possible effects. While not functioning as wholes to proper parts, moving artifacts contribute to cognitive decision making as they are integrated with action, perception and feeling. Thus links and chains are not proper parts of cognitive states. As Rupert suggests, causal spread is not sufficient to extend the mind. The artifacts of the cheap necklace do not set off insight. However, the cognitive process does use causal spread. Artifacts contribute to the actions of which they are part. While finding no reason to emphasize subjective experience, we echo Noë’s (2004) view of how acts of looking influence visual perception. When the non-visible aids insight, we use lived experience. As we act in an environment, we grasp what is possible. The task is restructured in the act. Our view of distributed cognition thus conforms to Järvilehto’s (1998) biological model. As organism-environment systems, we draw on biological, physical, and cultural events. Life history allows us to shift continuously between thinking and the world beyond the skin. Movements connect perception with events that prompt impasse, solutions or strategic change. While we may anticipate opportunistic solutions, movement is sufficient to connect simple heuristics with the anticipatory dynamics of body-world interaction. 5.3 The parallel with language In the CNP artifacts enrich a demanding cognitive task. Experimentally, links and chains contribute to solutions. This happens because, during manipulations, we seek (possible) ways out of impasse. As organism-environment systems we extend and retract our domain of attention. It is here that the CNP parallels how we create and construe language. For, as Clark (1998:â•›162) also sees, given artifacts, language need not be reduced to form-based representations (e.g., sentences, discourse, conversations, or utterances). For Clark, the ‘forms and structures’ of a language are themselves parts of cognitive states. Since material symbols are like artifacts, structural units are material. Thus language does not consist in events (e.g., conversations, shouting, or using written media). In recognizing that ‘98 is one more than 97’, material symbols are said to activate bits of mentalese/neuralese.4 Clark relies on comparing artifacts with symbols. In challenging Fodor (1987) and Churchland (1989), he denies that words map onto a content-matching code or state space. Verbal patterns differ from introspected objects (‘words’) and dictionary items (‘written words’) in not being



Insightful thinking: Cognitive dynamics and material artifacts

paired with a priori meanings. Clark rejects translation views. What, then, are the symbols? Avoiding terminological tangles, Clark (2008) exemplifies by appeal to calculation. While few can imagine 98-ness, we “appreciate that the number word 98 names a unique quantity between 97 and 99” (2008:â•›50). The ‘word 98’ (whatever that may be) is a material symbol. Suppose, then, that the token is a situated utterance of ‘novant’otto’. A person (or program) can manipulate ‘novant’otto’ to generate (what is construed as) a statement that it is one less than ‘99’. Like a computer’s physical symbol, ‘novant’otto’ functions without determinate meaning. Fair enough. Linguists will object that ‘novant’otto’ is an unusual token. While (most) readers of Italian will indeed be able to identify it with utterances of novant’otto, few will map it onto, say, nëntëdhjetë e tetë. Although ‘98’ describes both kinds of pattern, they come from different ‘languages’. The ‘98’ example relies on non-linguistic inscriptions that are associated with ‘98’ (e.g., XCVIII). In clarifying a world of words, ‘98’ contrasts with, say, ‘mama’ or ‘ancora’ in mapping onto a language-neutral written description. Numerical examples mask historical specificity. Appeal to 98-ness plays down that while Italians use ‘novant’otto’ sound-patterns, Albanians evoke ‘nëntëdhjetë e tetë’. In challenging translation models, Clark shows only that written tokens can function as material symbols. Fair enough. Yet this does not support the claim that human powers are extended by linguistic structures. Not only are number-tokens atypical, but they have striking independence from experience, development and culture. Clark thus ties language to a sub-set of inscriptions. By appeal to a written token (‘98’), he confuses a mode of description with the dynamics of language-in-action. The distributed view is simpler. The material basis of language is traced to perception and (inter)action. This is clearly seen in development. Like adult problem solvers, babies learn to talk by monitoring (joint) actions and their effects. Thus, joint attention is central in coming to construe and create what adults hear as ‘words’ (Tomasello 1999; Cowley 2007c). Mastering language uses — not material symbols — but clues to how to behave. However, once the rudiments emerge, children do come to hear speech in a verbal aspect. By age two, they repeat ‘words’ or, in Cowley’s (2007a; 2011) terms, begin to take a language stance. Given skills based in perception, they do not need material symbols. Unlike a printed ‘98’, utterances of ‘mama’ or ‘ancora’ are heard against circumstances, affect and activity. To speak, feel, and think children draw on dialogical activity. Given life-history, full-bodied dynamics are increasingly constrained by verbal patterns. While based in sensorimotor activity, effective social behavior also uses what we hear (including second-order cultural constructs). In treating language as a dynamical-verbal hybrid, a growing number follow Love (2004; 2007), Kravchenko (2007), and others in rejecting code views of language. It is lame to restrict one’s challenge to translation models. While Clark is correct to reject a priori meanings, he needlessly

73

74

Evridiki Fioratou and Stephen J. Cowley

tries to save a priori forms. Talk uses neither invariant structures nor material symbols. While tool-like, language does not consist in tools. By reifying material symbols Clark reduces experience, development and culture to disembodied skills. Extended mind remains a code view because it makes no attempt to address the question of how linguistic material is integrated with action. Clark assimilates the ‘content’ of talk to structures that can be ‘seen’ in texts (or Morse). As in translation models, he posits that ‘tokens’ (symbols) activate quasi-linguistic neural patterns. Even in connectionist tradition, such views are in retreat. For example, Elman (2004:â•›301) now thinks that ‘words’ (whatever they are) act as clues to meaning. Far from activating neuralese (or mentalese), they are physical events. Using experimental phonetics, Port (2007:â•›143) shows that neural dynamics map material utterances (and texts) onto both (inner) sense and socially defined forms. In coordinating action, we link dynamics with verbal patterns. What matter are not ‘symbols’ (in themselves) but, rather, perceived opportunities. In designing language-sensitive robots, no-one seeks to map the physical symbols of programs onto public linguistic tokens. Rather, tricks serve to ensure that the programmer and end-user adopt similar conventions. Increasingly, robots anticipate events using human dynamics (e.g., gaze). As Seabra-Lopes and Belpaeme (2008) report, they use “detailed analysis of multimodal, directed and dialogical features of language” (233). As in the CNP, far from making features part of the mind, robots monitor possible effects of human actions. Strikingly, this parallels Järvilehto et al.’s (2009) account of how anticipation drives reading aloud. Clark wrongly separates material symbols from bodies, coordination, and affect. By ignoring dynamics and life history, extended mind obscures how perceived dynamics shape opportunities for action. Linguistic material seems to function like links and chains. While making use of pattern recognition, life-history prompts us to anticipate. Linguistic events integrate real-time dynamics (speaking, hearing, looking, moving) with perceived verbal (and other) patterns. They are — not material symbols — but conversations, shouting and, in literate communities, ways of construing and creating written signs. Linguistic action is cued and constrained by Love’s (2004:â•›530) secondorder cultural constructs. Just as events move us to insight, language prompts us to anticipate what to see, hear or do. We link patterns with experience to trigger normative constraints on thinking and action. Use of predictable structure fuels cognitive powers based on anticipating meaning. In a dual mode of action, the feeling of thinking unites the physics of movement with verbal patterns. Using expression, we hear (what we call) ‘words’. Cognitive tricks ensure that insights and words do not reduce to physics.5 Organism-environment systems perceive as they act: insight and language arise from monitoring and anticipating the effects of actions (and thinking).



Insightful thinking: Cognitive dynamics and material artifacts

6. Insight spreads In the CNP, insights cannot be traced to pure computation. Since bodies and artifacts contribute to solutions, people make use of material resources. As Newell and Simon (1972) saw, insightful thinking is nothing special. The problem, rather, is that of grasping the problem. Necessary restructuring arises as inner processes are integrated with acts that generate contingencies. Insightful solutions arise as we anticipate effects. The approach thus challenges both cognitive internalism and extended mind. In real-time, artifacts can restructure events in the human parts of a person-link-chain system. This is temporary. Objects neither ‘supplement’ computation nor act as parts of a cognitive process. Rather they prompt us to use experience to anticipate solutions. The problem space extends across the body’s own lived world. In a distributed cognitive system, while causes spread, solutions emerge for a person. Our view is deflationary. Since insight uses nothing special, Clark is almost certainly correct that no new styles of neural processing subserve human language. Further, since encounters with material activate the brain, physics matters. The flaw of extended mind is that it makes links, chains and ‘words’ parts of cognitive states. For philosophical reasons, Clark overlooks real-time events. Instead of seeing that humans become skilled users of cognitive dynamics, he ascribes their powers to material symbols. In this brand of functionalism, the body can be replaced by mentalese/neuralese. Our findings, however, show the importance of how people act and, thus, anticipate (possible) effects. Playing down the necklace (and verbal) material, we emphasize how body-world coupling generates ways of acting, perceiving and inhibiting. In short, it prompts us to anticipate. This is ironic. Just as Rupert (2004) sees artifacts as an unexpectedly rich ‘add on’ to a computational space, Clark (2008) treats life-history as an ‘add on’ to a public language. Both erroneously trace reasoning to individuals. On the distributed view, by contrast, due weight is given to cultural practices. Using simple heuristics, lifehistory enables us to discover planning. By mid-childhood, we can integrate our activities with culturally derived strategies (e.g., we may read). Language and insight arise as we integrate bodily dynamics with feelings and events. Other evidence supports this claim. First, insight can arise in a paper-andpen version of the CNP. In these high quality cases, we inhibit use of action-driven feelings and accidents. Although we prefer to act, we can use strategies for looking ahead. Second, even when given links and chains, many perseverate with the naïve solution attempt. Clearly, we need cognitive skills to manage what we see and feel. Insight draws on rewards that prompt us to act on the basis of expectations. In the CNP, this happens when participants feel-think (roughly) “Darn it, I will just try every move”. They exploit how the world ‘leaks’ as they are prompted to use trial

75

76

Evridiki Fioratou and Stephen J. Cowley

and error. For Dartnall (2005) inward leakage gives us the mental imagery that serves to ‘discover’ more than we know. As with insight, our powers depend on monitoring (inner) action for (likely) beneficial effects. We not only use artifacts and the world but, as a result, exploit affect marked experience. We act to ensure that body-world dynamics prompt insights, imagined words, and pictures of the (im)possible. Our skills are ‘internalized’ or, more precisely, we draw on experience that has been shaped by the dynamics of normative events. As we learn to act in a cultural and normative world, mind seeps into the head.

Notes *╇ The experiments reported here were conducted as part of the PhD research of the first author, funded by a Lancaster University studentship. We also thank Ken Gilhooly, Fred ValleeTourangeau, and Karl MacDorman for their extensive comments on earlier versions of this paper. 1.╇ On the view presented here, we can see this in terms of two kinds of body-world embedding; in one case, we need to find a novel outcome; in the other, we draw on well-worn routines. 2.╇ Where insight does not depend on action, this may apply. By implication ‘insight problems’ would be detached from goals. Echoing Gick and Lockhart (1995), they might be defined as problems where (putative) extant representations serve to construct novel representations then the solver encounters trivial difficulty with their accessed (or explicit) counterparts. 3.╇ For example, Zhang (1997) tested people in the game of 15, where two players take turns selecting numbers from 0 to 9. The goal is to be the first to select three numbers that sum to 15. As in tic-tac-toe, the problem offers a rich external representation: people use a 3x3 grid to take turns writing their symbol, often X or O, until they achieve a row. The game of 15 actually proceeds from an internal representation that fosters poorer and slower appreciation of what is required for a draw than traditional (external) tic-tac-toe. Such representational effects illustrate that problems isomorphic at some level of description vary significantly in terms of the richness of the information embedded in their physical presentation. Hence the significant differences in problem solving strategies between the tic-tac-toe and the game of 15. 4.╇ It might be argued that links activate bits of mentalese/neuralese that correspond to abstract configurations. We read the experiments as showing that our action-perception model is likely to be simpler. 5.╇ For Dennett (1991) virtual patterns are ‘installed’ in the brain. Clark (2006) rejects Dennett’s view as too ‘developmental’ and posits, in its place, ‘shallow imagistic encodings’. In the distributed view, second-order constructs are latent in hearing (like opportunities glimpsed as we manipulate the necklace). After infancy, we live in a world of virtual patterns where bodies act as a medium through which computations function because their results enable us to use ‘reason’ to self-construct as persons who act, speak, and hear in accordance with their beliefs (see, Ross 2007; Cowley and MacDorman 2007; Cowley 2007a).



Insightful thinking: Cognitive dynamics and material artifacts

References Bowers, K.S., Farvolden, P., and Mermigis, L. 1995. “Intuitive antecedents of insight”. In S.M. Smith, T.B. Ward, and R.A. Finke (eds), The Creative Cognition Approach. Cambridge, MA: The MIT Press, 27–52. Churchland, P. 1989. The Neurocomputational Perspective. Cambridge, MA: The MIT Press, Bradford Books. Chronicle, E.P., MacGregor, J.N., and Ormerod, T.C. 2004. “What makes an insight problem? The roles of heuristics, goal conception, and solution recoding in knowledge-lean problems”. Journal of Experimental Psychology: Learning, Memory, and Cognition 30: 14–27. Clark, A. 1998. “Magic words: how language augments human computation”. In P. Carruthers and J. Boucher (eds), Language and Thought: Interdisciplinary Themes. Cambridge: Cambridge University Press, 162–183. Clark, A. 1997. Being There: Putting Brain, Body and World Together Again. Cambridge, MA: The MIT Press. Clark, A. 2006. “Material symbols”. Philosophical Psychology 19/3: 1–17. Clark, A. 2008. Supersizing the Mind: Embodiment, Action and Cognitive Extension. Oxford: Oxford University Press. Clark, A. and Chalmers, D. 1998. “The extended mind”. Analysis 58: 7–19. Cowley, S.J. 2007a. “How human infants deal with symbol grounding”. Interaction Studies 8 (1): 81–104. Cowley, S.J. 2007b. “Cognitive dynamics and distributed language”. Language Sciences 29, (5): 575–583. Cowley, S.J. 2007c. “The cradle of language: making sense of bodily connexions”. In D. MoyalSharrock (ed), Perspicuous Presentations: Essays on Wittgenstein’s Philosophy of Psychology. London: Palgrave MacMillan, 278–298. Cowley, S. J. 2011. “Taking a language stance”. Ecological Psychology, 23/3:185–209. Cowley, S.J. and MacDorman, K.F. 2006. “What baboons, babies, and Tetris players tell us about interaction: A biosocial view of norm-based social learning”. Connection Science 18(4): 363–378. Dartnall, T. 2005. “Does the world leak into the mind? Active externalism, internalism and epistemology”. Cognitive Science 29: 135–143. Deacon, T. 1997. The Symbolic Species: Co-evolution of Language and the Brain. London: Norton. Dennett, D. 1991. Consciousness Explained. Boston: Little Brown. Donald, D. 1991. The Origins of the Modern Mind. Cambridge, MA: Harvard University Press. Duncker, K. 1945. “On problem solving”. Psychological Monographs 58, Whole No. 270. Ellen, P. 1982. “Direction, past experience, and hints in creative problem solving: A reply to Weisberg and Alba”. Journal of Experimental Psychology: General 111: 316–325. Elman, J.L. 2004. “An alternative view of the mental lexicon”. Trends in Cognitive Science 8: 301– 308. Fioratou, E. 2006. Cognitive Processes in the Cheap Necklace Probem. Unpublished PhD dissertation, University of Lancaster. Fodor, J. 1987. Psychosemantics: The Problem of Meaning in the Philosophy of Mind. Cambridge, MA: The MIT Press. van Gelder, T. 1998. “The dynamical hypothesis in cognitive science”. Behavioral and Brain Sciences 21: 615–628.

77

78

Evridiki Fioratou and Stephen J. Cowley

Gick, M. L., R. S. Lockart. 1996. “Cognitive and affective components of insight.” In R. J. Sternberg, J. E. Davidson, (eds) The Nature of Insight. MIT Press, Cambridge, 157–196. Gilhooly, K.J. and Fioratou, E. 2007. Verbalisation and Working Memory Aid Effects on Insight and Non-Insight Problem Solving. Poster presented at the BPS Annual Conference, York, 21–23 March. Gilhooly, K.J., Fioratou, E., and Henretty, N. 2010. “Verbalization and problem solving: Insight and spatial factors”. British Journal of Psychology, 101: 81–93. Gilhooly, K.J., and Murphy, P. 2005. “Differentiating insight from non-insight problems”. Thinking & Reasoning 11: 279–302. Glenberg, M. 2008. “Radical changes in cognitive process due to technology: A jaundiced view”. Pragmatics & Cognition 14: 263–274. Goodwin, C. 2000. “Action and embodiment within situated human interaction”. Journal of Pragmatics 32: 1489–522. Greeno, J.G. 1978. “Natures of problem-solving abilities”. In W.E. Estes (ed), Handbook of Learning and Cognitive Processes: Human Information Processing (Vol. 5) Hillsdale, NJ: Erlbaum, 239–270. Harnad, S. 2005. “Distributed processes, distributed cognition and collaborative cognition”. Pragmatics & Cognition 13: 501–514. Hodges, B. 2009. “Ecological pragmatics: Values, dialogical arrays, complexity and caring”. Pragmatics & Cognition 17(3): 628–652. Hutchins, E. 1995. Cognition in the Wild. Cambridge, MA: The MIT Press. Järvilehto, T. 1998. “The theory of the organism-environment system 1. Description of the theory”. Integrative Behavioural and Physiological Science 33: 321–344. Järvilehto, T., Nurkkala, V.M., and Koskela, K. 2009. “The role of anticipation in reading”. Pragmatics & Cognition 17(3): 509–526. Kaplan, C.A. and Simon, H.A. 1990. “In search of insight”. Cognitive Psychology 22: 374–419. Knoblich, G., Ohlsson, S., Haider, H., and Rhenius, D. 1999. “Constraint relaxation and chunk decomposition in insight problem solving”. Journal of Experimental Psychology: Learning, Memory, and Cognition 25:1534–1555. Kravchenko, A. 2007. “Essential properties of language, or, why language is not a code”. Language Sciences 29: 650–671. Koffka, K. 1935. Principles of Gestalt Psychology. New York: Harcourt, Brace and World. Köhler, W. 1925. The Mentality of Apes. New York: Liveright. Laland, K. and Sterelny, K. 2006. “Seven reasons not to neglect niche construction”. Evolution 60: 1751–1762. Love, N., 2004. “Cognition and the language myth”. Language Sciences 26: 525–544. Love, N. 2007. “Is language a digital code?”. Language Sciences 29 (5): 690–709. MacGregor, J.N., Ormerod, T.C., and Chronicle, E.P. 2001. “Information processing and insight: A process model of performance on the nine-dot and related problems”. Journal of Experimental Psychology: Learning, Memory, and Cognition 27: 176–201. Maier, N.R.F. 1931. “Reasoning in humans: II. The solution of a problem and its appearance in consciousness”. Journal of Comparative Psychology 12: 181–194. Metcalfe, J. 1986a. “Feeling of knowing in memory and problem solving”. Journal of Experimental Psychology: Learning, Memory, and Cognition 12: 288–294. Metcalfe, J. 1986b. “Premonitions of insight predict impending error”. Journal of Experimental Psychology: Learning, Memory, and Cognition 12: 623–634.



Insightful thinking: Cognitive dynamics and material artifacts

Metcalfe, J. and Wiebe, D. 1987. “Intuition in insight and noninsight problem solving”. Memory & Cognition 15: 238–246. Noë, A. 2004. Action in Perception. Cambridge, MA: The MIT Press. Newell, A. and Simon, H.A. 1972. Human Problem Solving. Engelwood Cliffs, NJ: Prentice-Hall. Newell, A., Shaw, J.C., and Simon, H.A. 1958. “Elements of a theory of human problem solving”. Psychological Review 65: 151–166. O’Hara, K.P. and Payne, S.J. 1998. “The effects of operator implementation cost on planfulness of problem solving and learning”. Cognitive Psychology 35: 34–70. Ohlsson, S. 1992. “Information-processing explanations of insight and related phenomena”. In M. Keane and K.J. Gilhooly (eds), Advances in the Psychology of Thinking. London: Harvester-Wheatsheaf, 1–44. Ormerod, T.C., MacGregor, J.N., and Chronicle, E.P. 2002. “Dynamics and constraints in insight problem solving”. Journal of Experimental Psychology: Learning, Memory, and Cognition 28: 791–799. Port, R. 2007. “How are words stored in memory? Beyond phones and phonemes”. New Ideas in Psychology 25: 143–170. Ross, D. 2007. “H. sapiens as ecologically special: what does language contribute?”. Language Sciences 16 (1): 710–731. Rupert, R. 2002. “Challenges to the hypothesis of extended cognition”. Journal of Philosophy 101: 389–428. Schooler, J.W., Fallshore, M., and Fiore, S.M. 1995. “Epilogue: Putting insight into perspective”. In R.J. Sternberg and J.E. Davidson (eds), The Nature of Insight. MIT Press, Cambridge MA, 559–587. Seabra Lopes, L. and Belpaeme, T. 2008. “Beyond the individual: new insights on language, cognition and robots”. Connection Science 20 (4): 231–237. Silveira, J.M. 1971. Incubation: The Effect of Timing and Length on Problem Solution and Quality of Problem Processing. Unpublished PhD dissertation, University of Oregon. Simon, H.A. 1978. “Information-processing theory of human problem solving”. In W.K. Estes (ed), Handbook of Learning and Cognitive Processes; Vol. V. Hillsdale, NJ: Erlbaum, 271– 295. Smith, S.M. 1995. “Getting into and out of mental ruts: A theory of fixation, incubation, and insight.” In Sternberg and Davidson (eds), The Nature of Insight. MIT Press, Cambridge, MA, 229–251. Sternberg, R.J. and Davidson, J.E. (eds). 1995. The Nature of Insight. Cambridge, MA: The MIT Press. Thibault, P. 2004. Brain, Mind and the Signifying Body. Continuum: London. Tomasello, M. 1999. The Cultural Origins of Human Cognition. Cambridge, MA: Harvard University Press. Varela, F.J., Thompson, E., and Rosch, E. 1991. The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: The MIT Press. Weisberg, R.W. 1995. “Prolegomena to theories of insight in problem solving: A taxonomy of problems.” In R.J. Sternberg and J.E. Davidson (eds), The Nature of Insight. MIT Press, Cambridge, MA, 157–196. Weisberg, R.W. and Alba, J.W. 1981a. “An examination of the alleged role of ‘fixation’ in the solution of several “insight” problems”. Journal of Experimental Psychology: General 110: 169–192.

79

80

Evridiki Fioratou and Stephen J. Cowley

Weisberg, R.W. and Alba, J.W. 1981b. “Gestalt theory, insight, and past experience: A reply to Dominowski”. Journal of Experimental Psychology: General 110: 199–203. Weisberg, R.W. and Alba, J. W. 1982. “Problem solving is not like perception: More on Gestalt theory”. Journal of Experimental Psychology: General 111: 326–330. Weisberg, R.W. and Suls, J.M. 1973. “An information-processing model of Duncker’s candle problem”. Cognitive Psychology 4: 255–276. Wertheimer, M. 1959. Productive Thinking. New York: Harper & Row. Wickelgren, W.A. 1974. How to Solve Problems. San Francisco: Freeman. Zhang, J. 1997. “The nature of external representations in problem solving”. Cognitive Science 21: 179–217. Zhang, J. and Norman, D. 1994. “Representations in distributed cognitive tasks”. Cognitive Science 18: 87–122.

Actualizing semiotic affordances in a material world* Kristian Tyléna,b, Johanne Stege Bjørndahla, and Ethan Weedb,c University of Aarhus, Denmark

This paper investigates a special kind of social meaning-making manifest in how we experience static objects and properties of our everyday environment. This happens, for example, when we encounter objects like vacuum cleaners, sliced tomatoes, and sneakers that by their contextual arrangements seem to afford an intersubjective, semiotic style of exploration. It is argued that our intersubjective experience of such object configurations is brought about by attention to special ostensive qualities of composition: Certain compositional properties are experienced as purposefully constructed to afford an intersubjective mode of perception. This significantly changes the perceiver’s semiotic exploration of the scene. From a ‘private’ mode of sense-making mostly structured by reference to episodic, autobiographical experiential content, the perceiver adopts a qualitatively different meaning-constructing strategy: Engaging as an active meaning-pursuing recipient, the perceiver actualizes ‘public’ intersubjective meaning potentials of the scene. This process is guided by sensitivity and alignment to norms. Defending this claim, we present evidence from an empirical investigation of 20 participants’ construals of photographic images depicting everyday static objects. We show that a subset of these object configurations (signals) evoke a special kind of socially responsive attitude as manifested in participants’ introspective reports. The importance of these findings is brought out by discussion of parallels in neuro-cognitive work and how ostensive cues influence infant behaviour. *╇ This is a revised version of Tylén et al (2009). Taking a Language Stance in a Material World: a comprehension study, Pragmatics & Cognition, 17(3):573–595. (Corresponding author) Email: [email protected] a.╇ Center for Semiotics, University of Aarhus, Jens Chr. Skous Vej 7, 8000 Aarhus, Denmark; b.╇ Center for Functionally Integrative Neuroscience, Aarhus University Hospital, Nørrebrogade, 8000 Aarhus C, Denmark; c.╇ Department of Linguistics, University of Aarhus, Ndr. Ringgade, Building 1410, DK-8000 Aarhus C.

82

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

Keywords: material signals, semiotic affordances, intersubjectivity, conventionality, object cognition

1.0 Introduction In this investigation we focus on a special type of object-mediated communication, that is, those instances when we happen to perceive everyday objects like chairs, sprinkles of flour, or vacuum cleaners, as means of social coordination and construction of intersubjective meaning. This is accomplished when we recognize such objects or properties of our material environment as manipulated or arranged in a striking and deliberate manner that affords a special kind of semiotic exploration.1 For instance, we can encounter chairs put out in the street to reserve a parking lot, flour sprinkled out in the shape of an arrow to indicate a direction, or rows of vacuum cleaners lined up on a lawn to create a conceptual piece of art. On a fundamental plane, this novel mediating function depends on context. There is no such thing as a contextless perception of an object. An object is always perceived from a certain perspective, in a certain illumination, giving it a certain profile. Further, we bring all our previous experiences and current interests, valences and feelings to the encounter. Together, these give shape to perceptual experience. Perception of an object is thus jointly constituted by its inherent affordances and the immediate relevance brought to the encounter by the perceiver. Generally, this continuity between perception, thinking and action anticipation brings meaningfulness to our experiences (Gallagher and Zahavi 2008). However, some cases of object perception cannot be exhaustively described in terms of such “episodic”, first-person experiences. Sometimes we encounter objects and properties where the recognized intervention of another agent gives them a special relevance to us (Tylén, 2007). Specifically, the context is constructed for us in ways that profile social meaning-potential in the contextualized object. Examples of such constructed semiotic affordances include seemingly deliberate placement of objects in new and unfamiliar contexts, manipulation of properties such as color or shape and arrangement of common objects in a novel composition (e.g. a symmetrical configuration) (Tylén, 2007). In such cases, perception is structured by more than how the object’s affordances bear on the state of mind and past experiences that we bring to the subject-object encounter. Rather, we argue, the recognition of the scene’s purposeful compositionality sets off an intersubjective attitude in the perceiver. Making sense of the object scene is heavily scaffolded by another, addressing agent. The effect of the experienced ostensive quality of a situation on perceptual meaning-making is studied in a series of recent behavioral experiments of infant



Actualizing semiotic affordances in a material world

cognition. In these studies (e.g. Gergely et al., 2007; Senju & Csibra, 2008; Senju et al., 2008), infants are found to respond differently to adult object manipulations when they are themselves uninitiated observers in contrast to when they are engaged as active recipients. When an infant was experimentally framed as a thirdperson observer, adult behaviors were taken to express the dispositions of a specific experimenter (e.g. “Jane likes broccoli”). In contrast, when the experimenter addressed the infant ostensively (such as making eye-contact, nodding and vocalizing in ‘motherese’), object-oriented actions were comprehended as providing socially shared information about the world (e.g. “Broccoli is good!”). The participatory engagement of the infant as an active recipient (rather than an observing by-stander) thus has dramatic effects on her sense-making attitude (Gergely et al, 2007). Rather than relying on a ‘private’ and episodic style of perception, the interactive recontextualization of bodily actions and objects makes them mediators of shared ‘public’ meanings (Cowley et al 2004; Cowley 2007; Tylén & Allen, 2009). We argue that such ostensive qualities of intersubjective encounters can be extended to our engagements with particular types of static object configurations. Prompted by seemingly purposeful contextual and compositional properties, our perception ceases to be ‘private’ and we engage as second-person recipients who actualize semiotic potentials. Such active meaning-pursuing engagements arise from viewing static object configurations as acts of signaling. In effect, we direct attention to the objects’ situated, recontextualized value as mediating ‘public meanings’ whereby new details of the material scene become semiotically relevant (Tylén, 2007; Tylén & Allen, 2009). Perception becomes a kind of co-action whereby ostensive qualities of a scene’s compositionality afford receptive social engagement. Further, given the fundamental asymmetry of the situation (the material signals considered here are inherently monological) the perceiver comes to rely on normative strategies to regulate semiotic attunement and interpretative actions (Pickering and Garrod, 2004; Itkonen, 2008).2 This distributed approach to material signals finds support in recent neurocognitive studies showing that many forms of communicative mediation including bodily gestures and material objects resonate in brain areas, traditionally thought to be ‘language areas’ (cf. Tylén & Allen, 2009; Tylén et al, 2009; Roepstorff, 2008; Spurrett and Cowley, 2004). In parallel, there is a widening consensus that various expressive means may have common onto- and phylogenetic origins (Zlatev, 2008; Donald, 2002; Arbib, 2005).

83

84

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

2.0 The recognition and comprehension of material signals The conceptual idea gives rise to predictions that can be investigated empirically. In a recent event-related fMRI brain imaging study Tylén and colleagues presented two contrastive types of images to twenty-two persons in a MR brain imaging system (Tylén et al., 2009). The stimuli consisted of a hundred photographic images of static everyday objects-in-context that were each presented for five seconds in a randomized order. In the experimental condition, images depicted objects configurations that called for an intersubjective interpretation (e.g. chairs put out in the street to reserve a parking place or a bunch of flowers left on the doorstep of a private home to express a declaration of love, etc.). In the control condition the same objects were shown in canonical, instrumental, or accidental (non-communicative) contexts. The conditions thus contrasted objects experienced as intersubjective signals and objects that lack communicative relevance. Data analysis was based on the participants’ own post scan classification of the stimuli images (communicative vs. non-communicative) in consideration of potential subjective differences in their understanding of the scenes. The experimenters found that when participants made communicative interpretations and explored the object configurations as sources of intentional social meaning they had enhanced activity in part of the fusiform gyrus (a structure also known as the Visual Word Form Area3 and, bilaterally, in the pars triangularis of the inferior frontal gyrus (part of Broca’s area, BA 45). Further, the activation of right hemisphere inferior frontal gyrus showed sensitivity to the degree of conventionality of the expressive signals (the unconventional ones causing enhanced activity) as rated post hoc by the participants themselves. These same areas (especially left hemisphere Broca’s area) are consistently found in studies of verbal language and reading while the right inferior frontal gyrus has been shown to be modulated by novel unconventional metaphors and figurative language that tend to depend more on contextual integration.4 The results complement a growing number of cognitive neuroimaging studies that find Broca-activity associated with a variety of non-verbal, expressive activities like hand, body and facial gestures (e.g. Lotze et al, 2006, Lawrence et al, 2006), non-verbal vocalizations (Dietrich et al, 2007), musical patterns (Vuust et al, 2005, 2006), and joint action (Newman-Norlund et al, 2008). Rather than invoke an innate ‘verbal language-’ or ‘syntax module’ (c.f. Pinker, 1994 and Fodor, 1983), Broca’s area seems to be generally involved in interactive sense-making and meaning construction. Apparently, it contributes to a variety of modalities and expressive means including semiotic configurations of everyday objects (Tylén et al, 2009; Tylén & Allen, 2009).



Actualizing semiotic affordances in a material world

While there is strong evidence for qualitative differences in the perception of objects as signals vs. non-signals on the level of neuro-dynamics, in what follows, we extend the investigation to another index of participants’ (inter)subjective experiences of such scenes. In short, we use elicited verbal construals of depicted scenes to investigate meaning-constructing attitudes to signals and non-signals. We predicted that the perceptual differences found in the fMRI-study would parallel how people construe image descriptions in different conditions. Further, a more qualitative style approach was expected to facilitate detailed studies of how social norms and attitudes shape meaning construction. The study focuses on two central hypotheses. The first is categorical: – there will be fundamental differences in how people categorize signals vs. non-signals. The second is semantic: – different types of signals will elicit different meaning-pursuing attitudes. Informed by the brain imaging study, we also expected the degree of conventionality of expressive practices to predict variance in the meaning attributed to the signals. 2.1. The experiments The experiment was based on verbal construals by twenty healthy participants (10 females/10 males, mean age 26 ± 6 (STD)) who described visual scenes featuring various object configurations. Of these, only some were expected to afford the intersubjective exploration sketched above. Building on the hypotheses, we aimed, first, to ascertain whether the conceptually identified distinction between firstperson subjective (non-communicative) and second-person intersubjective (communicative) experiences was manifest in oral reports of the participants’ comprehension of scenes. Second, we aimed to identify trends and normative patterns in the meaning-constructing strategies applied in interpreting material signals with varying degrees of conventionality. Some view the comprehension of material signals as part of a perceiver’s (inter)subjective phenomenal experience which lies beyond experimental reach. However, following Gallagher (2003), we systematically investigated participants’ experiences of object scenes using verbal, phenomenological reports. The task was formulated with an ‘open’ instruction (“describe the stimuli and how you understand them”) that has been shown to motivate free descriptions that, without establishing strong interpretative biases, involve more than the listing of depicted objects. As shown below, the procedure gave us a rich data set suited to both qualitative and quantitative analyses.

85

86

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

Stimuli consisted of photographic representations depicting a variety of scenes featuring everyday objects collected from the local environment and photo sharing sites such as www.flickr.com and www.polfoto.dk. While no persons were depicted, we included only images of actually occurring scenes. The stimulus images were divided into three conditions based on a previous classification study. Twenty-two participants (not those of the present study) were instructed to make relevant distinctions in images of object configurations. Using an electronic questionnaire, they rated a raw sample of one hundred images on two parameters: 1) the presence/non-presence of communicative signals, and 2) the degree of conventionality of the signals (see Tylén et al., 2009, for details). Conventionality was defined by relative frequency or how often participants expected to encounter a certain type of material signal. Statistical analysis of the data allowed us to select the eighteen best exemplars, six for each of three predefined conditions: – Experimental condition I: images that were found to depict conventional ways of employing material objects as communicative signals. Examples: a buoy, marking the location of fishing nets, a large crowd of roses left on the pavement left in grief of a person killed there, and flour sprinkled out as an arrow to show directions. – Experimental condition II: images that were found to depict unconventional ways of using material objects as communicative signals. Examples: a crowd of vacuum cleaners arranged on a lawn to make a conceptual piece of art, thin slices of tomato in a patterned distribution on the windscreen of a car, and an iceberg covered with red paint. – The control condition: images that were found by the participants not to evoke any conspicuous communicative interpretations. Examples: colorful laundry hung out to dry, a blender on a kitchen table, and a fire extinguisher in a messy cupboard (see Figure 1). Stimulus images were presented in a randomized order on a 13.3” computer monitor using Microsoft PowerPoint. Each image was shown for sixty seconds and participants were instructed to speak while watching the images. Their oral reports were recorded using a Sony stereo IC Recorder (model ICD-SX56). The procedure provided a corpus of approx. six hours of audio recordings that were transcribed using CLAN software (MacWhinney, 2000). Analyses were carried out on the transcribed material. Overall, the methodology enabled us to test whether the conceptually grounded distinctions (partly validated in the classification study) elicit significant differences in participants’ unguided, casual experience of different types object scenes. We of course recognize that the experimental procedures may not reliably reflect



Actualizing semiotic affordances in a material world

Figure╯1.╇ Examples of stimulus images from each of the three conditions. First row depicts Experimental stimuli I: conventional ways of employing objects as signals. Second row depicts Experimental stimuli II: unconventional ways of employing objects as signals, and third row depicts Control stimuli: scenes without any conspicuous communicative relevance.

naturally occurring ecological subject-object encounters. Oral construal of images reflects on a participant’s cooperativeness, verbal proficiency and ability to access experiential content. Nonetheless, by using a within-subject design to present stimuli in contrasting conditions, we could contrast descriptions across identical experimental settings. While an open question is how such differences apply to ‘real-life’, we consider it important that our categories are validated by neuroscientific findings.

87

88

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

3.0 Data analysis Our data coding was informed by Content Analysis (Neuendorf, 2002). The procedure depends on a priori identification of variables that are associated with predetermined hypotheses. Raw data (e.g. participants’ verbal construals) can then be coded in relation to predetermined variables. Once this is done, the coded material can be summarized and tested for differences in distribution across conditions. In this study, two such analyses captured different aspects of image descriptions. In addition, we used qualitative semantic analyses to identify subtleties in the participants’ explorations of the scenes. The first coding procedure aimed to assess categorical intuitions (signal vs. non-signal) in relation to images from each of three stimulus conditions. We predicted this would be reflected in the distribution of specific lexical items in verbal reports. Accordingly, we sought communication- and meaning-related terms and discursive markers (e.g. references to addresser and addressee relations) of the kind that appear in the representative extracts below. These describe a control (Figure 1g) and an experimental scene (Figure 1c) respectively: – “Oh, it looks like your washing does when you’re on holidays at a cabin or somewhere in Sweden. I can definitely see some pretty polka dot underwear. It all looks very charming and idyllic.” – “You are in a forest and there is an arrow drawn in white chalk, I think, that points to the right, so you’re probably not supposed to go up the small path that is straight ahead (…)” Although the construals target static scenes in identical experimental settings, the participants construe what they see quite differently. The verbal construal of the control scene (cf. stimulus image Figure 1g) is apparently not aiming intersubjective contextualization. The participant’s sense-making relates what is depicted to private, experiential associations and preferences. In this sense, it is episodic or unmotivated by any appeal to intentional compositional or manipulatory intervention of another agent. In contrast, the interpretative attitude manifested in the second example (in the description of Figure 1c), is regulated by the recognized ostensive quality of the items depicted. Properties of the scene are seen as constructed for someone else, thus profiling a communicative context for perceptual exploration. Unlike the episodic sample, this is intersubjective. The depicted items are attributed with imperative, instructional semantic meanings. These are reflected, for example, in wordings like “pointing to the right …” or the modal, deontic “you’re probably not supposed to …”. The first coding procedure (see below) will focus on the contrast between episodic and intersubjective construals.



Actualizing semiotic affordances in a material world

Other evidence shows that experimental scenes afford socially normative contextualizations. This is seen, for example, in how participants tend to conform in use of interpretative strategies. When we go beyond the listing of items which initiates most construals (in all three conditions), descriptions of control scenes (e.g. the laundry scene above) take off in directions that seem to be motivated by individual dispositions and free associations (to the weather, autobiographical memories, holidays in Sweden, etc.). In contrast, descriptions of the same experimental scenes are much more similar. This is illustrated by representative extracts below of construals of fig.1d, depicting a crowd of old vacuum cleaners lined up on a villa lawn: – …And then there’s a little… a little…hmm, what is that? A little broom, I think. Some dictator or other that’s leading the vacuum-cleaner army to war… – And then there’s an arrow that’s pointing out toward the picture in the foreground so that it looks as though all of these vacuum-cleaners are about to march someplace. They’re sort of standing in rank, almost, and then there’s one in front, a vacuum out in front that looks like it’s leading the others. Both participants note figurative resemblances between the vacuum cleaners and an army of soldiers. Interestingly, this image elicits similar iconic analogies in most participants (14/20) who appeal to either an army or a cemetery. Further the tendency to adopt specific meaning-constructing strategies (e.g. an iconic strategy) applies generally in experimental conditions. To explore this, measurement was made of alignment effects in interpretative strategies Coding procedure I: categorical resolutions To capture fundamental differences in verbal construal of the object-scenes, three experimenters coded the reports using a predefined coding scheme based on five variables: a) communication; b) semantics/social meaning; c) addresser; d) addressee; e) non-communicative action/activity (see Table 1). To achieve a high level of reliability, the scheme was designed around easily recognizable, explicit lexical items. Verbal language is a flexible means of interpersonal meaning making, and explicit lexicalization is only one out of a series of parameters that could be used in this regard. However, in priority of methodological reliability, we chose this rather conservative strategy that gave us a stringent and objective mode of investigation. Codings by the three experimenters were summarized in a weighted fashion for each stimulus image and each participant. Each description of an image was rated in relation to the variables ranging from none (0%), to one (33%), or two (66%) or three experimenters (100%). In other words, a 100% score would be given if all three coders assigned the variable (e.g. communication) to an image

89

90

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

Table╯1.╇ Coding scheme I. Code

Description

Examples

A

Explicit references to communication terms

Nouns: ’sign’, ’art’, ’symbol’, ’message’, etc., or verbs: ’express’, ’say’, ’instruct’, ’show’, etc.

B

Explicit references to meaning

Social functions, modals, motivated figurative or metaphorical meanings, motivated references to objects or situation not present in the image.

C

Explicit references to the discoursive marker ’addresser’

Grammatical subjects for communication related verbs like ’someone’, ’an artist’, ’you’, etc.

D

Explicit references to the discoursive marker ’addressee’

Grammatical indirect objects of communication like ’someone’, ’you’, ’people’, etc.

E

Explicit references to non-communicative activity

Verbs like ’build’, ’hang up’, ’put on’, ’make’, etc.

Coding scheme for the multi-coder analysis of image descriptions. Three coders independently scored participants’ image descriptions in relation to the five variables, defined in the scheme. In order to accomplish a high level of objectivity (and inter-coder reliability) the scheme was designed primarily to designate easily recognizable, explicit lexical items.

description. Ratings were then averaged for each image across participants. Statistically significant differences across the three conditions (experimental I, experimental II and control) were measured for each variable using a one-way ANOVA. Furthermore, a set of planned post hoc analyses were executed using Tukey’s HSD multiple-comparison test. Both types of analyses were performed in MatLab (Mathworks inc., Sherborn, Massachusetts, USA). Inter-coder reliability between the three coders was measured pairwise for each variable and each participant in percent agreement (PA) and Cohen’s kappa (κ) (Cohen, 1960) performed in PRAM v 0.4.5 (Skymeg Software, Inc.) and then averaged across participants, coders and variables (cf. Neuendorf, 2002, for a discussion on reliability measurements). 3.1 Results of coding procedure I The one-way ANOVA showed no significant difference in the number of words used to describe images across conditions (F(2,15) = 2.86, p = .09). Therefore, significant differences in other measures cannot be attributed to variations in the length of participants’ reports. Each variable except ‘non-communicative activity’ showed significant differences across the conditions. Figures were as follows: communication: F(2/15) = 8.45, p = .0035; semantics: F(2/15) = 13.64, p = .0004; addresser: F(2/15) = 6.57, p = .0089; addressee: F(2/15) = 3.88, p = .0439; and non-communicative activity:



Actualizing semiotic affordances in a material world

F(2/15) = 0.68, p = .52 (see table 2). This supports our hypothesis. Next, to explore the two experimental conditions (conventional and non-conventional signals), we used set of pair-wise post hoc Tukey’s HSD multiple comparison tests (threshold = p < .05). No significant effects were found. Although the two conditions show a strong trend in relation to semantics (experimental I: M = 59, vs. experimental II: M = 39), the variability of the data made this non-significant. Thus, while the predefined variables of the coding scheme successfully captured differences between construing object scenes as communicative signals or non-signals, they did not show contrasts between the two experimental conditions (conventional vs. unconventional signals). Inter-coder reliability was measured to percent agreement (PA) = 86 and Cohen’s kappa (κ) 0.56 (Cohen, 1960), corresponding to a ‘fair to good’ agreement, (Banerjee et al., 1999). Since reliability is satisfactory, this gives validity to the statistical analyses (Neuendorf, 2002).

Figure╯2.╇ Diagram representing differences in the mean distribution of the three conditions in relation to each of the five variables (‘Test I’ corresponds to Exp. I; ‘Test II’ corresponds to Exp. II). The y-axis represents a summery of the three coders’ weighted ratings of participants’ image descriptions in percent. Error bars express standard error of the mean (SEM). All communication-related variables (communication, semantics, addresser, and addressee) showed significant effect in the one-way ANOVA (pâ•›<â•›.05), while the communication-neutral variable (non-semiotic activity) did not.

91

92

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

3.2 Coding procedure II: Semiotic strategies While the first procedure explores the ‘signal vs. non-signal’ contrast, the second focuses on different kinds of meaning-constructing strategies. The analysis is motivated by the differences in the participants’ relative inclination to align their interpretative strategies to images of the experimental and control conditions. As indicated by the results from the first coding procedure, properties of the experimental scenes (but not control scenes) afford semiotic styles of interpretative explorations. These seem guided by sensitivity to intersubjective norms. Next, therefore, we assess how participants attune to quite similar meaning-constructing strategies in their construal of images of the experimental conditions. Focusing on social/semantic meanings, we look more closely at the relevant image descriptions. Since such questions relate to subsamples of the data (i.e. small sample sizes), we proceed in a rather qualitative fashion. Hypotheses are thus formulated post hoc and analyses motivated more subjectively. The interpretative work involved in attributing social meaning to a red iceberg (cf. Figure 1f) differs fundamentally from that applied to a flour-arrow on a forest path (cf. Figure 1c) or roses left on the street (cf. Figure 1b). In coding we, therefore, asked how semiotic attitudes regulate meaning construction. In construing material signals, we find strategies that recall Peirce’s (1998) classification of sign types as iconic, indexical and symbolic. They are: 1) An iconic/aesthetic strategy where attention is put on objects’ resemblances and diagrammatic affordances (Stjernfelt, 2007): (e.g. a red iceberg is associated with blood). 2) An instructional strategy, where object configurations are experienced as social imperatives, pointing to something or instructing the perceiver to adapt his/her behavior, example: (e.g. a flour arrow shows direction). 3) A symbolic strategy, where material objects are appreciated for their conventional reference to arbitrary cultural meanings (e.g. roses express grief in relation to a sudden death). Each of the image descriptions previously tagged as expressing social, semantic meaning were rated by an expert coder in relation to the three interpretational strategies specified above. Ratings were then summarized and analyzed for participants’ relative alignment of interpretative strategy. The aim of this analysis was to establish the extent to which participants tend to choose similar interpretative strategies in construing a material signal. Alignment was measured by the percentage of agreement in semiotic attitudes to each of the images of the experimental conditions (that is, the relative propensity of participants to choose the same strategy in their exploration of an image).



Actualizing semiotic affordances in a material world

Table╯2.╇ Coding scheme II. Code

Description

Examples

Icon

An iconic/aesthetic strategy: objects are appreciated for their intended aesthetic qualities

Explicit references aspects of artistic compositionality such as symmetry, intended resemblances or figurative/metaphorical analogies

Index

An instructional strategy: objects are experienced as social imperatives for coordination or instruction

Explicit attribution of perceiver-directed deontic meanings for instance manifested in modal verbs such as ‘shall’, ‘ought to’, is supposed to’, etc.

Symbol

A symbolic strategy: objects are appre- Explicit references to meanings associated ciated for their conventional reference with specific cultural uses of objects as to arbitrary cultural meanings symbols

Coding scheme for the identification of different semiotic strategies applied by participants in their descriptions of material signals. An expert coder scored participants’ image descriptions in relation to the three variables, defined in the scheme.

3.3 Results of the second coding procedure Analyses revealed strong alignment effects in participants’ interpretative approach to images of the experimental conditions (overall agreement = 91,5%). Generally, participants independently adopted the same meaning-constructing strategies in their approaches to experimental scenes. For example, in the descriptions of the rose-scene (Figure 1b), 18/20 participants categorized the roses as a signal. Of these, 17 did a symbolic reading relating the roses to grief in relation to the recent, unnatural death of someone at the particular spot (while only one participant made an aesthetic reading). Interestingly, however, there were substantial differences between the two experimental conditions (conventional and unconventional signals). For example, the interpretative approach to the Figure 1d (vacuum cleaners lined up on a lawn) shows weaker alignment effects. Thus while 17/20 participants categorize the vacuum cleaner scene as a signal, only 12 participants agree in their choice of an iconic-aesthetic interpretative strategy (associating the scene with a cemetery or an army of warriors). Others are more hesitant in their strategies. This difference applies generally to the two experimental conditions. Thus, while we observe a high inter-subject alignment in the case of the conventional material signals (PA = 99%), the alignment effect is somewhat weaker for the unconventional material signals depicted in images of experimental condition II (PA = 81%) (See Figure 3). Participants’ interpretative approaches to unconventional material signals show more uncertainty. This is shown by a higher between-subject disagreement. Besides, this effect is supported by the observation that participants seem more indecisive in their construals of these scenes and often try out several possible interpretative strategies.

93

94

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

Figure╯3.╇ Diagram expressing the relative agreement (in percent) of participants in their choice of interpretative attitude across experimental conditions. Two types of agreement are represented: (1) inter-subject agreement (the extent to which two or more participants agree in their choice of strategy), and (2) intra-subject agreement (the extent to which each participant commits herself to a single interpretative strategy rather than indecisively drifts between several alternative strategies).

4.0 Discussion The results of both coding procedures show substantial differences between experimental and control scenes supporting our hypotheses. In parallel to the neurocognitive studies cited above, we found significant differences in how experimental participants actualize semiotic affordances in each of the three conditions. Coding procedure I shows a within-subject effect attesting to how the participants’ lexically construe experimental vs. control scenes as signals and non-signals respectively. In the between-subjects coding procedure, while descriptions of the control scenes showed massive variation, experimental scenes were generally characterized by a strong propensity to alignment, indicating conformity to social norms. Independently, participants do very similar descriptions of particular scenes applying distinctive semiotic strategies. While perception of non-signal objects is a more private affair that characterized by unregulated, episodic sense-making, certain objects scenes have an ostensive quality that alter participants’ attitudes to construal: they afford a special kind of social responsiveness constraining the perceptual explorations. As shown by verbal construals, they elicit more explicit references to communicational concepts, social meaning, and discursive roles. The second coding procedure used Peirce’s classification of sign types (icon, index and symbol) to explore normative aspects of meaning-construction. A striking



Actualizing semiotic affordances in a material world

example arose in comparing descriptions of two images used in the experimental conditions (Figure 1c and 1d). Both depict an arrow in a central position of the scene, but in different contexts. In Figure 1c flour is sprinkled out to form an arrow on a path in a wood, and in Figure 1d a road sign depicting an arrow is lying on a lawn surrounded by old vacuum cleaners arranged in rows. Though an arrow could be regarded as a conventional type of ‘symbol’ with an almost inherent diagrammatical meaning (Stjernfelt, 2007), the participants construed the meaning of the arrows quite differently. In their descriptions of Figure 1.c participants consistently adopted an instructional/imperative reading of the arrow. This is manifest in their use of more than twice as many modal verbs expressing deontic meanings (ex. should, ought to, have to, etc.) than in Figure 1d (Figure 1c: 26, Figure 1d: 9). Such interpretation is very rare in describing Figure 1d. In an aesthetic setting, the arrow is not experienced as addressing the perceiver in an instructional or imperative way. Rather, it “just points”. This is illustrated by extracts from descriptions of Figure 1c and Figure 1d, respectively: – “You are in a forest and there is an arrow drawn in white chalk, I think, that points to the right, so you’re probably not supposed to go up the small path that is straight ahead (…)” – “(…) In front of the vacuum cleaner formation you see a road sign with an arrow pointing ahead to the foreground and this sign lies upon something that looks like cardboard on which is drawn the American flag”. Contrary to instructional signals, the intersubjective relevance of aesthetic compositions (like the vacuum-cleaner scene) is not construed as intended to modify the behaviors of the perceiver. The arrow is thus taken only to relate to the other components of the configuration. Participants not only agree in construing certain object configurations as intersubjective signals, but also show enhanced sensitivity to compositional details that constrain semiotic exploration of the various scenes. Thus, participants end up assigning different meanings to otherwise quite similar objects (like the two arrows) in orientation to their socially constructed affordances. Interestingly in this respect, we observe how the degree of conventionality of semiotic object configurations influences the perceiver’s normative alignment to semiotic attitudes. While the distinction between conventional and unconventional object configurations showed no significant effects on the categorical level (signal vs. non-signal), in the second coding procedure it predicted participants’ inclination to align their semiotic attitudes. In contrast to the conventional style signals of the experimental I condition, descriptions of the unconventional signals show less closely aligned construals. Participants tend adopt competing interpretations.

95

96

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

An illustrative example is found in participants’ descriptions of figure 1.e. The image depicts a red car with a bunch of thin tomato slices distributed in an ordered pattern on the windscreen. The car is apparently parked among other cars in a parking lot. The image divides the participants between groups who use contrasting interpretative frames. This is illustrated by two descriptions: 1) “I don’t think they got there naturally, so I think it’s meant to be some kind of artistic statement, the way the slices of tomato have been put on that window. I don’t know exactly what it should mean. It could be some kind of parallel with the color red, or, I don’t really know. It could be health related, like the car has been infected with tomatoes, or something weird like that”. 2) “It’s some kind of hostile act, or somebody pulling somebody else’s leg by putting slices of tomato on the windscreen of their car. Either way it’s a pretty strange thing to do. And its not quite clear what’s going on. It could of course be somebody playing a practical joke with (or on) some friends”. The first example represents how a part of the participants apply an iconic, figurative attitude to the scene (in parallel to the red iceberg or the vacuum-cleaners). Some participants even suggest that the scene may be a “piece of art”. By contrast, a slightly smaller group appreciates the tomato configuration neither for its iconic resemblances, nor as a social instructional imperative (like the arrow). Rather, they see it as a symbolic message between interlocutors who use a more arbitrary kind of reference (comparable to the roses-scene). Accordingly, the tomato-scene becomes a ‘hostile act’ or a ‘practical joke’ addressed the owner of the car. The unconventionality of the expressive practice suggests a possible basis for the conflicting interpretative strategies observed. Given its novelty, the scene offers only weak constraints on intersubjective contextualization. As a result, there is variability in the distribution and weight of attention applied to different constituent parts of the scene (Rollins, 2004). Though participants tend to agree that the object configuration is intended as a communicative signal, the meaning potential is left too open and normatively unconstrained by its composition. However, the scene can easily be imagined as a part of a larger context where preceding events would ground the intended interpretation. In this respect, the object configuration may be addressing “initiates” who have the prerequisites needed for proper construal. Naïve to this preceding story, participants base their intuitions on the ‘unguided’ semiotic potentials. In this case, the result is two distinct types of interpretations (iconic/aesthetic and symbolic) that reflect different perceptual-attentional strategies: If we focus on the symmetrical compositionality of the vegetable slices and the sameness of color between car and tomatoes this can motivate an aesthetic reading. From this perspective, we see a well-ordered arrangement of



Actualizing semiotic affordances in a material world

the items. Those who take a global view and disregard compositional details, note the everyday scenery of the parking lot. In such cases, the artistic construal is less likely that one which focuses on a symbolic, “practical joke”. The example thus complements the general idea put forth in the introduction in interesting ways: the recognition of purposeful ostensive qualities of the composition increase sensitivity to the meaning potentials of the contextualized material structures (Tylén, 2007): By attunement to available intersubjective contextualizations and norms, the perceiver actualizes semiotic affordances. But to the extent that the object configuration relies on less conventional modes of expression, this actualization process becomes less regulated and potentially more polysemous. 5.0 Conclusion In this study we examined a special kind of social sense-making activity manifest in how we come to perceive novel configurations of everyday objects as mediating communicative interaction. This happens when such configurations are experienced as arranged or manipulated in striking and purposeful ways that afford a special style of exploration. Expanding upon a recent brain imaging study (Tylén et al., 2009), we conducted an experiment comparing participants’ casual comprehension of a series of images belonging to three conditions: conventional material signals, unconventional material signals and object configurations without any conspicuous communicative relevance. Analyses of participants’ oral construals suggest that they consistently pick up on certain ostensive qualities in compositional details of the experimental (and not the control) conditions and approach object configurations as ‘acts of signaling’. This shift in perceptual attitude is manifested in significantly more explicit references to communication and meaning related terms as well as by references to discursive roles (addresser and addressee) in descriptions of experimental relative to control scenes. Rather than a ‘private’ mode of perception structured by reference to episodic experiential content, compositional properties of these scenes call for a ‘public’ mode of perceptual sense-making: objects and properties of the material environment are actualized as sources of intersubjective meaning-making. These results are consistent with behavioral studies of how ostensive cues affect infant comprehension of object manipulations (Gergely et al., 2007; Senju & Csibra, 2008; Senju et al., 2008), and by neuro-cognitive findings (using very similar stimuli) showing that only the objectsignals elicit enhanced neural activity in meaning related brain regions like Broca’s area of the inferior frontal cortex (Tylén et al, 2009).

97

98

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

However, besides the categorical distinction between communicative and non-communicative object configurations, we show how social contextualizations of the objects regulate participants’ interpretative attitudes. From great variability in participants’ sense-making approaches to images of the control condition, these align in interesting ways when participants approach images of the experimental conditions. We speculate that these alignment effects are due to an orientation to shared norms for such expressive practices (not incomparable to the idea of top-top interactions, Roepstorff & Frith, 2004). Our findings suggest that our engagement with objects and properties of the material environment as mediating intersubjective meaning should be considered in continuation of other human expressive behaviors such as verbal language and gesture. Not unlike verbal utterances, some object configurations motivate the actualization of situated affordances that embody meaning. This process entails responsive sensitivity to dynamic matter and attunement to regulating norms that guide meaning construction.

Acknowledgements The authors would like to thank Stephen Cowley for his extensive and insightful comments on previous versions of this manuscript. Besides we would like to thank Chris Frith, Uta Frith, Mikkel Wallentin, Andreas Roepstorff, Kim Mouridsen, the Velux Foundation and the Danish National Research Foundation’s Center for Functionally Integrative Neuroscience.

Notes 1. As discussed in Tylén (2007), this can be conceptualized as semiotic agency which itself is a kind of manifest action (H. Clark, 2005). 2. Most material structures lend themselves to several, alternative meaning exploring interpretations (cf. e.g. Myers & Liben, 2008), which makes attention and attunement to addresser’s original intention crucial (Levinson, 1979). 3. Cf. e.g. Cohen et al, 2000, 2002; Dehaene et al 2002. 4. See Tylén et al, 2009 for extensive references.



Actualizing semiotic affordances in a material world

References Arbib, M.A. 2005. “From monkey-like action to recognition to human language: an evolutionary framework for neurolinguistics”. Behavioral and Brain Sciences, 28: 105–167. Banerjee, M., Capozzoli, M., McSweeney, L., Sinha, D. 1999. “Beyond kappa: a review of interrater agreement measures”. Canadian Journal of Statistics, 27: 3–23. Clark, A. 2006. “Material symbols”. Philosophical Psychology, 19: 291–307. Clark, H.H. 1996. Using Language. Cambridge: Cambridge University Press. Clark, H.H. 2005. “Coordinating with each other in a material world”. Discourse Studies, 7: 507–525. Cohen, J. 1960. “A coefficient of agreement for nominal scales“. Educational and Psychological Measurement, 20: 37–46. Cohen, L., Dehaene, S., Naccache, L., Lehericy, S., Dehaene-Lambertz, G., Henaff, M.A., Michel, F. 2000. “The visual word form area: spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients”. Brain, 123:291–307. Cohen, L., Lehericy, S., Chochon, F., Lemer, C., Rivaud, S., Dehaene, S. 2002. “Language-specific tuning of visual cortex? Functional properties of the Visual Word Form Area”. Brain, 125: 1054–1069. Dehaene, S., Le Clec, H.G., Poline, J.B., Le Bihan, D., Cohen, L. 2002. “The Visual Word Form Area: A prelexical representation of visual words in the fusiform gyrus”. Neuroreport, 13:321–325. Donald, M. 2002. A Mind So Rare: the Evolution of Human Consciousness. New York: Norton. Cowley, S.J. 2007. “How human infants deal with symbol grounding”. Interaction Studies, 8: 81–104. Cowley, S.J., Moodley, S. & Fiori-Cowley, A. 2004. “Grounding signs of culture: primary intersubjectivity in social semiosis”. Mind, Culture and Activity, 11: 109–132. Dietrich, S., Hertrich, I., Alter, K., Ischebeck, A., Ackermann, H., 2007. ”Semiotic aspects of nonverbal vocalizations: a functional imaging study”. NeuroReport, 18(18):1891–1894. Fodor, J. A. 1983. The Modularity of Mind. Bradford Books. Cambridge, MA: MIT Press. Gallagher, S. 2003. Phenomenology and Experimental Design: Toward a Phenomenologically Enlightened Experimental Science. In A. Jack and A. Roepstorff (eds), Trusting the Subject? The use of Introspective Evidence in Cognitive Science, Vol. 1, pp. 85–99. London: Imprint Academic. Gallagher, S., Zahavi, D. 2008. The Phenomenological Mind: An Introduction to Philosophy of Mind and Cognitive Science, London: Routledge Itkonen, E. 2008. The central role of normativity in language and linguistics. In The Shared Mind, Zlatev, Jordan, Timothy P. Racine, Chris Sinha and Esa Itkonen (eds), pp. 279–305. Amsterdam: John Benjamins Publishing Company. Lawrence, E.J., Shaw, P., Giampietro, V.P., Surguladze, S., Brammer, M.J., Davis, A.S. 2006. “The role of ‘shared representations’ in social perception and empathy: An fMRI study”, NeuroImage, 29(4):1173–1184. Lotze, M., Heymans, U., Birbaumer, N., Veit, R., Erb, M. Flor, H., Halsband, U. 2006. “Differential cerebral activation during observation of expressive gestures and motor acts”. Neuropsychologia, 44: 1787–1795. MacWhinney, B. 2000. The CHILDES Project: Tools for Analyzing Talk. 3rd Edition. Mahwah, NJ: Lawrence Erlbaum Associates

99

100

Kristian Tylén, Johanne Stege Bjørndahl, and Ethan Weed

Myers, L.J., Liben, L.S. 2008. “The role of intentionality and iconicity in children’s developing comprehension and production of cartographic symbols”. Child Development, 79: 668–684. Newman-Norlund, R.D., Bosga, J., Meulenbroek, R.G.J., Bekkering, H. 2008. ”Anatomical substrates of cooperative joint-action in a continuous motor task”: Virtual lifting and balancing. NeuroImage 41, 169–177. Neuendorf, K.A. 2002. The Content Analysis Guidebook, Sage Publications, Inc., USA. Peirce, C.S. 1998. Collected Papers, I-VIII, (ed. Hartshorne and Weiss; Burks) London: Thoemmes Press Pickering, M.J., Garrod, S. 2004. “Toward a mechanistic psychology of dialogue”, Behavioral and Brain Sciences, 27:169–226. Pinker, S. 1994. The Language Instinct: How the Mind creates Language. New York: W. Morrow. Roepstorff, A. 2008. “Things to think with: words and objects as material symbols”. Philosophical Transactions of the Royal Society B, 363: 2049–2054. Roepstorff, A., Frith, C.D. 2004. “What’s at the top in the top-down control of action? Scriptsharing and ‘top-top’ control of action in cognitive experiments”. Psychological Research, 68: 189–198. Rollins, M. 2004. “What Monet meant: intention and attention in understanding art”, The Journal of Aesthetics and Art Criticism. 62: 175–188. Spurrett, D., Cowley, S. 2004. “How to do things without words: Infants, utterance-activity and distributed cognition”. Language Sciences, 26: 443–466. Stjernfelt, F. 2007. Diagrammatology: an investigation on the borderlines of phenomenology, ontology, and semiotics, Netherlands: Springer Tomasello, M. 1999. The Cultural Origin of Human Cognition, Cambridge MA: Harvard University Press. Tomasello, M. 2003. Constructing a Language: A Usage-based Theory of Language Aacquisition, London: Harvard University Press. Tylén, K. 2007. “When agents become expressive: a theory of semiotic agency”. Cognitive Semiotics, 0: 84–101. Tylén, K, Wallentin, M., Roepstorff, A. 2009. “Say it with flowers! An fMRI study on object mediated communication”. Brain and Language, 108: 159–166. Tylen, K. Allen, M. 2009. Interactive sense-making in the brain. In A. Carassa, F. Morganti, and G. Riva (eds), Enacting Intersubjectivity: paving the way for a dialogue between cognitive science, social cognition and neuroscience, Proceedings from the International workshop, pp. 224–241, Lugano, Switzerland. Vuust, P., Roepstorff, A., Wallentin, M., Mouridsen, K., Østergaard, L. 2006. “It don’t mean a thing… Keeping the rhythm during polyrhythmic tension, activates language areas (BA 47)”. NeuroImage, 31(2), 832–841. Vuust, P., Pallesen, K.J., Bailey, C., van Zuijen, T.L., Gjedde, A., Roepstorff, A., Østergaard, L. 2005. “To musicians, the message is in the meter: pre-attentive neuronal responses to incongruent rhythm are left-lateralized in musicians”. NeuroImage, 24(2), 560–564. Zlatev, J. 2008. “From proto-mimesis to language: Evidence from primatology and social neuroscience”. Journal of Physiology - Paris, 102: 137–151.

Languaging in Shakespeare’s theatre Evelyn Tribble

University of Otago, New Zealand

The enshrinement of William Shakespeare’s plays in printed editions has led to the assumption that they were performed with an ideal of exact verbatim reproduction of the language. Evidence drawn from alternative versions of the plays circulating in Shakespeare’s lifetime and from our knowledge of the material practices of playing in early modern England presents us with a very different picture. Performing practices in this period were marked by a tension between improvisational here-and-now languaging practices, including the use of gesture in playing, and a new set of expectations based upon an emergent conception of plays as written documents. Keywords: distributed language, gesture, Shakespeare, theatre, written language bias

1. Written language bias and the code view of language Nigel Love, among others, has pointed to the power of inscription technologies to shape the “code” view of language: “It is not because language is a code that we are enabled to reify our thoughts. Rather, it is because (with the crucial aid of writing) we reify our utterances — treat them as instances of more abstract entities — that we (think we) develop a code” (Love 2004:â•›542). As Per Linell (2005:â•›viii) has argued, a persistent written language bias undergirds much of the discipline of linguistics and, by extension, literary studies. The implicit equation of language with written language models language as a “thing” rather than as an active, dynamic, distributed process produced by “co-acting bodies” (Cowley 2007:â•›577).1 The powerful back-formation that is the written language bias has far-reaching effects. This paper examines one of these: the implicit or explicit assumption that Shakespeare’s words are to be seen as codes for reproduction rather than a skeletal structure that constrains (but does not comprise or determine) performance. Written language bias would tell us that the plays consist of line and stage directions that can be reproduced in various ways, a view that the reification of the plays

102

Evelyn Tribble

in printed texts underscores. But if we instead take a multimodal dynamic view of the plays, we get a very different picture. Far from being a simple translation from page to stage, for actors in the 1590s mounting a play was an act as much of producing as of reproducing language. As I will argue, many assumptions in Shakespearean studies are essentially back-formations based on erroneous assumptions about the over-riding importance of the written word and the assumption that players held an ideal of verbatim reproduction of the author’s words. But the circumstances of play production in that period render such assumptions dubious. Indeed, they are often the result of a strong written language bias that assumes that players were attempting to conform as closely as possibly to a written version of the plays authorized by the playwright himself. This led editors and bibliographers at the turn of the last century to attribute disparities between early published versions of the plays and/or apparent inaccuracies and incoherencies within them to dishonest attempts on the part of so-called pirates to reconstruct the playtext from memory.2 In this view, marks of improvisation, repetition, hesitation, and incoherence are all stamps of the actor’s oral interference with the pristine written word of the playwright. 2. Background: The conditions of playing in the early modern theatre To understand the dynamics of playing, it is vital to understand the practical, material conditions of play production in this period, which differ enormously from those experienced by actors today. The conditions were roughly these: players performed up to six different plays a week, with an additional new play being learnt and mounted every fortnight. There were no long runs, as there are now; a play might be repeated every couple of weeks, and it was almost unheard of for the same play to be repeated on subsequent days. There were no directors, very few full group rehearsals, no special lighting, and no built sets. The company itself was relatively stable and consisted of about eight sharers — men such as Shakespeare himself who jointly financed and managed the company; eight or so “hired men” who filled out the cast; and eight or so boys, apprenticed to actors in the company, who played all female and juvenile parts. The actors themselves were given almost full authority to make decisions about movement across the stage, use of props, delivery of lines, and so forth; most preparation for plays consisted of the individual “study” of one’s own part (Stern 2000:â•›12). Three sets of written documents were used to mount the plays, spread unevenly across the company. Every player received his “part”: his own lines copied by hand onto a long scroll, separated by 2–3 word cues. Entrances and exits



Languaging in Shakespeare’s theatre

and major stage business were charted on a “plot”, a large A-3 size piece of paper, probably posted back-stage and available for consultation. Finally, the company scribe prepared the ‘play-book’, the only complete record of the drama. No player, then, had access to the entire play — only his own part, his cues, and a schematic outline of his entrances. The ‘play-book’ was very valuable to the company for several reasons. First, as the only complete record of the full play it was necessary, should any parts be lost or mis-transcribed. The book was also annotated with crucial production cues, especially cues for music and sound, which needed to be controlled centrally. Most importantly, it constituted an official legal document. No play could be produced in London without the signature of the “Master of the Revels”, a high-ranking court official who certified that the play did not contain any politically or religiously subversive material. Officially the rule was that the players could not utter any lines that were not set down in the play-book. In practice, however, this rule could not have been enforced in any meaningful way (Potter 1986:â•›87); rather, it was meant to penalize any attempts to bring in topically sensitive material after the play had been authorized. Any given play’s relationship to written documents is thus somewhat complex. On the one hand, the legal requirements for licensing placed a new-found importance on the written text, especially in contrast to earlier, more improvisational modes of playing (Wiles 1987; Weimann 2000). On the other, the sheer mnemonic demands of producing six different plays a week are such that it is necessary to question the possible level of ‘fidelity’ to the written text, or even what the question of ‘fidelity’ means in this context. As I shall show, the notion that the primary job of the players was to produce an accurate oral record of the written words grossly distorts the distributed, practical, here-and-now activities of the early modern playhouse. To understand this point, it is important to free ourselves of preconceptions derived from the cultural authority of Shakespeare, an authority that is in turn built upon veneration for the written word. Shakespeare’s words are among the best known of all writers — he is full of quotations, as the joke goes. The widespread knowledge of his plays is built upon multiple cheap editions, constant repetition of key phrases in the media, and exposure to films and theatrical productions. The cultural capital attached to the precise words of Shakespeare is very high and it is implicitly assumed that the task of mounting a play is primarily to reproduce them. This veneration is then implicitly transferred to the original production of the plays and is assumed to be the preoccupation of the original players. Discussions of “prompting” are a good example. Despite the paucity of evidence for verbal prompting, it is often assumed that a “prompter”, holding the one

103

104

Evelyn Tribble

copy of the play-book, would have held the players to the exact words of the text. A noted theatrical historian, for example, writes of the “multifarious and vital” nature of the “prompter’s chores” and suggests that “a letter-perfect rendition must have been unheard of, and prompting a constant necessity” (Bentley 1984:â•›82). But the very concept of a “letter-perfect rendition” reveals the written language bias behind this imagined scenario. Bentley assumes that the goal is exact verbatim reproduction, although he admits that such a goal would not be met, despite the prompter’s best efforts. But was this in fact the goal? The ideal of exact recall is a classic example of written language bias; it has no meaning in a culture without technologies of reproduction, including printing and audio and video recording (Hunter 1984; Small 1997). So accustomed are we to having printed books available that we forget how unusual the ability to compare precisely an oral performance to its written counterpart really is, and how rare it is to wish to do so. In today’s print saturated world, Shakespeare’s plays are taught to students who each are issued a full text of the play; productions begin with read-throughs, each actor holding on to his identical copy of the play. The “text”, between covers, becomes “the play”. But what would the play have been to early modern actors? Certainly, a play-book existed, as we have seen, but only in one copy, carefully kept as part of the official working documents of the company. Printed versions of the plays were a rare exception, not the rule and audiences were very unlikely to mentally frame a performance as a ‘version’ of a written play-book. Early modern players would have imagined the play as an event, occurring in real time and constructed during performance, distributed across the individual parts, more or less successfully remembered by the players; the physical props and the environment of the theatre; the interaction among the players; and the ‘plot’, the one-page summary of the play posted backstage. The play, then, would have been an action, not a thing. 3. Plays in action The plays were literally distributed, spread across a variety of artifacts and only realizable through the “human technology” that provides the “glue” to coordinate activity (Kirsh 2006:â•›250), and examining such activity becomes “very substantially the study of the variety and subtlety of coordination” (Kirsh 2006:â•›258; see also Sutton 2006). As I have argued elsewhere (Tribble 2005:â•›139; Tribble 2011), the act of producing plays can be usefully glossed through the model of distributed cognition as described by Hutchins (1995). Hutchins argues that the lines between “inside” and “outside” are frequently misdrawn or misidentified, “creat[ing] the



Languaging in Shakespeare’s theatre

impression that individual minds operate in isolation and encourag[ing] us to mistake the properties of complex sociocultural systems for the properties of individual minds” (Hutchins 1995:â•›81). In this account, human beings create ‘smart’ structures predicated on a system of constraints that, paradoxically, enable the execution of complex tasks. On board a modern naval vessel, these include charts, GPS systems, navigational electronic tools, the material configuration of the control room, and the social structures of shipboard life. Hutchins argues that [t]he environments of human thinking are not ‘natural’ environments. They are artificial through and through. Humans create their cognitive powers by creating the environments in which they exercise those powers. At present so few of us have taken the time to study these environments seriously as organizers of cognitive activity that we have little sense of their role in the construction of thought (Hutchins 1995:â•›169).

In making this argument, Hutchins imagines a nest of ants occupying the untracked surface of a beach after a storm has scoured it clean. Over time, their activity lays down trails that ultimately organize the activity of the ants, who now appear to be “smart through [their] simple interaction with the residua of the history of [their] ancestor’s actions” (Hutchins 1995:â•›169). The playing companies of the 1590s were in the process of developing these trails and tracks in the relatively untested environment of the purpose-built permanent playhouse. For the early modern playing company, analogous tools are physical space of the theatre, the bodies of the actors, including their movements in space, their gestures, their interactions with other actors and with their environment, their level of expertise, and the verbal features of the verse in each individual part, which formed the linguistic spine of the plays. To elaborate this metaphor of ‘spine’ at greater length: the written words indeed constitute the skeletal structure of the play, without which it cannot exist, or at least it would be indistinguishable from the non-verbal arts such as juggling or dancing. It is here perhaps where the analogy with shipboard navigation breaks down — the time-pressured, goal-driven aim of navigation is successful progress from point A to point B, while the analogous goal of the theatrical company is a successful production, a collaborative effort across actors, audience, and environment. Rather than reducing the act of performance to the representation of the written word, we should instead conceive of it as the successful “cognitive integration” (Menary 2007:â•›3) of a wide range of artifacts, practices, and social agents.

105

106

Evelyn Tribble

4. Multimodal dynamics on stage The beginning lines of Hamlet provide a deceptively simple example of the multimodal dynamics at work on the early modern stage: Enter Barnardo and Francisco two sentinels Barnardo: Who’s there? Francisco: Nay, answer me: Stand and unfold yourself. Barnardo: Long live the king Francisco: Barnardo? Barnardo: He. Francisco: You come most carefully upon your hour (Jenkins 1982:â•›416; 1:1:1–4).

A successful performance of these lines involves a complex distributed cognitive process that calls upon: (a) the individual actor’s ability to recall his lines and, even more importantly, the cues; (b) the knowledge of entrance conventions and the use of the space of the stage; (c) the ability to elaborate the implications of the first line for the entire scene — “who’s there?” implies an inability to see, which in turn means that it is dark, which in turn requires the actor to draw upon his embodied expertise in performing darkness, since the stage itself was fully lit by ambient daylight, as the play would have begun at 2:00 p.m. in an outdoor venue; (d) the ability to invoke both acting experience, a cultural schema, and the material supplies for actions, properties, and costume appropriate to “centinels”; (e) the ability to locate one’s entrance on the playhouse “plot” and time it correctly; (f) the subtle and constantly changing coordination between the two actors. Rather than simply reproducing the lines, actors are engaged in a hybrid process that engages material and social forces. Indeed, dialogism is built into the production by the practice of the cue scripts, for remembering both to say the cue and to listen for one’s own cue was the glue that held the entire production together — failure to remember cues could cause the entire scene to fall like “a house of cards” (Palfrey and Stern 2007:â•›85). The theatre then, constituted an “ecosocial semiotic environment” (Thibault 2005:â•›123) that worked only through the active dynamics of coordination across people, artifacts, and spaces. 5. “Authority” and the dynamics of performance The situation is even more complex than this account allows, because we forget how contingent the words on the page actually are. For a concrete example of the way that actual practice on the stage seems to have exceeded the mere reproduction of written documents, we can look at the evidence of alternative versions of



Languaging in Shakespeare’s theatre

Shakespeare’s plays circulating during his lifetime. This is an enormously complex issue in Shakespearean studies, but for our purposes it is important to know simply that: (a) thirty-seven of Shakespeare’s plays were published posthumously by members of his company in 1623, in an elaborately printed volume known as the “First Folio”; (b) about half of these plays were also published during his lifetime, but apparently without his oversight, in cheap editions similar to paperbacks today; (c) some of these editions differ markedly both from one another and from the Folio texts; (d) the case of Hamlet is particularly complex, since two very different cheap editions exist, commonly though probably erroneously dubbed the “bad quarto” (Quarto 1) and the “good quarto” (Quarto 2) (Werstine 1990:â•›65). Without addressing the myriad of theories about the relationship of these texts, it is fair to say that earlier Shakespearean studies were obsessed with establishing their “authority” (Werstine 1990:â•›83). “Authority” here means only their relationship to a written text — that is, to the hand of Shakespeare inscribing the words. Because we have no authorial manuscripts, attempts to establish “authority” are always conjectural and have tended to exclude any evidence of the actual enactment of the plays — all examples of repetitions, hesitations, confusions, improvisations were discarded as the actors’ interpolations and thus, by definition, unauthorized (forgetting of course that Shakespeare himself was an actor and that plays were in a fluid collaborative state). For example, Hamlet’s famous dying line is “the rest is silence”. Or is it? In one version, he says instead, “Farewell, Horatio, heaven receive my soul” (Irace 1998:â•›92; Sc. 17: 104) and in the First Folio he says, “The rest is silence. O, o, o, o” (Jenkins 1982:â•›416; 5.2.363n). Modern editions must choose among these, and almost all concur in using “the rest is silence” and ignoring “O, o, o, o” as the extra-linguistic utterance of an actor hamming up his death scene. For my purposes, though, it can equally be seen as a trace of the here-andnow languaging that marked performance. Any modern edition of Shakespeare silently occludes countless such examples, as decisions are made about what counts as the language of Shakespeare. Similarly, if we return to the example of the opening lines of Hamlet, one contemporary version (the so-called “bad quarto”) reads instead: 1. 2. 1.

Enter two Centinels. Stand: who is that? Tis I. O, you come most carefully upon your watch. (Irace 1998:â•›35; Sc. 1: 1–3).

It is difficult to read this as anything other than a gist memory for the more elaborate and complex lines of the more familiar opening. The version quoted earlier confusingly assigns the sentinel who is relieving his fellow the line, “Who’s there?”,

107

108

Evelyn Tribble

prompting the response, “Nay answer me” (emphasis added). This minor reversal of expectations — normally the sentry on duty issues the challenge — is omitted in the competing version, which exemplified some of the features of “omission, rationalization, and conventionalization” that Bartlett (1932:â•›125–127) associated with the processes of memory. In this case, two actors present the situation in its most economical form — one sentry relieving the other — with little apparent regard for verbatim reproduction. Shakespeare’s theatre can be seen as a hybrid system, where the demand for textual fidelity is mapped onto a strong residue of orality (Marcus 1996; Weimann 2000). As we have seen, verbatim recall is a product of literate culture, and it exists only within the context of “the pursuit of interests which involve a strong concern for verbatim fidelity” (Rubin 1984:â•›433). Obviously the early modern theatre is a good example of an institution, as the interest of the Master of the Revels in players’ fidelity to the written script suggests. But we need to place this demand within its historical context. The ideal might be to “speak no more than is set down”, but in reality, given the demands of performing up to six different plays a week, a framework for “fluent forgetting” (Tribble 2005:â•›150) must have been necessary. For Shakespeare, expert practice did not consist so much in exact fidelity to the written text, but to the cognitively fluid act of improvising within a strongly metrically controlled framework. In the representations of amateur practice and professional practice he presents in his plays, amateurs are always marked by their lack of fluency and their over-riding concern with getting the speech word-for-word. The excessive attention to the verbal dimension of the task robs them of the attention necessary to cope with the unexpected. The key to “remembering” your lines is never to give the appearance of forgetting — that is, never to stop. Amateur actors represented always stop; any interruption disrupts the brittle cognitive system precariously in place, as witnessed by the representations of failed amateur performances in Shakespeare’s plays. Shakespeare stages many examples of such failed performances, the most well-known of which is perhaps the woeful enactment of Pyramus and Thisbe by inexperienced working men that concludes A Midsummer Night’s Dream. 6. Action and accent: The gestural dimensions of the plays In particular, the amateur actor has difficulty integrating the two elements that were deemed absolutely vital to a good performance: action and accent, or gesture and word. Action (movement, including gesture) and accent (speech, pronunciation) were terms of art in this period, and these twinned categories governed both



Languaging in Shakespeare’s theatre

theatrical and oratorical practice. The twinning of “action” and “accent”, or movement/gesture and speech, seems to have been normal practice in this period and derives from classical oratorical practice. That is, when the actors studied their parts, they saw them not simply as words to be memorized, but as a skeletal structure for the integration of the verbal and the physical. This method of learning the part should be seen as dynamic, constructive, and embodied rather than a form of rote encoding. Taking their cue from Hamlet’s injunction “not to saw the air with thy arms thus”, many theatre historians have concluded that the early modern playing companies operated on the basis of a rigidly stylized set of conventional gestures learned and transmitted by rote (Hattaway 1982:â•›54). Even contemporary linguistic studies of gesture tend to neglect rehearsed gesture; for example, McNeill draws a sharp distinction between “choreographed” gestures such as those used by orators and actors, from the “spontaneous gestures” of online conversation (McNeill 1994:â•›3). Here rehearsed language sits in an uneasy limbo between naturalistically produced spontaneous speech and written discourse. However, I argue instead that the technique of incorporating action and accent into the rehearsal and production process did not result in the mechanical reproduction of a fixed written document, but instead was a means of bootstrapping somatic techniques in order both to retain and to produce meaning. A growing body of literature on gesture from linguists, cognitive psychologists, and cognitive anthropologists has dispelled the idea that gesture is simply random hand waving, mimetic matching of word and action, or a formal addon to thought. A number of studies have shown that “gesturing confers a significant benefit on verbal memory” (Goldin-Meadow 2004:â•›402). In a paper entitled, “Complementary strategies: why we use our hands when we think”, David Kirsh discusses the ways in which we use “our hands to help think, remember, and perceive” and to manage attention (Kirsh 1995:â•›212). Gestures thus become “complementary strategy”, defined as “any organizing activity which recruits external elements to reduce cognitive loads … hands, fingers, and surrounded material objects are recruited for cognitive use” (Kirsh 1995:â•›212, 216). In “Imagining the cognitive life of things”, Ed Hutchins suggests that “in some circumstances, the body itself can become a cognitive artifact, upon which meaningful environmentally-coupled gestures can be performed. In such settings, motion in space acquires conceptual meaning and reasoning can be performed by moving the body” (Hutchins 2006). Both material and somatic “anchors” for thought can be used to manage attention and memory. Actors, of course, must be experts in managing attention and memory, given the time-pressured nature of their activity and the importance of fluency and coping with the unexpected. Evidence suggests that the elaboration and encoding of

109

110

Evelyn Tribble

gesture while committing verbal material to heart was a primary means of accomplishing these tasks. Lines were not simply recited; if we see the words as the skeleton, then gesture functioned as the musculature system, the means by which the verbal artifacts were lifted into the body. If we recognize that gesture is “not just hand-waving” (Goldin-Meadows 2003:â•›12) or mere formalized stage business, what implications might we find? We have all perhaps seen novice speakers or actors painfully map an action onto a word, often a beat or two behind the proper moment. But this is precisely what we would expect of novices, because one of the major challenges in learning any skill is to get to a level at which performance becomes fluid and formerly disparate skills become integrated (as in, say, learning to serve a volleyball or bat at cricket) (Sutton 2007). If the relevant modes of performing were accent and action, these cognitive domains would have been used to encode the parts, the cue-scripts that were the primary cognitive artifacts for individual study. The requirement to suit the word to the action and the action to the word is not just advice for performance; it may describe the process by which the text itself is encoded in the body, which is in turn recruited as a complementary strategy used in performance to produce the part. Expert actors use this technique of study to perform deep encoding, allowing memory to fish with more than one hook and to attach a physical constraint to the verbal constraints of the verse. An immediate objection to this line of argument might be mounted: when Goldin-Meadows (2004), McNeill (1992), and Kirsh (1995) discuss gesture, they are explicitly not interested in rehearsed or “conventional” gesture — that is, for them, the gesture worthy of discussion is the ‘on-line’ spontaneous gesture that is produced in immediate, face-to-face discussions. Following the continuum of gesture identified by Kendon (1988), McNeill (1994) carefully distinguishes contemporary linguistic work on gesture from its counterpart in rhetoric. Classical rhetoricians such as Quintilian, he writes, “took care to specify gestures that could be designed in advance as deliberate elements in a choreographed presentation. They are thus quite distinct from the spontaneous gestures that are the subject matter of this book” (McNeill 1994:â•›3). This distinction between spontaneous and codified gesture is too starkly made. It is true that rhetoricians such as Quintilian wrote practical guides that codified gestures, and these have sometimes been taken as evidence that actors and rhetoricians relied upon a static stockpile of gestures learned by rote. In this view, encoding a part would involve the routine matching of action and accent. But to make this argument is to confuse the codification of a system with the messy particularities themselves — similar to confusing the systemization of a dictionary with the actual languaging practices it purports to represent. Indeed, Quintilian specifically advises against young orators memorizing “their gestures ahead of time” (Quintilian 2001:â•›109). The idea that because a system is subject to description,



Languaging in Shakespeare’s theatre

and because it is available as an “art”, that it is therefore by definition somehow insincere and merely mechanical represents a misunderstanding of the true dynamics of gesture in the early modern period. Instead gesture was a vital component of the here-and-now embedded act of languaging as practiced on the stage. 7. The limits of languaging: The writer and the clown A full account of the relationship of Shakespeare’s work to ‘languaging’ must take note of the tensions detectable between an emergent written theatre and the strongly residual physical “presentational” (Weimann 2000:â•›98) elements of the plays. To return to Hamlet, we can see these tensions played out within the fiction itself, in Hamlet’s famous advice to the visiting players, who are about to perform for the court as part of Hamlet’s attempt to entrap the King (3.2). Among other injunctions, Hamlet is concerned with the role of the Clown, whose job in the company was to provide physical and verbal humor, both within the fiction of the play and as the leader of the “jig” performed afterwards. Hamlet advises: Let those that play your clowns speak no more than is set down for them. For there be of them, that will themselves laugh, to set on some quantity of barren spectators to laugh too, though in the mean time, some necessary question of the play be then to be considered: that’s villainous, and shows a most pitiful ambition in the fool that uses it (Irace 1998:â•›67; Sc. 8:â•›17–21).

Here the tensions between presentational improvisation and the text of the play are made explicit. In this view, the clown and the playwright compete for control of the play. The clown, in attempting to exceed the writer’s instructions — to “speake . . more than is set downe” in the script — appeals to the audience by the simple act of laughing, a non-linguistic utterance which infects the spectators like a contagion, causing them to laugh as well. The ensuing chaos overwhelms the writer’s carefully constructed plot, the audience showing little interest in the “necessary question” of the plot. Here the languaging and the physical antics of the clown are placed into opposition with the words of the playwright. In the so-called bad quarto, Hamlet elaborates on the competition between written word and languaging practices: And then you have some [other clown] again who keeps one suit of jests, as a man is known by one suit of apparel, and gentlemen quote his jests down in their tables before they come to the play, as thus, “Cannot you stay till I eat my porridge?”, and “Your beer is sour,” and blabbering with his lips and thus keeping in his cinquepace of jests, when God knows, the warm clown cannot make a jest unless by chance (Irace 1998:â•›67–68; Sc. 8:â•›21–38).

111

112

Evelyn Tribble

Hamlet here describes a fundamental division over the question of the purpose of playing. For the prince, the play consists of its written instantiation, controlled by the writer, transmitted by the actors, and to be consumed by the audience, in a tidy process that ensures that “necessary points” of plot are conveyed in a model of communicative efficiency. But his account of what is actually valued in the play differs strikingly from this model. In this version, “gentlemen” go to plays not to follow complex plots, nor to admire carefully constructed poetry, but to engage in the pleasure of the familiar. These gentlemen have already written down what they go to hear — they have entered in their table books (small notebooks) the jests they expect the Clown to perform. Phrases such as “cannot you stay till I eate my porridge?” or “your beer is sour” have little representational content — and probably have become unmoored from dramatic representations of eating and drinking. They are instead taglines that bind together audience and actor in an experience, which cannot be seen as in any way advancing an argument or plot-line, but instead invokes pleasure at the circulation of familiar jests, just as concert-goers today demand the familiar old songs from their favorite bands. In this way the clown-audience dynamic can be seen as an instance of languaging as a means of realizing values (Hodges 2007:â•›585). In this instance, play-going is as much about a collaborative social dynamic between performer and audience as it is about passively ‘hearing’ a play. Existing evidence about the crucial role of the Clown in the Elizabethan theatre bears out this view. For an early modern audience, to attend a play was to do more than view a self-contained fictional dramatic representation. Plays at public theatres invariably ended with a “jig”, a cross between a farce and a sketch, accompanied by song, and most importantly, lewd dancing; these were not crude afterpieces, but essential to the playgoing experience (Wiles 1987). After the play proper ended, the stage was given over to the company clown, who presided over the jig, which often spread out to encompass the audience, sometimes with disorderly results (Baskerville 1929:â•›116). Of course, as Hamlet complains, the clown might also try to introduce his jigs and jests into the play itself, either by mugging (“blabbering with his lips”) or by bawdy dancing (“cinquepace of jests”). The disdain for authority draws the disapprobation of the intellectual university student, Hamlet. In fact, earlier in the play Hamlet demonstrates his contempt for Polonius’s theatrical taste when he says, “He’s for a jig, or a tale of bawdry, or else he sleeps” (Irace 1998:â•›65; Sc. 8:â•›322). Many Shakespearean scholars have been unable to resist the temptation to see Hamlet as a mouthpiece for the author. In this account, the playwright Shakespeare uses his character to denounce precisely the languaging practices I have been describing. But identifying Shakespeare with Hamlet is not necessarily warranted.



Languaging in Shakespeare’s theatre

Shakespeare’s own position — as an actor as well as a playwright, often performing for his social superiors — much more closely resembles that of the nameless actors than that of the aristocratic young amateur telling professionals how to do their jobs. At the very least, we need to acknowledge the tension between “author’s pen and actors voice” (Weimann 2000:â•›6) at work here. Indeed, in delivering these lines, the character Hamlet must “mimic the improvisatory, ‘oral’ theatre he despises” (Marcus 1996:â•›174; see also Johnson 2003). 8. Conclusion One of the reasons Shakespeare’s work is relevant to the field of distributed language is the pivotal historical moment in which his plays participate, in which plays stand uneasily between performance and poem. As Shakespeare was writing, the publication of his plays and those of others in both quarto and in large collected editions had begun the process of back-formation that would ultimately change plays from acts to objects. Printing plays was a new phenomenon in this period, and it ultimately altered the cognitive ecology of the theatre, shifting emphasis onto the verbal spine of the play rather than its presentational qualities. The shift in emphasis is apparent in Ben Jonson’s publication of his plays in a collected edition of his Workes (1616). Jonson was much derided by contemporaries, who mocked him for pompously calling his efforts “works” rather than “plays”. The process continued after Shakespeare’s death, when his plays were similarly collected in the Folio of 1623. During this period we find evidence of other playwrights attempting to fix their work as literary rather than performative; when publishing his play The Roman Actor in 1626, for example, the writer Philip Massinger distinguishes his “poem” from the “jigs” preferred by the ignorant (Massinger 1626). Describing the play as a “poem” is a means of situating it in the domain of the written, safely protected from its association with downmarket improvisational action-oriented genres such as jigs.3 Whether or not Hamlet represents Shakespeare’s own opinions — an impossible question to settle — it is Hamlet’s version of “the play” that ultimately prevails. The enshrinement of the text occludes the lived experience of the Elizabethan theatre, which included song, dance, improvisatory set-pieces, verbal combat between audience and actor, spectacles and jigs (Hattaway 1982). Apart from scattered references and eyewitness accounts, however, what we have of Shakespeare’s plays are the written texts that have come down to us. Because this is all we have, we have a tendency to conclude that this is all they were.

113

114

Evelyn Tribble

Notes 1.╇ For a discussion of the “distributed language” movement, see Cowley (2007); for the relationship between language and distributed cognition, see Love (2004). 2.╇ See Werstine 1990 for a critical account of these arguments. 3.╇ A similar process takes place in this period in respect to music. See Sloboda (2005) on the effects of notation upon conceptions of music.

References Bartlett, F. 1932. Remembering. Cambridge: Cambridge University Press. Baskerville, C. 1929. The Elizabethan Jig and Related Song Drama. New York: Dover. Bentley, G. 1984. The Profession of Player in Shakespeare’s Time. Princeton, NJ: Princeton University Press. Cowley, S.J. 2007. “The cognitive dynamics of distributed language”. Language Sciences 29: 575–583. Goldin-Meadows, S. 2003. Hearing Gesture: How Our Hands Help Us Think. Cambridge: Harvard University Press. Goldin-Meadow, S., Nusbaum, H., and Wagner, S. 2004. “Probing the mental representation of gesture: Is handwaving spatial?”. Journal of Memory and Language 50: 395–407. Hattaway, M. 1982. Elizabethan Popular Theatre: Plays in Performance. London and New York: Routledge and Kegan Paul. Hodges, B. 2007. “Good prospects: ecological and social perspectives on conforming, creating and caring in conversation”. Language Sciences 29: 584–604. Hunter, I.M.L. 1984. “Lengthy verbatim recall (LVR) and the mythical gift of tape-recorder memory”. In M. Kirsti, J. Lagerspectz, and P. Niemi (eds), Psychology in the 1990s. Amsterdam: North-Holland, 425–440. Hutchins, E. 1995. Cognition in the Wild. Cambridge, MA: The MIT Press. Hutchins, E. 2006. “Imagining the cognitive life of things”. Presented at the symposium: “The Cognitive Life of Things: Recasting the Boundaries of Mind”. Organized by Colin Renfrew and Lambros Malafouris at the McDonald Institute for Archaeological Research, Cambridge University, April, 2006. Irace, K. 1998. The First Quarto of Hamlet. Cambridge: Cambridge University Press. Jenkins, H. 1982. Hamlet: The Arden Shakespeare. London and New York: Thompson Learning. Johnson, N. 2003. The Actor as Playwright in Early Modern Drama. Cambridge: Cambridge University Press. Kendon, A. 1988. “How gestures can become like words”. In F. Poyatos (ed), Cross-cultural Perspectives in Nonverbal Communication. Toronto: Hogrefe Press: 131–141. Kendon, A. 2004. Gesture: Visible Language as Utterance. Cambridge: Cambridge University Press. Kirsh, D. 1995. “Complementary strategies: Why we use our hands when we think”. In J.D. Moore and J.F. Lehman (eds), Proceedings of the Seventeenth Annual Conference of the Cognitive Science Society, 212–217.



Languaging in Shakespeare’s theatre

Kirsh, D. 2006. “Distributed cognition: A methodological note”. Pragmatics & Cognition 14(2): 249–262. Linell, P. 2005. The Written Language Bias in Linguistics: Its Nature, Origins and Transformations. London and New York: Routledge. Love, N. 2004. “Are languages digital codes?”. Language Sciences 26: 525–544. Marcus, L. 1996. Unediting the Renaissance: Shakespeare, Marlowe, Milton. London and New York: Routledge. McNeill, D. 1994. Hand and Mind: What Gesture Reveals About Thought. Chicago: The University of Chicago Press. McNeill, D. 2005. Gesture and Thought. Chicago: The University of Chicago Press. Maguire, L. 1996. Shakespearean Suspect Texts: The ‘Bad’ Quartos and Their Contests. New York and Cambridge: Cambridge University Press. Massinger, P. 1626. The Roman Actor. London. Menary, R. 2007. “Writing as thinking”. Language Sciences 29: 621–32. Palfrey, S. and Stern, T. 2007. Shakespeare in Parts. New York and Oxford: Oxford University Press. Potter, L. 1986. “Nobody’s perfect: Actor’s memories and Shakespeare’s plays of the 1590s”. Shakespeare Survey 42: 85–97. Quintilian. 2001. The Orator’s Education. D. Russell (ed). Cambridge, MA: Loeb. Rubin, D.C. 1995. Memory in Oral Traditions: The Cognitive Psychology of Epic, Ballads, and Counting out Rhymes. New York: Oxford University Press. Sloboda, J.A. 2005. Exploring the Musical Mind: Cognition, Emotion, Ability, Function. Oxford: Oxford University Press. Small, J. 1997. Wax Tablets of the Mind: Cognitive Studies of Memory and Literacy in Classical Antiquity. London and New York: Routledge. Stern, T. 2000. Rehearsal from Shakespeare to Sheridan. Oxford: Clarendon Press. Sutton, J. 2006. “Distributed cognition: Domains and dimensions”. Pragmatics & Cognition 14 (2): 235–249. Sutton, J. 2007. “Batting, habit, and memory: The embodied mind and the nature of skill”. In special issue on cricket and philosophy: Sport in Society 10: 5. Thibault, P.J. 2005. “Brains, bodies, contextualizing activity and language. Do humans (and bonobos) have a language faculty, and can they do without one?”. Linguistics and the Human Sciences 1: 99–125. Tribble, E. 2005. “Distributing cognition in the Globe”. Shakespeare Quarterly 56(3): 135–55. Tribble, E. 2011. Cognition in the Globe: Attention and Memory in Shakespeare’s Theatre. New York: Palgrave Macmillan Weimann, R. 2000. Author’s Pen and Actor’s Voice. Cambridge: Cambridge University Press. Werstein, P. 1990. “Narratives about printed Shakespeare texts: “foul papers” and “bad” quartos”. Shakespeare Quarterly 41(1):â•›65–86. Wiles, D. 1987. Shakespeare’s Clown: Actor and Text in the Elizabethan Playhouse. New York and Cambridge: Cambridge University Press.

115

Semiotic cognition and the logic of culture* Barend van Heusden

University of Groningen, The Netherlands

In this paper I argue that semiotic cognition is a distinctive form of cognition, which must have evolved out of earlier forms of non-semiotic cognition. Semiotic cognition depends on the use of signs. Signs are understood in terms of a specific organization, or structure, of the cognitive process. Semiotic cognition is a unique form of cognition. Once this form of cognition was available to humans, the semiotic provided the ground structure for an evolutionary development that was no longer strictly Darwinian, but followed its own — semiotic — logic. In the increasingly abstract ways in which the ubiquitous difference is dealt with, we discover this logic of cultural evolution, which determines the course of long term cultural change. Keywords: cognition, cultural evolution, meaning, semiotic cognition

1. Cognition and representation In the course of evolution, through variation and selection, living species acquired a variety of patterns of reactions. These patterns, which secure the species’ survival in a certain environment, are mostly genetically encoded but sometimes also learned by single animals through mimicry, or imitation of other members of the species.1 The patterns of reactions constitute the memory of an organism (that of a single individual as well as that of a species). They provide an organism with a representation of its environment (cf. Changeux 1997:â•›115). We do not have to assume that representations are, as such, cultural or semiotic, or even ‘rational’, as for instance De Sousa (2007) would have it. Neither do we have to conceive representations as images ‘in’ the nervous system of the organism, or as coded information that has to be perceived, recognized, or decoded by a homunculus (cf. Clark 1997:â•›143ff.). Representations of the environment are patterns of neuronal sensory-motor activity. If such a pattern is activated, the environment acquires meaning for the organism (the acid for an amoeba, the skin for the tick, the fly for the frog, my walking to the door for my dog, the antelope for the lion, and

118

Barend van Heusden

this paper for you). Patterns of reactions are triggered and, as long as they match, an organism lives in a meaningful world, being a world which it ‘recognizes’. This activity allows it to maintain its homeostasis and survive — at least as long as is necessary for successful reproduction. Whether one is dealing with the unicellular, ticks, frogs, dogs, lions, or humans, when we say that something has meaning we assume, either implicitly or explicitly, that this something triggers a more or less adequate (re)action in the organism. Thus the amoeba moves away from the acid, the tick lets itself fall onto the skin, the frog unrolls its tongue to catch the fly, the dog runs toward the door for the afternoon stroll, the lion hunts the antelope, and the human sits down to read a scientific paper. The meaning of an environment is thus dependent on the bodily anatomy and the behavioral patterns of an organism. This view of the environment as a world full of meaning can be traced back at least to the work of Jakob von Uexküll (1920: 1940; cf. also Clark 1997:â•›23ff.). We should not forget, however, that the acid, the fly, etc., as we perceive them are such because of our patterns of reactions — the frog probably doesn’t know about flies (or about frogs, for that matter), nor do we know ‘what it is like to be a frog’. Its world, i.e., the affordances the frog gets, is determined by its frog-anatomy (Gibson 1979:â•›33–44 and 127–143).2 If something has meaning it triggers an action. No action, no meaning. At least so it seems, but as we will see, there is more at stake. We could of course get rid of the somewhat loaded term ‘meaning’ and decide that we are talking about patterns of behavior of organisms. If such a pattern of behavior is activated, the situation has meaning for the organism, if not, it has none. From this behaviorist perspective, ‘meaning’ becomes no more than a word for programmed or learned behavior. This meaningful interaction with an environment brings with it certain risks. Situations are either recognized or they ‘are not’ — in the sense that without a reaction they simply do not exist (except, of course, for an external observer). What if something happens which is not matched by the behavioral patterns available? What if a dangerous situation occurs that triggers no representation at all, and therefore no adequate reaction either? As I said above, for an organism such a situation simply cannot occur — but from the perspective of an observer, the organism will either be lucky, or dead. In the latter case, from the perspective of the organism, life stops. Thus the whole system works well only in a more or less stable environment. In the case of sudden changes, the organism stands with empty hands (or feet, tentacles, etc.,) and can only hope that genetic variation will help the species out. Eventually, if the animal is able to learn, it might succeed in adapting through a process of trial and error. If not, so much the worse, as so many species experience nowadays — being unable to develop an adequate response to human predation quickly enough.3



Semiotic cognition and the logic of culture

From a humanities perspective, this view of the animal world as a world full of meaning is far from self-evident. It requires some sort of a switch of mind. Scholars are used to thinking of meaning as something that pertains to the world of human culture, and is directly related to intentionality and the use of signs, to language, discourse, and texts. This may make it difficult to accept that, for instance, the world of a lion spotting, hunting, and catching his prey can be conceived of as being ‘meaningful’. If meaning, taken as a single phenomenon without any further specification, is something we find in all living beings, we may no longer have to bother about meaning in humans in particular. Importantly, we may be able to study meaning in humans with the same methodology as we study it in other living beings (and maybe even in machines, as it cannot be excluded that meaning exists for machines too). Humans are an animal species after all, and as a bird is known by its note and a man by his talk, why make a fuss? We may be a little more complicated in certain respects, but we are not essentially different. In semiotics, this perspective was firmly established by Charles Morris, who presented his theory of signs in 1938 in the framework of the Encyclopaedia of Unified Science (Morris 1971:â•›13–71). A similar perspective seems to justify attempts like the recent one by Mesoudi, Whiten, and Laland (2006), to elaborate a unified science of cultural evolution. These authors state, self-confidently, that “culture exhibits key Darwinian evolutionary properties”. “If this is accepted”, so they say, “it follows that the same tools, methods, and approaches that are used to study biological evolution may productively be applied to the study of human culture, and furthermore, that the structure of a science of cultural evolution should broadly resemble the structure of evolutionary biology” (Ibid.: 329). The transfer of biological methods and tools to the study of human culture seems to be justified on the basis of the similarity between the non-cultural (biological) and the cultural. According to the authors, in fact, the underlying processes in the two fields (biology and culture) are so similar as to lead us to expect a correspondence (Ibid.: 330). This similarity is made easier to accept if, in the biological realm, organisms also deal with meaning. Although nothing opposes this point of view in principle, it should not be grounded in a simple transfer of terminology from the one domain — that of the study of human culture — to the other — that of research into animal behavior and living beings. There are reasons to be wary of supposing that because the terminology is similar, the phenomena will also be similar. This would be wishful thinking. Rather, we have to ask ourselves what is meant by the terms and whether we are in fact dealing with one and the same phenomenon.

119

120

Barend van Heusden

2. Difference

Entia non sunt multiplicanda praeter necessitate From a methodological point of view the behavioristic approach described in the previous section may seem sound enough. If not necessary, we should avoid proposing a separate concept of ‘meaning’ in dealing with humans. This has consequences, however, for the definition of human culture. Thus, human culture becomes the set(s) of human patterns of (re)action. And again, nothing may seem to be wrong with that. As all other organisms, we too react upon our environment — we like or dislike, cherish or fear situations, and we act according to genetically or socially acquired patterns. A definition of culture corresponding to this point of view is that of Richerson and Boyd (2005), who define culture as “information capable of affecting individuals’ behavior that they acquire from other members of their species through teaching, imitation, and other forms of social transmission” (Ibid.: 5; the definition is also quoted by Mesoudi, Whiten, and Laland 2006:â•›331). If we rephrase the definition, it states that if information affecting the individual’s behavior is acquired from other members of the species, then it is culture. ‘Culture’ is thus a synonym for ‘learned behavior’ — the kind of definition a primatologist like Frans de Waal (2001) would endorse — as it means that many animals have culture too. Apparently, the way in which the information, acquired in a social context, affects an individual’s behavior is either irrelevant or self-evident. The problem with this definition, and with the underlying theory, is that while some forms of culture can certainly be learned, culture as such cannot. One can learn a language, but one cannot learn to be linguistic. One can learn a culture, but one cannot learn to be cultural. Humans can learn a culture because they are cultural animals. A theory of culture as learned behavior fails to explain the two related peculiarities of human culture: interpretation and creativity. There is something which humans seem to do a little differently — not always, but often — with respect to most, if not all, other organisms. Something in their patterns of reaction, in the way in which they use their memory, contrasts with what we find elsewhere in the biological world. It is a small difference with big consequences. Humans not only recognize situations and events on the basis of matching patterns of behavior but, strangely, may not recognize a situation or event.4 This, it will be argued, results in human culture. In our terms, humans experience an absence of meaning. This awareness of absence, or difference (in relation to the acquired patterns of behavior), even seems to be basic to human cognition: what we recognize is not, is never identical with the patterns used to recognize. We do not live in, and reality does not coincide with, our representations. Humans



Semiotic cognition and the logic of culture

not only recognize and act according to a more or less stable patterns but they can also not recognize a pattern. What makes this even more astonishing and disturbing is that they do so at the same time, most of the time. But how can an organism ‘recognize that not’, while ‘recognizing that’? Above I argued that for organisms an absence of meaning coincides with absence as such — it cannot exist, at least not for the organism itself. And yet, for humans, this difference between what they remember, their memory, and the ‘here and now’ (about which there will be much more to say) is commonplace. It relates directly to our sense of reality, our sense of time and space, and our sense of self. We can try to eliminate it — one may think of such different things as isolation cells in psychiatry or meditation techniques — but such attempts are generally doomed to fail. What distinguishes human cognition, and with it human culture, from that of other organisms is not meaning, but the absence of meaning. The presence of this absence in human cognition is extremely strange. First, one might ask what the ability is good for — isn’t it just a nuisance? Second, how can it be realized biologically? Third, how could it have emerged in evolution? A theory of human culture and cultural evolution faces the challenge of explaining how biological evolution could give rise to the specificities of human culture. The hypothesis I want to put forth here is that this absence of meaning, and with it the emergence of the human sense of reality, of time, space, and self, is the effect of a double processing of information. Human cognition draws on a very peculiar system of representation (or ‘pattern matching’) whereby we process what we see and hear5 twice. We process this information, at the same time, in terms of stable patterns, and in terms of global, concrete, and necessarily somewhat ‘fuzzy’ patterns. The consequence of this double processing is a difference between, first, the stable pattern, which corresponds to what we call memory and, second, an instable, continuously changing pattern that invokes what we call the here and now or the present. In a conscious human mind, the two patterns never merge completely. The deep ambiguity this ubiquitous difference creates is not the formal ambiguity we know from, for instance, two mutually exclusive interpretations of one figure (as in the famous duck-rabbit drawing in psychology textbooks). This semiotic ambiguity is inherent in the occurrence of a pattern (schema, script or type) — where ‘occurrence’ is an easy word whose correspondent, however, is very difficult to grasp. Something — in fact anything perceived — is both A (the stable pattern) and not-A (an instable pattern). Further, the occurring not-A is always ~A; it is always ‘more or less similar to A’. Humans deal with this ‘~’ all the time. The unity of sameness (the pattern) and difference (the instable, changing pattern constituting the occurrence) seems to be the starting point of human culture. By hypothesis, this applies neurobiologically, cognitively, and evolutionarily.6

121

122

Barend van Heusden

This difference between the simultaneously present stable and changing patterns, between memories and occurrences has been named God, Nothing, emptiness, Ding an sich, or a je ne sais quoi. It is, one might say, the grounding of the condition humaine. It burdens us, on the one hand, with a huge problem: to act in a coherent world, time and again this difference has to be overcome, solved, or eliminated. Further, we never succeed in doing away with it once and for all. Difference, first and foremost, generates doubt,7 uncertainty, fear and aggression, awe, and even madness. However, it also frees us from immediacy and the continuous ‘here and now’ in which most (or all) other organisms live. By so doing, it gives us the freedom to deal with our environment independently of what is simply ‘the case’; it gives us fantasy, myths, religion, technology, the arts and sciences, philosophy. The worlds which we construct in our imagination allowed us to cope with change in a more sophisticated way than other organisms do — it doesn’t take long to figure out the profit. If the hypothesis put forward here is correct, we should investigate further in two directions. ‘Backward’, in order to try to find out and explain how this double processing could have come about in the evolution of hominids. This is the subject of evolutionary psychology. And ‘forward’, to find out if, and how the peculiar structure of human cognition may have influenced the evolution of human culture. Let us first briefly look backward– speculating about the possible evolutionary development of the human capacity for double processing. It is probable that this development made use of the strong lateralization of the human brain. I do not want to suggest that lateralization caused human culture, but what I do suspect is that a lateralization which was already present (perhaps giving primates handedness8), allowed for the double processing, in terms of stable and changing patterns, of visual and acoustic information in animals with full stereoscopic vision, as this resulted in a 100% identity of the visual input to the two hemispheres. The coordinating process required a substantial enlargement of the equally present ‘chief executive’, as Elkhonon Goldberg (2001:â•›21ff.) dubbed it.9 The here and now of an occurrence is distinguished from memory by its comparative instability. While an occurrence must be recognized in terms of remembered patterns, the on-line processing of patterns is determined by the incoming information — the patterns ‘adjust’ towards what is perceived. ‘Recognition of the unique’ is, paradoxically, what is at stake. A pattern stored in memory that is not continuously adjusted to a changing actuality will become more precise and outlined in the course of time (this sort of processing might have become possible because of the very precise movements of the hand processed in the same hemisphere). The double processing of the stream of incoming information results in a combination of two types of patterns: stable structures on the one hand (which we can now identify as ‘schemata’, ‘scripts’, ‘concepts’, ‘structures’, or ‘signs’) and



Semiotic cognition and the logic of culture

a continuously changing situation on the other (identified as ‘reality’, ‘substance’, ‘object’, ‘the thing itself ’, etc.). Memories can now be stored and worked upon independently of the actual occurrence. Stable memories are used to recognize, to give form to or to ‘interpret’ an ever changing world of appearances.10 Evidently, the most could be made of this development if the set of available memories were considerable. Both factors (a large set of memories and a strong comparator) required brain space. Hypothetically, this may be the ‘compelling reason’11 why the human brain got so large between 2 and 1.5 million years ago, and why the costs of such a large brain were worth paying. 3. Cultural evolution A second set of issues relates to the evolution of human culture. I assume that biological evolution led to the double processing capacity. Further adaptations, such as the growth of the human neo-cortex — to accommodate elaborate linguistic and logical patterns, for instance — probably didn’t change the basic make-up, but elaborated upon it. Thus on the basis of a relatively small adaptation (compare the DNA of chimpanzees and humans12), a whole new structure could emerge: that of human culture. It allowed for the development of ‘memory content’ independently of the occurrence, thus enlarging enormously the range of possible (re)actions to a changing environment. But could this new biological structure also have influenced the evolution of human culture? And if so, how? As we have argued, in human cognition a stable pattern (memory) is related to a changing pattern (a situation or occurrence). Stable patterns or signs, are used to recognize (to give form to, to interpret, to signify) the more floating patterns that constitute occurrences. The here and now of the occurrence, in turn, is what is not a sign — it is ‘reality’, ‘the world’, ‘the object’ — though it is a construction based on memories. But the difference between the two is not a construction of our memory, and that seems to be of crucial importance. Cognition, in humans, becomes intentional, or semiotic.13 On the view presented here, this needs, not a new ‘semiotic’ organ, but a systematic reorganization of the information processing structure. A sign is thus not a ‘thing’ but, rather, a particular way of processing information, of representation or cognition. Once the double processing is in place, two different realms — that of the stable memory and that of a changing present — could appear together with the relation between them. It is this relation which we have become used to envisage as ‘intentionality’ or ‘aboutness’. The many forms into which this aboutness has developed bear witness to the complexity of human culture. Signs are about the world, they constitute an imagined world, but they are not the world — that is, the difference can never be eliminated. Difference is here

123

124

Barend van Heusden

to stay, functioning as a continuous trigger and motor of the semiotic process — of human culture. Until now, I have focused mainly on the processing of sensory information. But there obviously is a motor part to the story as well. Representations govern our actions — and therefore the double processing results in a new type of action as well: one in which we act out the stable patterns or memories. In fact, human actions also double up. They come to encompass a sign on the one hand and actuality on the other. The doubling of action results in the appearance of gestures, and also artefacts, that are recognized as ‘signs’ or ‘texts’. While they are always about something this may be, not the situation at hand (or actuality), but a situation elsewhere (and, perhaps in another time). While I am sitting at the dinner table, therefore, I may be thinking about what happened at work that day. The desire to share my thoughts about this earlier event can become very strong — ‘Out of the abundance of the heart the mouth speaketh’ (Luke 6:45). Signs are acted out or ‘externalized’14 and thus our thoughts become perceivable, we can communicate them and store them in external memory storage systems (Donald 1991). Traffic signs are particularly interesting, not only for their stable conventional meaning, but because they so clearly function as signs. Traffic signs stabilize patterns of behavior by giving a changing reality, (e.g., a crossing), form. What turns the traffic sign into a sign is not, as is so often asserted, its conventional meaning. Rather, what matters is the fact that it stabilizes, or signifies, a changing reality. This principle, I think, should be at the heart of a pragmatist theory of human culture: human culture is the cognitive process in which more or less stable memories are used to deal with the differences an ever-changing world confronts us with. We can now define signs as ‘stable memories that are used to recognize a changing reality’. Moreover, culture is the work that we carry out on these memories, in order to optimize them for this function. The double processing which underlies semiotic, or cultural cognition, saddles humans with the problem of a ‘double scope’,15 one that has to be solved if successful action is to be made possible. This is the basic motivation for culture: humans have to deal with the problems that arise in perception. And they will do so — in a variety of ways. But these problems also give them a certain freedom — the freedom of choice with respect to the memories available. The ways in which difference is dealt with will determine the evolution of human culture: how can memories be stored effectively, and how can they be related to a changing reality, as efficiently, that is, as safely and economically, as possible? Humans have tried to come to grips with this changing reality in a number of different ways, and the story of these attempts is the story of cultural evolution. Through the doubling of the information processing system — basically the



Semiotic cognition and the logic of culture

doubling of the perceptual representation — an ‘episodic culture’, as described by Donald (1991:â•›124–161) and studied in chimpanzees and bonobos by Frans de Waal, could become a semiotic culture. In a semiotic culture, the imitating behavior is different from that in an episodic culture. This is because, while episodic imitation involves the copying of behavior, imitative semiosis, or mimesis, permits a new way of representing behavior. The mimetic imitation is about the action imitated. Through mimesis, stable episodic memories, or signs, come to be acted out as audible and visible signs.16 In a second stage, then, the memories that have become autonomous vis-à-vis the perception of reality are the basis upon which imagination can flourish (Modell 2003:â•›111ff.). The acting out of the memories manipulated through imagination grounds technology: the fabrication and use of artefacts (tools). In a third stage, the acting out of stable patterns (in vocal gesturing, for instance) becomes more and more ‘schematic’. This permits the development of representations that we may identify as conceptual. These are signs which cease to be one-place entities (images or artefacts) as they became two-place entities that consist in an image of a bodily or vocal gesture and a prototypical image of a situation that this evokes. Abstract conceptual thinking came around, creating conceptual meaning. Ferdinand de Saussure (1916:â•›155–169) made it convincingly clear that concepts cannot arise apart from standardized images, and from the gestures related to these images (the famous ‘images acoustiques’). The emergence of vocal gestures, or spoken language, must have been a crucial phase in this evolutionary process. In a subsequent stage, the world of man-made concepts would hit upon what cannot be manipulated — upon necessity. The discovery of necessity, which probably occurred only around 70.000 to 50.000 years ago, and may have been related to the discovery and invention of graphic signs, became the basis of a new evolutionary development in culture — that of theoretical thought (Donald 1991:â•›269–360). It caused an unprecedented explosion of creativity (cf. Mithen 1996, 1998). Because of the cumulative character of the evolutionary process (earlier stages are not abandoned but incorporated), with each new phase, the semiotic complexity of the world, including the self, social interaction, and physical environment, increased.17 Semiosis

accommodation

assimilation

concrete / particular

Perception

Imagination (tool use)

abstract / general

(Theoretical) Analysis

Conceptualization

Figure╯1.╇ The four stages of semiosis. The arrow indicates the temporal sequence. The concepts ‘accommodation’ and ‘assimilation’ refer to the ‘direction’ of the cognitive process.

125

126

Barend van Heusden

Human culture is thus a cognitive process in which more or less stable memories are used to deal with the difference that the world forces upon us. The here and now presents itself as a deviance, a new and unknown combination of available patterns. It is this deviance that offers a certain freedom insofar that it calls for images that can be ‘grasped’. Thus the miracle of the creative image, of the metaphor and the model come about. First, there must be a situation, or occurrence, that doesn’t match a stable sign. To get hold of the difference, we search for other memories — other signs. Thus, for example, there is a face, and it has wrinkles, yes, but what more does it remind me of? Parchment, or maybe an old apple or cloth, or an arid landscape, cut through by deep canyons. The image that ‘fits’ the new situation best solves the problem created by the difference in perception. This image was not there in the first place, but came along, as an answer to the question: what is it that I perceive, how should I deal with this difference? The ‘blend’ created by the new image is not the problem but, rather, its solution. Indeed, we all acknowledge this when confronted with a meaningful metaphor, trope, image, model, or narrative. These enable us to grasp a difference experienced in life. Alternatively, take the following case: I want to express my feelings towards someone I hate — I could simply say so, of course, but I would have to acknowledge that the concept hate does not exhaustively cover the feelings I have (the ‘difference’!). For this reason, I turn to my memories to find images, concepts, or models that evoke these feelings. Indeed, I might come up with a list like that of The Epic of Gilgamesh (approx. 3000 B.C). In this oldest of our surviving literary works, Gilgamesh insults the goddess Ishtar by means of comparisons: “Your lovers have found you like a brazier which smoulders in the cold, a backdoor which keeps out neither squall of wind nor storm, a castle which crushes the garrison, pitch that blackens the bearer, a waterskin that chafes the carrier, a stone which falls from the parapet, a battering-ram turned back from the enemy, a sandal that trips the wearer” (Sandars 1960:â•›86). 4. Self-consciousness The cognitive process we have been discussing until now is itself one dimension of our continuously changing human reality. As such it is one more source of difference, and (to paraphrase Kant) of wonder: we remember and feel ourselves feeling, we remember and look at ourselves looking, we remember and hear ourselves hearing, and we remember and think about ourselves thinking, individually as well as collectively. Here self-consciousness as a distinctive human faculty arises. In, and as, a changing here and now, we are a difference to ourselves; we are a difference to be dealt with. And once more, we draw on stable memories.



Semiotic cognition and the logic of culture

Once children understand the concept of a false belief or the mind’s ability to incorrectly represent the world, they realize that beliefs are mental representations and hence can distinguish between mind and world. This comprehension often happens between the ages of 3 and 5 years. This awareness has been tested in nonhuman primates and stems from the assumption that along with comprehending the difference between mind and world comes an understanding of the sources of knowledge and the representation of beliefs by others (Gazzaniga et al. 1998:â•›547, italics mine).

Our identity as human beings is thus a process in which we continuously have to invest energy. It consists of a set of more or less stable memories (images, stories, ideas — both internal and external),18 a changing present, as well as the process in which the memories are used to deal with the change. Self-consciousness, it would seem, does not require, as Douglas Hofstadter (2007:â•›101ff.) assumes, ‘strange loops’. On the contrary, I would say: as individuals, or as groups, we are no different from the rest of the cultural world we inhabit: we exist as a set of memories and as a process in which these memories are used to deal with a dynamic environment. I am, and we are, ‘in the making’. Viewing the self as a process rather than as an object becomes more natural (Noble 2006). The self is always a constructed self as well as a remembered self or, rather, a self ‘under construction’. Self-consciousness, both individual and collective, is thus a form of recursion — not formal, but semiotic recursion. Whereas in formal recursion a form consists of elements that reproduce that same form (as in a triangle consisting of three smaller triangles, each of which again consists of three smaller triangles, etc.), in semiotic recursion the process of representation (of ‘aboutness’) is about that process itself. This is what we call reflexivity, meta-representation, or meta-cognition. Moreover, this meta-cognitive process is again remembered, which gives us a strong feeling of being, and having been, alive (cf. Metzinger 2003:â•›325ff.). This very basic form of dealing with difference in a self-referential or meta-cognitive way lies at the basis of a number of cultural domains. The arts, religion, and philosophy are important forms of meta-cognition in modern and in contemporary culture. Whereas with art we reflect upon our life through concrete images, sounds, and stories (cf. Donald 2006), in religion and philosophy, we do so through more abstract conceptualizations.19 Meta-semiosis

accommodation

assimilation

concrete / particular

Self-perception

Mimesis; the arts

abstract / general

Philosophy; theory

Myth; religion; ideology

Figure╯2.╇ The four stages of meta-semiosis. The arrow indicates the temporal sequence. The concepts ‘accommodation’ and ‘assimilation’ refer to the ‘direction’ of the cognitive process.

127

128

Barend van Heusden

5. Conclusion The intuition that the evolution of human culture depends upon an immanent logic, which is directly related to its semiotic structure is expressed, though of course in very different terms, in the work of Giambattista Vico (1744) and Ernst Cassirer (1923–1929; 1944), as well as in that of the art philosophers Erwin Panofsky (1939) and Alois Riegl (1966), and the historians Fernand Braudel (1969) and Hayden White (1973). Less obviously perhaps, it also grounds the evolutionary psychology of Merlin Donald (1991, 1998). Cultural evolution seems to be not so much a matter of continuous ‘progress’ — which would imply an external viewpoint from which a value-judgment can be made — but of an increasing diversity and complexity of the semiotic process. Though the basic semiotic structure is relatively simple, the resulting reality of culture turns out to be extremely varied and complex. The ‘double processing’ hypothesis presented in this paper would allow us to shed light upon the evolution of human culture, and of cultural evolution as well. Semiotic cognition became possible because of a change in the primate information processing system. Semiotic cognition may well be that “super module” which explains the specificity of human cognition better than the ones proposed until now — language and theory of mind.20 In fact, it explains both the emergence of language as one form of semiosis and of the theory of mind as a logical consequence of the semiotic process. We do not need to assume the existence of a physical symbol system — but what we do need to assume is the existence of an organization which allows for semiotic cognition. Once this system is put in place, a new logic of development can emerge. This logic of cultural evolution is based on what seems to be, in the end, a simple criterion. Specifically, we need to ask if a sign or sign system offers better chances of dealing with the difference that arises in a specific personal, social, and natural context. Further, in so doing, does it offer better chances of prediction and survival? Thus we come back to biology. The search for better solutions to the problem of difference can explain, I think, individual, social, and geographical variations in culture, as well as a steady increase in both abstraction and cultural complexity. ‘A more progressive and rigorous study of culture’ (Mesoudi, Whiten, and Laland 2006:â•›329) should, I think, beware of an all too simple transfer of terms and concepts from biology to the study of culture. Culture is not necessarily more complex than biology. In fact, I think it may be simpler, ‘structure-wise’. But although grounded in biology, it is something else. The ‘double processing’ hypothesis offers a synthesizing framework. It differs in important ways from ‘dual processing’ views in that no appeal is made to combination of two more or less autonomous



Semiotic cognition and the logic of culture

subsystems.21 Rather, the duality posited here is taken to be inherent to human cognition. Although culture certainly is a fact of human biology, what makes it so interesting is the fact that out of biology, in this case, emerges a form of cognition that became free — at least to a certain extent — of the laws of biology. This was achieved, I suggest, by adding a new dimension to these laws, namely the dimension of the semiotic.22 The approach is deeply and truly Darwinian, in the sense that it applies Darwin’s perspective where it can, and should be applied, and uses it to explain how and why a non-Darwinian evolutionary system, though nested in biological evolution and fulfilling the same goals, could emerge. (Neo-)Darwinism is fruitful, but only up to a certain point. Once semiosis arises, the laws of development change. Instead of ‘BVSR = Blind Variation + Selective Retention’, the logic of the semiotic process, involving perception, imagination, conceptualization, and analysis will determine the course of human evolution.23 Whereas selective retention may continue to function as it did in animal life, cultural variation is not blind, but informed by semiosis and driven by semiotic strategies.24

Notes *╇ I want to thank Stephen Cowley, as well as three anonymous reviewers, for their valuable comments, which certainly helped me to improve the presentation of my argument. 1.╇ In this context, it is sufficient to stress that I use imitation in a rather simple sense — as the copying of behavior. For the broad discussion on imitation in all its forms, see, for instance, Hurley and Chater (2005). 2.╇ Von Uexküll, with the help of Georg Kriszat, tried to give a figurative impression of the Umwelt of a number of different species (Von Uexküll and Kriszat 1940). 3.╇ Thus it seems, for instance, that fish in the North Sea are now adapting to human fishing practice by staying smaller and reproducing at a higher rate. 4.╇ While there may be some evidence of this in nonhuman primates, what matters here is cognitive structure and not species differentiation. For an elaborate discussion of the discontinuity between human and nonhuman minds, see Penn et al. (2008). 5.╇ About the other senses I’m very uncertain. 6.╇ But see Fauconnier and Turner (2002:â•›7) for a somewhat different point of view. 7.╇ Both William James (1983:â•›913–917) and Charles Sanders Peirce (1986:â•›14ff.) stressed the role of doubt as a bodily experience and as the origin of all inquiry (cf. also Mark Johnson 2007:â•›53–54). 8.╇ See Andrew and Rogers 2002; MacNeilage 2008.

129

130

Barend van Heusden

9.╇ The abundant scientific literature on hemispheric lateralization could be reread in the light of this hypothesis. 10.╇ See Gazzaniga et al. (1998:â•›542 ff.) on the emergence of the brain interpreter in human species: “It turns out that we humans have a specialized system to carry out this interpretive synthesis, and it is located in the brain’s left hemisphere. The interpreter is a system that seeks explanations for external and internal events in order to produce appropriate behaviors in response. We know it to be only in the left hemisphere, and it appears to be tied to our capacity to see how contiguous events relate to one another” (Ibid.: 543). 11.╇ This is a reference to a quote by Ian Tattersall: “There is simply no compelling reason we know of to explain why human brains got so large” (in Bryson 2003:â•›396). 12.╇ 2–5% differences, depending on the matter of counting. 13.╇ See Thure von Uexküll (1984) on ‘res’ and the human Umwelt. 14.╇ I use this term with a certain hesitation, as I have my doubts about the internal/external dichotomy. 15.╇ The reference is to Fauconnier and Turner’s theory of conceptual blending (2002). 16.╇ This corresponds to Piaget’s (1962) notion of “representative imitation”, which according to him emerges out of sensory-motor imitation, via deferred imitation (cf. Zlatev 2007). 17.╇ For a more elaborate treatment of the phases of cultural evolution, see van Heusden 2004. 18.╇ This is illustrated by Edelman’s work (1989) on how memories serve as patterns that have been enacted. 19.╇ I dealt with artistic and literary semiosis more extensively in van Heusden 2007. 20.╇ See Penn et al. 2008:â•›120–121. 21.╇ Cf. Evans (2003) for an overview of the dual processing views. 22.╇ Here my approach comes near to interactivism, which is premised on the notion that persons constitute an emergent ontological level that develops out of the biological realm (Bickhard and Christopher 2007). 23.╇ Lassègue (2007) develops a related ‘alternative’ view (to neo-Darwinism, that is), strongly oriented towards the work of Ernst Cassirer. See also Wolfgang Wildgen’s (2003) work on the evolution of human language. 24.╇ See Wheeler, Ziman, and Boden (eds) (2002) on cultural evolution and neo-Darwinism. See also Premack (2007), where the significant differences between human and animal cognition are stressed.



Semiotic cognition and the logic of culture

References Andrew, R.J. and Rogers, L.J. 2002. Comparative Vertebrate Lateralization. Cambridge: Cambridge University Press. Bax, M., van Heusden, B., and Wildgen, W. (eds). 2004. Semiotic Evolution and the Dynamics of Culture. Bern: Peter Lang. Bickhard, M.H. and Christopher, J.C. 2007. “Culture, self and identity: interactivist contributions to a metatheory for cultural psychology”. Culture and Psychology 13: 259–295. Braudel, F. 1980. On History. Translated from the French by S. Matthews. London: Weidenfeld and Nicolson. [Originally published 1969] Bryson, B. 2003. A Short History of Nearly Everything. London: Doubleday. Cassirer, E. 1944. An Essay on Man. An Introduction to a Philosophy of Human Culture. New Haven: Yale University Press. Cassirer, E. 1980. The Philosophy of Symbolic Forms. Vol 1: Language, Vol 2: Mythical Thought, Vol 3: The Phenomenology of Knowledge [Translated by R. Mannheim]. New Haven/London: Yale University Press. [Originally published 1923–1929] Changeux, J.-P. 1997. Neuronal Man. The Biology of Mind. (Translated by L. Garey. With a new preface by V.B. Mountcastle). Princeton, N.J.: Princeton University Press. [Originally published 1983] Clark, A. 1997. Being There: Putting Brain, Body, and World together again. Cambridge, MA: The MIT Press. Donald, M. 1991. Origins of the Modern Mind. Three Stages in the Evolution of Culture and Cognition. Cambridge, MA: Harvard University Press. Donald, M. 1998. “Mimesis and the executive suite: Missing links in language evolution”. In J.R. Hurford, M. Studdert-Kennedy, and C. Knight (eds), Approaches to the Evolution of Language. Social and Cognitive Bases. Cambridge: Cambridge University Press, 44–67. Donald, M. 2006. “Art and cognitive evolution”. In M. Turner (ed), The Artful Mind. Cognitive Science and the Riddle of Human Creativity. Oxford: Oxford University Press, 3–20. Edelman, G. 1989. The Remembered Present. A Biological Theory of Consciousness. New York: Basic Books. Evans, J.St.B.T. 2003. “In two minds: dual-process accounts of reasoning”. Trends in Cognitive Sciences 7(10): 454–460. Fauconnier, G. and Turner, M. 2002. The Way We Think. Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books. Gazzaniga, M.S., Ivry, R.B., and Mangun, G.R. 1998. Cognitive Neuroscience. The Biology of the Mind. New York and London: Norton. Gibson, J.J. 1979. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin. Goldberg, E. 2001. The Executive Brain: Frontal Lobes and the Civilized Mind. Oxford: Oxford University Press. van Heusden, B. 1999. “The emergence of difference: Some notes on the evolution of human semiosis”. Semiotica 127(1–4):â•›631–646. van Heusden, B. 2004. “A bandwidth model of semiotic evolution”. In M. Bax, B. van Heusden, and W. Wildgen (eds), Semiotic Evolution and the Dynamics of Culture. Bern: Peter Lang, 1–28. van Heusden, B. 2007. “Semiosis, art, and literature”. Semiotica 164(1-4):â•›133–147. Hofstadter, D. 2007. I Am a Strange Loop. New York: Basic Books.

131

132

Barend van Heusden

Hurford, J.R., Studdert-Kennedy, M., and Knight, C. (eds). 1998. Approaches to the Evolution of Language. Social and Cognitive Bases. Cambridge: Cambridge University Press. James, W. 1983. The Principles of Psychology. Cambridge, MA: Harvard University Press. Johnson, M. 1987. The Body in the Mind. The Bodily Basis of Meaning, Imagination, and Reason. Chicago and London: The University of Chicago Press. Johnson, M. 2007. The Meaning of the Body: Aesthetics of Human Understanding. Chicago: The University of Chicago Press. Lassègue, J. 2007. “Une réinterprétation de la notion de forme symbolique dans un scénario récent d’émergence de la culture”. Revue de métaphysique et de morale 2: 221–236. MacNeilage, P.F. 2008. The Origin of Speech. Oxford: Oxford University Press. Mesoudi, A., Whiten, A., and Laland, K.N. 2006. “Towards a unified science of cultural evolution”. Behavioral and Brain Sciences 29(4): 329–347. Metzinger, T. 2003. Being No one: The Self-model Theory of Subjectivity. Cambridge, MA: The MIT Press. Mithen, S. 1996. The Prehistory of the Mind. A Search for the Origins of Art, Religion and Science. London: Thames and Hudson. Mithen, S. 1998. “A creative explosion? Theory of mind, language and the disembodied mind of the Upper Paleolithic”. In S. Mithen (ed), Creativity in Human Evolution and Prehistory. London and New York: Routledge, 165–191. Mithen, S. (ed). 1998. Creativity in Human Evolution and Prehistory. London and New York: Routledge. Modell, A.H. 2003. Imagination and the Meaningful Brain. Cambridge, MA and London: The MIT Press. Morris, C. 1971. Writings on the General Theory of Signs. The Hague and Paris: Mouton. Noble, D. 2006. The Music of Life. Biology beyond Genes. Oxford University Press. Panofsky, E. 1939. Studies in Iconology: Humanistic Themes in the Art of the Renaissance. New York: Oxford University Press. Penn, D.C., Holyoak, K.J., and Povinelli, D.J. 2008. “Darwin’s mistake: Explaining the discontinuity between human and nonhuman minds”. Behavioral and Brain Sciences 31(2): 109–178. Piaget, J. 1962. Play, Dreams, and Imitation in Childhood. London: Routledge and Kegan Paul. Peirce, C.H. 1986. Writings of Charles S. Peirce: a Chronological Edition. M.H. Fisch (general ed); Vol. 3: 1872–1878, C.J.W. Kloesel (ed). Bloomington, IN: Indiana University Press. Premack, D. 2007. “Human and animal cognition: Continuity and discontinuity”. Proceedings of the National Academy of Sciences of the United States of America, vol. 104(35): 13861–13867. Richerson, P.J. and Boyd, R. 2005. Not by Genes Alone. How Culture Transformed Human Evolution. Chicago and London: The University of Chicago Press. Riegl, A. 2004 Historical Grammar of the Visual Arts. Translated from the German by J.E. Jung; foreword by B. Binstock. New York: Zone Books. [Originally published 1966] Sandars, N.K. 1960. The Epic of Gilgamesh. English version with introduction by N.K. Sandars. Harmondsworth: Penguin Books. de Saussure, F. 1974, Course in General Linguistics. Introduction by J. Culler; edited by C. Bally and A. Séchehaye in collaboration with A. Riedlinger; translated from the French by W. Baskin. London: Owen. [Originally published 1916] de Sousa, R. 2007. Why Think? Evolution and the Rational Mind. Oxford: Oxford University Press. Tattersall, I. 2008. The world form Beginnings to 4000 BCE. Oxford: Oxford University Press.



Semiotic cognition and the logic of culture

Turner, M. (ed). 2006. The Artful Mind. Cognitive Science and the Riddle of Human Creativity. Oxford: Oxford University Press. von Uexküll, J. 1973. Theoretische Biologie. Frankfurt/Main: Suhrkamp. [Originally published 1920] von Uexküll, J. and Kriszat, G. 1970. Streifzüge durch die Umwelten von Menschen und Tieren. Bedeutungslehre. Frankfurt/Main: Fischer. [Originally published 1940] von Uexküll, T. 1984. “Zeichen und Realität als anthroposemiotisches Problem”. In K. Oehler (ed), Zeichen und Realität. Vol. 1. Tübingen: Stauffenburg: 61–72. Vico, G. 1984. The New Science. Unabridged translation of the third edition (1744) with the addition of “Practice of the new science” [by] Th. G. Bergin and M.H. Fisch. Ithaca: Cornell University Press. [Originally published 1744] de Waal, F. 2001. The Ape and the Sushi Master: Cultural Reflections by a Primatologist. New York: Basic Books. Wheeler, M., Ziman, J., and Boden, M.A. (eds). 2002. The Evolution of Cultural Entities. Oxford: Oxford University Press, for the British Academy. White, H. 1974. Metahistory. The Historical Imagination in 19th century Europe. Baltimore and London: The Johns Hopkins University Press. Wildgen, W. 2003. The Evolution of Human Language. Scenarios, Principles and Cultural Dynamics. Amsterdam: Benjamins. Zlatev, J. 2007. “Embodiment, language, and mimesis”. In T.J. Ziemke, J. Zlatev, and R. Franck (eds), Body, Language, Mind. Vol. 1: Embodiment. Berlin: Mouton de Gruyter: 297–333.

133

Ecological pragmatics Values, dialogical arrays, complexity, and caring* Bert Hodges

Gordon College, Wenham, MA & University of Connecticut, Storrs, CT, USA

This paper explores the hypothesis that first-order linguistic activities are better understood in terms of ecological, values-realizing dynamics rather than in terms of rule-governed processes. Conversing, like other perception-action skills (e.g., driving) is constrained by multiple values, heterarchically organized. This hypothesis is explored in terms of three broad approaches that contrast with models of language which view it as a cognitive system: (1) conversing as a perceptual system for exploring dialogical arrays (Hodges 2007a); (2) conversing as an action system for integrating diverse space-time scales (Van Orden 2007); and (3) conversing as a caring system for embodying the context-sensitivity and interdependency necessary to realize values (Hodges 2007b). Approaching language as a caring action-perception system leads to a reconsideration of cognitive dimensions of linguistic activities, including consciousness, pragmatics, suffering, and hope. Keywords: action systems, attunement, consciousness, context-sensitivity, ecological psychology, heterarchy, nonlinear dynamics, suffering, values

1. Beyond rules It is widely believed that language is rule-governed, or at least, rule-following. A typical example is offered by Smith (1999:â•›29): “Any native speaker of English knows that I speak English fluently and I speak fluent English are acceptable, and that I speak fluently English is wrong”. But the example fails. One quickly thinks of the legitimate possibility of saying I speak fluently English, Swahili, and French. Smith might object that this is not the same (attempted) sentence, but the reply fails as well, since one can say: I speak fluently English. Swahili and French I speak only a little. This two-sentence sequence — which might be a reply to a question about one’s fluency — is not only allowable, it is elegantly stated. Only a fluent speaker would say such a thing.

136

Bert Hodges

The point of this counterexample is not that it refutes the claim that language is rule-following, but that both my ability to generate a counterexample, and Smith’s ability to pose it as an example of an improperly formed sentence in the first place, are dependent on something other than rules. If language were dependent solely on rules to function, then it would not be possible for Smith to generate intentionally and competently an ill-formed sentence. In doing so he would violate the rules he claims language follows. Similarly, I would have had no basis for generating an alternative possibility. Such supposedly rule-following sequences point beyond themselves: Our ability to speak competently and fluently requires more than rules. What beyond rules is required? A variety of answers might be offered to this question, but the one I will explore in this paper is that rule-following stabilities point to and depend on values-realizing dynamics (Hodges 2007a, 2007b; Hodges and Baron 1992). Exploring the nature of these values-realizing dynamics will lead to a consideration of three broad approaches to rethinking how we should understand language. The first approach is to look at language as a perceptual system (Gibson 1966) for exploring dialogical arrays (Hodges 2007a). This will provide an opportunity to consider how dialogue enables both locating and way-finding. The second approach is to work out what it might mean to claim language is an action system (Reed 1982, 1996) for coordinating diverse space-time scales (Thibault 2005). Consideration of the activities of speaking and listening (and/or signing and looking) will reveal the complexity of their dynamics, including the context-sensitivity, global interdependence, and non-causal nature of these dynamics. The third approach is to embody the concern that language is a caring system (Hodges 2007b), rather than (primarily) a representational system or a control system. This approach affords consideration of the hypothesis that conversations embody a complex set of action-perception skills, including the pragmatics of consciousness, that allow humans to take care of each other and the environments that surround and support them. Before going further it is important to clarify the term language. To refer to language as a perception-action system for caring is to consider what has been called first-order language (Love 2004), or languaging (Cowley 2005). Language in this first-order sense is a diverse and distributed set of activities that involve multiple speakers over time, enmeshed in cultural histories that unfold over an array of time scales. Most of these diverse, distributed activities are not represented in individual brains; they are collective phenomena. Second-order language is what is often in view in linguistics, which is a series of reflections on various stabilities across these speakers, scales, and collective activities. Various patterns and dependencies that have been described as phonological, syntactic, or semantic have been



Ecological pragmatics

identified. The identification of these patterns and dependencies is what has made the “language is rule-governed” claim appealing. These patterns and dependencies are frequently reified, counted as codes (Kravchenko 2007; Love 2007), and located in individuals, so that language is treated as an acquisition of individuals or a property of neural modules. To avoid confusing these second-order claims with first-order activities, I will sometimes refer to conversing, or speaking and listening, or linguistic activities. On the other hand, I am unwilling to give up the term language as referring to first-order activities of conversing with others. On the traditional view, language is viewed as a system, and conversation as the use of that system. The argument of this paper is that language is use and that the use is systematic, but in ways more complex than previously assumed (i.e., rule-governed ways). Language as a pragmatic, first-order activity that emerges in use and is more extensive and more integrated (i.e., systemic) than most current cognitive models of language envision, will be discussed further in considering language as an action system. This article will proceed as follows: To explore language as a distributed set of dialogical activities (Linell 2007) — beyond rules — I will first sketch an ecological (Gibson 1979), values-realizing approach to the nature of action and perception (Hodges and Baron 1992) that may be applied to language (Hodges 2007a). Then I will explore in more detail how such an ecological, values-realizing approach might be fleshed out, following the three avenues mentioned earlier, language as a perceptual system, as an action system, and as a caring system. The hope is that looking at linguistic activities from these varying perspectives — caring, action, perception — will provide an integrated view that yields a richer, more comprehensive appreciation of what it means to converse with others. In integrating these three perspectives, issues relating to consciousness and pragmatics will come to the fore, along with their emotional overtones. 2. Ecological perception and action: Realizing values in driving and conversing The theoretical framework used in this paper to consider linguistic activities is an ecological, values-realizing psychology, sometimes referred to as values-pragmatics theory. This is a general approach to psychology (Hodges 2007b; Hodges and Baron 1992), which has been applied to issues in social psychology (Hodges 2004; Hodges and Geyer 2006), perception-action studies (Hodges 2007b; Hodges and Lindhiem 2006), developmental psychology (Hodges and Baron 1992), and language studies (Hodges 2007a). According to values-realizing theory, all actions, whether driving a car or conversing with a colleague, are constrained and

137

138

Bert Hodges

legitimated by multiple values. Values are the real goods that actions must realize sufficiently for an ecosystem to exist; thus, values are obligatory demands that define what constitutes good driving or a good conversation (Hodges 2007a 2007b; Hodges and Baron 1992). These multiple constraints on action are often in tension (e.g., accuracy and safety in driving may pull against each other; Hodges 2007b), but overall they work cooperatively, since each value can only be realized jointly with all the others (Hodges and Baron 1992). One of the crucial sources of values-realizing psychology is Gibson’s (1966, 1979) ecological theory of perception with its focus on agency and affordances. Perceiving is an activity that depends on an animal’s embodied movement in and through various arrays (e.g., optic, acoustic) that yield information (i.e., invariants revealed through variance), that is sufficiently specific to its sources that the animal can locate food, shelter, potential mates, and the other goods (i.e., values) for its existence. Perceiving and acting are reciprocal: Neither is possible without the other, though neither causes the other to occur. Actions can be exploratory, performatory, or both. Perceiving is not fundamentally conceptual: It is a matter of an animal acting so as to detect affordances, not objects or categories or any other product of thinking. Affordances are the possibilities for the animal’s interacting with its environmental surroundings to realize goods. Cats, for example, do not see windowsills, but see/feel jumping-on and lying-down affordances. They see a field of potential actions, defined with respect to the goods of the ecosystem as a whole and its demands on them. The animal acts as an agent of the ecosystem, realizing affordances through action. Before turning to linguistic activity in humans, consider the example of humans driving vehicles, described more fully in Hodges (2007b:â•›154–163). Gibson and Crooks (1938) proposed that driving is organized and guided by a field of safe travel. As Figure╯1 illustrates, the field of safe travel is a very real physical field that can be described as a continuously unfolding set of possible paths opening before the vehicle, bounded by an ever-changing array of physical obstacles that serve as lawful constraints. It is a socially constrained field as well: There are lane markings, various signals and signs, all of which function as rules that may be followed (or not), to assist the driver in maintaining the field. It is a jointly enacted field: The movement of any one vehicle contributes to the size and shape of the field of other nearby vehicles. Thus, the field is also moral, one defined by faith: We must trust that others’ movements, as well as our own, are intended to realize values. What are the values to be realized in driving? Gibson and Crooks’s naming of the field makes clear that it is defined by values, namely safety and travel. The whole reason humans invented/discovered driving was because comprehensiveness is a value (e.g., greater range of movement; i.e., travel). But the value of travel bears a complex relationship to safety. Greater range



Ecological pragmatics

Figure╯1.╇ Example of a field of safe travel, as shown in Gibson and Crooks (1938).

of movement can increase safety (e.g., more resources, greater protection from predation), but more travel also increases the number of hazards and the risk of harm. Although they are not named, there are other values that constrain the field as well, such as accuracy, tolerance, and efficiency. According to Hodges and Baron (1992), values operate in heterarchical fashion, providing for the “conservation of the integrity of the ecosystem as a whole” (Hodges 2007b:â•›155). What is meant by heterarchical relations among values is that actions are mutually constrained by all the values, so that there is no fixed, hierarchical ordering of values; rather, across time and task, values vary in their ordering for the sake of the joint realization of all the values (i.e., the ecosystem as a whole). There are real tensions among values, but it is this ongoing tension that gives energy to action, emotion, and cognition. It is easy enough to think of these tensions as tradeoffs, but that misses the necessary reciprocity of values as they work cooperatively to enhance each other (Hodges and Baron 1992:â•›281–282). For example, without accuracy (e.g., direction) and efficiency (e.g., speed), driving would be of little use, but driving as close as possible to potential hazards or as fast as possible is not good driving either. Safety usually demands reduced speeds and accuracies (or greater tolerances), but in an “emergency” it might demand increased speed and smaller tolerances. Good driving is a continuous balancing act, jointly realizing all the values. To summarize, driving is an ecosystem defined by values, which are demands that both constitute and evaluate the activity. Values make it possible to develop or revise rules, and/or to take advantage of newly discovered lawful relations, all in the service of the activity. These revisions of rules and rearrangements of natural law assemblies open up new opportunities for realizing the values embodied in the activity.

139

140

Bert Hodges

Conversing, like driving, is an ecosystem defined by values. Among the values that define it are clarity, coherence, comprehensiveness, and complexity (Hodges 1990; Hodges and Baron 1992). Both in our speaking and our hearing we work to make utterances differentiated, integrated, flexible, and rich enough (first-order language) to be labeled as articulate, grammatical, meaningful, and useful (second-order language). These four values line up reasonably well with Slobin’s (1979) hypothesized constraints guiding the evolution and development of languages: Be clear; be processible (e.g., grammatical and pragmatic coherence); be quick and easy (e.g., blurring of inflections); and be expressive (e.g., semantic and rhetorical diversity and richness). These constraints, he argues, are in “constant flux because of the dynamic equilibrium maintained between these competing pressures” (Slobin 1979:â•›191). No utterance can fulfill perfectly all four of Slobin’s mandates because they are in tension, but over time the values (to which the mandates point) jointly provide for the sufficiency of language. While clarity and coherence, and perhaps comprehensiveness, will strike most readers as intuitively reasonable constraints, complexity may be more surprising. Why would our utterances be attracted to complexity, rather than simplicity or transparency? Perhaps the simplest and most powerful reply is to point to the pragmatic character of language. Pragmatics highlights how people complicate the ways in which they speak with each other, shaping and reshaping linguistic forms and patterns to increase the force, beauty, and subtlety of what is said and what is understood. Metaphor, irony, politeness, and all the other phenomena of pragmatics make matters more complicated for speakers and hearers, but paradoxically they also make it more effective and interesting (what Cowley 2007:â•›1 referred to as the “magic of wordings”). Wray and Grace (2007:â•›543) have argued that complexity is the “default setting” for human linguistic activity. Conversations in small intimate groups are more semantically and grammatically esoteric and irregular than conversations among strangers, and children have greater tolerance for complexity and inconsistency of phonological forms than adults. Rule-based systematicity may be a cultural development, not a natural starting point in conversing with others. In fact, Evans and Levinson (2009) have proposed that the diversity of language is its “most remarkable property — there is no other animal whose communication system varies both in form and content” (Ibid.: 446). This stress on diversity contradicts the usual claims to uniformity and universality of linguistic forms (i.e., rule-following regularity). Over time and circumstance, speakers and hearers work pragmatically to juggle the demands of complexity, as well as clarity, coherence, and comprehensiveness. Depending on the specifics of who is being addressed and in what situation, clarity may take the lead (e.g., a guide might over-articulate, using simplified syntactic forms with a single visitor inexperienced in the language), or it might recede



Ecological pragmatics

relative to comprehensiveness (e.g., a guide speaking to a group of people whose expertise and interests were diverse would vary the clarity and complexity of form and content). In ecological approaches to the activity of conversing, the focus is on the physical and the pragmatic (Hodges 2007a): What physical activities occur as two or more people converse with each other in a specific physical setting, and how are those activities coordinated and directed over time? Speaking and listening are seen as cooperative gesturing, and the relation of articulatory gestures to other gestures and movements, both intentional (e.g., reaching) and unintentional (e.g., postural sway, eye movements), are what is typically explored (Chambers, Tanenhaus, and Magnuson 2004; Galantucci, Fowler, and Goldstein, in press; Rosenblum 2005; Shockley, Baker, Richardson, and Fowler 2007). Gestures are physical movements pointing to other movements, to the results of those movements, and/ or to the possibilities of future movements. Conversing with others is not a matter of deciphering an abstract structure or of producing some outcome; rather, it is a matter of ongoing agency. Agency is the initiation of unforced actions that activate environmental potentials that constitute a real physical-social-moral field that constrains and guides the ongoing activity. That agency is directed toward the goods of the ecosystem. Listening and talking with others generates affordances that, if actively explored, reveal their structure, which allows them to function as signs of further affordances. Through perceiving and acting linguistically, humans can make use of linguistic affordances to help realize other affordances (Hodges 2007a, 2007b). What are values on an ecological accounting, and how are they related to affordances? Ontologically, values are the global constraints on self-organizing ecosystems, the boundary conditions that “provide not only for the initial conditions for the system but … also underwrite the system dynamics” (Hodges and Baron 1992:â•›270). As such, they motivate and obligate the activities of intentional agents within the system. However, the system as a whole is intentional: Values are “the intentions of the world as a self-organizing system” (Hodges and Baron 1992:â•›270, italics deleted). Thus, values are not properties of persons or of objects, as is often assumed or argued (e.g., Reed 1996; Rokeach 1973), but are about relationships and the demands that the ecosystem places on those relationships. Values are ecological, not subjective or objective. Affordances are possibilities of particular layouts of events and objects for a particular animal. If they are realized, affordances are a partial enactment and embodiment of ecosystem values. Values are inherently plural, diverse, and developmental (i.e., revealing themselves over time). They function as a heterarchy, pulling against each other in cooperative fashion. As such, that they cannot be defined by fixed hierarchical relationships, as is the case in rule-governed systems. Temporary hierarchical

141

142

Bert Hodges

relationships may develop in a given task or at a given point in a developmental trajectory, but they are reversible, so that what leads at one time or place, follows at another time or place. Values cannot be characterized as goals. Values “underwrite the self-organizing constitution of niches (i.e., ways of life) that guide the selection, coordination, and revision of goals … [and thus] have priority over goals, rather than being means for” realizing goals (Hodges 2007a:â•›7). Goals, like rules, are legitimated by values, rather than the reverse (Hodges and Baron 1992). Values constrain not only the choice of goals, but the paths by which they may be properly attained (Martin, Kleindorfer, and Brashers 1987). What might an ecological, values-realizing psychology offer to our understanding of language, of people conversing and the forms of gesturing embodied in them? What is the relation of language to perception and action, to cognition and emotion? To begin to explore such questions, we will take the three approaches identified earlier, considering language as a perceptual system, an action system, and a caring system. 3. Dialogical arrays: Conversing as a perceptual system Hodges (2007a) considered the hypothesis that language is a perceptual system and why it is not usually thought of as such (e.g., no dedicated sensory anatomy). If language is thought of in perceptual terms, the focus is usually on how phonemes or syntactic structures are perceived, or how semantic inferences influence the perception of speech, for example. Occasionally, the focus is on the perception of voice qualities that indicate something about the speaker (e.g., identity, mood). What Hodges was proposing, though, was to consider speaking as part of a perceptual system that was fundamentally dialogical. Through conversation with others one can perceive, partially and fallibly to be sure, information about the layout of the physical environment in which he or she is situated that is frequently superior to what could be learned by personal exploration, no matter how extensive. Furthermore, one can perceive through dialogue a considerable amount about the prior events in which one’s conversational partners have participated, and about future events in which they intend to participate. The specific proposal offered was that conversing allows for the perceptual probing of dialogical arrays. Somewhat like bats, humans produce complex acoustic patterns that are broadcast, yielding complex acoustic patterns in return, that provide information about their surroundings. A crucial difference between a normal acoustic array and one generated by human speech is that while the former is caused, the latter is not. People may speak in informative ways, but they may not.



Ecological pragmatics

The basis for speaking of a dialogical array is a comparison with optic or acoustic arrays. An optic array is the different intensities of light in different directions, reflected from various bounded surfaces, ordered around an observation point. The differing intensities provide optical structure that a visual system can explore. What Gibson (1979) calls the ambient optic array is the sum of all these, the layout of all points and paths, surrounding an observer that can be explored by moving one’s eyes, head, and body. Visual systems do not respond to light, but to optical information, revealed in movement through the ambient optic array. A dialogical array is a group of hearer-speakers surrounding a given speaker-hearer, listening and talking in ways that reveal, inevitably, something of their perspectives, their intentions, and their histories relative to the present place and time. Like light, the ordered gestures of the array, as well as their disordering and reordering over time, allow a participant in the array to have their own orderings restructured on various scales. It is an array of partners, actual and potential, who provide information, not just about themselves as intentional agents and as objects, but about objects, events, and agencies beyond the physical and temporal horizons of the immediate physical surround. As we probe with questions, declarations, exclamations, proposals, and all the other modes of linguistic “looking,” we can increasingly come to see (much as in ordinary vision) the real prospects for futures to be realized, and the real histories that have given bodily shape to the persons and place of the dialogical array that is being enacted as a mode of perception. Language as a perceptual system provides for “distance perception” in a way even vision cannot match. One of the tradeoffs, though, it that it is less transparent as a medium than light, and it is far less continuous. It is full of gaps and often opaque. Nonetheless, dialogical arrays provide for retrospective and prospective perception that give real direction to our ongoing actions, as well as cognitive preparations for what might be future actions. Participation in dialogical arrays is a crucial perceptual skill for the way-finding of humans. A dialogical array also provides a way of measuring the immediate environmental surround more completely and accurately than can be done by an individual perceiver, even one that moves around, putting herself in multiple positions over time. It is for this simple reason that we value multiple observers and descriptions of events in scientific laboratories, courtrooms, as well as at the family dinner table. What emerges from dialogue is not a super-vision that is equivalent to what would be possible if we were omniscient, looking at the world from every possible position in the optic array. But since many events are irreversible and of sufficiently short-duration, it is measurably better to share — if only partially and often painfully — the intersections of our words so that we can orient ourselves more completely and clearly in the world in which we find ourselves. Dialogical arrays help to orient us, to locate us, to give us a sense of place and event.

143

144

Bert Hodges

A dialogical array is jointly created and sustained. Only in our active probing and being willing to be probed is it possible to learn about our situation and its prospects, and something about what lies beyond the horizon of the situation. But it is in using the horizon (literal and metaphorical) of my conversation partners that I can begin to see what lies beyond. Horizons both hide and reveal a larger reality that frames what is done immediately and locally. But through conversing over time, the pragmatics of any particular exploration of the dialogical array involves a kind of movement that reframes things — just as the horizon shifts when I move — some things go out of view and others come into view. The horizon that bounds our view locates us reassuringly, but it invites us to go beyond to discover what supports, surrounds, and cares for that field of vision and action that are present to us. Language bespeaks a presence — ours, our partners, our field of action and view, our horizon with its prospects of orientation and discovery. It is a presence that invites an agency of accountability and answerability. It hints at, but does not prove, an ontology of care-fulness, and an ethic of risk-giving. We risk ourselves, and others, when we speak and listen together. In dialogue we reach beyond our horizons, through others, to discover the source of ontological care that has yielded our existence, our conversation, our purpose. 4. The complexity of fractal dynamics: Conversing as an action-system Viewed from an ecological, values-realizing perspective, linguistic activity is not only perceptual, but enactive. While it is common to think of language as a cognitive skill, it is a form of action (Melser 2004). Since “actions are realizations of what the environment affords” (Reed 1982:â•›101), they are distributed, “as much ‘in’ the environment as ‘in’ the organism” (Ibid.: 125). As the discussion of dialogical arrays has revealed, the ecosystem relations within which utterances function as a perceptual system (Hodges 2007a) are both local and distant, both present and historical. Dialogical utterances are enacted as perceptual systems to locate and orient us to the possibilities, opportunities, and obligations that surround us. In locating and orienting us, linguistic activity functions, like all perceptual systems, to make us context-sensitive. As actions, utterances are also prospective. They are intentional. As we saw earlier, dialogical arrays can only be explored by being enacted through dialogical fields (fields of caring conversation). The creation and development of dialogical fields, as people move in and through dialogical arrays, requires that language functions as an action system (e.g., Hodges 2007a; Reed 1982). That is, the movements we designate as linguistic cannot be so many separate actions, independently and reactively cobbled together, oriented only to local signs. The movements



Ecological pragmatics

must be coordinated and directed. Values, I have claimed, are the motivating force and the large-scale dynamical constraints that provide for direction and coordination. The coordination of movement must occur on many scales. Thibault (2004, 2005) has proposed that language is a “multi-modal contextualizing activity which is embedded in an ecosocial semiotic environment and which integrates diverse space-time scales” (Thibault 2005:â•›123). This suggests that language may have a special role to play in the integration of scales of action and perception for humans, but it also means that if language is to work pragmatically, linguistic activity itself will be scaled at multiple levels. These issues of context-sensitivity and prospective coordination across multiple scales will be the focus of exploration in this section. Recent work in complex dynamical systems elaborates and clarifies how radical the claim is that language is distributed. It points to an understanding of linguistic activities as being far more complex than previously imagined, and of context-sensitivity and global interdependence as central features of that complexity. Along the way, we will learn more about ecological approaches to perception and action, about the inadequacy of rules to guide linguistic activities, and about the importance of realizing values, not just goals. Recent work in ecological psychology (e.g., Van Orden, Moreno, and Holden 2003; Van Orden, Holden, and Turvey 2005; Van Orden, Kello, and Holden, 2010) using non-linear dynamical analyses, have revealed pervasive, long-range patterns in linguistic performances (e.g., word pronunciation, lexical decisions, semantic categorization). These data are startling and have important implications for our understanding of intentional activities. If, for example, someone repeatedly says a word thousands of times, and measures such as the latency and duration are taken, aperiodic waves of variation in amplitude occur across time. Instead of random variation around a “true score” of latency or duration, the waves of variation at different scales of times (e.g., blocks of 10 trials, 100 trials, 1000 trials, 10,000 trials), are similar at every scale. The similarity is so precise that if the amplitudes are graphed relative to frequencies on a log/log scale, the relationship is linear. Such a pattern is fractal (sometimes referred to as 1/f scaling). This fractal patterning means that there is no one scale (or unit of measurement) that defines or captures the phenomenon. This means that the phenomenon (i.e., the activity as measured) is not caused in the usual sense; that is, there is no single, proximate cause (or level of causes) that explains the act (Gibbs and Van Orden 2003; Shaw 2001). While such performances are often referred to as scalefree (Kloos and Van Orden, 2009), a better description might be that such wave patterns are global, collective, and paradoxical.

145

146

Bert Hodges

The fractal pattern is global: It is as large as the sample measured, so that if we collected more data, we would discover even larger, longer patterns than those observed previously. Instead of reducing variability, increased precision of measurement reveals more variability (Van Orden, Kello, and Holden, 2010). The fractal pattern is collective: It emerges over time because of interactions among an uncountably large number of activities/components, which are linked in a network of interdependencies. The dynamics are interaction-dominant; the phenomena come into existence in a specific space-time configuration, but do not exist before or after in any of the component systems (Van Orden, Kello, and Holden, 2010). The skill, knowledge, or ability demonstrated cannot be located in a brain, a body, a set of instructions, a set of cultural practices, an experimental setting, or an evolutionary history. All of these and more may participate, but only in the integrity of collective action can they generate the phenomena. Because of the uniqueness and integrity of the configurations, phenomena cannot be predicted from component measurements, in principle, nor can they be measured in a way that separates the influence of various participating components. The fractal pattern is paradoxical: The phenomenon emerges from the repetition of a single, “identical” set of movements (e.g., saying the word), yet the result is a pattern of increasing diversity. The diversity is not random; rather, there are correlations across scales. These long-range correlations suggest that human actions are like the wave-particle duality of quantum physics: If one electron is fired per minute through a tiny slit and strikes a screen leaving a mark, over time a bulls-eye type pattern (concentric bands of light and dark) will emerge (Van Orden 2007). Human actions seem to be generated by “not simply interacting components but interdependent components” (Van Orden and Kloos 2003:â•›165). Actions that occur on the scale of 0.2 sec show dependencies on other activities that are on the order of 11,000 sec (Van Orden, Holden, and Turvey 2005). Interdependence implies that each part reflects something of the whole in its behavior. The interdependence of fractal patterns suggests that human performances, linguistic and otherwise, are exquisitely context-sensitive, playing out on all scales simultaneously. Just as is true for quantum dynamical systems, human actions are marked by context-sensitive, global interdependence, where the measurement activity itself is one of the contexts that constrains the performance. Like other aspects of context, measurement is an essential co-author of action. It is a crucial system interacting with other interdependent systems, revealing their joint actions. The fundamentally social, distributed character of action, cognition, and skill is thus highlighted (Kloos and Van Orden, 2009; Van Orden, Kello, and Holden, 2010).



Ecological pragmatics

An interesting example of the importance of measurement is Evans and Levinson’s (2009) call for greater diversity in the sampling of languages used by linguists when making generalizations about linguistic patterns (e.g., claims about universal features). They argue that most of the languages previously sampled were more closely related in their historical development than first realized, yielding an insufficiently representative sample of all existing human languages. In addition, language groups coming into contact with each other lead to changes in each other, as each takes the measure of the other. Over time, most of these differing language groups have vanished altogether, making it even more difficult to sample language(s) as a whole accurately. The research on fractal patterning points to the possibility that “contemporary models grossly underestimate the number of temporal scales on which cognitive activity is actually assembled” (Kloos and Van Orden, 2009: 263). Complexity is fundamental. Word pronunciation tasks, along with lexical decisions and semantic categorization tasks, all show interaction-dominant dynamics in which control is distributed, not localized. These findings create considerable difficulty for any view that gives priority to the central nervous system and to any purported modular processes in explaining linguistic and related cognitive skills (Van Orden and Kloos 2003). Processes that are traditionally seen as separate and encapsulated, capable of independent analysis, are in fact part of a complex, unified, dialogical system (Van Orden, Holden, and Turvey 2005:â•›121). Complexity is not the mark of a distressed dynamical system, but a vibrant one. Disease is marked by a “departure away from healthy fractal variability and toward a loss of complexity” (Van Orden 2007:â•›1). For example, healthy hearts show fractal, aperiodic rhythms. It is often proposed by cognitive researchers and linguists that simplicity, economy, and repetition are the markers of language proper, but as noted earlier, complexity and variability seem to be better candidates for identifying the systematicity and usefulness of language. One way to try to evade the force of fractal patterning is to claim that certain tasks (i.e., goals) should be treated as canonical in establishing the nature of a competence or skill, while other interaction effects are treated as peripheral to the competence, that is, as oddities of particular performance conditions. The difficulty with this is that there appears to be no principled, empirical basis on which to decide which tasks should be canonical (Kloos and Van Orden, 2009; Van Orden, Holden, and Turvey 2005). What has happened in numerous areas of research is that arguments about which task is right for carving up the competences or the modules correctly have reached “stalemates” (Kloos and Van Orden, 2009). Instead of arguing about which task and the rule system describing it produce more “pure” data (i.e., revealing “context-free competences”), an alternative would be to

147

148

Bert Hodges

acknowledge that all the data are relevant, and that the underlying competence is inseparable from its contextual supports. Context is constitutive of competences, not just performances. Softly-assembled, context-sensitive dynamic performances are what human competences, such as language, are: Language is use. Repetition and recursion are not the genius of language, but fractal variability. In mounting their argument against linguistic universals Evans and Levinson (2009) claim that the “more we discover about languages, the more diversity we find” (Ibid.: 436) and that “almost every new language that comes under the microscope reveals unanticipated new features” (Ibid.: 432). This is a powerful example of the global nature of linguistic dynamics; that is, the more precise and comprehensive our measurements, the greater the variability in the shaping of sound, syntax, and semantics that is observed. If conversing is assembled across many more scales than previously assumed, and yet is also more integrated than expected, and if the diversity and complexity of natural human languages grows larger as linguists examine a greater array of exemplars, it suggests two things. One is that language is more of a system than even traditional cognitive models making claims about universals has recognized. Second, this larger, more integrated definition of language indicates that linguistic forms “reflect cultural preoccupations and ecological interests” (Evans and Levinson, 2009: 436), as well as general biological and psychological constraints (Christiansen and Chater 2008; Locke and Bogin 2006). Furthermore, a values-realizing approach (Hodges 2007b) suggests that the ecological interests go beyond any given culture and deeper than any specific biology (Hodges and Baron 1992:â•›270–271). Language, like other complex systems, bespeaks a “global dependence” (Van Orden, Moreno, and Holden 2003:â•›51). Language embodies the ontological conviction that we are not alone. To conclude the discussion of fractal variability, consider again the fluently English example. The more positive and profound side of the claim that rules alone are inadequate is that language is context-sensitive and interdependent, often exquisitely so. A slight change in context shifted what appeared to be a fixed regime into another region of possibilities. The counterexample was generated by extending the range of domains to which the skill, speak fluently, applied. This extension shifted the focus from a particular domain, English, to the skill (fluency) and its range, changing the grammatical affordances (where the adverb could fit). This shift in focus tipped the balance slightly; what had been impossible or awkward became possible and elegant. Similarly, although it has seemed impossible to many psychologists (ecological ones included) to approach language in any way other than as a rule-following regime, perhaps the balance is tipping in new directions, creating and revealing new affordances.



Ecological pragmatics

5. Realizing values through affordances: Conversing as a caring system The themes of context-sensitivity and interdependency highlight a central theme of an ecological, values-realizing account of language: They are distinguishing marks of caring, which Hodges (2007a, 2007b) hypothesized is a primary function of conversing (i.e., language). Caring arises out of interdependency and demands context-sensitivity, which can be contrasted with traditional metrics of autonomy and control. An ecological approach to language gives primacy to the physical and the pragmatic, that is, to first-order activities. From this perspective conversing is a form of orienting, integrating, and way-finding (Hodges 2007b:â•›174), helping humans to recognize and realize existing affordances, and to modify and create new ones that invite responsible action. As the earlier discussion of dialogical arrays illustrated, the particular power of language may be its ability to assist way-finding by extending our perception retrospectively and prospectively, enlarging our horizons and our ability to realize values as a consequence. Given that the purpose of way-finding is to realize affordances that embody and enact values, linguistic activities physically alter humans and their surroundings, although less directly and immediately than many perception-action systems. Linguistic activity that does not engage dialogical arrays and does not alter the orientation and direction of other performatory and exploratory activities would not realize values and would not be maintained. Linguistic activity is maintained, adjusted, and altered as a “way of caring for others, for our self, and for the world” (Hodges 2007b:â•›174). Caring can be defined as working (i.e., being discomforted) to realize values in some encounter (Hodges 2007b; Hodges and Lindhiem 2006). There is some biological and psychological evidence that suggests caring may function as a physical-social-moral system (e.g., Taylor 2002). To be caring is to be anxious and attentive, meticulous and prudent, in assessing and addressing a situation — whatever or whomever is cared for. It is to serve the good of that event or the participants in it, in light of the larger circumstances within which the event is embedded. Ultimately, the creative choices made will need to realize values, if the caring is to be good. To care for someone or something is to become dependent on them, to put oneself at their service, but it requires their dialogical willingness to be cared for. Each serves the other in a way, and both serve values, if the relationship functions in the flourishing way self-organizing, complex systems do in states of criticality (Kloos and Van Orden, 2009). Claiming that conversing intends to realize values and embody and enact caring does not mean that all linguistic activity is judged good. We can and do make claims about utterances being inarticulate, awkward, false, irrelevant, careless,

149

150

Bert Hodges

inconsiderate, and cruel. Ecological theory does not take these claims to be idle impositions of personal preference or oppressive strictures of social power, although they may be, but judgments that are themselves meant to realize values, and are themselves subject to judgment. Speaking and listening to each other (or ourselves) is always about acting appropriately, in ways that seek to make appropriate use of prior resources and to find appropriate ways to “go on” both in the conversation and in life (Hodges 2007a:â•›597–598). Values are what makes it possible for linguistic activity to be creative, to challenge as well as acknowledge societal practices and their authorizing powers, and to violate or alter rules, not as an assertion of autonomous will, but as an act of social solidarity and trust (Hodges 2007a; Hodges and Geyer 2006). Approaching conversing as a caring system helps to avoid two influential, but limited models for understanding language: One is to posit language’s function in terms of representation; the other is to frame language in terms of tool use. Both see language as a means of fixing reality: language-as-representation aspires to freeze reality into some idealized structure; language-as-tool-use works to manipulate reality into the ego’s image or toward some societal goal. On the representational view, which has dominated cognitive psychology and linguistics, autonomy has been the watchword (e.g., the autonomy of syntax; I-language). The tool-use model, itself an attempt to get past the passivity of the representational view, recognizes the importance of coordination (Clark 1996:â•›72–91) and even cooperation (Grice 1975:â•›47), but usually with the point of maintaining control or achieving a goal. As noted earlier, an ecological account sees goals as means to realizing values; by themselves, goals and control are insufficient to account for why we listen and speak with each other. If we did not care, we would not speak or listen, and if our caring were limited to achieving goals and controlling actions, then what passes for conversation would be reduced to a code and a tool after all. To explore further the nature of language as a caring action-perception system, and the centrality of complexity and dialogical relations to its dynamics and integrity, we need to move in a surprising direction, at least for ecologically oriented researchers. We need to consider cognition and more specifically consciousness. However, what emerges from this move will be surprising from the perspective of traditional cognitive theory: The focus will be on pragmatics and dialogue rather than on computations and control.



Ecological pragmatics

6. Attunement and alienation: Consciousness, suffering, and hope Thus far, language has been considered as a perceptual system for exploring dialogical arrays, and an action system for coordinating diverse space-time scales. Both have pointed toward the integrity of conversing as a pragmatic activity that is concerned to realize appropriateness (i.e., values) from beginning to end. Such pragmatic activity is not caused, but jointly-enacted within the integrity of the ecosystem as a whole. But values-realizing is not automatic and inevitable. It requires work. Only through cooperative (argumentative as well as agreeing) dialogue can the world be prospected, and our ability to enact and embody the good be increased. The task is a shared one that demands creativity, not conformity, but it does require following as well as leading (Hodges 2007b). With its focus on joint action, an ecological approach sees language less as a tool of exclusion or manipulation than many pragmatic theorists (de Saussure 2005:â•›6; Mey 2003:â•›343–345) would have it. The very act of conversing moves us toward sharing, rather than conforming. But it is a sharing that is complex, difficult, and surprising. Although ecological psychology approaches language in terms of action and perception, there is a need for cognition too. Thibault (2004:â•›1) rightly claims we can have no serious account of language if we ignore human consciousness. Developmental research suggests that the first perceptual awarenesses emerge in early childhood through dialogical interactions with caregivers (Bråten 2007; Reddy and Morris 2004; Trevarthen 1998). Thibault (2004:â•›2) states: “the principle of the other — the nonself … is an affordance — perhaps the most fundamental one of all — of the ecosocial semiotic environment … This observation suggests that the ontological basis of our bio-social being is the principle of alterity”. Here Thibault is being perfectly ecological. Shaw and Turvey (1999) have discussed the ecological foundations of consciousness as follows. Taking the “new physics” as their point of departure — in which there is a forceful, but unforced, coupling of organismic biological dynamics and environmental physical dynamics by information — they suggest that the question is not whether Schrödinger’s cat is alive or dead, but whether the cat observes us as well [as being observed]. If so, there is a social dyad, sharing an environment … Hence an ecosystem exists… As partners, they make the ontological descent together, coupled by information, satisfying the same conservations, governed by the same laws … [In this system of mutual constraints] their wave functions are perfectly in phase … [and] harmony is achieved a posteriori (Shaw and Turvey 1999:â•›119–120).

151

152

Bert Hodges

For Thibault, as well as for Shaw and Turvey, the focus of the alterity is attunement. Thibault proposes that when a person inhabits her environment in a system-sustaining way (i.e., attunement), there is “dialogic closure” and “no self per se of which we may be self-aware” (2004:â•›230). This suggests that the consciousness of attunement differs from self-consciousness. I think he is right about this. What Thibault has identified is something like what Csikszentmihalyi (1991) has called flow, or what might be described as lostness — as in “lost in the moment”, completely given to the activity. A fabulous conversation over a great meal with friends illustrates such unselfconscious closure. Even the burden of time seems absent. The experience is like a “taste of heaven”. But that, I suggest, presents a problem. Consciousness is marked — weighted, really — by a sense of something other than attunement. The contextualizing and recontextualizing that Shaw and Turvey read as intrinsic and harmonious, pragmatic theorists (e.g., Mey 2001) read as extrinsic, and marked by asymmetry, exclusion, and alienation. Here, I think, pragmatic theorists have a point, and one that ecological theorists need to heed. The hypothesis that I (Hodges 1997) find most convincing about the origins of consciousness is that its roots are historical (Taylor 1989), social (Mead 1934; Vygotsky 1978), and moral (Jopling 1993), inextricably related to the rise of a certain sense of self — not the ecological self which is the reciprocal of the environment, but rather a self marked by a sense of separation and alienation. The conditions of emergence for consciousness, according to many theorists, are forms of interruption. “We all have had experience of automatically driving a car, typing a letter, or even participating in a cocktail party conversation, and of being suddenly brought up short by some failure such as a defective brake, a stuck key, or a ‘You aren’t listening to me’↜” (Mandler 1985:â•›72). Interruption, Mandler notes, calls for restructuring, choice, or trouble-shooting. The consciousness arising from interruption can be painful; in fact, Mandler suggests that pain may be the most insistent form of consciousness (Ibid.: 67). More generally, consciousness is marked by a kind of suffering, a discomforting that challenges the habitual and conventional. Reed (1996) refers to the experience as one of being disjoint from our environment. According to Baars (1997:â•›146), perhaps the most profound and universal of conditions that provoke consciousness is that of loss. The most compelling form of loss is death. Knowledge of our impending deaths, I once suggested (Hodges 1997:â•›8), may be the focal point of consciousness, and may be why scientists are sometimes reticent to address consciousness. Deacon (1999:â•›22) observes that only humans ponder “what it will be like not to be” and claims that knowledge of “the inconceivable possibility” (436) of one’s own end creates a “foreboding sense of … impending loss” (437). Death could be viewed as the dialogical horizon for the pragmatics of life.



Ecological pragmatics

Consciousness, then, might be viewed as pragmatics under the constraint of alienation. Research by Kagan (1989) and Lamb (1991) indicates that 18-month old children spontaneously begin to notice and bemoan (“uh-oh”) the flaws, cracks, and breaks in things. Their pragmatic sensitivity to gaps (breaks, distortions, interruptions) may be the beginning of a consciousness that presents the world as in need of repair. Much of language is provoked by this sense of unfulfilled intentionality of the world, and/or our own awkwardness and alienation with respect to our surroundings. This is especially true with respect to others, whom we learn cannot always be trusted. Even more painful, we become aware that we ourselves cannot always be trusted. Consciousness, then, is what allows us to “take care” (i.e., be cautious and circumspect) in what we feel, do, and say. Of course, caution and circumspection are the very stuff of pragmatics. In fact, the primary reason for the necessity of pragmatics in the first place is that there is alienation as well as attunement. Attunement is a project, not a given. Alienation is not a given either, but the consequence of failures to fulfill the project of attunement. Cognitivism assumes humans begin from alienation (Reed 1996). Ecological theory assumes attunement is fundamental, but not assured. The work required is not to imagine a world with which we have no direct contact, but to explore thoroughly and jointly the world we inhabit and are to care for. Once again we come to caring as central to language, and nowhere is that more evident than in the care that defines pragmatics. Politeness, truthfulness, relevance, and all the other principles described in pragmatics are grounded in the need to be caring and careful. Pragmatics requires that we think about what we say, or about what another means, because caring “entails suffering, a burdensome or anxious sense of responsibility that borders on grief ” (Hodges 2007b:â•›167). Caring is a willingness to be discomforted, interrupted, and troubled by another. How do we negotiate consciousness as attunement and as alienation? How do we take care? How do we discern what could and should be repaired? The puzzle of consciousness brings us back to dialogical arrays. Bakhtin (1993) suggested that the fundamental psychological fact for humans, located in their dialogical relation to the other, is not attunement or alienation, but answerability. Humans, he claims, answer to others, which he considered variously as the person’s community, the “court of history”, even God (Bakhtin 1986:â•›126). For him, the social and the moral are the defining context for the conscious acts of persons; thus, consciousness is a matter of ecology, not of Cartesian ego. As he puts it: I am conscious of myself and become myself only while revealing myself for another, through another, and with the help of another … To be means to be for the other; and through him for oneself. Man has no internal sovereign territory; he is all and always on the boundary (Bakhtin 1984:â•›287).

153

154

Bert Hodges

This suggests that consciousness functions as con-science, literally “together knowing” (von Foerster 1986; Järvilehto, 2009); that is, consciousness is a form of conversing that encourages conviction and that criticizes carelessness. It points to the possibility that it is in the dialogue of attunement and alienation within dialogical arrays that humans become conscious and caring. Pragmatics, I have argued, is values-realizing. Within those heterarchically organized activities, we may set goals and generate rules in an attempt at ordering our efforts. But, since language must be chaotic as well as orderly, goal-seeking and rule-following by themselves are too precise and repetitive to account for the coordination necessary for pragmatics. Values, on the other hand, provide an edginess, a playfulness, that makes it possible for language to work properly (Hodges 1995, 2007b). Thibault (2004) suggests that it is through triadic, cooperative play that consciousness develops: Through play, the child’s world is significantly enlarged in one sense, and significantly entrained in another. More specifically, the child learns that language is not itself reality, but is a form of play (Thibault 2004:â•›298). But it is play of a serious sort. How so? Paradoxically, we often find rule-breaking utterances more interesting and informative than conventional, rule-following ones. It is as if we find language more useful when it is playful rather than when it is “working to rule” (a phrase used by labor unions to denote abiding by contractual obligations in a legalistic way rather than a values-realizing way; Schwartz 1986). Cowley (2004) discussed ways in which language is by turns playful and serious, arguing that utterances (i.e., “words”) cannot come to function as tools until the child takes them seriously. Thus, in order to use the language, the child has to care. But to get children to care, parents often play. Goodnow and her colleagues (Goodnow 1988, 1990; Goodnow and Warton 1991) noted that parents, intent on encouraging their children to be responsible, kind, and caring, often say strange, difficult to comprehend things to them. Rather than clearly explaining what they want their children to do and why (as nearly all developmental experts encourage them to do), parents seem intentionally to occlude, at least partially, their intent. The purpose of this partial occlusion may be that it invites the child into a game of sorts, a kind of hide-and-seek, but with the serious intent of getting them to move from their egocentric position to positions that make it possible for them to see more clearly the other. In having to work (and play) to understand the parent’s utterance, the child will have begun to embody the very virtues the parent wants the child to learn. The parent’s playful occlusion invites the child to make the serious move of entering into the dialogical array. Representational consciousness (i.e., the disengaged self) offers itself as a sufficient guide to perception and action, to learning and social achievement. It is



Ecological pragmatics

not. It partially occludes ecological relationships that provide the resources for acknowledging dependencies and obligations that make our actions possible. Skilled activities, from juggling balls to “juggling family and career”, seem to balance on the edge of laws and rules (Hodges 1995:â•›6–7), since the very existence of these skills depends on the intention to realize values. The juggling inherent in speaking and listening also lives on the edge. Beek, Turvey, and Schmidt (1992:â•›91) have argued that biological action systems are “to a considerable extent ‘frustrated’ … subject simultaneously to very many different physical requirements that they cannot possibly satisfy fully”. This is precisely the heterarchical nature of values. Conversing with others always yields a kind of suffering: As we engage in conversations over time, we come to realize that no matter how hard we work, we cannot be completely clear, coherent, comprehensive, and complex all at once. This frustration is sometimes painful, but, paradoxically, it is also the basis for hope. It is because of dialogical arrays, and the long-range correlations they embody, that we are able to share a world that is larger than any one of us can see directly. Dialogue acts as our con-science (i.e., consciousness) and allows us engage in joint actions that negotiate the demands of existence. Only in appreciating the opacity of persons, in actually caring for them as persons, is it possible for a dialogical array to function as a substantive horizon, and for it to become a relatively transparent medium for the prospects and history beyond the horizon. Only if we engage in the indirectness of pragmatics will it be possible to be careful of and caring for the ecosystem that sustains, obligates, and informs us. Dialogical arrays reveal metaphorically. The reality that is nearer is used to probe the reality that lies farther, beyond the horizon. Only if we have the humility to submit to the dialogical array, and the confidence to risk being answerable to the other (Bakhtin 1986), will we have the hope necessary to trust language as a medium of perception and as a mode of action.

Note *╇ An earlier version of portions of this chapter was presented at the Symposium on Language dynamics and the phenomenology of individual experience at Agder University College, Grimstad, Norway, May, 2007, at the kind invitation of Paul Thibault. My thanks to Reuben Baron, Stephen Cowley, Damon DiMauro, Timo Järvilehto, Alex Kravchenko, James Martin, and Guy Van Orden for comments and suggestions on earlier versions of this paper, as well as to Stein Bråten, Lynne Cameron, Barbara Johnstone, Jay Lemke, Per Linell, Derek Melser, and other symposium participants. Special thanks to Joanna Rączaszek-Leonardi, Sune Steffensen, and Kristian Tylén for their careful reviews. Support from Initiative and Faculty Development grants of Gordon College is gratefully acknowledged.

155

156

Bert Hodges

References Asch, S.E. 1956. “Studies of independence and conformity: I. A minority of one against a unanimous majority”. Psychological Monographs 70, no. 9 (Whole No. 416). Baars, B.J. 1997. In the Theater of Consciousness: The Workspace of the Mind. New York: Oxford University Press. Bakhtin, M.M. 1984. Problems of Dostoevsky’s Poetics. Minneapolis, MN: University of Minnesota Press. Bakhtin, M.M. 1986. Speech Genres and other Late Essays. V.W. McGee (trans). Austin, TX: University of Texas Press. Bakhtin, M.M. 1993. Toward a Philosophy of the Act. Austin, TX: University of Texas Press. Beek, P.J., Turvey, M.T., and Schmidt, R.C. 1992. “Autonomous and nonautonomous dynamics of coordinated rhythmic movements”. Ecological Psychology 4: 65–95. Bråten, S. 2007. “Participant perception of the speaker by conversation partners and infants: On the origins of (m)other-centered participation”. Paper presented at Symposium on Language Dynamics and the Phenomenology of Individual Experience, Grimstad, Norway. Chambers, C., Tanenhaus, M., and Magnuson, J. 2004. “Actions and affordances in syntactic ambiguity resolution”. Journal of Experimental Psychology: Learning Memory and Cognition 30: 687–696. Christiansen, M.H. and Chater, N. 2008. “Language as shaped by the brain”. Behavioral and Brain Sciences 31: 489–509. Clark, H.H. 1996. Using Language. Cambridge: Cambridge University Press. Cowley, S.J. 2004. “Simulating others: The basis of human cognition?”. Language Sciences 26: 273–299. Cowley, S.J. 2005. “Languaging: How humans and bonobos lock on to human modes of life”. International Journal of Computational Cognition 3: 44–55. Cowley, S.J. 2007. “The magic of wordings”. Paper presented at Symposium on Language Dynamics and the Phenomenology of Individual Experience, Grimstad, Norway. Csikszentmihalyi, M. 1991. Flow: The Psychology of Optimal Experience. New York: Harper. Deacon, T.W. 1997. The Symbolic Species: The Co-evolution of Language and the Brain. New York: Norton. Evans, N. and Levinson, S. 2009. “The myth of language universals: Language diversity and its importance for cognitive science”. Behavioral and Brain Sciences, 32: 429–492. von Foerster, H. 1986. “From stimulus to symbol”. In V. McCabe and G. Balzano (eds), Event Cognition: An Ecological Perspective. Hillsdale, NJ: Lawrence Erlbaum Associates, 79–91. Galantucci, B., Fowler, C.A., and Goldstein, L. 2009. “Perceptuomotor compatibility effects in speech”. Perception & Psychophysics 71: 1138–1149. Gibbs, R.W., Jr. and Van Orden, G.C. 2003. “Are emotional expressions intentional? A self-organizational approach”. Consciousness & Emotion 4: 1–16. Gibson, J.J. 1966. The Senses Considered as Perceptual Systems. Boston: Houghton- Mifflin. Gibson, J.J. 1979/1986. The Ecological Approach to Visual Perception. Hillsdale, NJ: Lawrence Erlbaum Associates. Gibson, J.J. and Crooks, L.E. 1938. “A theoretical field analysis of automobile-driving”. American Journal of Psychology 51: 453–471. Goodnow, J.J. 1988. “Children’s household work: Its nature and functions”. Psychological Bulletin 103: 5–26.



Ecological pragmatics

Goodnow, J.J. 1990. “Adding social factors to the study of cognitive development”. Paper presented at the meetings of the American Psychological Association, Boston. Goodnow, J.J. and Warton, M.P. 1991. “The social bases of social cognition: Interactions about work and their implications”. Merrill-Palmer Quarterly 37: 27–58. Grice, H.P. 1975. “Logic and conversation”. In P. Cole and J. Morgan (eds), Syntax and Semantics, Vol. 3: Speech Acts. New York: Academic Press, 41–58. Hodges, B.H. 1990. “Social and values-realizing constraints on ecological accounts of perceptual warrant”. Paper presented at Perception-Action Workshop, University of Connecticut, Storrs. Hodges, B.H. 1995. “Beyond goals: The place of values in a world of postmodern affordances”. Paper presented at Eighth International Conference on Perception & Action, Marseilles. Hodges, B.H. 1997. “The importance and inadequacy of consciousness”. Paper presented at the Ninth International Conference on Perception and Action, Toronto. Hodges, B.H. 2004. “Asch and the balance of values”. Behavioral and Brain Sciences 27: 343–344. Hodges, B.H. 2007a. “Good prospects: Ecological and social perspectives on conforming, creating, and caring in conversation”. Language Sciences 29: 584–604. Hodges, B.H. 2007b. “Values define fields: The intentional dynamics of driving, carrying, leading, negotiating, and conversing”. Ecological Psychology 19: 153–178. Hodges, B.H. and Baron, R.M. 1992. “Values as constraints on affordances: Perceiving and acting properly”. Journal for the Theory of Social Behaviour 22: 263–294. Hodges, B.H. and Geyer, A.L. 2006. “A nonconformist account of the Asch experiments: Values, pragmatics, and moral dilemmas”. Personality & Social Psychology Review 10: 2–19. Hodges, B.H. and Lindhiem, O. 2006. “Carrying babies and groceries: The effect of moral and social weight on caring”. Ecological Psychology 18: 93–111. Järvilehto, T. 2009. “The theory of the organism-environment system as a basis for experimental work in psychology”. Ecological Psychology 21: 112–120. Jopling, D. 1993. “Cognitive science, other minds, and the philosophy of dialogue”. In U. Neisser (ed), The Perceived Self: Ecological and Interpersonal Sources of Self-knowledge. Cambridge: Cambridge University Press, 290–309. Kagan, J. 1989. Unstable Ideas: Temperament, Cognition and Self. Cambridge, MA: Harvard University Press. Kloos, H. and Van Orden, G.C., 2009. “Soft-assembled mechanisms for the unified theory”. In J.P. Spencer, M. Thomas, and J. McClelland (eds), Toward a unified theory of development: Connectionism and Dynamic Systems Theory Re-Considered. Oxford: Oxford University Press, 253–267. Kravchenko, A.V. 2007. “Essential properties of language, or, why language is not a code”. Language Sciences 29: 650–671. Lamb, S. 1991. “Internal state words: their relation to moral development and to maternal communications about moral development in the second year of life”. First Language 11: 391–406. Linell, P. 2007. “Dialogicality in languages, minds and brains: Is there a convergence between dialogism and neuro-biology?”. Language Sciences 29: 605–620. Locke, J.L. and Bogin, B. 2006. “Language and life history: A new perspective on the development and evolution of human language”. Behavioral and Brain Sciences 29: 259–325. Love, N. 2004. “Cognition and the language myth”. Language Sciences 26: 525–544. Love, N. 2007. “Are languages digital codes?”. Language Sciences 29: 690–709.

157

158

Bert Hodges

Mandler, G. 1985. Cognitive Psychology: An Essay in Cognitive Science. Hillsdale, NJ: Lawrence Erlbaum Associates. Martin, J.E., Kleindorfer, G.B., and Brashers, W.R. Jr. 1987. “The theory of bounded rationality and the problem of legitimation”. Journal for the Theory of Social Behaviour 17: 63–82. Mead, G.H. 1934. Mind, Self, and Society. Chicago: The University of Chicago Press. Melser, D. 2004. The Act of Thinking. Cambridge, MA: The MIT Press. Mey, J.L. 2001. Pragmatics. 2nd ed. Oxford: Basil Blackwell. Mey, J.L. 2003. “Context and (dis)ambiguity: A pragmatic view”. Journal of Pragmatics 35: 331– 347. Reddy, V. and Morris, P. 2004. “Participants don’t need theories: Knowing minds in engagement”. Theory & Psychology 14: 647–665. Reed, E.S. 1982. “An outline of a theory of action systems”. Journal of Motor Behavior 14: 97–134. Reed, E.S. 1995. “The ecological approach to language development”. Language and Communication 15: 1–29. Reed, E.S. 1996. Encountering the World: Toward an Ecological Psychology. Oxford: Oxford University Press. Rokeach, M. 1973. The Nature of Human Values. New York: Free Press. Rosenblum, L.D. 2005. “The primacy of multimodal speech perception”. In D. Pisoni and R. Remez (eds), Handbook of Speech Perception. Malden, MA: Blackwell, 51–78. de Saussure, L. 2005. “Manipulation and cognitive pragmatics: Preliminary hypotheses”. In L. de Saussure and P. Schulz (eds), Manipulation and Ideologies in the Twentieth Century: Discourse, Language, Mind. Amsterdam: John Benjamins, 113–146. Schwartz, B. 1986. The Battle for Human Nature. New York: Norton. Shaw, R.E. 2001. “Processes, acts, and experiences: Three stances on the problem of intentionality”. Ecological Psychology 13: 275–314. Shaw, R.E. and Turvey, M.T. 1999. “Ecological foundations of cognition: II. Degrees of freedom and conserved quantities in animal-environment systems”. Journal of Consciousness Studies 6: 111–123. Shockley, K., Baker, A.A., Richardson, M.J., and Fowler, C.A. 2007. “Articulatory constraints on interpersonal postural coordination”. Journal of Experimental Psychology: Human Perception and Performance 33: 201–208. Smith, N. 1999. Chomsky: Ideas and Ideals. Cambridge: Cambridge University Press. Taylor, C. 1989. Sources of the Self. Cambridge, MA: Harvard University Press. Taylor, S.E. 2002. The Tending Instinct. New York: Henry Holt. Thibault, P. J. 2004. Agency and Consciousness in Discourse: Self-other Dynamics as a Complex System. London: Continuum. Thibault, P.J. 2005. “Brains, bodies, contextualizing activity and language: Do humans (and bonobos) have a language faculty, and can they do without one?”. Linguistics and the Human Sciences 1: 99–125. Trevarthen, C. 1998. “The concept and foundations of infant intersubjectivity”. In S. Bråten (ed), Intersubjective Communication and Emotion in Early Ontogeny. Cambridge: Cambridge University Press, 15–46. Van Orden, G.C. 2007. “The fractal picture of health and wellbeing”. Psychological Science Agenda 22: 2, 1–5. Van Orden, G.C., Holden J.G., and Turvey, M.T. 2005. “Human cognition and 1/f scaling”. Journal of Experimental Psychology: General 134: 117–123.



Ecological pragmatics

Van Orden, G.C., Kello, C.T., and Holden, J.G. 2010. “Situated behavior and the place of measurement in psychological theory”. Ecological Psychology 22: 24–43 Van Orden, G.C. and Kloos, H. 2003. “The module mistake”. Cortex 39: 164–166. Van Orden, G.C., Moreno, M.A., and Holden, J.G., 2003. “A proper metaphysics for cognitive performance”. Nonlinear Dynamics, Psychology, and Life Sciences 7: 49–60. Vygotsky, L.S. 1978. Mind in Society. Cambridge, MA: Harvard University Press. Wray, A. and Grace, G.W. 2007. “The consequences of talking to strangers: Evolutionary corollaries of socio-cultural influences on linguistic form”. Lingua 117: 543–578.

159

Symbols as constraints The structuring role of dynamics and self-organization in natural language* Joanna Rączaszek-Leonardi University of Warsaw, Poland

The paper draws a parallel between natural language symbols and the symbolic mode in living systems. The inextricability of symbols and the dynamics with which they are functionally related shows that much of their structuring is due to dynamics and self-organization. It is also stressed that important factors that determine the shape of language structure lie outside individual mind/brains and they draw on time-scales quite different from those of phenomenological experience. Analysis of language into units and subsystems is thus not straightforward, since they show functionality on many levels and many time-scales. Finally it is recognized that, as a specific and specialized system of inter-individual coordination, natural language is many hierarchical levels away from simpler forms of information transmission in biological systems. Keywords: control, constraints, dynamics, measurement, natural language, symbol, symbolic structure

1. Introduction Investigations of natural language are conducted from perspectives that depend on the domain of science, on available methodology, and on cognitive and practical motivations. Cross-talk among the domains is often difficult in that terms are often defined differently. This is because they are rooted in diverse theoretical traditions and, looking bottom up, by different modes of operationalization and data gathering. Differences concern even those terms which are crucial for specification of the phenomena under study. This applies, for example, to the terms language and linguistic symbols. For those fascinated with how the evolution of human communication gave rise to beautiful and intricate sound-systems, emphasis falls on accurate formal description of the system itself and, where needed,

162

Joanna Rączaszek-Leonardi

models of its ontogenetic and/or on-line generation. On this view, the term language identifies a system of symbols (linguistic elements corresponding to words) that are lawfully connected by grammatical rules; the main goal of linguistics is to discover those rules. By contrast, those interested in how language shapes human interactions tend to focus on dynamical processes and mechanisms which make language an important social-regulatory and coordinating device. From this perspective the term language identifies both the sound patterns exchanged and dynamical processes in which these sound patterns are engaged. It is inextricably bound up with the dynamical variables that shape interactions. It is unlike an artificial code. Far from being an elegant system of units connected by rules of grammar, a language is a meshwork of entities of different types and processes that function on many levels and time-scales. This perhaps is better termed languaging (e.g., Cowley 2005; Kravchenko 2007). Yet the two perspectives do look at one single phenomenon, and the most fascinating challenge is that of coming to understand how they relate to each other. Out of dynamical languaging, we come to discern a pattern of sounds that yields to formal (albeit “leaky”) descriptions. In this way, an ordered system of sounds connects with rich dynamics. This paper aims to trace the structuring influences that dynamical events exert on the patterns of sounds, without relying on explanations in terms of inborn and/or learned rules. In so doing it accepts that language is primarily a rich system for coordinating human interactions in their physical, emotional and cognitive aspect. 1.1 Language without symbols? Dynamic processes have become crucial to explanations in a range of theories that bear on natural language. Once their role is acknowledged one can develop new kinds of measurements that range widely across time-scales and levels. One can focus on cognitive processes in individuals as in speech perception research (Tuller et al. 1994; Rączaszek-Leonardi et al. 1999; Port and Leary 2005; Port 2007) and on-line sentence understanding (Rączaszek-Leonardi et al. 2008). Alternatively, one can turn to inter-individual processes such as emotional and motive coordination (Trevarthen 1979, 1998; Cowley 2006), physical coordination through speech (Shockley, Baker, Richardson, and Fowler 2007), or coordination of conceptual categories through language in development (Steels and Belpaeme 2005). Finally, one can turn to investigations at levels of the group and species. Thus, dynamic process models can be used to investigate language evolution, e.g., evolution of speech sounds (Lindblom et al. 1984; for more recent work see Oudeyer 2006), or the emergence of particular syntactic properties (Smith, Brighton, and Kirby 2003; Culicover and Nowak 2003). Research and theory thus span time scales that



Symbols as constraints: Dynamics and self-organization in language

range from the milliseconds of on-line language processing by the brain, through the seconds and minutes of interactions, the months and years of linguistic development, the hundreds of years of cultural history, up to the millions of years of biological evolution. Further, they engage systems at levels from those of an individual’s neural networks, to dyads, cultural groups, and whole societies. In relating symbolic and dynamic aspects of language, one could take an absolute dynamicist view by placing sound patterns of a language on a par with the physical dynamics of communicative exchange. In principle, it might be supposed that our tendency to see symbols in the resulting patterns can be sacrificed in the name of a unified dynamical description.1 But is such description possible at all in any living system engaged in information transmission? This issue is tackled at a basic level by theories of informational processes in living organisms. These enable one to ask whether symbolic properties are indispensible to organisms. If this proves to be the case, what are these properties? And, just as importantly, what is their relation to dynamics? One theorist who pioneered such lines of enquiry is Howard Pattee who, in the last 40 years, has developed a comprehensive account of how symbols contribute to the informational processes that are necessary to the existence of living organisms (see Pattee 1969, 1972, 1982, 1987, 1996, 1997, 2001). In so doing, he began with a simple question: Which processes, if any, require description involving more than the rate-dependent laws of physics?2 In pointing to the role that symbolic information plays in organisms, Pattee was obliged to reconstruct the notion of symbol. He moves away from how the term is used in formal systems and, for that matter, most linguistic theories and defines symbols by the function they have with respect to the dynamics within which they evolved. If we adopt Pattee’s view that symbols are indispensable in living organisms, we open up new ways of thinking about language. It becomes possible that the symbolic nature of the repetitive sound patterns of natural languages may stem not only from the fact that external observers can describe these as symbols (and thus develop formal linguistic descriptions), but also for more biodynamic reasons. The symbolic may be non-eliminable from any naturalized model of language. Given the importance of symbols to living things, Pattee’s work can be used in rethinking the ‘symbols’ of natural language. Of course, the entities that are considered to have a symbolic role in natural language differ from what Pattee regards as symbols in living organisms. Indeed, they function with a quite different dynamics. Nonetheless by adopting Pattee’s view of how symbols and dynamics interact, one can develop new strategies for understanding natural language. In previous work, emphasis has been placed on empirical issues concerning the contextual dependence of meaning and language efficiency (Rączaszek-Leonardi and Kelso 2008). The prime concern of this paper is the crucial role of dynamics in

163

164

Joanna Rączaszek-Leonardi

the emergence of symbolic structures. In parallel to emergentist views of language (e.g., MacWhinney 2002; Elman 1999), most of the weight falls on iterative dynamical processes and self-organization. These, however, can be extended by Pattee’s view of why symbols are indispensable and how they are reciprocally linked with dynamics. Accordingly, the paper begins with implications of Pattee’s view for the definitions of crucial terms. Then it analyzes the structuring role of the dynamics that operates within embedded dynamical systems on several time-scales, and illustrates this process with examples from computer models of language evolution. Several issues are emphasized: – Self-organization comes to play a structuring role when dynamical variables are used in explaining the emergence of ordered patterns. – Dynamics links individual mind/brains with interactions and events in populations of individuals. As a distributed phenomenon, natural language is shaped not only by processes within individuals but also by forces operating on levels and time-scales far removed from language production or understanding. – Analyzing the observed structures of verbal patterns depends on the dynamics with which they evolved. It is less straightforward than reducing language to a system of elements (words). Different units function on different time-scales and for different processes. Two points perhaps should be stressed. First, the aim is not ontological reduction. Rather, it is that of establishing how symbolic and dynamic descriptions of language can be reconciled. Second, no simple analogy links processes in a cell with the communicative and coordinative processes of natural language. The two systems are separated by both qualitative transitions in evolution and layers of hierarchical organization. That said, the paper proceeds by arguing that both sets of processes exploit similar relations between symbols and dynamics. For this reason, it is important to specify how informational processes function in living systems. 2. Pattee’s framework: Meaning, symbols, and rules Pattee’s framework for relating symbols to dynamics (1969, 1972, 1982, 1987, 1997, 2001) is built upon his interest in differentiating physical and biological systems. Elsewhere, Cariani (2001), Umerez (2001), Rączaszek-Leonardi (2003), and Rączaszek-Leonardi and Kelso (2008) have begun to explore its implications for language and cognition. For current purposes, however, it will suffice to take a narrower focus. Specifically, using Pattee’s framework, the paper focuses on how our view of meaning is changed by his redefinition of symbol.



Symbols as constraints: Dynamics and self-organization in language

Pattee (1972, 1987, 1997) finds two functional reasons for not describing living systems in terms of physical, time-dependent, universal laws. The first is that of control. During morphogenesis this permits selective control of the rate of protein synthesis and, by extension, the ontogenetic formation of functional structures (morphology). The second process is measurement. This is a selection process which depends on choosing some of a multitude of dynamical processes. The result, which occurs at a particular moment, is symbolic. Indeed, since according to Pattee, symbols can only arise through measurement, the processes are functionally connected.3 It is because the constraints must carry (be remembered) across situations that the symbols must be physically instantiated. This ensures that when measurements give efficient control of structures and behaviors in an environment the results are preserved. Physical structures which necessitate description in terms of symbols are thus the basis for the displacement of measurement and control processes. The DNA of a cell exemplifies the physical realization of a structure that has symbolic function, i.e., the symbolic mode of a dynamical system. Its two descriptions, 1) as symbols and 2) in terms of physical laws, are in Pattee’s terms disjoint but complementary (Pattee 1996:â•›262–263). Since Pattee emphasizes the necessity of the two modes of description, his work represents a type of dualism. However we should stress that it is a non-Cartesian dualism: one type (mode) of description cannot function independently of the other. It would, therefore, be impossible to rely solely on the symbolic description to formulate either the rules of organism construction (in the case of DNA) or information processing (in the case of the “symbolic mind” of classic cognitive science). The claim about the fundamental role of symbols in the informational processes in living organisms makes Pattee’s approach consonant with approaches in biosemiotics that take the relation of meaning (here that between symbols and dynamics) to be a primitive element of the description of living organisms (see, e.g., Barbieri 2007 or Favareau 2007). Emphasis on the co-presence of the descriptions in accounting for information in living systems has profound consequences for Pattee’s concepts of symbol and meaning. Thus symbols do not give detailed specifications of natural dynamics but, rather, constrain what happens. They neither contain detailed programs for the actions they control nor give blueprints for structures that they build. Just like a cell’s DNA, a symbol serves to harness dynamics that are already there. By definition, symbols function as “hierarchical constraints which harness matter to perform coherent functions” (Pattee 1972:â•›248). On such a definition symbols share little with elements that can be manipulated independently of semantics. They differ from the syntactic units that are used in computational systems. This is because, in Pattee’s view of symbols in biology, they have meaning based on their function as constraints on dynamics. It follows,

165

166

Joanna Rączaszek-Leonardi

therefore, that no symbol can be abstracted from the meshwork in which it (or, indeed, the symbol system) has evolved. To account for meaning, therefore, one has to consider both the dynamics that the symbols constrain and, just as crucially, the dynamics that sustain symbolic structures. Thus there is no entity that the symbol captures, identifies, indicates, or otherwise indexes (viz. an external referent or mental representation). Instead, a functional definition gives meaning a dynamic, processual dimension (e.g., Pattee 1987). On the other hand, Pattee suggests that it would be an error to put symbols on a par with other physical entities. While symbols are physical stimuli, their selective role makes them special. More specifically, their relation to dynamical events marks a history of previous choices, selections, through which they have come to set up functional constraints on dynamics. Since these structures carry constraints over time, they are what, following von Neumann (1966), Pattee calls an internal description of an organism. Indeed, were it not for their physical structure there could have been no increase in the adaptive complexity of the living (see Pattee 1982).4 To view symbols in terms of constraining and measuring dynamical events has consequences for the structures that they form. This, of course, has theoretical implications. How symbols co-function depends on the dynamics they measure, the control they exert, and the structure of the measuring device. The functional definition of a single symbol’s meaning is thus awkward if not nonsensical. In many (perhaps all) cases, various units and substructures of the system are likely to be functional at different levels, or dynamical systems. The symbolic and systemic nature of the constraints on dynamics made Pattee deem the symbolic mode of description in living organisms linguistic. Some have criticized this vague term as metaphorical when applied to biological informational processes (Oyama 2000:â•›56). While there is truth in this, using linguistic, symbolic, and meaning in biological contexts can lead to powerful new insights, for the benefit of both biology and linguistics. Given a physical biologist’s model of how symbols impact on dynamics, one can establish important parallels with natural language. The parallels are not based on a superficial metaphor but on the recognition of the similarity of this basic relation in all living systems. In this way, language ceases to be a completely novel system. Rather, it can be shown to rely on principles of information transfer that are shared with other living organisms. 2.1 Symbols of natural language: Constraints on dynamics in multiple time-scales Taking this view on the relation between symbols and dynamics has dramatic implications for the study of language. Indeed, since symbols cannot specify in detail the dynamics they constrain, their role in the communicative system is, in a way,



Symbols as constraints: Dynamics and self-organization in language

reduced. Conversely, the importance of dynamics (on many time-scales and levels) comes to the fore. This is consistent with the view that symbols evolved by taking advantage of rich dynamics on many time-scales. In psycholinguistics and developmental psychology, many researchers are rethinking the nature of linguistic symbols. Mainstream researchers in language development are increasingly emphasizing the interactional context of languageactivity and, thus, reconsidering work by functionally oriented theorists of the 50s and 60s (e.g., Halliday 1975; Bruner 1975). For example, Tomasello stresses how utterances contribute to mutual control of attention during interaction. By so doing, he has focused on methods that investigate the dynamical processes in which a symbol participates (Tomasello and Akhtar 1995; Tomasello 1999; for a discussion of the notion of symbol see Tomasello 2000). Researchers more firmly grounded in the functional tradition increasingly trace skills in human interaction to the communicative and coordinative capacities of newborns and very young children (Trevarthen 1979, 1998). These capacities form a basis for language and, on the other hand, the child’s increasing engagement with language (and other people) is instrumental in ensuring the self-organization of its own co-ordinative (and cognitive) powers (Cowley 2006). In linguistics, the formalist model of linguistic symbols remains stronger. In this field, however, it has been challenged by theoretical and empirical research on the context-dependency of meaning. As Barwise and Perry (1983) pointed out, philosophers of language were long preoccupied with entailments between (seemingly) eternal sentences and, for this reason, considered the ambiguity of natural language as a rare imperfection. This made it difficult to acknowledge that ambiguity may be one of the central features of language. Thus, as they put it over 20 years ago, expressions (…) can be recycled, can be used over and over again in different ways, places and times and by different people, to say different things. This is what we mean by the efficiency of language (Barwise and Perry 1983:â•›32; authors’ emphasis).

Efficiency depends on ambiguities that enable context to shape meaning. Once this is acknowledged, one can ask how context contributes to utterance semantics. Many authors have tried to systematize and formalize how pragmatic factors contribute to communication by using the interplay between speaker and hearer. Thus dynamic theories of meaning emerged in the 1980s (Heim 1982; Kamp 1981) and, more recently, led to game-theoretic approaches to communication (e.g., Benz et al. 2006). In all such theories, it is acknowledged that there is a complex relation between linguistic symbols and their situated communicative meaning.

167

168

Joanna Rączaszek-Leonardi

Context dependency is necessary in any model where symbols function as constraints. This is because entities can take on symbolic functions in measuring and constraining dynamic processes. They do not constitute meaning. Rather, they participate in its situated construction and, for this reason, adapt to the context. Abstract linguistic “meaning” may be a generalization from many concrete, situated constructions, but meaning of an expression cannot be — once and for all — specified (be it only in a linguist’s head). In a sense it is possible only post-factum, when symbols have acted in a given situation. Human interaction draws on many factors that are independent of language, establishing its own dynamics. Language ‘relies’ on this dynamics and this makes the action of symbols always contextually flexible, and, in evolution, gives symbols only as much power as to harness omnipresent dynamics. 2.2 The interplay of dynamics Acknowledging natural context-dependency through the action of dynamics constrained by symbols holds not only for the time-scale of on-line communication. Dynamics function in multiple, intertwined time-scales. In applying this view to language it is useful to focus on, at least,5 the time-scales of cultural evolution, language development, and on-line human communication. Their interdependency means that events on one time-scale will serve as constraints for events in the others while, of course, also constrained by them. For example, the evolution of linguistic symbols “counts on” reliably present on-line interaction dynamics. Language thus evolves to control only what needs to be controlled. As noted above, it does not need to specify the full content of an exchange. There is a parallel with the case of DNA. Just as this takes into account, for example, gravitation and gradients and, for this reason, does not “code for” their action, the symbols of natural language evolve to participate in meaning construction. They too “count” on the dynamics of cognition, interaction, and coordination. Language is thus a system of constraints which evolved around both human cognitive capabilities (for perception and learning) and natural interaction dynamics among people (MacWhinney 1999; Deacon 1997). It is beyond doubt that, with the emergence of language, these dynamics were thoroughly transformed. In this context, however, what matters is not only that dynamics influences language evolution but that its effects also plays out today. Language can be seen as evolving in natural human interactions in a variety of social situations. It transformed these interactions making new ways of coordination possible by providing only the information that was “missing”. It slowly, but never entirely, transformed them into linguistic interactions.



Symbols as constraints: Dynamics and self-organization in language

The functions of natural language integrate dynamics on numerous timescales by connecting systems from different hierarchical levels. Thus the dynamics which maintains and is maintained by linguistic symbols occurs at the levels of individuals, communicative interactions, and those which ensure the inter-generational transmission of language. Accordingly, many forces that shape language depend on interactions and collective processes that are outside an individual’s brain/mind. From an individual perspective, therefore, language is a distributed phenomenon (Cowley 2007; Love 2007). In a discussion on distributed “mental” or “cognitive” capacities (in the sense of Hutchins 1995), one of the arguments against such concepts was the phenomenology of knowing, cognizing etc., that should be the main target for explanation (Harnad 2005). However, this phenomenology does not prevent individual cognition from subserving (and influencing) how members of a society collectively cognize the world. Similarly the claim that individual phenomenology is primary seems inapplicable to language. The feeling that we know language, i.e., what words “mean”, and what are the “proper” grammatical rules is no reason to confine language to the individual’s head. Rather, phenomenological content contributes to the overarching regulatory function of language. A comparison might be made with collective behaviors in social animals. Thus, neither what occurs in an individual termite’s mind/brain, nor the proximal stimuli that prompt a specific individual termite behavior (e.g., “drop a piece of soil here”) are sufficient to explain the resulting mound. The cognitive and behavioral results of encountering a pheromone template are subservient to a collective result. Therefore, neither the pheromone, nor the “cognitive” or memory capacities of the termite that allow the appropriate reaction to it, need contain information about the mound’s functional role (that of protection of the colony and thermoregulation). 2.3 Syntactic rules describe emergent patterns If a symbol, as accepted by Pattee (see Section╯2), acts to constrain dynamics (in different time scales), the co-action of symbols is decisive for forming structures. Such symbolic structures are quite different from formal symbol structures. Since these symbols are inseparable from dynamics, the emergence of structures can be explained, at least in part, by dynamics which unfolds according to laws of physics (including those which lead to self-organization). Such view motivated reflection on the ontological status of the rules that describe those structures. For Pattee, functional interdependencies between symbols determine the configurations in which they occur. Symbols co-function because they come to control compatible aspects of dynamical events or, perhaps, to measure closely

169

170

Joanna Rączaszek-Leonardi

related aspects of dynamical phenomena. Finally, they might co-occur because of the properties of the measuring device, i.e., the human perceptual and cognitive system. The systematic co-occurrence of symbols is thus, in part, an outcome of the repetitiveness of the stabilized controlling/measuring function with respect to dynamics on multiple scales. Rules formulated to capture these systematicities are thus descriptions of these superficial co-occurrences, prone to change when dynamics radically changes. On this view, it is mistaken to posit that symbols “obey rules”: symbols, being involved in measurement and control, must “obey” the dynamics of the system within which they evolved. Thus rules, which describe the co-ocurrences of symbols, cannot be used as explanatory constructs. Indeed, this error creates a false sense of understanding where, in fact, our models grossly oversimplify the underlying multi-time-scale processes. While without explanatory value, much is nonetheless gained from noticing regularities in symbolic function (and co-function). Our descriptions are able to identify persistent biases in the underlying processes and, by so doing, mark phenomena that demand explanation. In the long term, this may be done by specifying how dynamical events are integrated across time-scales. The view that linguistic structures result from dynamical processes operating on multiple time-scales is congruent with recent views on the emergence of language. Deacon (1997) shows that such processes can account for emergence of systematicities that others have ascribed to inborn capacities they called universal grammar (Chomsky 1965). According to Deacon, structural properties of language may result from processes outside the brain that adapt language to both human cognitive capacities and society. In his terms, “Language structures at all levels are the product of powerful multilevel evolutionary processes to which innate mental tendencies contribute only one subtle source of the Darwinian selection biases” (Deacon 1997:â•›122). The tendency to attribute to children’s minds inborn grammatical rules is understandable but misleading: “Human children appear pre-adapted to guess the rules of syntax correctly, precisely because languages evolve so as to embody in their syntax the most frequently guessed patterns” (ibid.). Thus, besides many other structuring forces, this one “prunes” the unlearnable from language. Once one recognizes that symbols function as constraints on dynamics, it becomes clear that learning to talk (at least at the beginning) has little need for abstract structures. Children start off by doing things with words. By incorporating sound-patterns in situated action, they create rich structured activity. Patterned social interactions that take place in a physical world are gradually, and functionally, incorporated into structure of language. Another consequence of bringing dynamics into the explanation of linguistic structures is that new importance is given to self-organization. Just as in other



Symbols as constraints: Dynamics and self-organization in language

pattern formation processes in nature, dynamical, non-linear, local, and iterative interactions lead to spontaneous order formation. Observable linguistic patterns (apparently rule-based) may be, at least to some extent, due to such processes. Let’s consider one example from nature. As is well known (see, e.g., Murray 1988), it is difficult to use geometrical formulae to describe the surface shape of a leopard’s spots. However, once one acknowledges the dynamics of pattern formation (using reaction-diffusion equations), these can be traced to a morphogenetic model. As Murray (1988) shows, while it is impossible to predict the details of a given pattern (viz. where a spot will appear on the skin), Turing’s (1952) principles of morphogenesis can predict the shape and distribution of spots. Reaction-diffusion equations thus provide understanding by showing a possible mechanism of spot generation. The model, based on dynamical equations, constitutes a theory: by attuning the model’s parameters one can predict similar patterns in nature (e.g., not leopard’s spots but tiger’s stripes). Self-organization has been used to throw light on linguistic phenomena already in the early models of the emergence of phonological systems (Linblom, MacNeilage, and Studdert-Kennedy 1984). Since then, appeal to “emergence” in the explanation of structures in language has been on the rise (e.g., MacWhinney 1999; Hurford 1998; Oudeyer 2006). However, contrary to the early hopes of dynamical systems theory, Pattee’s work suggests that self-organization is not enough to account for the adaptive increase of complexity in any system. Thus the mechanisms of self-organization do not provide mechanisms for evolution. As D’Arcy Thompson (1992:â•›201) wrote: “In the order of physical and mathematical complexity there is no question of the sequence of historic time. (…) A snow-crystal is the same today as when the first snows fell”. Something is needed to remember the selection outcomes and this something might be physical structures with symbolic function. Both biological and cultural evolution, while relying, “counting on”, dynamic spontaneous order-formation, need symbols. On the other hand, dynamics and self-organization are present — both in on-line processes as well as in learning and evolution. Symbols do not need to bring a great deal of structuring information. It is sufficient that they give the dynamics “light pushes” in certain directions. To repeat, their role is that of harnessing, not of constructing structures or behaviors. With self-organization, explanations of the emergence of linguistic systematicities need to rely to an even lesser extent on “formal properties” of symbols and rules. There are many advantages in positing that dynamic, self-organized pattern formation contributes to the emergence of linguistic structures: – A link is established between local (fast) processes and their global counterparts. – The system can be co-shaped by both internal and extra-systemic forces.

171

172

Joanna Rączaszek-Leonardi

– Self-organization and dynamics assume a structuring role in pattern emergence, enriching the explanations by providing plausible mechanisms. – There is no need to propose a set of inborn rules or homunculi. To tackle the emergence of linguistic patterns in many time-scales and levels, the phenomena can be clarified by using computer models. Next, therefore, it is shown how such models can be used to throw light on the evolution of syntax. While of value in capturing the relevant dynamics, these models (more or less consciously) neglect certain time-scales. Further, they typically make formalist assumptions about symbols and communication. In principle, however, such methods might be linked with the framework developed by Pattee by viewing communication as social coordination. 3. Identifying dynamics: Examples from computer modeling Computer simulation serves to model multiscale and multiagent complexity (see e.g., Cangelosi and Parisi 2002). There are now simulations of various aspects of language production, understanding, development, and evolution. In this context, the focus is on the latter category or “computational evolutionary linguistics” (Kirby 2001; Smith, Brighton, and Kirby 2003; Cangelosi 2006; Steels and Belpaeme 2005). Brighton and colleagues echo Jackendoff in describing their goal: If some aspects of linguistic behavior can be predicted from more general considerations of the dynamics of communication in a community, rather than from the linguistic capacities of individual speakers, then they should be (Jackendoff 2002:â•›101; cited in Brighton, Smith, and Kirby 2005:â•›291).

The evolution of language is seen as exploiting intertwined complex dynamical systems, operating in at least the time-scales of biological evolution, cultural language evolution, and language acquisition (phylogenetic, glossogenetic, and ontogenetic, respectively). What is often missing, therefore, is attention to on-line communication. In one model, Smith et al. (2003) focus on how inter-generational transmission might shape language structuring factors. While this limits the model to cultural and learning time-scales, they use these idealizations to show that compositionality can emerge as an adaptation to the “bottleneck” in language transmission (i.e., to a situation where a next generation experiences only a subset of language). They conclude that only languages that are learnable are stable in evolution and, as a result, posit a process of “cultural selection for learnability” (Smith et al. 2003; Brighton et al. 2005:â•›291). In light of the debates that dominate especially developmental linguistics (e.g., Pinker 1994), this is an important



Symbols as constraints: Dynamics and self-organization in language

conclusion. It shows that the poverty of stimulus can give rise to compositionality without proposing “rules in the head”.6 Further demonstrating the power of the method, processes on one scale can be shown to lead to the emergence of compositional structure on another (see Batali 1998; De Beule and Bergen 2006). While not challenging intergenerational transmission, the aforementioned papers show that systematic syntactic structures can emerge from repetitive instances of communication in a structured environment. Semantic factors can thus contribute to the emergence of structure. Smith, Brighton, and Kirby (2003) focus on minimal requirements for structure. To do this they use only a subset of relevant variables to examine whether structuring can be explained without evoking the properties of individual cognition or properties of the communication process. By emphasizing processes of intergenerational transmission, they underplay the dynamics of interpersonal communication (and its time-scale). Certain claims, therefore, should be treated with caution. Rather than concur that “were there no poverty of the stimulus, compositional language would have no advantage over unstructured holistic language” (Kirby 2003:â•›7), we might posit that some aspects of structure draw on other factors. They might depend, for example, on a history of body-world interactions. Indeed, just because one feature is sufficient to evoke certain aspects of a phenomenon, this is no reason to think that its genesis (and nature) can have no other basis. Both individual cognition and on-line communication are likely to matter for several reasons. First both the results of behavior and communicational effectiveness will favor selection of structure. This is likely to be quite independent of its (theoretical) learnability. In other words, certain elements or structures may be selected due to their value for the organism or for the efficiency of communication. Further, this may apply even if they require effortful learning. Second, the rich social-coordination dynamics of human interaction provides a likely source for ‘mutations’. As we encounter each other, we creatively use linguistic symbols to exert control over what happens. In this context, moreover, creativity does not reduce to ‘generativity’. While there are many reasons for this, one is that people often do violate extant grammatical rules. Third, inter-individual communicative events do shape both linguistic, and as studies increasingly show, nonlinguistic aspects of individual cognition (see, e.g., Bowerman and Levinson 2001; for computer models, see Steels and Belpaeme 2005). Clark puts this strongly. “We cannot hope to understand language use without viewing it as joint actions built on individual actions” (Clark 1996:â•›4). But the influence is bidirectional: communication and co-action may have just as big a role in constituting individual (especially linguistic) cognition, motivational systems, and actions, as these have in constituting language (Cowley 2006; Trevarthen and Aitken 1994).

173

174

Joanna Rączaszek-Leonardi

Thus, while theoretically interesting to ignore on-line communication in order to focus on other factors and scales, this must be done with care. Indeed, simulations of language evolution cannot avoid assumptions about communicative functions, even though they often remain implicit. These are revealed in the tasks which agents simulate. For example, Smith et al. (2003) posit that agents learn the ‘output’ of previous generations’ interactions. On this view, learning reduces to passing symbol-meaning correspondences across the generations. Similarly, while Batali (1998) posits agents that seek to establish a common reference (i.e., have to point to the same scene), De Beule and Bergen (2006) give them the task of computing a speaker’s output vector (i.e., mapping the form of a speaker’s utterance). Both simulations involve simple and complex (composed) scenes (vectors). It seems, therefore, that communicative compositionality exploits the compositionality of the scenes (vectors). While compositionality may to some extent reflect on the structure of a world, it is also possible that it is based upon more complex relations such as the structure of the world filtered through the exigences of social co-action in this world. Computational models show the importance of dynamics and self-organization on various scales. However, they tend to reduce language to form-meaning pairings which can be transferred across generations or between agents. From this point of view, symbols become forms whose static meanings have no impact on dynamics. 4. Communication: Meaning mapping or coordination? Pattee’s view of symbols is incompatible with positing that symbols “map” any “meaning”. Rather, their primary role is that of constructing functional structures and shaping coordinative behavioral patterns. Applied to natural language, this supports a functional approach which prioritizes not the individual but how language sustains physical and social coordination. Mapping meaning from hearer to speaker, i.e., establishing common reference, which is usually taken to be the goal of communication, is no more than one linguistic function (Halliday 1975; Hymes 1972). Using language, parties achieve emotional and cognitive coordination. In such symmetrical interactions, each speaker both provides and seeks out information (Hodges 2007). Indeed, the sameness of reference is itself brought into doubt when one recognizes that a specific use of an expression depends not only on objects important in a given communication but also on their subjectively, temporarily, and contextually important aspects. Thus, if some meaning mapping occurs, it may well serve the goal of coordination.



Symbols as constraints: Dynamics and self-organization in language

Physical symbols can be thought as constructing a cognitive niche (Clark 2006). Given Pattee’s view of symbols, however, this is not to be explained by our skills in referring to (or representing) objects. Rather, linguistic symbols function, in the first place, to constrain interactional dynamics. Living in the language niche brings about new ways of coordinating action and shapes individual cognition. While Clark (2006) correctly notes that the representational functions of symbols give a role to the self-regulating potential of self-directed speech, inter-individual communication is likely to be primary. Indeed, if language has a representational aspect, this is presumably subordinated to the coordinative function of language. This is possible because live ongoing action can be coordinated without shared referents for symbols (if referents are objective entities; cf. Galantucci 2005). If language functions as social coordination achieved by sharing, linking (co-dividing) cognition distributed in social practice (Hutchins 1995), then communication need not depend on negotiating “sameness of meaning”. In coordinated social action, it often matters little if meanings (viz. external references or mental representations) differ. Further, there are also situations where coordination depends on there being differences in individual, psychological “meanings”. For example, we expect a fireman’s concepts and action programs that relate to the word ‘fire’ to differ from the welder’s. Linguistic on-line communication is more than a matter of establishing reference (for example, by conveying intentions of a speaker vis-à-vis reference with the goal to shape intentions of a hearer vis-à-vis reference). However, it may also be much less. In communication, reference can be established by any means that gives enough ‘sameness’ for co-action. Indeed, this is the value of exploiting symbols which are ambiguous and underspecified. Linguistic forms, therefore, do little on their own. They function to control rich interpersonal dynamics in ways that exploit the current situational context. Once we take this view of ‘symbol’ seriously, we recognize that communication does not happen because of how we use language. Rather language directs and constrains coordination, both on-line and in its evolution, by enabling us to use extant forms of communication. While simulations often define ‘communicative success’ in terms of meaning mapped between hearer and speaker, other criteria could be used. If they are focused on the results of coordinated action, emergent structure might be traced to features other than the (usually arbitrarily constructed) external world. At least in part, it might reflect on how, in a given environment, we structure co-action. After all, the only gauge of communicative success is behavioral and emotional coordination. It is by no means obvious that these types of coordination depend on the sameness of reference (or even connotation).

175

176

Joanna Rączaszek-Leonardi

If language is shaped not only by cognitive factors (Deacon 1997), learning and transmission properties (Steels, Brighton, and Kirby 2003) but also by the requirements of interactive coordination, similar physical and social worlds should give rise to biases for shaping language. Thus, just as in Deacon’s explanation for “universal grammar” cited above, we should expect there to be similarities across human cultures and, for this reason, commonalties in structure. In other words, we should expect the world to create “common adaptive contexts” for language evolution. Conversely, we can begin to think differently about differences in structure. Among other factors, these may point to and stem from how tasks are coordinated in a given society. This general view has many implications. Once we cease to give priority to symbol-meaning mappings, we see that children begin by learning to control interpersonal events through language. From this it follows that language can be used creatively for on-line purposes: when situational factors exert an influence, innovations are likely to appear. In some cases, moreover, these may violate the ‘rules of grammar’ shown in linguistic descriptions. In Steels’ terms, “according to this dynamic coordination view, language is forever changing. It is a complex adaptive system shaped and reshaped by the members of a population in order to satisfy their needs” (2006:â•›349). Language thus adapts to current coordinative needs, relying on the process of innovation generation and selection. As shown in linguistic and sociolinguistic studies, adults often deviate from what is semantically and syntactically “correct” for pragmatic reasons (Hodges 2007; Garrod and Pickering 2007). Galantucci (2005) shows creative processes in how we invent communication systems that are suited to various tasks. In cooperative settings, adults can develop a complex communication system that functions in spite of mismatches in reference. Language effectuates coordination on many levels — from individual conceptual coordination (on-line and in development), to physical coordination by strictly physical, acoustic, and temporal variables that mutually constrain posture and participant attention (Shockley et al. 2007), to emotional alignment (in development and on-line), and cognitive coordination (interpersonal alignment of the situation view). The relevance of a multitude of shaping dynamical factors raises questions about the units of linguistic structure. One might expect that selective pressures will act on various units, depending on their overall functionality. 5. Units of linguistic structure Viewing words as elements of phrases, themselves elements of sentences and larger texts may mislead in subtle ways. There is, it seems, no reason to suppose that this



Symbols as constraints: Dynamics and self-organization in language

reflects on the functional division of the intricate social coordination that we call language. In other words, the phenomenological units into which structure is readily analyzed do not necessarily match the division into functional units that stem from how language controls and is maintained at different levels and time-scales. The view of language as a means of interaction, regulation, and coordination points to structural principles that are quite different from those of the compositionality of the world, of learning biases, and of vertical (intergenerational) transmission. For example, in a seminal work on dyadic interaction in the first days of a child’s life, Trevarthen (1979) showed how prosodic features of language tune the baby to social stimulation and, just as importantly, how the baby’s prosody elicits behavior from a parent. Language serves to regulate interaction from birth, and thus serves as a means of seeking information. Building on this work, Cowley (2004) shows how primary coordination can shape the motivational systems that help mother and infant to coordinate emotionally and then cognitively. Such studies show two important things. First, the units subjected to evolutionary pressures might be quite different from “words”. Among other things, they depend on prosodic contours and the timing of exchange. Second, variables that make a functional contribution to socially coordinated action in a given environment are better candidates for structuring language than are, say, the purely perceptual variables, accessible to phenomenal experience, seemingly used to categorize the world. To use language “correctly” in early interaction, a child needs to sensitize to these patterns and, in some way, to distinguish them. These are complex, non-obvious, dynamical variables pertaining to the interaction with the world that need to be conveyed to a conspecific in order to coordinate. Research on pragmatic functions of linguistic forms is pursued within functional approaches to language, linguistic anthropology, and discourse analysis. In their book Interaction and Grammar, Schegloff and colleagues write: [O]nce we register that language figures in the actual, practical activities of the lives of people and societies, and that how the language is configured is more than incidentally related to its involvement in those activities, it is readily apparent that, at the very least, attention must be paid to what the relationship is between activity, action, and the orderly deployment of language called grammar (Schegloff, Ochs, and Thompson 1996:â•›21).

A few pages later, they add: [The] interactional matrix of grammar requires a different understanding of what should enter into a linguistic description and/or a different model of linguistic structure (Schegloff et al. 1996:â•›24).

Numerous examples are adduced to show how syntax and prosody co-regulate mutual attention and turn alternation. These forces shape utterances or larger

177

178

Joanna Rączaszek-Leonardi

fragments of discourse, to which, words may have to “conform”. An individual linguistic symbol might be thus an abstraction: symbols always appear in a system and what is maintained and functional is a whole system rather than individual elements. It is an a priori division of language into elements that leads to questions like: “what syntactic rules order those elements?” Indeed, there may be better ways to discover the ordering forces, focused on how language is functional on various levels of its systemic properties. Various substructures may be functional on various levels and time-scales: e.g., some shaping and being shaped by developmental processes, others by on-line communication. Pattee’s view of how symbols constrain dynamics allows us to rethink how coordinative dynamics influence symbolic function (and vice versa). This enables a greater role to be given to the dynamical processes that, during human interactions, are always present. Indeed, applying Pattee’s view of biological symbols back to language aids, first, in recognizing the relevant dynamics. Second, it can promote a grasp of how relevant functional substructures emerge. At a more theoretical level, the approach suggests that the shape of a specific language is unlikely to ‘reflect’ either the structure of the physical world or an individual’s perceptual system. This is because the forces that structure language manifestly function on many levels and timescales. While both physics and human perceptual powers are likely to have a role on structuring language, the resulting ‘mapping’ depends on mediating functions that specify how perception of the physical world plays out in (social) action that depends heavily on interindividual coordination. 6. Conclusion: The specificity of natural language There are advantages in following Pattee’s positing of a particular relation between symbols and dynamics in natural language. Above all, because it affords theory to follow work on information in living systems by recognizing that dynamics and self-organization have an explanatory role in shaping symbolic structure. By admitting that several kinds of dynamics bear on this structure, a theorist is able to recognize that her focus lies on one scale or level. At the same time, it can be acknowledged that each of these is embedded in dynamical systems that have their own time scales. Far from being reducible to a system ‘in the brain’, language must be viewed as radically heterogeneous and as spread across space, time and bodies. It is thus mistaken to focus exclusively on either individual or collective processes. Indeed, the theoretical challenge lies in showing how these can be connected. By recognizing that symbols function to constrain dynamics, we can begin to ask how it is that individual dynamics gains from being embedded in (shaping



Symbols as constraints: Dynamics and self-organization in language

and being shaped by) the dynamics that takes place in higher level interpersonal scales. On this view both levels will exert pressures to stabilize structures in evolution by processes of selection and self-organization. Importantly, many processes that shape language are not accessible to individual consciousness. Indeed, the phenomenology of language is likely to be subservient to dynamics at the level of inter-individual coordination. The above account does not explain what marks out natural language from nature’s other informational processes. It is, however, clear that language contrasts with other symbolic processes of information transfer that characterize living beings. It is thus only a beginning to describe natural language as a system of symbols that constrain the dynamics of development and interactions. Maynard Smith, and Szathmary (1999:â•›16–18) claim that five major qualitative changes in the way of storage and transmission of information had to take place in the evolution from an adaptively self-reconstructing unicellular organism to a society organized through language. The issue of the specificity of natural language with respect to other information storing and transmitting systems is the next step in linking Pattee’s work to cognitive and linguistic theories. In the area of cognitive functioning, the task has already been undertaken both by Pattee (1982, 1997) and others (see, e.g., Cariani 2001). One aspect of what makes language special is clear. We have descriptive grammatical rules. While lacking explanatory value in the larger scale of things, these rules do capture systematicities in linguistic signals. As a result, once identified, they can perform many functions. Given cognitive systems that excel in extracting statistical regularities, they can be used to scaffold learning. Children notice and use systematicities and, in this sense, the ‘rules’ which shape their knowledge (see e.g., Saffran, Aslin, and Newport 1996). As a result, the approach can throw new light on statistical models of language learning, and construction learning (Goldberg 2006), assigning to them a stabilizing role in the overall dynamics. Being able to treat language as a phenomenon in itself, being able to talk about it and — apparently — identifying its elements on some level, indubitably transforms the way language is used (see Cowley’s “taking the language stance” e.g., 2006, 2007, 2011, and Love’s (2004) “second-order” constructs). However, even if acquiring rules as petrified systematicities is a learning strategy, language remains a biological information transmission system that sustains a living organization. It exploits a “basic”, “primitive” relation between symbols and dynamics. Resorting to this primary relationship is indispensable for two reasons. First, without grasping it, we could not understand how language originated. Second, no matter how much the present shape and function of language depends on self-reflecting agents and second-order constructs, its grounding lies in the dynamical forces that sustain it as a symbol system. These forces are still present. Language has never broken free to become an abstract object.

179

180

Joanna Rączaszek-Leonardi

Notes *.╇ This work was funded by a Marie Curie Fellowship MEIF-CT-2007–042086 to the author. I wish to thank Stephen Cowley for many discussions and invaluable comments on this paper, and to Paul Thibault for his insightful comments on the earlier version. 1.╇ Attempts at developing non-symbolic approaches to cognition have been made in neuroscience (Freeman and Skarda 1990), some neural network models, and dynamical systems theory (van Gelder and Port 1995). The “symbols” that were under attack in these approaches were the mental symbolic representations (and the view of cognition as computation over these). As it will be shown later, the symbols that seem necessary for a Patteean view of information in biological systems are not of the same kind and do not have the same, representing function. 2.╇ In Pattee’s words, laws of physics are expressed as rate-dependent equations (e.g., Pattee 2001:â•›11), i.e., they are directly related to physical time and space. 3.╇ A symbol is an outcome of a selection process, compressing dynamical information into a variable relevant for an organism — this, in Pattee’s writings, is a function of measurement. 4.╇ The term “description” should be interpreted with caution. It was used by von Neumann and others to characterize the conditions for adaptable self-reproduction or self-reconstruction. Description, being one of such conditions, does not have a straightforward relation to “representation” that the term “internal description” usually evokes in the context of cognitive sciences. 5.╇ Other time-frames are certainly involved in the emergence of language. Recently MacWhinney (2005) has made a proposal for their typology. 6.╇ In generative grammar the very same “poverty of the stimulus” argument was used in Chomsky’s claims that since it is logically impossible to infer the rules of grammar from the input, children must rely on a set of inborn grammatical rules. (Chomsky 1980:â•›34-38)

References Barbieri, M. 2007. “The challenge of biosemiotics”. In M. Barbieri (ed), Introduction to Biosemiotics. Dordrecht: Springer, ix–xii. Barwise, J. and Perry, J.R. 1983. Situations and Attitudes. Cambridge, MA: The MIT Press. Batali, J. 1998. “Computational simulation of the emergence of grammar”. In J.R. Hurford, M. Studdert-Kennedy, and C. Knight (eds), Approaches to the Evolution of Language. Cambridge: Cambridge University Press, 405–426. Benz, A., Jäger, G., and Rooij, R.V. (eds). 2006. Game Theory and Pragmatics. Houndmills, Basingstoke, Hampshire: Palgrave Macmillan. Bowerman, M. and Levinson, S. 2001. Language Acquisition and Conceptual Development. Cambridge: Cambridge University Press. Brighton, H., Kirby, S., and Smith, K. 2005. “Cultural selection for learnability: Three principles underlying the view that language adapts to be learnable”. In M. Tallerman (ed), Language Origins: Perspectives on Evolution. Oxford: Oxford University Press, 291–309. Bruner, J. 1975. “The ontogenesis of speech acts”. Journal of Child Language 2: 1–19.



Symbols as constraints: Dynamics and self-organization in language

Cangelosi, A. 2006. “The grounding and sharing of symbols”. Pragmatics & Cognition 14(2): 275–285. Cangelosi A. and Parisi, D. 2002. “Computer simulation: A new scientific approach to the study of language evolution”. In A. Cangelosi and D. Parisi (eds), Simulating the Evolution of Language. London: Springer, 3–28. Cariani, P. 2001. “Symbols and dynamics in the brain”. Biosystems 60, 59–83. Chomsky, N. 1965. Aspects of the Theory of Syntax. Cambridge, MA: The MIT Press. Chomsky, N. 1980. Rules and Representations. Oxford: Basil Blackwell. Clark, A. 2006. “Language, embodiment, and the cognitive niche”. Trends in Cognitive Sciences 10: 370–374. Clark, H.H. 1996. Using Language. Cambridge: Cambridge University Press. Cowley, S.J. 2004. “Contextualizing bodies: How human responsiveness constrains distributed cognition”. Language Sciences 26: 565–591. Cowley, S.J. 2005. “Languaging: How humans and bonobos lock on to human modes of life”. International Journal of Computational Cognition 3(1): 44–55. Cowley, S.J. 2006. “Distributed language: Biomechanics, functions and the origins of talk”. In C. Lyon, C. Nehaniv, and A. Cangelosi (eds), The Emergence and Evolution of Linguistic Communication. London: Springer, 105–129. Cowley, S.J. 2007. “The cognitive dynamics of distributed language”. Language Sciences 29: 575– 583. Cowley, S. J. 2011. “Taking a language stance”. Ecological Psychology, 23/3: 185–209. Culicover, P. and Nowak, A. 2003. Dynamical Grammar. Oxford: Oxford University Press. De Beule , J. and Bergen, B.K. 2006. “On the emergence of compositionality”. Proceedings of the 6th evolution of language conference, Rome, 35–42. Deacon, T. 1997. The Symbolic Species. New York: Norton & Company. DeLoache, J.S. 2004. “Becoming symbol-minded”. Trends in Cognitive Sciences 8: 66–70. Elman, J.L. 1999. “The emergence of language: A conspiracy theory”. In B. MacWhinney (ed), Emergence of Language. Hillsdale, NJ: Lawrence Earlbaum Associates, 1–28. Favareau, D. 2006. “The evolutionary history of biosemiotics”. In M. Barbieri (ed), Introduction to Biosemiotics. Dordrecht: Springer, 1–67. Freeman, W.J. and Skarda, C.A. 1990. “Representations: Who needs them?”. In J.L. McGaugh, N.M. Weinberger, and G. Lynch (eds), Brain Organization and Memory. New York: Oxford University Press, 375–380. Galantucci, B. 2005. “An experimental study of the emergence of human communication systems”. Cognitive Science 29: 737–767. Garrod, S. and Pickering, M.J. 2004. “Why is conversation so easy”?. Trends in Cognitive Sciences 8: 8–11. Goldberg, A. 2006. Constructions at Work: The Nature of Generalization in Language. Oxford: Oxford University Press. Halliday, M.A.K. 1975. “Learning how to mean”. In E.H. Lenneberg and E. Lenneberg (eds), Foundations of Language Development, Vol. 1. Paris: UNESCO Press and Academic Press. Harnad, S. 2005. “Distributed processes, distributed cognizers, and collaborative cognition”. Pragmatics & Cognition 13(3): 501–514. Heim, I. 1982. The Semantics of Definite and Indefinite Noun Phrases. Unpublished PhD. thesis, University of Massachusett, Amherst.

181

182

Joanna Rączaszek-Leonardi

Hodges, B.H. 2007. “Good prospects: Ecological and social perspectives on conforming, creating, and caring in conversation”. Language Sciences 29: 584–604. Hurford, J. 1998. “Introduction: The emergence of syntax”. In J.R Hurford, M. Studdert-Kennedy, and C. Knight (eds), Approaches to the Evolution of Language: Social and Cognitive Bases. Cambridge: Cambridge University Press, 299–304. Hutchins, E. 1995. Cognition in the Wild. Cambridge, MA: The MIT Press. Hymes, D. 1972. “Models of the interaction of language and social life”. In J. Gumperz and D. Hymes (eds), Directions in Sociolinguistics: The Ethnography of Communication. New York: Holt, Rinehart and Winston, 35–71. Jackendoff, R. 2002. Foundations of Language. Oxford: Oxford University Press. Kamp, H. 1981. “A theory of truth semantic representation”. In J. Groenendijk, T. Janssen, and M. Stokhof (eds), Formal Methods in the Study of Language. Amsterdam: Mathematisch Centrum, 277–322. Kirby, S. 2001. “Spontaneous evolution of linguistic structure: An iterated learning model of the emergence of regularity and irregularity”. IEEE Transactions on Evolutionary Computation 5:102–110. Kravchenko, A. 2007. “Essential properties of language, or, why language is not a code”. Language Sciences 29(1): 650–671. Lindblom, B., MacNeilage, P., and Studdert-Kennedy, M. 1984. “Self-organizing processes and the explanation of phonological universals”. In B. Butterworth, B. Comrie, and O. Dahl (eds), Explanations for Language Universals. New York: Mouton, 181–203. Love, N. 2004. “Cognition and the language myth”. Language Sciences 26(6): 525–544. Love, N. (2007). “Are languages digital codes?”. Language Sciences 29: 690–709. MacWhinney, B. 1999. “The emergence of language from embodiment”. In B. MacWhinney (ed), Emergence of Language. Hillsdale, NJ: Lawrence Earlbaum Associates, 213–256. MacWhinney, B. 2002. “Language Emergence”. In P. Burmeister, T. Piske, and A. Rohde, (eds), An integrated View of Language Development — Papers in Honor of Henning Wode. Trier: Wissenshaftliche Verlag, 17–42. MacWhinney, B. 2005. “The emergence of linguistic form in time”. Connection Science 17(3–4):â•›191–211. Markman, A. and Dietrich, E. 2000. “Extending the classical view of representation”. Cognitive Psychology 40: 138–171. Maynard Smith, J. and Szathmáry, E. 1999. The Origins of Life: From the Birth of Life to the Origins of Language. Oxford: Oxford Univesity Press. Murray, J.D. 1988. “How the leopard gets its spots”. Scientific American 258: 80–87. von Neumann, J. 1966. Theory of Self-Reproducing Automata. Edited and completed by A.W. Burks. Urbana, IL.: University of Illinois Press. Nisbett, R. 2003. The Geography of Thought. New York: The Free Press. Ochs, E., Schegloff, E.A., and Thompson, S. (eds). 1996. Interaction and Grammar. Cambridge: Cambridge University Press. Oudeyer, P-Y. 2006. Self-Organization in the Evolution of Speech, Studies in the Evolution of Language. Oxford: Oxford University Press. Oyama, S. 2000. The Ontogeny of Information, 2nd edition. Durham, NC: Duke University Press. Pattee, H.H. 1969. “How does a molecule become a message”? Developmental Biology Supplement 3: 1–16.



Symbols as constraints: Dynamics and self-organization in language

Pattee, H.H. 1972. “Laws and constraints, symbols and languages”. In C.H.Waddington (ed), Towards a Theoretical Biology 4, Essays. Edinburgh: Edinburgh University Press, 248–258. Pattee, H.H. 1982. “Cell psychology: An evolutionary approach to the symbol-matter problem”. Cognition and Brain Theory 5: 325–341. Pattee, H.H. 1987. “Instabilitites and information in biological self-organization”. In F.E. Yates (ed), Self-organizing Systems: The Emergence of Order. London: Plenum Press, 325–338. Pattee, H.H. 1996. “The problem of observables in models of biological organizations”. In E.L. Khalil and K.E. Boulding (eds), Evolution, Order, and Complexity. London: Routledge, 249–264. Pattee, H.H. 1997. “The physics of symbols and the evolution of semiotic controls”. Proceedings of the Workshop on Control Mechanisms for Complex Systems: Issues of Measurement and Semiotic Analysis. Redwood City, CA: Santa Fe Institute Studies in the Sciences of Complexity, Addison Wesley (Downloaded from http://www.ssie.binghamton.edu/faculty_pattee.html/semiotic.html). Pattee, H.H. 2001. “The physics of symbols: Bridging the epistemic cut”. Biosystems 60: 5–21. Pinker, S. 1994. The Language Instinct: How the Mind Creates Language. New York: Harper Collins. Port, R. 2007. “How are words stored in memory? Beyond phones and phonemes”. New Ideas in Psychology 25: 143–170. Port, R. and Leary, A. 2005. “Against formal phonology”. Language 81: 927–964. Rączaszek, J., Tuller, B., Shapiro, L.P., Case, P., and Kelso, J.A.S. 1999. “Categorization of ambiguous sentences as a function of a changing prosodic parameter: A dynamical approach”. Journal of Psycholinguistic Research 28: 367–393. Rączaszek-Leonardi, J. 2003. “The interrelation of time-scales in a description of language”. Views & Voices 1: 93–108. Rączaszek-Leonardi, J. and Kelso, S.J.A. 2008. “Reconciling symbolic and dynamic aspects of language: Toward a dynamic psycholinguistics”. New Ideas in Psychology 26: 193–207. Rączaszek-Leonardi, J., Shapiro, L., Tuller, B., and Kelso, J.A.S. 2008. “Category names in context: On-line adaptation”. Journal of Psycholinguistic Research 37: 87–113. Saffran, J., Aslin, R., and Newport, E. 1996. “Statistical learning by 8-month-old infants”. Science 274: 1926–1928. Schegloff, E.A., Ochs, E., and Thompson, S. 1996. “Introduction”. In E. Ochs, E.A. Schegloff, and S. Thompson (eds), Interaction and Grammar. Cambridge: Cambridge University Press, 1–51. Shockley, K., Baker, A.A., Richardson, M.J., and Fowler, C.A. 2007. “Articulatory constraints on interpersonal postural coordination”. Journal of Experimental Psychology 33: 201–208. Smith, K., Brighton, H., and Kirby, S. 2003. “Complex systems in language evolution: The cultural emergence of compositional structure”. Advances in Complex Systems 6: 537–558. Steels, L. 2006. “Experiments on the emergence of human communication”. Trends in Cognitive Sciences 10: 347–349. Steels, L. and Belpaeme, T. 2005. “Coordinating perceptually grounded categories through language: A case study for colour”. Behavioral and Brain Sciences 28: 469–489. Thompson, D.W. 1992 (1917). On Growth and Form. Cambridge: Cambridge University Press. Tomasello, M. 1999. Cultural Origins of Human Cognition. Cambridge, MA: Harvard University Press. Tomasello, M. 2000. “The social-pragmatic theory of word learning”. Pragmatics 10: 401–14.

183

184

Joanna Rączaszek-Leonardi

Tomasello, M. and Akhtar, N. 1995. “Two-year-olds use pragmatic cues to differentiate reference to objects and actions”. Cognitive Development 10: 201–224. Trevarthen, C. 1979. “Communication and cooperation in early infancy: A description of primary intersubjectivity”. In M. Bulowa (ed), Before Speech: The Beginning of Human Communication. Cambridge: Cambridge University Press, 321–347. Trevarthen, C. 1998. “The concept and foundations of infant intersubjectivity”. In S. Braten (ed), Intersubjectivity Communication and Emotion in Early Ontogeny. Cambridge: Cambridge University Press, 15–46. Trevarthen, C. and Aitken, K.J. 1994. “Brain development, infant communication and empathy disorders: Intrinsic factors in child mental health”. Development and Psychopathology 6: 599–635. Tuller, B., Case, P., Kelso, J.A.S., and Ding, M. 1994. “The nonlinear dynamics of categorical perception”. Journal of Experimental Psychology: Human Perception and Performance 20: 3–16. Turing, A.M. 1952. “The chemical basis of morphogenesis”. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences 237: 37–72. Umerez, J. 2001. “Howard Pattee’s theoretical biology: A radical epistemological stance to approach life, evolution and complexity”. Biosystems 60: 159–177.

Beyond mind An extended ecology of languaging* Sune Vork Steffensen

University of Southern Denmark, Denmark

This chapter contrasts a distributed perspective on language with how Andy Clark applies the Extended Mind Hypothesis to language in Supersizing the Mind (2008). While sympathetic to Clark’s broad picture, his understanding of language is shown to be inadequate. The Chapter outlines five main reasons for rejecting his conservative view: 1) Language cannot be explained with reference to a ‘language system’, i.e. ‘a semiotic system for communication’. 2) Language is irreducibly bound up with real-time metabolic activity (e.g. voice dynamics, gaze and task-oriented modes of action). 3) Language gains its cognitive power from, not how individual cognizers handle it, but from how they adapt to socially moulded ecological niches. 4) Language functions, metaphorically, as airborne synapses in distributed cognitive systems. 5) Language provides an extended ecology within which human cognizers engage in languaging. The chapter concludes that how language contributes to human activities cannot be squared with Clark’s surprising view that the extended human mind is organism-centred. Keywords: extended mind hypothesis; language; distributed cognition; interaction; extended ecology

Introduction The purpose of this chapter is to contrast how two post-Cartesian approaches to cognition differ when it comes to language. The two approaches are the extended mind hypothesis (Clark and Chalmers, 1998; Clark, 2008) and distributed cognition (Hutchins 1995; Hollan et al., 2000, Kirsh, 2010). Whereas the former has its disciplinary home in the philosophy of mind (though drawing heavily on experimental cognitive science), the latter arose from cognitive anthropology. But they

186

Sune Vork Steffensen

share a central tenet, namely that human cognition is non-local (Steffensen and Cowley, 2010), i.e. it draws on brains, bodies and surroundings, including other cognizers, artefacts, social relations and environmental structures. In their own words, this is how Clark and Hutchins, respectively, present their views on cognition as both internal and external: The actual local operations that realize certain forms of human cognizing include inextricable tangles of feedback, feed-forward, and feed-around loops: loops that promiscuously criss-cross the boundaries of brain, body and world. The local mechanisms of mind, if this is correct, are not at all in the head. Cognition leaks out into body and world. (Clark, 2008:xxviii) Human beings are adaptive systems continually producing and exploiting a rich world of cultural structure. […] This heavy interaction of internal and external structure suggests that the boundary between inside and outside, or between individual and context, should be softened. [...] Thinking consists of bringing these structures [“of artifacts and social organizations”] into coordination so that they can shape and be shaped by one another. The thinker in this world is a very special medium that can provide coordination among many structured media—some internal, some external, some embodied in artifacts, some in ideas, some in social relationships. (Hutchins, 1995:288, 316)

These accounts emerged in the 1980s and 1990s as a revolt against the Cartesian regime in cognitive science. Thus, Western cognitive science has for four centuries conceived of human cognition in terms of the metaphor mind as machine, as meticulously fleshed out in Margareth Boden’s opus magnus “Mind as machine” (Boden, 2006). By viewing mind as machines, one compares human cognition with the computational use of effective procedures: mind becomes neural machinery that, given appropriate input, produces relevant output. In the second half of the 20th century, cognitive science thus flourished with computational input-output models and a representational theory of mind “according to which cognitive states and processes are constituted by the occurrence, transformation and storage (in the mind/brain) of information-bearing structures (representations) of one kind or another” (Pitt, 2008). In linguistics, Chomsky (e.g. 1965) used the model to argue that language should be pictured in terms of grammars in mind/brains that ran alongside systems of speech production and processing. In claiming explanatory power for classic cognitive theories, performance was subordinated to competence (Chomsky, 1965). What people do with language, their performance, was kept theoretically separate from what they know about language, their competence. The change in the field, away from Cartesian models of the symbol-processing mind towards an extended or distributed view, logically unfolds as two arguments.



Beyond mind: An extended ecology of languaging

The first is the argument that, whatever the role of symbol systems in human life, our daily living in a lived environment depends, first of all, on metabolism: we are metabolic first, symbolic second. According to this biogenic premise, cognition is Embodied (and not merely procedural), Embedded (and not merely representational), and Enacted (and neither subjective nor objective).1 The second is a logical continuation of the first argument (though it emerges simultaneously in the field). Taking one step further away from the Cartesian mind, this argument denies any sharp distinction between inner and outer. In this view, cognition is not confined to the inner realms of organisms, but makes active use of external artefacts. Building on this argument, we can add that cognition is not merely Embodied, Embedded and Enacted; it is also Extended, Distributed and Situated (cf. Robbins and Aydede, 2009). These claims are built on a wealth of empirical findings, both experimental and ethnographic. Hitherto, as pointed out by Kiverstein and Clark (2009), there has been a tendency to rubric all of these post-Cartesian lineages together as “one church” of Embodied, Embedded, Enacted, Extended, Distributed and Situated cognition. The present discussion is written in the same spirit as their paper: I too value the discussion of the differences rather than the celebration of the similarities, and I agree that such a discussion marks out “an important moment in the evolution and maturing of the study of the embodied, embedded mind. It marks the moment of critical self-awareness and questioning distinctive of an emerging paradigm” (Kiverstein and Clark, 2009:6). As mentioned, the current chapter focuses on how language is, or can be, treated according to the extended mind hypothesis and the distributed cognition approach. The main tenet of the discussion is a critique of Clark’s view on language; whilst I find his general views on mind and cognition groundbreaking, I find his treatment of language quite inadequate. Some of his weaknesses are shared by Hutchins who also holds a representationalist view on language. Thus my errand is not to oppose cognitive anthropology with philosophy of mind. Rather, I make the claim that, if we adopt these tenets of cognitive science (cf. Steffensen, in press), we find that they necessitate a new approach to the dynamics of linguistic cognition. This claim is defended by presenting five claims that foreground relevant and critical differences. These will be elaborated one by one in the following five sections in this order: 1. Language cannot be explained with reference to a ‘language system’, i.e. ‘a semiotic system for communication’. 2. Language is irreducibly bound up with real-time metabolic activity (e.g. voice dynamics, gaze and task-oriented modes of action).

187

188

Sune Vork Steffensen

3. Language gains its cognitive power from, not how individual cognizers handle it, but from how they adapt to socially moulded ecological niches. 4. Language functions, metaphorically, as airborne synapses in distributed cognitive systems. 5. Language provides an extended ecology within which human cognizers engage in languaging. 1. Language beyond the language system Claim 1: Language cannot be explained with reference to a ‘language system’, i.e. ‘a semiotic system for communication’. Andy Clark presents his view on language in Chapter 3, Material Symbols in his recent book Supersizing the mind (Clark, 2008:44-60).2 Clark’s starting point is the pivotal question: “Where does language fit into our emerging picture of the plastic, environmentally exploitative, ecologically efficient agent?” (Clark, 2008:44). While the question is highly relevant, Clark’s strategy for answering may surprise the reader: One useful way to approach this question is to consider language itself as a form of mind-transforming cognitive scaffolding: a persisting, though never stationary, symbolic edifice whose critical role in promoting thought and reason remains surprisingly ill understood (44).

The devil lies in appeal to “language itself ” which, for a linguist, echoes Ferdinand de Saussure’s structuralist notion of “la langue elle-même” (and “la linguistique elle-même”).3 Language is, in his own words, an edifice that scaffolds cognition, i.e. the importance of language depends on how it appears to the individual cognizer. In so saying, Clark places his work in Western grammatical tradition by adopting a naïve realist view of language. Clark too chooses to ignore the embodied dynamics that gives life to the phenomena under scrutiny (language or languaging). Reduced to a structural entity, language is placed on a par with the famous artifacts of the extended mind literature, e.g. Otto’s notebook (cf. Clark and Chalmers, 1998), Sheba’s plastic tags (cf. Clark, 2008:45) and Tetris players’ computers (cf. Kirsh and Maglio, 1994). Language, in Clark’s view, is best understood in the relation between cognizer and “language itself ”. Indeed, it is because of this conservative view that his work shows a biased preference for non-dialogical and non-conversational examples, such as labelling, recalling and reasoning:



Beyond mind: An extended ecology of languaging

Simple labeling thus functions as a kind of augmented reality trick by means of which we cheaply and open-endedly project new groupings and structures onto a perceived scene. (Clark, 2008:46) linguaform rehearsal enables experts to temporarily alter their own focus of attention, thus fine-tuning the patterns of inputs that are to be processed by fast, fluent, highly trained subpersonal resources. (Clark, 2008:48) Linguaform reason […] emerges as a key cognitive tool by means of which we are able to objectify, reflect upon, and hence knowingly engage with our own thoughts, trains of reasoning, and cognitive and personal characters. (Clark, 2008:59)

Even when several agents are involved, Clark’s examples retain a single-minded logic. We never meet human beings engaged in linguistic interaction or, for that matter, communicative troubles. Rather, we meet chimpanzees that have received numerical training, experts who perform linguaform rehearsal, prelinguistic infants who navigate in an artfully designed room, and Russian-English bilinguals doing maths in both languages. All examples deal with how one cognizer uses language, and all examples build on experimental work on individual language users. While enlightening, the examples mask how language serves us in the ecological settings of human interaction. Thus, from a linguistic point of view, Clark’s view of language is conservative and strangely ‘un-extended’. This can be clarified by contrasting Clark’s extended view to the distributed position. Both concur that, in Clark’s words, “the material structures of language play a cognitive role that in some way actually depends on, and exploits, that very materiality” (ibid). One crucial difference can be traced to a seminal paper by Nigel Love (2004), in which the distinction between the classical view of mind and the distributed view of mind is shown to have a parallel in linguistics. This arises from comparing the classical view of language with the integrationist view of language (cf. Love 2004:525f.; Love, 2007; Harris, 1981; see also Clark’s (2004) response to Love) by drawing on Roy Harris’ critique of the so-called “language myth.” Whereas Clark avoids linguistic theory, Harris challenges the orthodox view that determinate language systems (consisting in determinate forms) are used for problem-free communication of determinate messages (or ‘telementation’).4 While Clark also rejects telementation, he sticks with the determinate forms. The distributed and extended view thus differ on how language is stabilized and, hence, how it informs human cognition. In linguistic theorizing, especially in Saussurean structuralism, stability emerges when different expressions invoke the same linguistic form. Thus, you may say [tәʹmejtou] and I may say [tәʹma:tou], but on the content plane we have the same red nightshade vegetable in mind. In the terms of Hjelmslev, a difference on the “expression plane” is said to make no difference on the “content plane”

189

190

Sune Vork Steffensen

(1943:44-55). Stability is said to be a feature of the content plane that is warranted by a “translation” of the differing expressions into the same (Mentalese or Neuralese) content. Opposing this, both distributed (e.g. in Love, op.cit.) and extended (Clark, 2006, 2008) approaches reject any “translation theory of language” (Clark, 2006:304). However, whereas Clark attributes a stable materiality to language (in itself?), those in the distributed movement trace this to human phenomenology sustained by social institutions (alphabets, dictionaries/grammars, religious and scientific traditions, schools etc.). First, it is simply a fact that main modalities of language – speaking and writing – share no single (abstract) materiality (cf. Kravchenko, 2007, this vol.). Second, the materiality of language is not either the alphabet or a phonological equivalent: it is highly variable. In Love’s words: The linguistic sign, whether spoken, written or manifest in any other medium, is not an object, or a permanent property of an object. It has no fixed or determinate semiotic value. It becomes a sign as and when used as such, and its signification is a function of that use. [...] When it comes to treating a language as a consistently identifiable set of recurrent ‘sames’, the problem is that there are indefinitely many dimensions in which one thing may be deemed the same as, or different from, another. (Love, 2004:531, 541)

Clark fails to understand that forms are as indeterminate as meanings. To retain his notion of material stability, Clark invokes an alleged capacity to recognize the same words: Of course, there is plenty of variability here [i.e. in “the actual sound streams and inscriptions”] too, but we are usually quite well able to recognize, for example, the same word in different fonts and as uttered by different speakers, and that is all the stability I need. (Clark 2004:723)

Clark’s argument conflates relational stability with objective stability. While we certainly perceive what we call ‘the same words’, this cannot ground a scientific explanation of how we speak or act. There is a parallel with human colour vision (Giere, 2006): we can normally recognize a colour as ‘the same’, even under different circumstances of lightness and darkness. This is neither because objectively we see the “same” electromagnetic waves, nor because subjectively we have the ‘same’ experience. Rather visual systems adapt to various degrees of ambient light and, by so doing, stabilize a relation between organism and environment. In this instance, one cannot deduct from the fact that we behave as though two varieties are “the same colour” to the conclusion that these are the same, materially or by other means. Nor, of course, can one conclude that brains (or minds) represent discrete colour categories that correspond to what we see.



Beyond mind: An extended ecology of languaging

2. From language to languaging Claim 2: Language is irreducibly bound up with real-time metabolic activity (e.g. voice dynamics, gaze and task-oriented modes of action). Clark’s assumption that language has a “material stability” is contradicted by experimental phonetics (cf. Port and Leary 2005; Port 2010). Linguistic vocalizations are so diverse that we must question any theory that attributes “material stability” to language. By way of illustration, Port (2010) invites us to compare a single speaker’s pronunciation of the syllables [di] and [du] (as in Dee and Dew).5 He suggests that, as literates, we believe that these consist of the same consonantal onset and two different vowels. In fact, as Port demonstrates, They sound to us like they have the same initial “sound,” but do not have anything obvious that is a physical invariant. This situation is not rare, but is actually the norm wherever you look in speech. (Port, 2010:49)

Port continues: The conclusion I have drawn from attempts over the last half century to find physical correlates of units like phones and phonemes is that they obviously do not have invariant physical correlates. Languages work anyway despite the fact that speakers have idiosyncratic auditory systems as well as idiosyncratic speech articulation. (Port, 2010:43f.)

Clark falls victim of what Per Linell terms the written language bias in linguistics (Linell 2005), i.e., the view that language structurally – phonologically, morphologically, and syntactically – behaves like an alphabet where smaller buildingblocks are combined to constitute larger blocks. Since, phonetically, this is false, it is unwarranted to rely on ‘internal’ linguistic units. In Port’s terms, “linguistic memory does not extract an abstract, context-free, non-overlapping invariant for each consonant and vowel” (Port, 2010:48). In a distributed view, this line of argument leads to the insight that the stability that we experience is not based on an individual’s (representational) skills, but on our ability to make and track what Fowler terms “phonetic gestures” (cf. Fowler and Rosenblum, 1991; Thibault, 2011). This ability emerges through “education of attention” (Gibson, 1966): in interactions with caregivers, infants attend to specific patterns of activities, and in doing so they learn to take a language stance (Cowley, 2007b; 2011). Indeed, without stance-taking, people would be unable to control speech by choosing what to say, just as they would lack a phenomenological experience of apparent linguistic stability. What we perceive as materially stable in language is deeply influenced by sociocultural norms (cf. Linell, 2009; Cowley and Zheng, 2011).

191

192

Sune Vork Steffensen

Whereas Clark’s extended mind hypothesis pictures language as scaffolding that is used by an individual cognizer, the distributed view treats language as part of activity. In doing so, the distributed program adopts the integrationist insight that a language is a second-order cultural construct, perpetually open-ended and incomplete, arising out of the first-order activity of making and interpreting linguistic signs, which in turn is a real-time, contextually determined process of investing behaviour or the products of behaviour (vocal, gestural or other) with semiotic significance. (Love 2004:530)

This view brings another crucial difference between the extended and the distributed approach to the fore. Thus, Paul Thibault takes the integrationist distinction between first and second order further than Love. Instead of restricting first-order activity to “making and interpreting linguistic signs,” Thibault emphasises distributed activity: there is no way of distinguishing ‘linguistic’ human activities from their ‘non-linguistic’ counterparts: First-order languaging is whole-body sense-making activity that enables persons to engage with each other in forms of co-action and to integrate themselves with and to take part in social activities that may be performed either solo or together with other agents. (Thibault, 2011: 215)

Ironically, in his discussion of gesture Clark shows how this can be done. Thus, when it comes to gesture, Clark observes that “we do it when talking on the phone. We do it when talking to ourselves. We do it in the dark when nobody can see” (123). Notice the agent: We do it – gesture is done, it is activity. The point in Clark’s treatment of gesture is, in McNeill’s succinct formulation, that “the actual motion of the gesture itself is a dimension of thinking” (McNeill 2005: 98; quoted in Clark 127). So it is not the case that certain pre-defined gestural forms, gaze directions, and body postures enact thinking; it is the process of doing the gesture, the actual motion, that is thinking. Echoing Alač and Hutchins (2004), Clark (128) treats “action as cognition”. What if Clark had adopted the same approach to language? What if he viewed language as “the actual motion” (with McNeill 2005) and regarded “language as cognition” (with Alač and Hutchins 2004)? Had he done so, he would conclude that language is activity performed as human beings do things together. Language is like gesture, i.e. gesturing, in that there is a real-time aspect to language: language is action. In fact, language is a form of gesture, albeit “phonetic gestures” (cf. Fowler and Rosenblum, 1991). In not reducing language to material scaffolding, those in the distributed language movement focus on the cognitive dynamics of languaging (cf. Cowley 2007a, 2010; Steffensen et al., 2010; Thibault, 2008, 2011). Human beings engage in “whole-bodied communication” which, in many circumstances, has more to do



Beyond mind: An extended ecology of languaging

with the bodily (or body-worldly) co-actions by interlocutors than with what they (think they) say. On this view language is a heterogeneous set of physical, cognitive and social activities that unfold in real-time on many time-scales. It arises as we adjust to each other and co-ordinate our life-worlds, behaviourally and cognitively. Examples of this are Cowley’s (1994; 1998) demonstration of how voice dynamics enact interpersonal relationships, and Steffensen et al.’s (2010) demonstration of how a senior doctor, playing the role of a nurse in a simulation, directs the attention of a junior doctor using only subtle inter-bodily dynamics. How people interact depends more on inter-bodily coupling than on rules, turn-taking or the ‘content’ of what is actually said. For instance, Cowley demonstrates how, by extraordinarily sophisticated timing, interlocutors orchestrate speech while relying on anticipating, responding, and resonating with each other (Cowley, 2010). By contrast, whether Saussurian or post-Saussurian, form-oriented linguistics depends on naïve realism. It treats a second order phenomenological pattern as-if it was something in itself. Indeed, by so doing, the complexities of first-order languaging (embodied expression) become ‘non-linguistic’. Ironically, Clark relies on the same folk views. While professing scepticism about phenomenology, he relies on the literate’s phenomenological experience of handling language.6 However, merely saying that language is action is insufficient. Such claims date from the times of Wilhelm von Humboldt who, in Vološinov’s (1973) succinct recapitulation, held the view that Language is activity, an unceasing process of creation (energeia) realized in individual speech acts; […] Language as a ready-made product (ergon), as a stable system (lexicon, grammar, phonetics), is, so to speak, the inert crust, the hardened lava of language creativity, of which linguistics makes an abstract construct in the interests of the practical teaching of language as a ready-made instrument. (Vološinov, 1973:48)

While Humboldt and the distributed approach concur that second-order language (Humboldt’s ergon) is a frozen depiction of first-order languaging (Humboldt’s energeia), they differ on two main points: first, whereas Humboldt seems to assume a closed circuit between energeia and individual speech act, the distributed language movement proposes a meshwork view (Steffensen et al., 2010; Thibault, 2011). From this perspective no specifically linguistic creativity is needed to engender activities such as speaking or writing. Language is not a monolithic ‘system’, but rather a plenitude of pebbles that we count on in human existence.7 Second, to Humboldt, “The laws of language creativity are the laws of individual psychology” (Vološinov, 1973:48), while the distributed approach takes a naturalized sociocultural approach. Appeal to a distributed and naturalized meshwork thus chal-

193

194

Sune Vork Steffensen

lenges Clark’s view of language. Exactly how will be shown in the next sections which explore the socio-cultural anchoring of language. 3. The social dimension of languaging as human niche construction CLAIM 3: Language gains its cognitive power from, not how individual cognizers handle it, but from how they adapt to socially moulded ecological niches. Niche construction is “the process by which animals build physical structures that transform problem spaces in ways that aid (or sometimes impede) thinking and reasoning about some target domain or domains” (Clark, 2008:62). Classical examples are the structures made by beavers, bees, and termites. But human beings also exploit their environment in this way, and Clark features some entertaining examples, especially Tribble’s (2005) study of how the scenic structures of the Elizabethan theatre assisted the actors in remembering plots and lines (see also Tribble, this vol.). Such artefacts and actions that aim at facilitating thinking are called epistemic artifacts and epistemic actions (cf. Kirsh and Maglio 1994; Kirsh 1995), respectively, and they point to the looping interactivity between brain, body, and world, showing that “the inner-outer boundary is both analytically unhelpful and computationally far less significant than one might have pre-theoretically supposed” (Clark, 2008:73). Interestingly, Clark investigates such structures in Chapter 4, World, Incorporated (61-82), i.e. after having presented language in Chapter 3 (Material Symbols). Had Clark reversed the order of these chapters, he might have asked whether language enriches the niche as do material structures and epistemic artefacts? Rather than a priori assuming that language scaffolds cognition, we can ask how language (strictly, stabilized patterns of languaging in social contexts) influences the human ecological niche. This question is the topic of the current section. Just as beavers “collectively build structures [i.e. dams] that persist beyond their own lifetime” (61), phenomenologically stabilized patterns of languaging persist beyond an individual lifetime. What doesn’t work communicatively is selected out through the evolutionary history of languaging, not blindly, but as a result of behaviour that emerges from doing things with other people. Therefore, far from consisting in material symbols vis-à-vis the individual cognizer, linguistic phenomena shape, and are shaped by, a much larger community of individuals moulded in and by human sociality. However, while a beaver dam is material, human languaging draws on both material props and phenomenological experience: not only are we moved by the movements of the other but, after early infancy, we come to hear utterances as wordings or, in Cowley’s terms, learn to take a



Beyond mind: An extended ecology of languaging

language stance (Cowley 2007; 2011). We are able to master the bodily-worldly skills of languaging because we link our metabolism with symbols (see RączaszekLeonardi, this volume). Over time, we come to hear (external) words-in-languaging as meanings. Gradually, as this becomes familiar, language becomes a part of who we are as living beings. For this reason, the ‘individual cognizer’ is a contradictio in adiecto when it comes to linguistic cognition. When we observe how human beings draw on linguistic resources, it is simply false, that “Individual cognizing, then, is organism centered “even if it is not organism bound”â•›” (Clark, 2008:123). Clark adopts this individualistic bias only because he makes “the (surely uncontroversial) assumption that the biological brain is, currently at least, the essential core element in all episodes of individual human cognitive activity” (Clark, 2008:118; emphasis added). On the distributed view, language is deeply collective. By linking dynamics with the affordances of the language stance we become biological individuals who act and think while drawing on a plenitude of other persons’ voices, values and verdicts in (polyphonic) reasoning. When Clark argues that “linguaform reason” allows us to “engage with our own thoughts, trains of reasoning, and cognitive and personal characters” (59; emphasis added), he overlooks the deeply social character of language. He underplays our skills in making use of other peoples’ thoughts and fails to see that, given linguaform reasoning, our thoughts cease to be strictly our own. Interactions exist between the first person and the second person in the medium of the third person (cf. Bang and Døør 2007): There is always an anonymous third party present when we use language. The anonymous third expresses the cultural and social order that has pre-organized the language to a certain degree. This means that the child learning a language is forced to consider the anonymous third. Often we do not reflect on these matters, because it is so tempting to believe that our inner speech is a conversation with ourselves and no-one else. We are tempted to believe that we are engaged in a ‘free’ conversation (Døør 1998: 40-41; cited in Steffensen 2007: 24).

Human beings rely on cognitive implications that they can draw from the social world. While language and the social order reshape our subjectivity, they also lessen our individuality. Using the anonymous third, we learn to monitor and adjust our behaviour and thoughts as we see fit; however, we pay the price of being moulded by the norms and values of social reality. Wittgenstein is halfway correct in stating that “A concept forces itself on us” (Wittgenstein, 1958:204; quoted after Cowley, this vol.); the other half of the story is that concepts (as well as grammatical constructions, etc.) act as proxies for the full suite of human agents and their lived environments (our forms of life). Furthermore, though we enact the words voices and values of our forefathers, if prompted (or self-prompted) to do so, we can also challenge these.

195

196

Sune Vork Steffensen

Once we take the social or socio-logical (Bang and Døør 2007: 51-55) dimension of languaging seriously, we are bound to consider how the extended mind is embedded in social reality. Drawing on the instructive parable of Otto, who finds his way to a museum by using, not his biological memory, but a notebook as part of his extended cognition (cf. Clark, 2008:226-232), we raise new questions: Where did Otto get his notebook from? Who taught him to use it? Who put the museum on the map, and why? Many persons are integrated in, and integral to, the dynamical processes that enable Otto to find his way around.8 Such issues lack serious treatment in Clark’s exposition because language is reduced to an abstract relation between a cognizer and a symbolic edifice. For the same reason, Clark tends to ignore the social nature of human life, forgetting that languaging is anchored in socio-cognitive modes of acting. The difference between extended individualism and socially distributed cognition can be clarified by reconsidering Clark’s examples. Relying on experimental work by Boysen et al. (1996), Clark (2008:45-47) introduces Sheba, one of five numerically trained chimpanzees: In an experiment, the chimps can choose from one of two plates that are piled with candies, which “were highly preferred items, and the animals were highly motivated to procure them” (Boysen et al. 1996:77). One of the plates had a large pile of candies, the other only a small pile. The setup was that the choosing chimp receives the other plate, while another chimp receives the one chosen. So in order to procure the preferred large pile, the chimp had to choose the small pile. The results of the experiment are straightforward. Presented with two piles of candy, the chimps cannot learn to choose the small pile. They consistently choose the large one even when this means that they fail to get the big reward. However, in an ensuing set-up the two piles are exchanged with cards with numerals. In this setup, the numerically trained chimps can choose the small number and thus receive the large pile corresponding to the un-chosen card. Clark concludes that “the material symbol impacts behaviour […] providing a new target for selective attention and a new fulcrum for the control of action” (Clark, 2008:45). The facts are clear: Language-trained Sheba reliably gets more candy when pointing to the small Arabic numeral than she does when pointing to the large pile. However, it is a non sequitur to claim that this tells us about numerals – let alone language! Thus, if we compare the interactional dynamics across the two experiments, we see different patterns. The first setup features the unfolding of a chimp-candy relation where the chimps point to what they prefer: confronted with such yummy treats, they are unable to sidestep their instinctual drive. The second set of relations is more complex. While presented as a chimp-numerals situation, this socially complex experimenter-chimp interaction is rather ‘mediated’ by a numeral. Even when the human is not visible, the numeral marks a history



Beyond mind: An extended ecology of languaging

of human-chimp coupling. It indexes a meticulous process of training a chimp to do things with numbers during shared activity in a distinct ecological niche. This history is re-activated by the functionality of the numerals. it is precipitous to argue that Sheba’s performance is altered by “language” (symbols or numerals) per se. It is more likely that this depends on a history of the chimps’ experience of coupling with trainers. So, the candy problem is solved, not by Sheba alone, but cooperatively by Sheba and the human experimenter, drawing on a shared history of training. Though both parties depend on second-order constructs (cards with numerals), their cooperation is not warranted by the numerals (or by knowledge of numbers). It depends on an ecology that has been fostered by a history of coupling and co-ordinating while using artefacts that include cards with numerals. Clark claims that labelling allows Sheba to “project new groupings and structures onto a perceived scene” (Clark, 2008:46). The claim leads one to wonder if Sheba could pick the “right” plate when presented simultaneously with candy piles and the numerals. If that were so, it would show that brief training with numerals was able to short-circuit the chimps’ phylogenetic history and the evolved preference for large piles of candy. My conjecture, however, is that Sheba would still point to the large pile. After all, overcoming instinctual drives is hard work! Sheba’s dependence on a history of complex social coupling in a distinct language-laden ecology resembles that of another protagonist in another of Clark’s examples, the American sociologist and jazz pianist, David Sudnow (cf. Clark, 2008: 237; Sutton, 2007; Sudnow 2001). As a piano novice he felt it very frustrating when his teacher told him to “go for the jazz” or to play with “jazz hands”. But having learnt to master the instrument these instructions became “very detailed practical talk, a shorthand compendium of ‘caretaking practices’ for toning and reshaping the grooved routines” (Clark, 2008:237). Such “instructional nudges” (Sutton, 2007:773) are expressions that are grounded in a community of jazz musicians. In other words, just as the piano novice uses a musical community (incarnated in the teacher’s instructional terminology) to solve the problem of playing jazz, so Sheba relies on the human experimenter in order to indicate the small pile. Having demonstrated a distributed reinterpretation of the cases of Sheba and Sudnow, I conclude with Clark’s thought experiment, featuring a hypothetical snakelike creature, Adder, that lives on an advanced touch-screen environment. In this flat-screen setting, every little wriggle of the snake can cause specific external symbolic tokens to appear elsewhere on the screen – tokens that are themselves apt for perceptual uptake (perhaps via a kind of Braille) (Clark, 2008:203).

On the basis of this thought experiment, Clark argues that cognition is not embodied in the strong sense that thought and reason are inextricably and nontrivially tied to the details of the human body (Clark, 2008:204). We can easily imagine that,

197

198

Sune Vork Steffensen

by wriggling on its touch-screen, Adder carries out “the same complex accounting as the standard, pen-and-paper accountant Ada” (ibid.). The Adder thought experiment illustrates Clark’s principle of Distributed Functional Decomposition (Clark, 2008:203). In his terms, cognitive processes use “the full suite of encodings and operations made available by some combination of neural, gross bodily, and worldly opportunities” (ibid.). In others, cognition depends on the entire ecological niche. However, if we accept his conclusion (which I certainly do), it is contradictory to claim that Sheba solves the cognitive problem by drawing on one tiny part of her ecological niche. Like Adder and Sudnow, Sheba’s cognitive powers use “the full suite” of brain, body, and environment. 4. Ecologically extended cognition CLAIM 4: Language functions, metaphorically, as airborne synapses in distributed cognitive systems. Our kinds of minds depend on social and phenomenal aspects of language. They give us an ecology in which we become parts of each other’s environment and, thus, each other’s extended brain-body-world systems. Therefore, as mind extends via languaging, an individual’s problem solving activities become linked with those of other agents. However, in our everyday life, many problems are not purely individual but, rather, appear as common problems that require collective solutions. This is less clear in experimental work, partly because of the methodological individualism (Udehn, 2002) of traditional psychology. The fourth claim is that once languaging comes to the fore, new importance falls to events within a network of communicating agents. Languaging is an emergent, collective phenomenon: an agent’s mind is extended by encounters with other agents in a reciprocal flow of influence. Metaphorically, languaging sets off airborne synapses that link a community of individuals. Its function is to connect whole-bodied dynamics with verbal patterns (based on taking a language stance). In other words, human agency is partly collective. We draw on shared resources (e.g. numbers and verbal patterns) that lie ‘outside’ an individual’s borders – and indeed beyond the mind. The distributed view challenges Clark’s Hypothesis of Organism-Centered Cognition: Human cognitive processing (sometimes) literally extends into the environment surrounding the organism. But the organism (and within the organism, the brain/ CNS) remains the core and currently the most active element. Cognition is organism centered, even when it is not organism bound (Clark, 2008:139).



Beyond mind: An extended ecology of languaging

For Clark, the brain extends cognition into the world: “it is indeed the biological brain (or perhaps some of its subsystems) that is in the driver’s seat” (Clark, 2008:122). In order to investigate such claims, we can ask: is the capability of extending cognition itself a cognitive skill? If the answer is affirmative, then this cognitive skill can be extended by a naked brain. Thus, in Clark’s extended program, some crucial cognitive processes are, it seems, inherently brainbound. This is rather surprising, as it actually runs counter to the substrate-neutral functionalism that allows the mind to extend beyond the brain and body. When some of Clark’s critics (e.g., Adams and Aizawa, cf. Clark 2008:85-99) accuse him of advocating the view that, because it is part of a cognitive system (the extended mind), a notebook is intrinsically cognitive, they fail to understand the functional principle. This is because, if taken seriously, what is true for the whole is not necessarily true for each of its parts: even if my car is capable of auto-mobility, this does not imply that the transmission or the fuel tank can move by itself. The functionalist principle prompts us, whenever we meet a phenomenon that exhibits signs of cognition, to investigate if this is because the phenomenon is intrinsically cognitive, or if it is part of a larger cognitive system. Thus, Clark’s functionalism bars us from the a priori claim that because we experience ourselves as cognitive, the individual cognizer always is the centre of (extended) cognition. We need to consider the possibility that human beings constitute a single cultural cognitive network (Donald, 2001) that achieves a high degree of co-ordination through languaging. Whereas the term extended implies a point of origin, the term distributed carries no such connotations. Indeed, from a distributed perspective, there is a simpler and more radical way to understand the social dimensions of the extended mind. This can be done by dismissing the hypothesis of organism-centered cognition altogether. Rather, if cognition is extended into a social domain, the extending may be controlled by many brains and bodies. Human cognition is often not organism-centered, which of course does not imply that cognition take places independent of organisms. In a similar vein, Timo Järvilehto argues that “mental activity cannot be localized in any part of the organism; it is not an activity of the brain, for example, although it may not exist without the brain” (Järvilehto, 1998:330). This difference between the two positions arises because Clark focuses on epistemic artefacts, which are, of course, less active than organisms. However, in stating his Hypothesis of Organism-Centered Cognition, Clark ignores ontological differences between epistemic artifacts and human co-cognizers. According to the Hypothesis of Organism-Centered Cognition, if other cognizers become part of an individual’s extended mind, they are on a par with epistemic artefacts like notebooks and sextants. This means that Clark’s hypothesis supposes that, in

199

200

Sune Vork Steffensen

the cognizer’s environment, artefacts and cognizers function analogously. In other words, Clark’s hypothesis unashamedly instrumentalizes other human beings. The alternative is to start from the simple fact that, outside the laboratory, we do things with other people who do things with us. As a result, we set up intricate processes of interaction and, very often, these mask the “centre of cognition” – if any such centre exists. Both mass psychology and political history provide many examples where patterns of interaction (including patterns of languaging) make agents perform odd, horrible or glorious actions. For instance, it is hard to tell who does what when the British mentalist and entertainer Derren Brown demonstrates his stunning abilities to influence random passers-by. Take the Russian scam clip as an example.9 Here Brown asks a passer-by for directions, and through various subtle bodily actions he leads his “victim” into a disoriented condition. Having done so, he asks the man to pass him his watch, house keys and wallet. Bizarrely, the man gives Brown all three things, ‘voluntarily’. Only when several moments have passed, does the victim realize that he has been scammed. What is the centre of the cognitive process where the passer-by “decides” to give his house keys to a stranger who just asked for directions? His brain? Brown’s brain? Their interaction? In Clark’s mono-directional approach, the extended mind, “although not ‘brainbound,’ remains neurocentric. The brain becomes a lazy genius at the centre of a “loose confederacy of tricks and tweaks” (Lloyd, 2009: 340). Far from portraying it as part of a balanced ecosystem, Clark tears the brain from its setting, and makes it the overlord of the ecology.10 Clark’s neurocentrism is also visible in his metaphors: “It is the brain’s great plasticity and thirst for cheap, outsourced labor that drives the distributed engines of sociotechnological adaptation and change” (Clark, 2008:162). At best, the outsourcing metaphor is ill-chosen. At worst, Clark’s description of the extended mind resembles socio-political neo-colonialism. Thus, the development which is euphemistically known as “globalization” is only global in the sense that its tentacles reach every spot on earth. In the same vein, Clark holds a neo-colonial view on the brain and mind, where individual cognition colonizes the environment, human or non-human. 5. Ecologically extended cognition CLAIM 5: Language provides an extended ecology within which human cognizers engage in languaging. To avoid a colonial view on human cognition, one can ask how language-laden human cognition equips us with an extended ecology. In so doing, we move well



Beyond mind: An extended ecology of languaging

beyond Clark’s mono-directional extension of the mind. For this reason, in presenting the fifth claim, less attention is given to the extended mind hypothesis. Indeed, in seeking a fuller understanding of how language shapes human ecology, invocation of material symbols has little to offer. Rather, the idea of an extended ecology combines Clark’s extended mind hypothesis with Richard Dawkins’ theory of extended phenotypes. As worked out in an illuminating paper by Dennis Waters (in press), the extended phenotype can be used to develop the distributed perspective on language. Classic examples of extended phenotypes are spiders’ webs and beavers’ dams. A web is “a temporary functional extension of her body, a huge extension of the effective catchment area of her predatory organs” (Dawkins, 1982:192; quoted in Waters, in press). Unlike the spider’s web, the beavers’ dam (which is also mentioned by Clark, 2008:61f.) is not “a direct metabolic product” (Waters, in press). Further, again unlike the web, the dam “outlast[s] the beavers that built it and continue[s] to function for succeeding generations and populations” (ibid.). The examples give us an apt metaphor for understanding how language functions in an extended human ecology. Language vastly enhances the individual human phenotype’s capacities for perception and action: like the spider, we can rely on second-hand perception (cf. Gibson, 1966:26) and action at a distance. If I hear a warning call (“fire!”), I can react immediately even if my phenotypical senses gives me no indications of any immediate danger. Likewise, had I sufficient equanimity I might be able to act at a distance, yelling to someone: “call the fire brigade!” Like the spider, we depend on exteriorising, though the exteriorised air is not metabolically produced (but metabolically modulated). This means that we do not depend on our tactile pick-up of information, which limits the distance at which our extended actions and perceptions function. Rather, while the spider’s phenotypical extension is self-sustained, we depend on conspecifics to react to our bodily dynamics. Were this the full story, though, we would be in no better position than vervet monkeys (another featured example of Waters’, in press) who can perform alarm-response calling by means of only three distinct alarm calls. However, as human beings who take a language stance, we have a vast inventarium of developed socio-cultural resources at our disposal. In this respect, we are more like beavers than spiders: we can take advantage of our ancestors’ experientially developed ways of coping with their everyday existence. We normally rely on their experiences as these are compressed in verbal patterns: we trustfully rely on the “wet paint” sign, allowing our behaviour to be shaped by the arduous lessons of our forefathers. As pointed out by Clark, we normally trust the information in our environment because the chances of us being cheated “are sufficiently low that they may be traded against the efficiency gains of (for some cognitive purposes)

201

202

Sune Vork Steffensen

leaving some information ‘out in the world’ and relying on just-in-time access” (Clark, 2008:103f.). Language gives us some of the same advantages that spiders, beavers and monkeys get from their webs, dams and calls: it has ensured that each of us is equipped with an extended phenotype. However, as argued above, language is not organism-centred. Rather, the language-induced extension of the human mind and phenotype depends on a cultural meshwork that is constantly renewed by the interaction and co-action of human beings. In this respect the human use of language is similar to the biological phenomena of superorganisms and parasites. First, superorganisms are “Colonies of the eusocial ants, bees, wasps and termites” (Wilson and Sober, 1989:345) that exhibit “colony-level thermoregulation and patterns of information processing that transcend single brains” (ibid.). An incredibly high number of human processes and activities depend on group-level and society-level regulations which depend on processes of self-organization that extend beyond the individual (cf. Heylighen, 2007). Just ask yourself how many people have been involved in your being able to read these lines? Software programmers, hardware engineers, university administrators, teachers, publishers, editors, collectives of scientists, etc., etc. – and as just another piece of the puzzle, an author. In a biological perspective, such polyhuman aggregates act as superorganisms: cognitive processes take place on multiple timescales, both within the individual and distributed across the total system, and language stabilizes the social co-ordination on which such life forms depend. Since we have phenomenological experience of being relatively autonomous agents with a (relatively) free will, acknowledging superorganismic systems gives us a creepy feeling of being reduced to a cog in totalitarian societies. However, as Heylighen (2007) argues, both biological individuals and social groups are “self-organizing, adaptive systems. Most processes in such system are decentralized, indeterministic and in constant flux. They thrive on ‘noise’, chaos, and creativity” (Heylighen, 2007:59). As Garfinkel (1967) suggests, it has been clear that such self organizing systems depend on languaging (and language) since the first breaching experiment in world history, when God said “Go to, let us go down, and there confound their language, that they may not understand one another’s speech” (Gen. 11,7). Thus, organisms and superorganisms are self-organized systems that draw on biosemiosis (cf. Favareau, 2009) which in the human species includes languaging. Don Ross concurs in addressing the question: “How do humans achieve coordination approximating that of eusocial animals despite their standard mammalian population genetics?” (Ross, 2007:729). He replies that “immersion of behavior in language performs the job that hapladiploid [sic] genetics does in bees” (ibid.).



Beyond mind: An extended ecology of languaging

In another ecological niche, parasites provide us with insights into how “a creature’s phenotype is extended not by construction but by instruction” (Waters, in press). Some parasites thus instruct their host how to behave, more often than not in a way that favours the parasite’s life cycle at the expense of the host’s. For instance, the trematode worm Leucochloridium reproduces in birds, and sheds its eggs along with the faeces of its host. Upon hatching the larvae of the worm needs to re-enter a bird’s digestive system in order to reproduce, which is obviously a challenge when you live in bird dropping. How does the trematode solve this problem? They get eaten by snails and in the snails they take on their parasitic existence. The challenge now is to secure that the snail is eaten by a bird, but obviously, snails try to avoid catching the attention of birds. So... The trematode works its way up to the snail’s eye stalks, where it has two effects. First, it causes the snail to become phototropic, attracted to light instead of avoiding it, which is its usual mode. As a result, the snail wants to climb up into the sunshine. Second, the worm itself begins to pulsate, animating the snail’s eye stalks to appear like a small insect. This attracts [...] an avian predator that ingests the snail and the enclosed worm. (Waters, in press)

As demonstrated by this example, extended phenotypes may function through constraining other phenotypes’ behaviour. As a way of constraining other’s behaviour, languaging is pervasive in inter-human relations, not just in face-to-face encounters, but also diachronically. Just as the trematode enter the eye stalks of the snail, language-laden values and norms quite literally make their way into infants, as they auditorily digest what their world offers. What both superorganisms and parasites testify is that collectives, voluntary or not, take on agentive properties. Behavioural constraints on the bodily dynamics of other organisms are ubiquitous in the animal kingdom, and linguistic constraints on dynamics (cf. Rączaszek-Leonardi, this vol.) are an important subclass for human beings. This insight emphasises the need to discard the monolithic distinction between language and non-language: talk, and other semiotic types of behaviour, does not constitute a realm of meaning making that is sealed off from other human behaviour. Though one can methodologically describe the relation between speech acts, or turns, as brought forth, and bringing forth one another in a strict sequential pattern, this does not provide us with a realist explanation of what language is or does: when languaging, human beings create a meshwork of bio-socio-ecological relations (cf. Thibault, 2011; Steffensen et al., 2010), and any claim of a distinct semiotic realm of sense-making relies purely on a specific perspective (cf. Giere, 2006) projected onto a poly-human scenery by a semiocentric observer.

203

204

Sune Vork Steffensen

The extended ecology lies on a middle path between reducing language to biology and treating language without recurrence to biology. Both extremes are deleterious. Language unfolds in human Organism-Environment-Systems (cf. Järvilehto, 1998, 2009), and as such it is part of the human ecology, just like the web is a part of the spider’s ecology. But admittedly, language has catapulted and stabilized human cognition in ways that are unequalled in the ecology of any other species. The human ecology is radically extended. All living systems realize values (cf. Hodges, this vol.) by categorizing its environment in terms of what they can eat, with whom they can mate and what jeopardises their survival. Language allows human beings to share categories, reflect upon them, and use them to animate distributed cognitive systems. Importantly, by languaging with each other, we are more than parts of each other’s environment; we develop shared agency in a shared world. As Bert Hodges observes, this allows us to develop deep emotional relations to each other. Our shared agency depends on, not just what each of us can gain individually, but on our ability and willingness to distribute our agency across larger social groupings. In Hodges’ terms, “Social solidarity with those who speak to us and listen to us in caring ways is a crucial dimension of why and how we speak at all” (Hodges 2007: 598). 7. Conclusion: If you want to learn about language, forget about language! The American structural linguist, Leonard Bloomfield, once made the sarcastic remark that “If you want to compare two languages, it helps to know one of them!” (quoted in Crystal, 1997:412). The target of his wryness was “unscientific impressionism in language studies” (ibid.), or what is often called naïve realism. This chapter has questioned what it means to “know a language.” It suggests that much work in linguistics suffers from naïve realism based on treating language as an object that can be ‘taught’, ‘known’ and ‘used’. Likewise, it has criticised Andy Clark (2008) for adopting the same conservative view and presenting “language” in a way parallel to “unscientific impressionism in language studies.” Are we to understand how mind and cognition extend beyond brains, we need a careful examination of how language saturates human cognition in our everyday existence. Using integrational theory as a springboard (Harris, 1981; 1998; Love, 2004), the distributed language movement seeks an understanding by looking beyond our (abecedaric) phenomenological experiences of ‘using language’. Summarized à la manière de Bloomfield, this can be stated in the slogan: “If you want to learn about language, forget about language!” Thus, a fuller understanding requires that we give up the naïve idea that our idea of ‘language’ corresponds to a



Beyond mind: An extended ecology of languaging

unitary phenomenon in the natural world. Rather we can take a panoramic view of how human interactivity based on making and tracking vocal and bodily gestures impact our lives. This contrasts with Clark’s view of an individual cognizer who uses linguistic (or linguaform) structures: by clinging to an impressionistic view of language, Clark propagates an individualistic (disguised as ‘organism-centred’) view on human cognition. This chapter has suggested that the path to a deeper insight into human cognition, languaging and interactivity derive from a distinctly human extended ecology. Like the beavers’ dam, languaging has a history that is influenced by situation-transcending third parties (Bang and Døør, 2007:58-61; Linell, 2009:99105), just as what is left of our voices will silently contribute to the interactions of our descendants. Being the domain of third parties, our dam of language is heritable, variable and amenable to selection. The historically grounded, situationtranscending dimension of human cognition scatters its distribution in time (cf. Hollan et al., 2000). However, languaging also allows us to engage in collective and reciprocal problem-solving: in our extended ecology, language co-constitutes a cultural network (cf. Donald, 2001), and, equally, distributed cognitive systems (cf. Hollan et al., 2000). In this sense, language functions as airborne synapses: it contributes to inter-human co-ordination, thus allowing us to be smarter, more creative and more flexible, just like the brain is smarter and more flexible than a pile of neurons. Alas, at the same time it also makes us do the most stupid, bizarre and horrendous mistakes, misdeeds and misdemeanours. That’s life. Since language is real-time metabolic activity (e.g. voice dynamics, gaze and task-oriented modes of action), it can be explained neither with reference to a ‘language system’ nor a complex of microsocial rules and norms. We need to acknowledge the role of human biology in how it unfolds in an extended ecology. However, appeal to bio-ecology does not imply that we discard microsocial normativity altogether, because, borrowing a term from Stephen Cowley (this vol.), language is symbiotic: real-time bio-dynamics are partially reconfigured by phenomenology. It plays out in an extended ecology where, odd as it may seem, biology and phenomenology cease to be individual. This claim challenges present views of human interaction. In doing so it calls for a deeper view of the real-time bodily activities in and between human beings. If this is to be achieved, there is a need to reject pre-theoretical ideas of ‘language’, ‘gesture’, ‘meaning-making’, etc. (cf. Steffensen 2008, in press). It is ontologically, epistemologically and methodologically unwarranted to assume that we understand these phenomena when we try to understand what people do when they do things, alone or together. In order to fully grasp the complexities of human interaction and cognition, the entire extended ecology will have to become our unit of analysis.

205

206

Sune Vork Steffensen

Notes *╇ This chapter is a thoroughly revised version of a review article (Steffensen, 2009) of Andy Clark’s Supersizing the Mind (2008). I am grateful to Nigel Love, Susan Stuart, and Stephen Cowley for comments on the previous version. Dennis Waters and Stephen Cowley provided helpful and valuable comments on the current version. 1. This is an all too coarse-grained description, of course. An overview of this movement can be found in, for example, Boden, 2006, Robbins and Aydede, 2009 and Kiverstein and Clark, 2009. 2. Chapter 3 is a revised version of Clark, 2006. In the current context I squint at the original paper which spells out some of Clark’s assumptions. 3. “La tâche de la linguistique sera : […] de se delimiter et de se definir elle-même” (Saussure 1916/1972: 20). And “la linguistique a pour unique et veritable objet la langue envisage en ellemême et pour elle-même”. (Saussure 1916/1972:317). Saussure uses this term with two interdependent purposes (Steffensen 2000): the one is to place linguistics in a privileged position as the sole science with access to language (the “real” language, i.e., la langue), the other is to make an ontological claim that insists that la langue – as a system (Hjelmslev’s 1943 term) of linguistic competences (Chomsky’s 1965 term) – is an ontologically real phenomenon: “La langue n’est pas moins que la parole un objet de nature concrete” (Saussure 1916/1972: 32). 4. Cf. Bade and Pable, 2011. 5. Even repeated versions of such utterances by the same speaker would never have ‘material stability’, cf. the study by Van Orden et al. (2003, 2005), reported by Hodges (this vol.). 6. Clark is not fond of phenomenological arguments. He states: “I do not mean, here or elsewhere, to advance any arguments of the form ‘it seems to us as if we are/are not cognitively extended; therefore, we are/are not cognitively extended’!” (Clark 2008:238). But as Kyselo and Walter (2009) point out, the extended mind hypothesis is actually supported by phenomenological arguments, e.g.: “If the coupling between an agent and an external tool becomes intimate enough, the agent may no longer feel as though she is using a tool, but instead experience the (former) tool as part of herself” (Kyselo and Walter, 2009:806). 7. in Latin “pebbles” is calculi, cf. Clark’s view that language is a cognitive scaffolding, that makes our environment accountable. 8. As demonstrated below, we can ask the same questions in relation to Clark’s Inga: who taught her how to find the way to the museum, etc.? 9. See: http://www.youtube.com/watch?v=DR4y5iX4uRY, or search for “Derren Brown Russian scam complete”. 10. A philological note: ecology is derived from the Greek oikos, which means ‘house’. Dominator and dominion is derived from Latin dominus (‘lord’) which is derived from domus, which means … ‘house’! Two very different views of our shared planetary household...



Beyond mind: An extended ecology of languaging

References Alač, M. and Hutchins, E. 2004. “I see what you are saying: Action as cognition in fMRI brain mapping practice”. Journal of Cognition and Culture 4(3): 629–661. Bade, D. and Pable, A. (Eds.). 2011. Linguistics Out of Bounds: Explorations in Integrational Linguistics in Honour of Roy Harris on his 80th Birthday. Special issue of Language Sciences, 33(4). Bang, J.C. and Døør, J. 2007. Language, Ecology and Society – a Dialectical Approach. S.V. Steffensen and J. Nash (eds). London: Continuum. Boden, M. 2006. Mind as Machine: A History of Cognitive Science. Oxford: Oxford University Press. Boysen, S.T., Bernston, G., Hannan, M., and Cacioppo, J. 1997. “Quantity-based inference and symbolic representation in chimpanzees (Pan troglodytes)”. Journal of Experimental Psychology: Animal Behaviour Processes 22: 76–86. Chomsky, N. 1965. Aspects of the Theory of Syntax. Cambridge, MA: The MIT Press. Clark, A. 2004. “Is language special? Some remarks on control, coding, and co-ordination”. Language Sciences 26(6): 717–726. Clark, A. 2006. “Material symbols”. Philosophical Psychology 19(3): 291–307. Clark, A. 2008. Supersizing the Mind. Embodiment, Action and Cognitive Extension. Oxford: Oxford University Press. Clark, A. and Chalmers, D. 1998. “The extended mind.” Analysis 58(1): 7–19. Cowley, S.J. 1994. “The role of rhythm in conversations: a behavioural perspective.” Language and Communication, 14: 353–376. Cowley, S.J. 1998. “Of timing, turn-taking, and conversation.” Journal of Psycholinguistic Research, 27(5): 541–571. Cowley, S.J. 2007a. “The cognitive dynamics of distributed language.” Language Sciences 29(5): 575–583. Cowley, S.J. 2007b. “How human infants deal with symbol grounding.” Interaction Studies 8 (1): 83–104. Cowley, S.J. 2010. “Beyond symbols: Interaction and the enslavement principle.” In J. Streeck (Ed.). New Adventures in Language and Interaction. John Benjamins: Amsterdam, pp. 47–70. Cowley, S.J. This volume. “Distributed language.” In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 1–14. Cowley, S.J. 2011. “Taking the language stance.” Ecological Psychology, 23/3: 185–209. Cowley, S.J. and Zheng, D. 2011. “The turning of the tide: Rethinking language, mind and world.” Journal of Multicultural Discourses, 6/2: 197–210. Crystal, D. 1997. The Cambridge Encyclopedia of Language. 2nd ed. Cambridge: Cambridge University Press. Dawkins, R. 1982. The Extended Phenotype. San Francisco: W.H. Freeman. Donald, M. 2001. A Mind so Rare. New York: Norton. Døør, J. 1998. Moralske Meditationer. To Essays. Odense: Odense Universitet. Favareau, D. (Ed.) 2009. Essential Readings in Biosemiotics. Dordrecht: Springer Science + Business Media B.V. Fowler, C.A. and Rosenblum, L.D. 1991. “The perception of phonetic gestures.” In Mattingly, I.G. and Studdert-Kennedy, M. (eds.). Modularity and the motor theory of speech perception. Hillsdale, NJ: Lawrence Erlbaum, pp. 33–59.

207

208

Sune Vork Steffensen

Garfinkel, H. 1967. Studies in Ethnomethodology. Cambridge: Polity Press. Gibson, J.J. 1966. The Senses Considered as Perceptual Systems. Boston: Houghton-Mifflin. Giere, R.N. 2006. Scientific Perspectivism. Chicago: Chicago University Press. Harris, R. 1981. The Language Myth. London: Duckworth. Harris, R. 1998. Introduction to Integrational Linguistics. Oxford: Pergamon Press. Heylighen F. 2007. “The Global Superorganism: an evolutionary-cybernetic model of the emerging network society.” Social Evolution & History, 6(1): 58–119. Hjelmslev, L. 1943. Omkring Sprogteoriens Grundlæggelse. Copenhagen: Hans Reitzels Forlag. Hodges, B.H. 2007. “Good prospects: Ecological and social perspectives on conforming, creating, and caring in conversation”. Language Sciences 29(5): 584–604. Hodges, B.H. This vol. “Ecological Pragmatics: Values, dialogical arrays, complexity, and caring.” Hollan, J., Hutchins, E., and Kirsh, D. 2000. “Distributed cognition: Toward a new foundation for human–computer interaction.” In: ACM Transactions on Human-Computer Interaction, Vol. 7(2), pp. 174–196. Hutchins, E. 1995. Cognition in the Wild. Cambridge, MA: The MIT Press. Järvilehto, T. 1998. “The theory of the organism-environment system: I. Description of the theory.” Integrative Physiological and Behavioral Science 33: 321–334. Järvilehto, T. 2009. “The Theory of the Organism-Environment System as a Basis of Experimental Work in Psychology.” Ecological Psychology, 21(2): 112–120. Kirsh, D. 1995. “The intelligent use of space.” Artificial Intelligence, 73(1-2): 31–68. Kirsh, D. 2010. “Thinking with external representations.” AI & Society, 25(4): 411–454. Kirsh, D. and Maglio, P. 1994. “On distinguishing epistemic from pragmatic action.” Cognitive Science. Vol. 18(4), pp. 513–549. Kiverstein, J. and Clark, A. 2009. “Introduction: Mind embodied, embedded, enacted: One church or many?” Topoi 28: 1–7. Kravchenko, A.V. 2007. “Essential properties of language, or, why language is not a code.” Language Sciences 29(5): 650–671. Kravchenko, A.V. This vol. “The experiential basis of speech and writing as different cognitive domains.” Kyselo, M. and Walter, S. 2009. “Supersizing the Mind”. Philosophical Psychology, 22(6): 803–807. Linell, P. 2005. The Written Language Bias in Linguistics: Its Nature, Origins and Transformations. London: Routledge. Linell, P. 2009. Rethinking Language, Mind, and World Dialogically: Interactional and Contextual Theories of Human Sense-making. Charlotte, NC: Information Age Publishing. Lloyd, D. 2009. “Outsourcing the Mind.” American Scientist 97, July-August 2009, 340–342. Love, N. 2004. “Cognition and the language myth.” Language Sciences 26(6): 524–544. Love, N. 2007. “Are languages digital codes?” Language Sciences 29(5): 690–709. McNeill, D. 2005. Gesture and Thought. Chicago: The University of Chicago Press. Pitt, D. 2008. “Mental Representation.” The Stanford Encyclopedia of Philosophy (Fall 2008 ed.), ed. by E.N. Zalta. URL: http://plato.stanford.edu/archives/fall2008/entries/mental-representation. Port, R.F. 2010. “Rich memory and distributed phonology.” Language Sciences, 32(1), 43–55. Port, R.F. and Leary, A.P. 2005. “Against formal phonology.” Language 81 (4): 927–964. Rączaszek-Leonardi, J. This volume. “Symbols as constraints: the structuring role of dynamics and self-organization in natural language.” In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 161–184.



Beyond mind: An extended ecology of languaging

Robbins, P. and Aydede, M. (Eds.). 2009. The Cambridge Handbook of Situated Cognition. Cambridge: Cambridge University Press. Ross, D. 2007. “H. sapiens as ecologically special: what does language contribute?” Language Sciences 29 (5): 710–731. Saussure, F. 1916/1972. Cours de linguistique générale. Paris: Payot. Steffensen, S.V. 2000. Strukturalisme, Marxisme og Dialektik. Dialektiske og dialogiske læsninger i diskurser om diskurser. Odense: Odense Universitet. Steffensen, S.V. 2007. “Language, ecology and society: An introduction to Dialectical Linguistics.” In J.C Bang and J. Døør, Language, Ecology and Society – a Dialectical Approach. London: Continuum, 3–31. Steffensen, S.V. 2008. “The ecology of grammar. Dialectical, holistic and autopoietic principles in ecolinguistics.” In M. Döring, H. Penz, and W. Trampe (eds), Language, Signs and Nature. Ecolinguistic Dimensions of Environmental Discourse. Essays in Honour of Alwin Fill. Tübingen: Stauffenburg, 89–105. Steffensen, S.V. 2009. “Language, languaging, and the Extended Mind Hypothesis.” Pragmatics & Cognition, 17(3): 677–697. Steffensen, S.V. In press. “Care and Conversing in Dialogical Systems.” Language Sciences (special issue: Caring and conversing: The distributed dynamics of dialogue, ed. by B. Hodges, J. Martin and S.V. Steffensen). Steffensen, S.V. and Cowley, S.J. 2010. “Signyfying bodies and health: a non-local aftermath” In: Cowley, S., Major, J.C., Steffensen, S.V. and Dinis, A. (Eds.) Signifying Bodies: Biosemiosis, Interaction and Health, pp. 331-356. Braga: Catholic University of Portugal. Steffensen, S.V., Thibault, P.J., and Cowley, S.J. 2010. “Living in the social meshwork: the case of health interaction.” In: S.J. Cowley, J.C., Major, S.V. Steffensen and A. Dinis, (Eds.), Signifying bodies: Biosemiosis, Interaction and Health. Braga: Catholic University of Portugal, 207–244. Sudnow, D. 2001. Ways of the Hand: A Rewritten Account. Cambridge, MA: The MIT Press. Sutton, J. 2007. “Batting, habit, and memory: The embodied mind and the nature of skill”. Sport in Society 10 (5): 763–786. Thibault, P.J. 2008. “Face-to-face communication and body language.” In K. Knapp and G. Antos (eds.), Handbooks of Applied Linguistics (HAL) Linguistics for Problem-Solving: Perspectives on Communication Competence, Language and Communication Problems, and Practical Solutions. Volume 2: Interpersonal Communication. G. Antos & E. Ventola (eds.). Berlin: Mouton, 285–330. Thibault, P.J. 2011. “First order languaging dynamics and second-order language: the distributed language view.” Ecological Psychology, 23/3: 210–245. Tribble, E. 2005. “Distributing cognition in the globe.” Shakespeare Quarterly 56(2): 135–155. Tribble, E. This vol. “Languaging in Shakespeare’s theatre.” In S.J. Cowley (ed.) Distributed Language. Amsterdam: Benjamins, 101–115 Udehn, L. 2002. “The Changing Face of Methodological Individualism.” Annual Review of Sociology, 28: 479–507. Van Orden, G.C., Holden, J.G. and Turvey, M.T. 2005. “Human cognition and 1/f scaling.” Journal of Experimental Psychology: General 134: 117–123. Van Orden, G.C., Moreno, M.A., and Holden, J.G. 2003. “A proper metaphysics for cognitive performance.” Nonlinear Dynamics, Psychology, and Life Sciences 7: 49–60.

209

210

Sune Vork Steffensen

Vološinov, V.N. 1973. Marxism and the Philosophy of Language. New York and London: Seminar Press. Waters, D. In press. “From Extended Phenotype to Extended Affordance: Distributed Language at the Intersection of Gibson and Dawkins.” Language Sciences (special issue: Caring and conversing: The distributed dynamics of dialogue, ed. by B. Hodges, J. Martin and S.V. Steffensen). Wilson, D.S. and Sober, E. 1989. “Reviving the Superorganism.” Journal of Theoretical Biology, 136: 337–356. Wittgenstein, L. 1958. Philosophical Investigations. 2nd ed. Oxford, Blackwell.

Subject Index A absence of meaning, 120, 121 accent, 108, 110 acoustic array, 142, 143 action, 2, 4, 5, 8, 9, 11, 12, 16, 24, 25, 26, 27, 28, 29, 57, 58, 59, 62, 63, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 82, 84, 89, 98, 99, 100, 104, 108, 109, 110, 113, 118, 120, 124, 125, 135, 136, 137, 138, 139, 142, 144, 145, 146, 149, 150, 151, 154, 155, 157, 158, 168, 170, 175, 177, 178, 185, 188, 191, 192, 193, 196, 201, 205, 207, 208 action results, 24, 25, 26, 27, 124 acts of writing, 11, 41 adaptation, 13, 48, 123, 172, 183, 200 affordances, 9, 13, 14, 81, 82, 92, 94, 97, 98, 118, 138, 141, 148, 149, 156, 157, 195 agency, 10, 13, 53, 72, 98, 100, 138, 141, 144, 158, 198, 204 alienation, 9, 10, 151, 152, 153, 154 alignment, 81, 89, 92, 93, 94, 95, 98, 176 anonymous third, 195 answerability, 144, 153 anticipation, 13, 15, 16, 18, 24, 25, 63, 74, 78, 82 anticipatory dynamics, 1, 7, 24, 72 anticipatory process, 21, 27, 28, 58

articulatory gestures, 141 artifact, 44, 51, 69, 109 artificial intelligence, 34, 208 assumption of simple location, 6 attention, 38, 62, 72, 73, 81, 83, 92, 96, 98, 100, 108, 109, 115, 167, 172, 176, 177, 189, 193, 196, 201, 203 attunement, 97, 98, 135, 151, 152, 153, 154 auditory perception, 37 authority, 45, 46, 47, 102, 103, 106, 107, 112 autonomy, 45, 53, 149, 150 autopoiesis, 42, 52, 53, 54 B background knowledge, 41, 49 bio-cognitive, 51, 54 biological symbol, 1, 178 biosemiotics, 8, 13, 14, 165, 180, 181, 207 brain imaging, 17, 84, 85, 97 C caring, 8, 13, 53, 78, 114, 135, 136, 137, 142, 144, 149, 150, 153, 154, 155, 157, 182, 204, 208, 209, 210 caring system, 8, 135, 136, 137, 142, 149, 150 causal spread, 72 caution, 69, 153, 173, 180 cheap necklace, 57, 63, 64, 66, 69, 72, 77 circumspection, 153

clarity, 140 co-acting bodies, 101 co-action, 83, 169, 173, 174, 175, 192, 193, 202 code, 3, 7, 13, 33, 34, 38, 39, 41, 43, 45, 46, 47, 48, 51, 53, 63, 72, 73, 78, 90, 91, 92, 101, 150, 157, 162, 168, 182, 208 code model, 34, 41, 47, 51 code view, 3, 73, 101 codified gesture, 110 coding, 12, 16, 88, 89, 90, 91, 92, 93, 94, 95, 207 cognition, 1, 3, 5, 6, 7, 8, 9, 12, 13, 14, 30, 31, 33, 34, 35, 42, 43, 52, 53, 54, 55, 57, 58, 62, 65, 77, 78, 79, 82, 83, 100, 114, 115, 117, 120, 121, 122, 123, 124, 127, 128, 129, 130, 131, 132, 139, 142, 146, 150, 151, 156, 157, 158, 164, 168, 169, 173, 175, 180, 181, 182, 183, 185, 186, 187, 188, 189, 192, 194, 195, 196, 197, 198, 199, 200, 204, 205, 207, 208, 209 cognitive anthropology, 186, 187 cognitive coordination, 174, 176 cognitive development, 39, 48, 51, 157, 184 cognitive domains, 7, 33, 35, 38, 41, 47, 48, 51, 52, 110, 208 cognitive dynamics, 6, 7, 11, 12, 13, 30, 34, 35, 38, 39, 43, 48, 52, 53, 57, 63, 75, 77, 114, 181, 192, 207

212

Subject Index

cognitive ecology, 113 cognitive integration, 105 cognitive niche, 35, 175, 181 cognitive state, 48, 71, 72, 75, 186 cognitivism, 153 cognizer, 42, 43, 45, 188, 189, 192, 194, 195, 196, 199, 200, 205 coherence, 140 collective, 1, 6, 7, 8, 127, 136, 145, 146, 169, 178, 195, 198, 205 communication, 6, 12, 14, 30, 34, 37, 42, 44, 50, 55, 82, 88, 89, 90, 97, 100, 114, 140, 158, 167, 172, 173, 174, 175, 176, 181, 182, 184, 185, 187, 188, 189, 207, 209 communicative dysfunction, 33, 46, 53 communicative success, 175 complex adaptive system, 176 complexity, 4, 13, 18, 29, 49, 78, 123, 125, 128, 135, 136, 140, 144, 145, 147, 148, 150, 166, 171, 172, 183, 184, 208 composition, 81, 82, 96, 97 compositional structure, 173, 183 compositionality, 82, 83, 96, 172, 174, 177, 181 comprehension, 18, 30, 44, 58, 84, 85, 97, 100, 127 comprehensiveness, 138, 140 computational model, 174 conceptual thinking, 125 conformity, 94, 151, 156 consciousness, 4, 30, 53, 77, 78, 99, 131, 135, 136, 137, 150, 151, 152, 153, 154, 156, 157, 158, 179 consensual coordination, 43 consensual domain, 9, 11, 33, 35, 41, 43, 44, 45, 46, 47, 48, 50, 51, 52

constraint relaxation, 60, 61, 78 context, 7, 15, 21, 25, 27, 28, 31, 43, 44, 46, 48, 60, 69, 82, 84, 88, 96, 103, 108, 120, 128, 129, 135, 136, 144, 145, 146, 147, 148, 149, 153, 158, 167, 168, 172, 173, 175, 180, 183, 186, 191, 206 context dependency, 168 control, 3, 5, 8, 11, 24, 28, 65, 84, 86, 88, 89, 90, 92, 94, 97, 98, 100, 105, 111, 136, 147, 149, 150, 161, 165, 166, 167, 168, 169, 173, 175, 176, 183, 191, 196, 207 conventional meaning, 124 conventionality, 82, 84, 85, 86, 95 conversing, 135, 137, 140, 141, 142, 144, 148, 149, 150, 151, 154, 155, 157, 209, 210 coordination, 14, 74, 104, 106, 142, 145, 150, 154, 158, 161, 162, 168, 173, 174, 175, 176, 177, 178, 179, 183, 186, 202 coordination dynamics, 173 coordinative capacities, 167 creativity, 120, 125, 131, 132, 133, 151, 173, 193, 202 cultural capital, 103 cultural cognitive network, 199 cultural evolution, 38, 117, 119, 121, 123, 124, 128, 130, 132, 168, 171 cultural practice, 1, 75, 146 culture, 9, 13, 14, 33, 38, 43, 45, 48, 49, 50, 53, 55, 62, 73, 74, 99, 104, 108, 117, 119, 120, 121, 122, 123, 124, 126, 127, 128, 131, 132, 148, 207 D Darwinian, 117, 119, 129, 170 decoding, 15, 16, 19, 23, 24, 39 dependencies, 136, 146, 155

dialogical, 2, 4, 5, 6, 13, 24, 34, 43, 44, 48, 54, 55, 73, 74, 78, 135, 136, 137, 142, 143, 144, 147, 149, 150, 151, 152, 153, 154, 155, 188, 208, 209 dialogical activity, 5, 44, 73 dialogical array, 13, 78, 135, 136, 142, 143, 144, 149, 151, 153, 154, 155, 208 dialogical field, 144 dialogue, 4, 5, 7, 10, 11, 28, 44, 54, 100, 136, 142, 143, 144, 150, 151, 154, 155, 157, 209, 210 difference, 19, 23, 34, 48, 49, 65, 66, 68, 90, 93, 117, 120, 121, 122, 123, 124, 126, 127, 128, 131, 142, 189, 192, 196, 199 digital signalling, 3 distance perception, 143 distributed cognition, 3, 12, 13, 14, 53, 57, 58, 69, 71, 72, 78, 100, 104, 114, 115, 181, 185, 187, 196, 208 distributed cognitive process, 106 distributed cognitive system, 7, 8, 13, 58, 62, 65, 71, 72, 75, 185, 188, 198, 204, 205 distributed language, 1, 3, 4, 12, 13, 14, 24, 30, 34, 53, 57, 77, 101, 113, 114, 181, 192, 193, 204, 207, 209, 210 distributed perspective, 1, 2, 4, 9, 185, 199, 201 distributed phenomenon, 2, 164, 169 distributed system, 63 diversity, 4, 38, 52, 128, 140, 146, 147, 148, 156 diversity of language, 140 dna, 123, 165, 168 double dialogicality, 5 double processing, 8, 12, 121, 122, 123, 124, 128



double processing hypothesis, 8, 12 dyadic interaction, 177 dynamic process, 162, 168 dynamical process, 162, 164, 165, 167, 170, 178, 196 dynamics, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 24, 25, 26, 29, 33, 38, 40, 41, 44, 49, 52, 57, 59, 63, 73, 74, 75, 79, 102, 106, 111, 131, 133, 135, 136, 144, 146, 147, 148, 150, 151, 155, 156, 157, 158, 161, 162, 163, 164, 165, 166, 168, 169, 170, 171, 172, 173, 174, 175, 178, 179, 181, 185, 187, 188, 191, 193, 195, 198, 201, 203, 205, 208, 209, 210 E ecological niche, 185, 188, 194, 197, 198, 203 ecological psychology, 7, 13, 77, 135, 145, 151, 156, 157, 158, 159, 181, 207, 208, 209 ecological self, 152 ecology, 4, 7, 9, 35, 50, 52, 153, 197, 198, 200, 201, 204, 205, 206, 207, 209 ecosystem, 138, 139, 140, 141, 144, 151, 155, 200 education of attention, 191 efferent innervation, 25 efficiency, 18, 112, 139, 167, 173, 201 emergence, 30, 35, 37, 43, 63, 121, 125, 128, 130, 131, 152, 162, 164, 168, 169, 170, 171, 172, 173, 180, 181, 182, 183 emotional alignment, 176 enactive, 144 encoding, 39, 51, 109, 110 episodic culture, 125 epistemic artifact, 194, 199 evolution, 8, 33, 35, 53, 55, 77, 78, 99, 117, 119, 121, 122, 123,

Subject Index

124, 128, 129, 130, 131, 132, 133, 140, 156, 157, 161, 162, 164, 168, 171, 172, 174, 175, 176, 179, 180, 181, 182, 183, 184, 187, 208 evolutionary development, 117, 122, 125 evolutionary history, 6, 146, 181, 194 evolutionary linguistics, 172 evolutionary properties, 119 evolutionary psychology, 122, 128 exact recall, 104 experience, 2, 3, 5, 7, 10, 11, 15, 19, 23, 24, 34, 35, 37, 40, 41, 42, 43, 44, 47, 49, 51, 52, 57, 58, 60, 61, 63, 65, 67, 68, 70, 71, 72, 73, 74, 75, 76, 77, 79, 80, 81, 82, 85, 86, 106, 112, 113, 118, 120, 129, 152, 155, 156, 177, 190, 191, 197, 199, 206 expression, 3, 5, 28, 33, 46, 74, 97, 168, 174, 189, 193 extended ecology, 1, 9, 11, 14, 185, 188, 200, 204, 205 extended individualism, 196 extended mind, 52, 57, 58, 69, 70, 71, 74, 75, 77, 185, 187, 188, 192, 196, 199, 200, 201, 206, 207, 209 extended phenotype, 11, 201, 202, 203, 207, 210 external memory storage, 124 external resource, 2, 59, 65, 69, 70 externalism, 12, 77 externalist, 58, 62 F feeling of thinking, 1, 2, 5, 7, 63, 74 field, 59, 113, 138, 141, 144, 156, 167, 187 first-order activity, 3, 6, 7, 11, 137, 192

first-order language, 4, 136, 140 fixation, 10, 15, 16, 17, 18, 19, 21, 23, 26, 28, 30, 79 fixation-speech interval, 15, 18, 19, 23, 28, 30 formal ambiguity, 121 formalist assumption, 172 form-based representation, 72 fractal patterning, 145, 147 fractal variability, 147, 148 function, 1, 2, 6, 8, 10, 34, 37, 42, 46, 49, 50, 57, 72, 73, 74, 76, 82, 115, 124, 129, 136, 138, 141, 144, 149, 150, 154, 155, 162, 163, 164, 165, 166, 168, 169, 170, 171, 174, 175, 178, 179, 180, 183, 190, 198, 200, 201, 203 G game theory, 180 gaze, 7, 23, 74, 185, 188, 191, 192, 205 gesture, 2, 5, 98, 101, 108, 109, 110, 111, 114, 115, 125, 192, 205, 208 goal state, 59, 61, 64, 66 grammatical rule, 162, 169, 170, 173, 179, 180 graphic image, 37, 44, 46 graphic markings, 33, 46 H heterarchy, 135, 141 heuristic, 61, 67, 69, 70 hierarchical levels, 50, 161, 169 hierarchical models, 26 history, 1, 3, 4, 6, 7, 9, 10, 15, 17, 27, 28, 30, 36, 42, 47, 50, 51, 72, 73, 74, 75, 105, 131, 153, 155, 157, 163, 166, 173, 196, 197, 200, 202, 205, 207, 208 human communication, 5, 161, 168, 181, 183, 184 human technology, 104 hybrid system, 108

213

214

Subject Index

I iconic strategy, 89 images, 1, 8, 10, 37, 54, 81, 84, 86, 87, 88, 90, 92, 93, 95, 97, 98, 117, 125, 126, 127 imbumba, 9, 10, 11 imitation, 117, 120, 125, 129, 130, 132 improvisation, 10, 102, 111 individual, 1, 7, 8, 33, 35, 45, 48, 49, 50, 51, 59, 79, 89, 102, 104, 105, 106, 110, 117, 120, 127, 128, 136, 143, 155, 156, 161, 163, 164, 169, 172, 173, 174, 175, 176, 178, 185, 186, 188, 189, 191, 192, 193, 194, 195, 198, 199, 200, 201, 202, 205 individualism, 198, 209 inferencing, 51 inferential skills, 7 influence, 5, 7, 8, 24, 25, 37, 45, 46, 59, 63, 69, 71, 72, 81, 142, 146, 173, 176, 178, 198, 200 information processing, 15, 16, 17, 18, 24, 25, 31, 62, 70, 78, 123, 124, 128, 165, 202 information transmission, 161, 163, 179 initial state, 59, 62, 64, 70 input-output model, 186 inscription, 18, 19, 24, 26, 27, 101 insight, 1, 7, 10, 11, 19, 57, 58, 59, 60, 61, 62, 63, 64, 65, 69, 70, 72, 74, 75, 76, 77, 78, 79, 80, 191, 192, 203, 205 integrational, 3, 14, 204, 207, 208 intentionality, 100, 119, 123, 153, 158 interaction, 7, 13, 14, 28, 30, 34, 42, 44, 45, 52, 53, 54, 69, 72, 77, 78, 97, 99, 104, 105, 118, 146, 147, 167, 168, 173, 177, 182, 183, 185, 186, 189, 196, 200, 202, 205, 207, 208, 209

interactional dynamics, 175, 196 interaction-dominant, 146, 147 interactive sense-making, 10, 84, 100 inter-bodily dynamics, 193 interdependent components, 146 inter-individual, 161, 162, 173, 175, 179 internal description, 166, 180 internalism, 12, 70, 75, 77 interpretative action, 83 intersubjective, 3, 8, 81, 82, 83, 84, 85, 88, 92, 95, 96, 97, 98, 158 intersubjective contextualization, 88, 96, 97 intersubjective encounter, 83 intersubjective interpretation, 84 intersubjective meaningmaking, 97 intersubjective signal, 84, 95 intersubjectivity, 14, 82, 99, 100, 158, 184

language acquisition, 39, 45, 172, 180 language activity, 34 language development, 158, 167, 168, 181, 182 language efficiency, 163 language itself, 188 language myth, 13, 14, 54, 78, 157, 182, 189, 208 language niche, 175 language stance, 3, 10, 13, 73, 77, 179, 181, 191, 195, 198, 201, 207 language system, 185, 187, 188, 189, 205 language transmission, 172 language-activity, 2, 7, 167 languaging, 3, 4, 5, 7, 9, 10, 11, 14, 33, 34, 35, 37, 39, 40, 41, 42, 43, 44, 47, 48, 51, 58, 101, 107, 110, 111, 112, 136, 156, 162, 181, 185, 188, 191, 192, 193, 194, 196, 198, 199, 200, 202, 203, 204, 205, 209 lateralization, 122, 130, 131 learning, 3, 5, 28, 38, 40, 41, L 48, 51, 53, 64, 77, 78, 79, 109, language, 1, 2, 3, 4, 5, 6, 7, 8, 9, 110, 114, 154, 156, 168, 170, 10, 11, 12, 13, 14, 15, 18, 24, 28, 171, 172, 173, 174, 176, 177, 29, 30, 31, 33, 34, 35, 37, 38, 179, 181, 182, 183, 195 39, 41, 42, 45, 47, 48, 49, 50, lexicalization, 89 51, 52, 53, 54, 55, 57, 58, 62, lexicon, 17, 77, 193 63, 72, 73, 74, 75, 77, 78, 79, linguaform, 189, 195, 205 83, 84, 99, 100, 101, 104, 107, linguaform reason, 189, 195 109, 114, 115, 119, 120, 125, 128, linguistic action, 74 130, 131, 132, 133, 135, 136, 137, linguistic activity, 34, 138, 140, 140, 142, 143, 144, 145, 147, 144, 145, 149, 150 148, 149, 150, 151, 153, 154, 155, linguistic autonomy, 1 156, 157, 158, 161, 162, 163, 164, linguistic creativity, 193 166, 167, 168, 169, 170, 171, linguistic event, 6, 74 172, 173, 174, 175, 176, 177, 178, linguistic interaction, 34, 35, 41, 44, 46, 49, 50, 168, 189 179, 180, 181, 182, 183, 185, linguistic material, 6, 74 186, 187, 188, 189, 190, 191, linguistic sign, 2, 5, 11, 33, 37, 39, 192, 193, 194, 195, 196, 197, 42, 44, 46, 48, 179, 190, 192 198, 200, 201, 202, 203, 204, linguistic signifier, 2, 11 205, 206, 207, 208, 209, 210



linguistic symbol, 161, 167, 168, 169, 173, 175, 178 literacy, 38, 39, 41, 45, 48, 49, 51, 52, 53, 54, 115 living system, 6, 8, 36, 42, 51, 161, 163, 164, 165, 166, 178, 204 loops, 127, 186 M magic of wordings, 140, 156 managing attention, 109 material practice, 101 material signals, 82, 83, 84, 85, 92, 93, 97 material stability, 190, 191, 206 material symbol, 6, 10, 35, 38, 58, 63, 72, 73, 74, 75, 77, 99, 100, 188, 194, 196, 201, 207 materiality, 189, 190 matrix of grammar, 177 maximization heuristic, 70 meaning, 5, 7, 9, 10, 11, 15, 16, 17, 18, 19, 27, 28, 29, 30, 33, 37, 41, 42, 43, 47, 48, 53, 55, 73, 74, 77, 81, 82, 83, 84, 85, 88, 89, 92, 93, 94, 96, 97, 98, 104, 109, 117, 118, 119, 120, 121, 124, 125, 132, 163, 164, 165, 166, 167, 168, 174, 175, 176, 203, 205 meaning mapping, 174, 176 meaning spaces, 10, 11 meaning-making, 43, 48, 205 mechanism, 171 medium, 35, 42, 43, 48, 49, 76, 143, 155, 186, 190, 195 memory, 4, 14, 34, 49, 54, 58, 60, 77, 78, 79, 102, 107, 109, 114, 115, 117, 120, 121, 122, 123, 156, 169, 181, 183, 191, 196, 208, 209 meshwork, 2, 8, 9, 11, 162, 166, 193, 202, 203, 209 metabolic activity, 185, 188, 191, 205

Subject Index

mimesis, 38, 100, 125, 131, 133 mimetic imitation, 125 mimetic matching, 109 mimetic semiosis, 8 mimicry, 117 monitoring, 4, 7, 10, 57, 71, 73, 74, 76 monological, 83 morphogenesis, 165, 171, 184 multimodal, 74, 102, 106, 158 multimodal dynamics, 106 N naïve realism, 10, 11, 193, 204 natural language, 14, 37, 43, 46, 161, 162, 163, 164, 166, 167, 168, 169, 174, 178, 179, 208 neurocentric, 200 neurocentrism, 200 neurocognitive studies, 83, 94 neuro-dynamics, 85 neurons, 25, 205 niche, 35, 42, 62, 78, 194, 198 niche construction, 62, 78, 194 non-linear dynamical analyses, 145 nonlinear dynamics, 135, 159, 184, 209 non-local, 2, 4, 6, 8, 9, 10, 14, 46, 186, 209 non-locality, 6, 14 nothing special, 60, 75 O object configuration, 81, 83, 84, 85, 86, 92, 95, 96, 97, 98 object-oriented action, 83 on-line communication, 168, 172, 173, 174, 175, 178 ontology, 6, 48, 100, 144 optic array, 143 oral construal, 87, 97 organism-centred, 1, 4, 5, 185, 202, 205

organism-environment system, 4, 7, 13, 15, 17, 24, 25, 27, 29, 30, 53, 57, 72, 74, 78, 157, 208 ostensive qualities, 81, 83, 97 other orientation, 5 P parasites, 202, 203 pattern formation, 171 patterns of reaction, 117, 118, 120 perception, 4, 13, 16, 17, 25, 27, 28, 29, 30, 31, 37, 55, 63, 70, 71, 72, 73, 76, 79, 80, 81, 82, 83, 85, 94, 97, 99, 124, 125, 126, 129, 131, 135, 136, 137, 138, 142, 143, 145, 149, 150, 151, 154, 155, 156, 157, 158, 162, 168, 178, 184, 201, 207 perception-action, 135, 136, 137, 149, 157 perceptual awareness, 151 perceptual meaning-making, 82 perceptual process, 17 perceptual system, 135, 136, 137, 142, 143, 144, 151, 156, 178, 208 performance, 8, 9, 21, 23, 26, 30, 57, 61, 65, 71, 78, 101, 104, 105, 106, 107, 108, 110, 113, 114, 146, 147, 158, 159, 184, 186, 197, 209 performing practice, 101 persons, 3, 76, 84, 86, 130, 141, 143, 153, 155, 192, 195, 196 phenomenological experience, 161, 191, 193, 194, 202, 204 phenomenological reports, 85 phenomenology, 6, 33, 53, 99, 100, 131, 155, 156, 169, 179, 190, 193, 205 pheromone, 169 phonetic gesture, 4, 191, 192, 207

215

216

Subject Index

physical symbol, 12, 73, 74, 128, 175 pirates, 102 planning, 30, 70, 75 play, 4, 5, 37, 39, 46, 57, 63, 102, 103, 104, 105, 106, 111, 112, 113, 132, 145, 154, 164, 189, 197 play production, 102 polyhuman aggregate, 202 poverty of the stimulus, 173, 180 practical activities, 177 pragmatics, 12, 13, 52, 53, 78, 115, 135, 136, 137, 140, 144, 150, 152, 153, 154, 155, 157, 158, 180, 181, 183, 208, 209 presence, 52, 86, 121, 144, 165 principle of non-locality, 6 problem solving, 10, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 69, 76, 77, 78, 79, 80, 198 processing, 16, 17, 18, 19, 23, 24, 25, 26, 28, 31, 44, 59, 60, 61, 71, 75, 79, 80, 121, 122, 123, 124, 128, 130, 163, 186, 187, 198 prompting, 103, 108 public presentation of self, 5 R rate-dependent, 163, 180 reading, 1, 2, 7, 9, 10, 11, 13, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 38, 39, 40, 46, 47, 50, 51, 52, 54, 74, 78, 84, 93, 95, 96, 99 reading skills, 16, 31, 38 realizing values, 1, 8, 10, 112, 137, 145, 149, 150 reasoning, 39, 75, 78, 109, 131, 188, 189, 194, 195 recognize that not, 121 reference, 1, 37, 43, 46, 48, 50, 81, 92, 96, 97, 130, 174, 175, 176, 184, 185, 187, 188, 205

regulation, 177 reification, 101 relational stability, 190 remembered patterns, 122 remembering, 106, 108, 114, 194 representation, 58, 61, 71, 76, 99, 105, 112, 114, 117, 118, 121, 123, 125, 127, 150, 166, 180, 182, 207, 208 representational change theory, 60 reproducing language, 102 reproductive thinking, 59 result, 6, 8, 24, 25, 27, 29, 37, 38, 39, 41, 46, 49, 50, 51, 60, 76, 96, 102, 109, 146, 165, 169, 170, 172, 179, 194, 200, 203 rule, 103, 104, 135, 136, 137, 140, 141, 147, 148, 154, 171 rules, 8, 39, 60, 135, 136, 137, 138, 139, 142, 145, 148, 150, 154, 155, 162, 164, 165, 169, 170, 171, 172, 173, 176, 178, 179, 180, 181, 193, 205

self-consciousness, 126, 127, 152 self-organization, 14, 156, 161, 164, 167, 169, 170, 171, 172, 174, 178, 179, 182, 183, 202, 208 self-organizing, 141, 142, 149, 182, 183, 202 semantic meaning, 88, 92 semiotic ambiguity, 121 semiotic attunement, 83 semiotic cognition, 1, 8, 9, 14, 117, 128 semiotic exploration, 81, 82, 95 semiotic recursion, 127 semiotic system, 34, 185, 187, 188 semiotics, 9, 13, 100, 119 sense of self, 5, 121, 152 sense-making, 2, 5, 8, 11, 44, 81, 83, 88, 94, 97, 98, 192, 203, 208 sentence, 16, 17, 19, 22, 23, 27, 40, 41, 135, 136, 162 serendipity, 10, 57, 69 S shared agency, 204 scaffold, 58, 65, 179 shared resource, 198 scaffolding, 41, 44, 47, 48, 188, sign, 5, 9, 37, 39, 43, 46, 53, 92, 192, 206 94, 95, 123, 124, 126, 128, scale-free, 145 190, 201 second-hand perception, 201 sign types, 92, 94 second-order construct, 7, 76, signal, 8, 10, 20, 21, 86, 88, 92, 179, 197 93, 94, 95, 96 second-order cultural consilent reading, 28, 29 struct, 3, 34, 43, 73, 74, 192 simulation, 172, 180, 181, 193 second-order language, 14, situations, 5, 10, 48, 60, 118, 136, 140, 193, 209 120, 165, 168, 175, 180 second-order living system, 51 skill, 18, 110, 115, 143, 144, 146, self, 5, 13, 14, 44, 76, 112, 119, 147, 148, 199, 209 120, 121, 125, 126, 127, 131, social coordination, 82, 172, 132, 141, 142, 149, 152, 154, 174, 175, 177 156, 157, 158, 161, 164, 167, social dynamic, 112 social interaction, 39, 125, 170 169, 170, 171, 172, 174, 175, social meaning, 81, 82, 84, 89, 178, 179, 180, 182, 183, 187, 92, 94 195, 201, 202, 208



social meaning-making, 81 social reality, 7, 11, 195, 196 socially constructed affordances, 95 socially normative contextualization, 89 society, 4, 5, 35, 45, 49, 50, 51, 52, 100, 114, 115, 158, 159, 169, 170, 176, 179, 184, 202, 207, 208, 209 sociocultural norm, 191 space-time scales, 135, 136, 145, 151 speech, 2, 10, 12, 13, 14, 15, 16, 19, 21, 23, 24, 27, 28, 29, 30, 31, 33, 34, 35, 37, 38, 39, 41, 42, 46, 48, 51, 53, 73, 108, 132, 142, 156, 157, 158, 162, 175, 180, 182, 184, 186, 191, 193, 195, 202, 203, 207, 208 spontaneous gesture, 109, 110 stability, 189, 190, 191 statistical model, 179 stimulus, 16, 17, 24, 26, 86, 88, 89, 156, 173 structural dynamics, 42, 49, 51 structure determined system, 42, 49, 50 suffering, 135, 151, 152, 153, 155 superorganisms, 202, 203 symbol, 1, 2, 7, 8, 11, 30, 37, 39, 52, 53, 59, 73, 76, 77, 94, 99, 156, 161, 163, 164, 165, 166, 167, 169, 174, 175, 176, 179, 180, 181, 183, 187, 207 symbol processing, 1, 59

Subject Index

symbol-grounding problem, 37 symbolic message, 96 symbolic mode, 161, 165, 166 symbolic strategy, 92 symbolic structure, 161, 164, 166, 169, 178 system dynamics, 141 T technology, 8, 33, 38, 51, 78, 122, 125 text, 7, 8, 11, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 30, 33, 34, 35, 41, 44, 46, 49, 50, 51, 103, 104, 107, 108, 110, 111, 113, 115 text-world, 41, 51 thinking, 2, 4, 5, 10, 11, 13, 18, 30, 35, 41, 44, 51, 52, 54, 57, 58, 59, 62, 70, 72, 74, 75, 78, 79, 80, 82, 105, 115, 119, 124, 126, 138, 158, 163, 186, 192, 194, 208 third-order living system, 33, 35, 36, 45, 48, 51 time-scales, 2, 3, 4, 6, 161, 162, 164, 166, 167, 168, 169, 170, 172, 177, 178, 183, 193 tool-use model, 150 top-top interactions, 98 translation model, 73, 74

163, 164, 170, 171, 172, 177, 185, 201, 204 unfinalizability, 11 universal grammar, 170, 176 utterance-activity, 3, 4, 6, 7, 100 V values, 7, 8, 9, 11, 13, 33, 44, 45, 46, 61, 63, 71, 78, 135, 136, 137, 138, 139, 140, 141, 142, 144, 145, 148, 149, 150, 151, 154, 155, 157, 158, 195, 203, 204, 208 values realizing, 7, 9 values-pragmatics, 137 Van Heusden, 6, 8, 9, 12, 14, 117, 130, 131 verbal language, 84, 89, 98 verbal pattern, 1, 2, 3, 5, 7, 8, 9, 11, 15, 57, 58, 63, 72, 73, 74, 164, 198, 201 verbatim recall, 108, 114 vibrant, 147 virtual patterns, 76

W way-finding, 136, 143, 149 whole-bodied communication, 192 wording, 67 writing, 4, 8, 13, 30, 33, 34, 35, 37, 38, 39, 40, 41, 44, 46, 47, 48, 49, 51, 53, 54, 65, 76, 101, U 113, 115, 190, 193, 208 understanding, 5, 25, 27, 31, 33, writing system, 35 42, 43, 44, 48, 49, 84, 100, written language bias, 5, 10, 14, 127, 132, 142, 145, 150, 162, 34, 54, 101, 104, 115, 191, 208

217

Name Index A Altmann, G.T.M., 16 B Bakhtin, M.M., 5, 153,155 Bang, J.C., 35,195, 196, 205 Barbieri, M., 165 Belpaeme, T., 37, 74, 162, 172, 173 Bickhard, M.H., 130 Brighton, H. 162, 172, 173, 176, 180, 183 Bruner, J., 167 C Cangelosi, A., 172 Cariani, P., 164, 179 Changeux, J-P., 117 Chomsky, N., 170, 180, 186, 206 Chronicle, E.P., 60, 61 Clark, A., 7, 35, 37, 38, 39, 42, 48, 58, 62, 63, 65, 71, 72, 73,74,75,76, 117, 118, 150, 173, 175, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 204, 205, 206 Cook-Gumperz, J., 40 Cowley, S.J., 2, 3, 4, 5, 6, 7, 10, 12, 24, 27, 34, 37, 48, 58, 62, 63, 73, 76, 83, 101, 114, 136, 140, 154, 162, 167, 169, 173, 177, 179, 186, 191, 192, 193, 194, 195, 205 Csikszentmihalyi, M., 152 D Dartnall, T., 76

Deacon, T.W., 62, 152, 168, 170, 176 Dewey, J., 17 Donald, M. 8,9, 35, 37, 38, 62, 83, 124, 125, 127, 128, 199, 205 Dror, I.E., 38 Duncker, K., 58, 59 E Elman, J.L., , 74, 164 Evans, N., 130 F Favareau, D., 165, 202 Fioratou, E., 6, 7. 10, 58, 63, 64 Fowler, C.A., 4, 141, 162, 191, 192 Frith, C.D., 98 G Gallagher, S., 82, 85 Gibson, J.J., 12, 17, 118, 136, 137, 138, 139, 143, 191, 201 Giere, R.N., 3, 7, 190, 203 Goffman, E., 51 Goldin-Meadow, S., 109, 110 Goodnow, J.J., 154 H Halliday, M.A.K., 167, 174 Harnad, S., 5, 35, 37, 63, 169 Harris, R., 3, 49, 189, 204 Heidegger, M., 11 Hodges, B.H., 4, 7, 8, 9, 45, 62, 112, 135, 136, 137, 138, 139, 140, 141, 142, 144, 148, 149, 150, 151, 152, 153, 154, 155, 174, 176, 204, 206

Hofstadter, D., 127 Hollan, J., 3, 7, 10, 12, 185, 205 Hutchins, E., 1, 3, 35, 58, 62, 104, 105, 109, 169, 175, 185, 186, 187, 192 J James, W., 129 Järvilehto, T., 4, 7, 9, 10, 15, 17, 18, 24, 25, 28, 29, 58, 72, 74, 154, 199, 204 K Kagan, J., 153 Kelso, J.A.S., 6, 163, 164 Kendon, A., 110 Kirsh, D., 3, 7, 104, 109, 110, 185 Knoblich, G., 60, 61 Koffka, K., 58 Kravchenko, A., 3, 4, 6, 7, 9, 28, 33, 34, 37, 38, 42, 43, 46, 52, 63, 73, 137, 162, 190 L Levinson, S., 98, 140, 147, 148, 173 Linell, P., 15, 4,5, 34, 43, 44, 48, 101, 137, 191, 205 Lloyd, D., 200 Love, N., 3, 4, 6, 34, 43, 58, 63, 73, 74, 101, 114, 136, 137, 169, 179, 189, 190, 192, 204 M MacWhinney, B., 86, 164, 168, 171, 180 Maier, N.R.F., 59, 62

220

Name Index

Martin, J.E., 142 Maturana, H.R., 4, 34, 35, 42, 43, 52 McNeill, D., 109, 110, 192 Mead, G.H., 5, 152 Melser, D., 159 Menary, R., 34, 37, 41, 105 Metzinger, T. 127 Mithen, S., 125 Morris, C.W. 34, 119 N Newell, A., 58, 59, 60, 75 O Ochs, E., 177 Ohlsson, S., 60, 61 Olson, D.R., 39, 40 Ormerod, T.C., 60, 61, 62, 66, 67, 69, 70 P Pattee, H.H., 12, 163, 164, 165, 166, 169, 171, 172, 174, 175, 178, 179, 180 Peirce, C.S., 5, 12, 92, 94, 129

Port, R.F., 5, 63, 74, 162, 180, 191 Q Quantz, J.O., 16

T Thibault, P.J. , 2, 4, 43, 58, 63, 106, 136, 145, 151, 152, 154, 191, 192, 193, 203 Tomasello, M., 45, 62, 73, 167 Tribble, E. 7, 8, 9, 10, 104, 108, 194 Trevarthen, C., 3, 151, 162, 167, 173, 177 Tylen, K., 8, 10, 82, 83, 84, 86, 97

R Rączaszek-Leonardi, J., 6, 8, 10, 11, 162, 163, 164, 195, 203 Rayner, K., 16, 17, 18, 25, 28 Reed, E.S., 136, 141, 144, 152, 153 Ross, D., 3, 10, 12, 62, 63, 76, V 202 Van Heusden, B., 8, 9, 130 Rupert, R., 70, 71, 72, 75 Van Orden, G.C., 135, 145, Russell, B., 42 146, 147, 148, 149, 206 S Schegloff, E.A., 177 Simon, H.A., 58, 59, 64, 75 Sober, E. 202 Steels, L., 162, 172, 173, 176 Steffensen, S.V., 1, 2, 6, 7, 8, 9, 186, 187, 192, 193, 195, 203, 205, 206 Stjernfelt, 92, 95 Sutton, J., 104, 110, 197

W Weisberg, R. W., 59, 60 Wertheimer, M., 59 Whitehead, A.N., 6 Wilson, D.S. 202 Wittgenstein, L.W., 11, 195 Z Zhang, J., 58, 66, 76 Zlatev, J., 42, 83, 130