BILINGUAL SENTENCE PROCESSING
ADVANCES IN PSYCHOLOGY 134 Editor:
G. E. STELMACH
ELSEVIER A m s t e r d a m - L o n d o n - N e w Y o r k - O x f o r d - Paris - T o k y o Boston- San Diego - San Francisco - Singapore - Sydney
BILINGUAL SENTENCE PROCESSING
Edited by Roberto R. Heredia Department of Psychology Texas A &M International University Laredo, TX, U.S.A. Jeanette Altarriba Department of Psychology University at Albany State University of New York Albany, New York, U.S.A.
2002 ELSEVIER Amsterdam- London- New York- Oxford- Paris - TokyoBoston- San Diego - San Francisco- Singapore- Sydney
ELSEVIER SCIENCE B.V. Sara Burgerhartstraat 25 p.o. Box 211,1000 AE Amsterdam, The Netherlands 9 2002 Elsevier Science B.V. All rights reserved. This work is protected under copyright by Elsevier Science, and the following terms and conditions apply to its use: Photocopying Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for advertising or promotional purposes, resale, and all forms of document delivery. Special rates are available for educational institutions that wish to make photocopies for non-profit educational classroom use. Permissions may be sought directly from Elsevier Science Global Rights Department, PO Box 800, Oxford OX5 IDX, UK; phone: (+44) 1865 843830, fax: (+44) 1865 853333, e-mail:
[email protected]. You may also contact Global Rights directly through Elsevier's home page (http://www.elsevier.nl), by selecting 'Obtaining Permissions'. In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; phone: (+1) (978) 7508400, fax (+1) (978) 7504744, and in the UK through the Copyright Licensing Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London W 1P 0LP, UK; phone (+44) 207 6315555; fax: (+44) 207 6315500. Other countries may have a local reprographic rights agency for payments. Derivative works Tables of contents may be reproduced for internal circulation, but permission of Elsevier Science is required for external resale or distribution of such material. Permission of the Publisher is required for all other derivative works, including compilations and translations. Electronic Storage or Usage Permission of the publisher is required to store or use electronically any material contained in this work, including any chapter or part of a chapter. Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher. Address permissions requests to: Elsevier Science Global Rights Department, at the mail, fax and e-mail addresses noted above. Notice No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made.
First edition 2002 Library of Congress Cataloging in Publication Data A catalog record from the Library of Congress has been applied for.
ISBN: 0 444 50847 3 ISSN: 0166-4115 (Series) The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper). Printed in The Netherlands.
Acknowledgements We would like to thank the many people who helped us directly and indirectly in the completion of this book. First, we thank Fiona Barron of Elsevier for catching the vision of this unique work. Also, we would like to express our gratitude to Greg B. Simpson for providing a thoughtful and positive Preface to the current work. Our thanks to the contributors themselves who were cooperative in meeting our sometimes ambitious deadlines. I, Roberto, would like to acknowledge the Center for Research in Language (CRL), The Cognitive Science Department, and The Center for Information Processing at the University of Califomia, San Diego for sparking my interest in this very important field of bilingual sentence processing. I thank my excellent friend and colleague Jeffrey M. Brown for his unconditional support and his insightful comments and suggestions on every chapter of this volume. We deeply appreciate those dedicated students at Texas A&M International University for their support and assistance in the preparation of this book. Among them, we particularly thank Liza Michelle Garcia, Gretel Arredondo, Selma Nanette Maldonado, Patricia Gutierrez, Florentina Aguallo, Graciela Yafiez and Erika Rodriguez. I, Jeanette, feel grateful that the colleagues who contributed to this book acknowledge the importance of the topic in the field of bilingual research and continue to serve as role models for other scientists in this area. Finally, we wish to acknowledge our friends who are too numerous to name and our wonderful families. It is their love and support that always motivates and encourages us. Este libro est~i dedicado para todos ustedes.
Con Carifio, Roberto R. Heredia and Jeanette Altarriba
This Page Intentionally Left Blank
Preface
The presentation of a volume on bilingual sentence processing is timely (indeed, it seems long overdue) for several reasons. Perhaps the most obvious of these, though not necessarily what might first come to mind, is the recognition of the prevalence of multilingualism around the world. Except in the United States, bilingualism generally is the rule rather than the exception. Even in the United States, which has come slowly to this recognition, greater attention than ever before is being paid to bilingualism. This interest is reflected in studies of second-language acquisition and pedagogy. As it becomes clearer that multilingualism is increasingly important in a shrinking world, greater understanding of the processes of language learning and teaching becomes critical. Interest in bilingualism is reflected as well in the growing body of literature, exemplified by the present volume, concerning cognitive processes in bilingual speakers. Researchers and educators interested in issues of bilingualism will find the present treatment to be comprehensive in its consideration of language processes in bilingual speakers, from the representation of individual words in two languages, to the processing of figurative language. For many years, research in this area was dominated by questions regarding the representation in memory of lexical information in two languages. One can see from the chapters here that research and theory have progressed far beyond the word level. The topics span the entire range of issues commonly found in psycholinguistics texts, including syntactic and semantic processing, memory representation, and language acquisition. The present book, therefore, provides for the first time a comprehensive treatment of issues of bilingualism and their implications for issues of language processes in general, and for sentence processing, in particular. After consideration of methodological issues in the study of bilingual language processing, two connectionist models are presented. Like other models of this type, these two go far beyond consideration of a single level of representation or processing, and should prove fruitful for researchers studying interactions among processing domains by bilinguals. The third section of the book, on Memory Representation in Sentence Processing, represents what is perhaps the most traditional area in bilingual research. The chapters here, however, again exceed the questions of word representation that have been dominant in the past, and also introduce new procedures for researchers in bilingualism (such as the repetition-blindness paradigm). The use of context in the recognition of words in two languages is a well-traveled ground, but here these studies involve consideration of cross-language homographs and of context greater than the single word or sentence. In Psycholinguistic Theory and Research, the implications for issues outside bilingualism itself are perhaps the clearest. Hypotheses concerning, for example, parsing strategies are usually based on assumptions of universal aspects of language and language processing. Studies of bilingual speakers are therefore critical to tests of these hypotheses of syntactic processes. Similarly, the studies of figurative
viii
Preface
language that are reported here tell us as much about figurative language as they tell us about bilingual speakers, dealing with issues such as the processing of syntactic ambiguity in phrasal verbs, and visual imagery and idiom processing. Finally, the acquisition of two languages in children is given its proper due. Indeed, no topic could be more central to bilingualism. Here again, however, the discussion is not restricted to issues concerning characteristics of the learner and learning environment. Written language comprehension and production have not been widely considered in the past, and these chapters show how much research in bilingual language acquisition has progressed in recent years. Partly because of the promise of its educational applications, bilingualism is a legitimate field of study in its own right. This fact might be enough to capture the passion of psycholinguists generally, but alas, to date it seems not to have done so. What could capture this passion is the recognition that the bilingual language processor might offer a unique medium for understanding fundamental language processes that go beyond bilingualism itself. There is a disappointing tendency, certainly among many American researchers, to consider studies conducted in any language other than English to be about that language. Their own research, conducted in English, is naturally assumed to be concemed with the architecture of language processing in general. Similarly, research on bilingual processing may be neglected because it is perceived to shed light only on the issue of bilingualism per se. Obviously, the present authors are not guilty of such provincialism. Ultimately, promoting the recognition of the relevance of the issues discussed here in the context of the bilingual speaker to broader issues of language processing may be one of this volume's greatest contributions to the field.
Greg B. Simpson University of Kansas October 28,2001
Table of Contents
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
v
Preface Greg B. Simpson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vii
Introduction and Overview Jeanette Altarriba and Roberto R. Heredia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
Part I. Methods in Bilingual Research Chapter 1. On-Line Methods in Bilingual Spoken Language Research Roberto R. Heredia and Mark T. Stewart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
Part II. Connectionist Models of Second Language Processing and Bilingualism Chapter 2. Extending the Competition Model Brian MacWhinney . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
Chapter 3. A Self-Organizing Connectionist Model of Bilingual Processing Ping Li and Igor Farkas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
Part III. Memory Representation in Sentence Processing C h a p t e r 4. Cross-Language Facilitation, Repetition Blindness, and the Relation Between Language and Memory: Replications of Altarriba and Soltano (1996) and Support for a New Theory Don G. MacKay, Lori E. James and Lise Abrams . . . . . . . . . . . . . . . . . . . . . . . 89
Chapter 5. The Use of Sentence Contexts in Reading, Memory, and Semantic Disambiguation Jeanette Altarriba and Jennifer L. Gianico . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Chapter 6. Exploring Language Asymmetries in Early Spanish-English Bilinguals: The Role of Lexical and Sentential Context Effects Arturo E. Hem~ndez . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
137
Chapter 7. Text Comprehension in Bilinguals: Integrating Perspectives on Language Representation and Text Processing Gary E. Raney, Sharon M. Obeidallah and Timothy K. Miura . . . . . . . . . . . .
165
Part IV. Psycholinguistic Theory and Research Chapter 8. Relative Clause Attachment in Bilinguals and Monolinguals Eva M. Fem~indez . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
187
Chapter 9. An On-Line Look at Sentence Processing in the Second Language Cheryl Frenck-Mestre . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Chapter 10. Cross-Linguistic Aspects of Anaphor Resolution Dieter Hillert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
237
Part V. Figurative Language Processing Chapter 11. Understanding Phrasal Verbs in Monolinguals and Bilinguals Teenie Matlock and Roberto R. Heredia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Chapter 12. What Native and Non-Native Speakers' Images for Idioms Tell Us About Figurative Language Heather Bortfeld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Part VI. Language Skill Development in Bilingual Children Chapter 13. The Role of Formal Definitions in Reading Comprehension of Bilingual Students Aydin Y. Durguno~lu, Zehra F. Peynircio~,lu and Montserrat Mir . . . . . . . . . 299 Chapter 14. Syntactic Structure, Grammatical Accuracy, and Content in Second-Language Writing: An Analysis of Skill Learning and On-Line Processing Wendy S. Francis, Laura F. Romo and Rochel Gelman . . . . . . . . . . . . . . . . .
317
Chapter 15. Code Switching in Preschool Bilingual Children Zehra Peynircio~,lu and Aydm Y. Durguno~lu . . . . . . . . . . . . . . . . . . . . . . . .
339
Authors' Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
359
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
363
Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
373
Introduction and Overview
Bilingual research has increased a great deal in the past ten years. Within the field of psycholinguistics, bilingual research has focused on a broad range of topics frtSm bilingual memory representation to language processing issues such as the nature of word representation, bilingual word recognition, and reading. Because of the universality of bilingualism, being much more the rule than the exception, interest in the field of bilingual sentence processing is growing significantly. Indeed, the study of bilingualism attracts a broad range of individuals from disciplines such as linguistics, psycholinguistics, cognitive science, communication, and artificial intelligence, just to name a few. A comprehensive volume such as the current one was intended to keep students and researchers interested in bilingual sentence processing issues and motivate them to pursue investigations into various related areas in bilingualism. To date, there are no other books or specialized texts devoted exclusively to the topic of bilingual sentence processing. The aim of the current work was to fill this void in the literature by collecting works that emphasize theoretical issues and the presentation of empirical findings or evidence in support of different theoretical positions. Research from both the visual and spoken modalities was included along with a comprehensive selection of paradigms and methods that have been applied in the cross-linguistic domain. A volume such as this one provides both the beginning researcher and the seasoned researcher with the latest on bilingual investigations. It serves as a technical handbook in the field as well as a tutorial reader that can provide a starting point for almost any direction in the bilingual language domain. We would be remiss if we did not cite some of the earlier books published on bilingual research as they helped to shape the direction and focus of the work reported in the current volume. Books such as Language Processing in Bilinguals edited by Vaid (1986), Cognitive Processing in Bilinguals edited by Harris (1992), and The Bilingual Lexicon, edited by Schreuder and Weltens (1993) introduced researchers and theoreticians to the basic issues and debates that structured the framework for exploring bilingual language issues that has been expanded upon in recent years. Most of these works contained papers that focused on the word level of language representation and its various attributes such as orthography, phonology, semantics, pragmatics and the like. They laid the groundwork for the investigations that are presented in the current work, as issues related to the processing of words in sentential contexts were informed by the previously published investigations. The central focus of Bilingual Sentence Processing was the presentation of an overview of the literature on bilingual sentence processing from a psycholinguistic and linguistic perspective. To do this, noted researchers in the field of bilingual language processing agreed to contribute works ranging from the development of connectionist models, to the representation of cross-linguistic figurative language, to investigations of reading and comprehension in bilingual children and adults. The volume is divided into six sections each describing work that can be viewed as a
2
Jeanette Altarriba and Roberto R. Heredia
major theme within the current thinking in the field. The work opens with a discussion of the methods used in bilingual research. This work provides an overview of the various on-line tasks that have been used to investigate language processing in bilinguals at the sentence level. The second section begins with the fundamental issue of mental representation and the nature of the structure of the two languages in memory, with the presentation and description of two connectionist models of bilingual language processing. Next, the interplay between language contexts, memory, and reading is explored by examining effects such as repetition blindness, lexical ambiguity, and sentence constraint. The fourth section has its strengths in the presentation of linguistic theory and research and examines those questions that surround the syntactic parsing and strategic processing that occurs specifically within sentence contexts. The nuts and bolts of the linguistics surrounding sentence comprehension are included here. A fifth grouping of chapters emphasizes an area that has often been overlooked in previous volumes on language processing--the representation and use of figurative language (i.e., metaphors, idioms and the like). This novel inclusion should draw the attention of individuals who are interested in the broader questions of language acquisition, the implicitness of language representation, and the interpretation of inferences and linguistic implicatures. They make for interesting reading, as well, as the examples provided should alert the reader to the fact that figurative language exists across all languages and is often an interesting link to the representation of thought within a particular language group or culture. Last but not least, a growing interest in second language acquisition prompted the inclusion of the final section of the current volume on the development of language skills in bilingual children. Questions regarding the representation of vocabulary in the two mental lexicons of bilinguals, to the extraction of semantic meaning from stories and narratives, to the implications of code-switching in verbal descriptions was included, rounding out a complete selection of topics in the area of sentence processing. Several debates abound in the area of bilingual sentence research. Quite notable is the issue regarding on-line versus off-line processing. Measures that record behavior in an on-line fashion are aimed towards examining language processing as it o c c u r s in a natural state of reading and listening, and word identification (cf. Chapters 1 & 6). Tasks such as those recorded by eyetracking systems and selfpaced reading tasks that may involve moving windows are an example of on-line procedures (cf. Chapters 5, 9, & 11). Other studies, particularly those interested in the processing occurring after encoded material is processed and integrated may rely on measures that are considered "off-line." Work in recall of words from mixed-language presentations or the measure of semantic comprehension after material is read are examples of this type of processing (cf. Chapters 4 & 7). It is clear that while some information can be gained within each paradigm, the ultimate decision as to which to employ should depend on the research question under investigation. Would one want to capture processes that are pre-lexical or those that occur early in the stages of language processing, or, would issues regarding memory retrieval and the time course of forgetting or interference in memory be of interest? Both types of approaches have value and research should continue to focus on existing tasks and the development of new tasks in both domains. Another concern seems to reside with the factors implicit in language learning that contribute to the
Introduction and Overview
3
mental representations of a bilingual's lexicons The relative significance of age of acquisition is not well understood and is often either overlooked or squarely emphasized as an important source of variance in language learning (see e.g., Chapter 2 vs. Chapter 9). Another intriguing focus that currently motivates quite a bit of research in the bilingual domain centers on the question of whether or not language use is selective or non-selective. That is, can an individual functionally process information in one language without interference from a second language? Work involving the Stroop paradigm in which words that name colors in a given language are presented in varying ink colors, indicates that languages may not be entirely separable when processing. The processing of ambiguity within and between languages also addresses the question as to the separateness of multiple languages (cf. Chapters 1, 5, 9, & 12). Finally, as in the monolingual domain, questions regarding the nature of the architecture of the mental lexicon(s) in the brain has led to the development of models of representation that are neurally inspired. The current volume includes two such models that challenge the current capacity models of information processing across languages by describing both localist and distributed properties of language. These connectionist models not only describe representation but also describe the processes involved in language learning. They can aptly predict findings for both priming effects and effects of interference across languages. Despite the varied approaches and abundance of data described in the current volume, the need ultimately remains to provide evidence regarding bilingual sentence processing in cases in which the exact same stimuli are used across a variety of experimental paradigms. Likewise, an investigation into the relative similarities and differences in processing as a function of languages from different historical origins is in order. Despite these issues in the field, a strength in the current volume lies in the fact that sentence processing has been reviewed at various levels from the most basic and theoretical to applications within education and specifically, within the classroom. Whereas researchers may typically focus on the level of theoretical constructs and their empirical counterparts, the current book also offers ways in which this work can be applied to a most pressing and timely topic-the acquisition of a new language. In this way, we hope this volume becomes a type of "handbook" for researchers wanting to know how to conduct work that will have practical implications as well as answer theoretical queries of general interest. Finally, we hope that this work will serve as a springboard for new research that moves this area of intrigue ahead by having summed up what we know up until the present.
References
Harris, R. J. (Ed.). (1992). Cognitive processing in bilinguals. Amsterdam: Elsevier Science Publishers. Schreuder, R., & Weltens, B. (Eds.) (1993). The bilingual lexicon. Philadelphia, PA: John Benjamins. Vaid, J. (Eds.) (1986). Language processing in bilinguals: Psycholinguistic and neuropsychological approaches. Hillsdale, NJ: Lawrence Erlbaum Associates.
This Page Intentionally Left Blank
Part I: Methods in Bilingual Research
This Page Intentionally Left Blank
Bilingual Sentence Processing- R.R. Heredia and J. Altarriba (Editors) 9 2002 Elsevier Science B.V. All rights reserved.
On-Line Methods in Bilingual Spoken Language Research Roberto R. Heredia Texas A&M International University Mark T. Stewart Willamette University
Abstract
In this chapter, we review several methodological approaches to studying spoken language comprehension in bilinguals. In particular, we focus on those tasks that allow for closer scrutiny of sentence level processing, including gating (Grosjean, 1980, 1996), cued shadowing (Bates & Liu, 1996), crossmodal lexical priming (Swinney, 1979), and the auditory moving-window (Ferreira, Henderson, Anes, Weeks, & McFarlane, 1996). Throughout the chapter, we offer suggestions as to how these techniques might be used to investigate a host of issues important to researchers in bilingualism, including but not limited to grammatical priming, ambiguity resolution, and contextual priming. We highlight the potential strengths and weaknesses of each paradigm and, whenever possible, offer suggestions as to future work that might be conducted using a particular experimental procedure. In the end, our hope is that the reader will come away with a stronger sense of the sorts of techniques that are available and are being used by researchers in the field of bilingual sentence processing. While research at the word level, and primarily in the visual modality, has examined the question of whether bilinguals organize their two languages into one or two memory systems (e.g., Durguno~lu & Roediger, 1987; Glanzer & Duarte, 1971; MacNamara & Kushnir, 1971), and most recently, the organization of the bilingual lexicon (e.g., de Groot, Dannenburg, & van Hell, 1994; Heredia, 1997; Kroll & Stewart, 1994), considerably less is known about how bilinguals process connected speech, both at the sentential and discourse levels, during the communicative process. This state of affairs is interesting given how much we rely on our auditory sense for our linguistic information in our everyday communication in our first and second languages (cf. Ferreira, Henderson et al., 1996). More important, however, is the fact that in most bilingual communities, communication between bilinguals is largely spoken, with comparatively little at the written level. For example, it is not unusual for Spanish-English bilinguals in the Southwest of the United States to show high language proficiency levels in understanding spoken language and in their ability to speak their two languages. However, these same bilinguals show limited ability in their written and
8
Roberto R. Heredia and Mark T. Stewart
reading skills (e.g., Heredia & Altarriba, 2001; cf. Hem~indez,/~vila, & Bates, 1996; see also Favreau & Segalowitz, 1982, 1983; Segalowitz, 1986). Moreover, it could be argued that certain linguistic forms of bilingualism occur at the spoken language modality and not necessarily at the written language level (e.g., Ferreira, Henderson et al., 1996). For instance, code switching or language mixing in which bilinguals substitute a word or a phrase in one language for a word or phrase in a second language (Li, 1996a) is mainly a spoken language phenomenon. This is not to say, that code switching can only be studied in the spoken language modality because there are impressive research findings that have used reading to study this phenomenon (e.g., Altarriba, Kroll, Sholl, & Rayner, 1996; see also Peynircio~lu & Durguno~lu, this volume). In this chapter, we review some of the main methodologies in the existing bilingual literature that have taken the sentence or discourse level information as their focus to examine bilingual language processing and semantic memory issues. Our discussion is limited to only those studies that have used the auditory modality (for an overview of other tasks using spoken language see, Guillelmon & Grosjean, 2001; Grosjean & Frauenfelder, 1996). We conclude by describing two research techniques that are currently being explored in our laboratories to study issues related to bilingual lexical access and the phenomenon of code switching. We now turn to our discussion of the various on-line psycholinguistic tasks used in examining bilingual sentence processing.
The Sentence Interpretation Task One of the first paradigms to be used to investigate bilingual sentence processing was the sentence interpretation task (e.g., Hem~indez, Bates, & Avila, 1994; Kilbom, 1987, 1989; Li, Bates, Liu, & MacWhinney, 1992; MacWhinney, 1987; MacWhinney & Bates, 1989; McDonald, 1987a; McDonald & Heilenman, 1991, 1992;Vaid & Pandit, 1991). In this task, participants listen to a sentence such as (1 a) below and are asked to report aloud as quickly and as accurately as possible who or what in the sentence is doing the action (Kilborn, 1989). Thus, in the example below, the correct verbal response would be dog. One variation has been to ask participants to make a timed choice between a pair of pictures (e.g., dog vs. cup) (Hern~indez et al., 1994). (1 a) The dog is chasing the cups This particular version of the task has been used predominantly in cross-linguistic studies (e.g., Bates, Devescovi, & Wulfeck, 2001; MacWhinney & Bates, 1989) seeking to understand how speakers utilize the various sources of information provided by a particular language, and how speakers incorporate and use these sources of information during the language comprehension process. For example, sentence (la) above provides important sources of information that the English speaker must consider during sentence processing. These sources of information are, (a) the position of the first noun that is doing the action (e.g., dog); (b) the agreement between subject and verb in person and number; and (c) a contrast in animacy between the subject dog,
On-line Methods in Bilingual Spoken Language
9
which is animate and the object cups, which is inanimate (Li et al., 1992). How these sources of information are utilized depends on the particular language. Languages such as English rely more on word order (e.g., SVO: subject-verb-object), than other languages such as Spanish that rely mainly on morphological information such as noun and verb agreement and less on word order. For instance, sentence (2a) is grammatically correct in English because it follows the SVO word order. Sentence (2b) on the other hand, is grammatically incorrect because it does not follow the English SVO word order, and without the subject/, it is unclear who wants the television. In Spanish, on the other hand, sentence (2b) is correct because the verb carries information about the subject. That is, in Spanish the subject is optional and word order is not restricted to SVO. (Please note that grammatically incorrect sentences are indicated by double asterisks "**"). (2a) I want the television (Yo quiero el televisor) (2b) **Want the television (Quiero el televisor) Hern~indez 9 et al. (1994) used a version of this task to examine sentence interpretation strategies in Spanish-English bilinguals compared to Spanish and English monolinguals. The question of interest was whether Spanish-English bilinguals would behave like English monolinguals during the comprehension of English sentences, or like Spanish monolinguals during the comprehension of Spanish sentences. SpanishEnglish bilinguals received both Spanish and English versions of sentences such as (3a3d) below that experimentally manipulated different variations of word order (e.g., NVN: noun-verb-noun; VNN: verb-noun-noun, and NNV: noun-noun-verb), agreement between noun and verb (sentences 3a-3d), and animacy, where the first noun was animate (e.g., 3a, 3b, and 3c) or inanimate (e.g., 3d), and the second noun was animate (e.g., 3a and 3d) or inanimate (e.g., 3b and 3c). (3a) The dog is chasing the cow (both nouns agree (3b) **The dog are chasing the cups (second noun (3c) The dog is chasing the cups (first noun agrees (3d) **The cup are chasing the dogs (second noun
with verb) agrees with the verb) with verb) agrees with the verb)
Spanish and English monolinguals, on the other hand, received the sentences in their respective language only. Participants were instructed to press a button corresponding to the side on which a picture of the noun appeared (e.g., for sentence 3a, a picture of a dog or a cow). In general, Hern~indez et al. (1994) found that English monolinguals were faster in sentence interpretation when sentences conformed to the SVO or NVN word order, followed by verb agreement and animacy. In other words, English monolinguals were faster to decide who was doing the action when the sentences followed the English SVO canonical word order. For the Spanish monolinguals, on the other hand, noun verb agreement produced the fastest sentence interpretation reaction times, followed by animacy and word order. More important, a comparison between bilinguals and English monolinguals showed remarkable similarities. B ilinguals, like English monolinguals were very sensitive to word order,
10
Roberto R. Heredia and Mark T. Stewart
and somewhat sensitive to verb agreement. A comparison between bilinguals and Spanish monolinguals showed that both groups were highly sensitive to verb agreement. However, unlike Spanish monolinguals, bilinguals were affected by word order. That is, bilinguals were faster in sentence interpretation than Spanish monolinguals when the sentences followed the SVO word order. In short, SpanishEnglish bilinguals and English monolinguals exhibited similar sentence processing strategies during the comprehension of English sentences. Both groups, in this case, were sensitive to word order and verb agreement, respectively. Although bilinguals showed similar processing strategies as Spanish monolinguals during the comprehension of Spanish sentences, unlike monolinguals, bilinguals were actually more sensitive to word order effects as opposed to animacy. These results suggest that at least for the bilinguals in this experiment, the second language (L2) influenced the processing of the first language (L1), in this case, the comprehension of Spanish sentences (see Liu, Bates, & Li, 1992 for similar results). This finding contrasts with Kilborn's (1989) results in which the participant's first language (German) influenced the processing strategies of the second language (English). This difference, we believe, may be due to language dominance. It is possible that Kilborn's (1989) bilinguals were more dominant in their first language and their sentence processing strategies reflected this dominance. In contrast, the bilinguals in Hern~indez et al.'s experiment were more dominant in their second language (cf. Heredia, 1997; Heredia & Altarriba, 2001), and this dominance is reflected by the influence of the second language in the processing of Spanish sentences. One important aspect to note here is that the bilinguals described here were to some extent more English-monolingual-like, than Spanish-monolingual-like in their sentence processing strategies. In addition to demonstrating the reliability of this task (for a review see Bates, Devescovi, & Wulfeck, 2001), this study underscores the importance of examining the interaction of the bilingual's two languages during the comprehension of language. The results reviewed here, in conjunction with findings from other bilingual language groups such as German-English (Kilborn, 1989), Chinese-English and English-Chinese (Liu et al., 1992), Hindi-English (Vaid & Pandit, 1991), Dutch-English and EnglishDutch (McDonald, 1987b), and English-French (McDonald & Heilenman, 1991) suggest that language processing by bilinguals can be influenced by the structural constraint of one or both of their languages, depending on such factors as language proficiency and language dominance. In relation to bilingual sentence processing, the usefulness of this task lies in its reliance on spoken language. As shown by Liu et al. and others (e.g., Kilborn, 1989) this task can be useful in studying the development of bilingualism and the various sentence processing strategies utilized throughout the second-language learning process. More important, this task can be used to study second-language developmental issues with children (A. E. Hern~indez, personal communication, May 12,2001; see also McDonald, 1986). Furthermore, this task may be suitable to examine the effects of gender in bilingual language processing (cf. Sera, Berge, & del Castillo Pintado, 1994). For example, in Spanish (and other Romance Languages) all objects are categorized in terms of "masculine" or "feminine." For instance, the word for flower in Spanish Oqor) is categorized as "female." Thus, a
On-line Methods in Bilingual Spoken Language
11
question of interest would be to examine whether or not Spanish-English bilinguals assign or transfer gender from Spanish to English objects. Anecdotal evidence suggests that when bilinguals code-switch between Spanish and English, they engage in this type of categorization. For example, in sentence (4a) below, peanuts take the masculine unos because cacahuates or manis are masculine in Spanish. In contrast, in sentence (4b) peanut butter takes the feminine la because crema de cacahuate is feminine for Spanish speakers. A possible experiment might involve English sentences such as I want peanuts in which participants determine if the noun peanuts is masculine or feminine (cf. Guillelmon & Grosjean, 2001). (4a) Quiero unos peanuts (I would like peanuts) (4b) Quiero la peanut butter (I want peanut butter) One prediction would be that if gender transfer were to take place, Spanish bilinguals would be faster (in milliseconds) deciding or judging that peanuts are masculine rather than feminine. Notice that it would also be possible to analyze responses in terms of choice data percentages (whether participants choose "masculine" or "feminine"). Of interest in this experiment would be a potential statistical interaction between gender and animacy, where things that are animate would be more likely to be given conceptual gender during language transfer.
The Phoneme-Triggered Lexicai Decision Task
The next methodology we consider is the phoneme triggered lexical decision task (PTLD). The PTLD has been used to study lexical access and semantic facilitation during sentence processing. This task is a combination of phoneme-monitoring (e.g., Connine & Titone, 1996; Foss, 1970) and lexical decision. In this task, listeners make a word or nonword judgment for a particular target occurring within the sentence. The target word begins with the sound specified for each particular sentence. The phoneme is used to trigger the lexical decision. For example, participants listening to sentence (5a) below may be asked to listen for a word or nonword that begins with the phoneme /k/. Once that target is identified, participants decide as fast and as accurately as possible whether the target is a word or a nonword. In this case, the participant identifying the phoneme/k/for the target word cake in sentence (5a) would respond that it is a word. Target phonemes for each sentence may be provided auditorily before the presentation of each sentence (Blank, 1980) or on a sheet of paper specifying the target phoneme and the sentence number. Response times are measured from the onset of the target phoneme until a response is recorded.
(5a) After lunch, the children asked for a piece of CAKE for dessert (5b) Depois do almogo os miudos pediram uma fatia de BOLO para sobremesa (5c) Depois do LUNCH os miudos pediram uma fatia de CAKE para DESSERT
12
Roberto R. Heredia and Mark T. Stewart
Soares and Grosjean (1984) utilized this task to examine lexical access between Portuguese-English bilinguals in comprehending Portuguese and English sentences, and English monolinguals listening to English sentences. Their objective was to address findings by Kolers (1966) and Macnamara and Kushnir (1971) suggesting that bilinguals were significantly slower to retrieve information from mixed-language sentences such as the Portuguese-English code-switched sentence in (5c) as opposed to monolingual sentences in English (5a) and Portuguese (5b). At issue was whether bilinguals would be faster at making lexical decisions to sentences (5a-5b) than to (5c). Another objective was to determine differences (if any) between English monolinguals and Portuguese-English bilinguals listening to English sentences. The results showed that when both monolinguals and bilinguals listened to English sentences, lexical decisions to real word targets were comparable for both groups. However, bilinguals were slower to respond to nonwords. When bilinguals listened to English and Portuguese monolingual sentences (5a and 5b) and code-switched sentences (5c), results showed no differences in the two monolingual sentence conditions. However, lexical decision times to word targets in the code-switched condition were slower, compared to both monolingual conditions. Lexical decisions by bilinguals for nonwords in the three language conditions were identical. In sum, Soares and Grosjean's results replicated the general finding that bilinguals are typically slower at retrieving information from one language, especially when this information is embedded in a second language, than accessing information in only one language (e.g., Kolers, 1966; Macnamara & Kushnir, 1971; see also Altarriba et al., 1996; Heredia & Altarriba, 2001). Their results also show that lexical access between monolinguals and bilinguals during sentence processing was quite similar. Lexical decisions to real words in English, while listening to English sentences, were identical for both groups. However, Portuguese-English bilinguals took longer to respond to English nonword targets, while listening to English monolingual sentences. As argued by Soares and Grosjean, it is possible that bilinguals took longer to respond to nonword targets because they had to search two mental lexicons instead of one. Thus, if in the initial search, while listening to English sentences and searching the English lexicon the target is not encountered, the bilingual defaults to the other language lexicons, in this case Portuguese, to identify the target. Although, this is a theoretically viable explanation, it is not clear as to why this same difference is not reflected in the real word data. Alternatively, this difference in the nonword data could be due to the manner in which the nonword targets were created and possible task demands. Perhaps, bilinguals were quick to respond to real words because these targets were clearly marked as belonging to English (cf. Grosjean, 1988; Li, 1996a). Nonwords, on the other hand, may have been too phonologically similar to English and Portuguese, thus making it difficult to decide whether the nonword was more like English or Portuguese, or both. This ambiguity in conjunction with the sensitivity of the PTLD to the sound unit (i.e., phoneme) may have caused the bilinguals to activate their second language lexicon. Indeed, this methodological issue cannot be rejected as a possible explanation. Overall, the PTLD appears sensitive to differences between lexical access in the monolingual (e.g., English or Portuguese) and bilingual (code-switched) mode. Given its reliance on the sound unit to trigger the lexical search during sentence
On-line Methods in Bilingual Spoken Language
13
comprehension, this task is ideal for studying issues related to how bilinguals process homophones (i.e., words that share phonological similarities, but whose meaning is different) across languages (cf. Li & Yip, 1998). For example, how do Spanish-English bilinguals process ambiguous sentences such as (6a-6b)? The English target word YELLOW is a homophone of the Spanish word HIELO which is related to ice. What are the effects of context and the interpretation of cross-language homophones? To what extent does language in conjunction with semantic context determine bilingual lexical search (6b)? That is, during the comprehension of sentence (6a), are bilinguals more likely to engage in exhaustive search in which both the English and Spanish lexicons become simultaneously activated? Is the preceding context of sentence (6b) more likely to re-direct the lexical search to the English lexicon only? In this example, we are assuming that participants would be in a bilingual mode. i (6a) She liked YELLOW because it was a pleasant color (6b) She liked [blue more than] YELLOW because it was a pleasant color An important aspect of this task is that it could be also employed simply as a phoneme-monitoring task (Connine & Titone, 1996) in which bilinguals would be asked to respond when they find the/y/sound. However, researchers utilizing this task should pay careful attention to the possibility that this task may be too sensitive to phonotactic structural factors (i.e., sound pattems) of a particular language (see discussion on the gating and cued shadowing sections for more information about the interaction of sound patterns and language). Clearly, empirical studies utilizing this task, both at the monolingual and bilingual domains are limited. More research is needed to determine the usefulness of this task in the field of bilingual research.
The Cross-Modal Naming Task
A task similar to the PTLD is the cross-modal naming task, used by Hern~indez et al. (1996, and this volume) and Hern~indez, Fennema-Notestine, and Udell (in press). It has been argued that this task is sensitive to both sentential and lexical priming. Participants in this task are presented with an auditory sentence such as (7a). At a predetermined point within the sentence, the sentence stops momentarily and a visual target appears in the middle of the computer screen. The participants' objective is.to name, as quickly and accurately as possible, the visually presented related or unrelated target word. For example, in sentence (7a) and (7b) the sentence stops at the offset of Actually, given the appropriate comparisons in which both English and Spanish sentences are utilized, we would predict little if any activation (as measured by lexical priming) of the Spanish meaning (e.g., ice) while listening to English sentences (e.g., yellow), and activation of the English meaning (e.g., color) while listening to Spanish sentences, in situations in which English is the bilingual's dominant language regardless of which language was learned first (for further details about the importance of language dominance see Heredia & Altarriba, 2001).
14
Roberto R. Heredia and Mark T. Stewart
the definite article the (depicted by "[**1]"), and the participant names the visual English street or Spanish calle target. Note that it would be possible to have participants do lexical decisions on the visually presented target. (7a) John was walking down thet**~1STREET when he remembered his appointment (7b) John was walking down thet**~1CALLE when he remembered his appointment (8a) Juan caminaba por lat,,~ 1CALLE cuando record6 su cita (8b) Juan caminaba por Iat,,~ISTREET cuando record6 su cita Hem~indez et al. (1996) investigated the effects of expectancy and predictability in cross-language priming. In cross-language priming, naming a word in an L2 (peace) is faster when preceded by a related word in the L 1 (guerra: war), than an unrelated word in the L1 (boteHa: bottle). Similarly, it is faster to name a word in the L 1 (guerra) when followed by a related word in the L2 (war) than an unrelated word in the L2 (bottle) (Altarriba, 1992). Hem~indez et al's. purpose was to determine the optimal conditions in which cross-language priming would be more likely to occur. Accordingly, during the comprehension of a sentence, for cross-language priming to occur, the bilingual must anticipate that a word in the second language will be presented in order to tune themselves into a "bilingual mode" and be able to access the lexical item in the second language (cf. Amrhein, 1999). Because cross-language priming may be strategic in nature, ifbilinguals are unable to tune themselves into this mode, cross-language priming would not occur (Hem~indez et al., 1996). Hern~indez et al. had Spanish-English bilinguals listen to sentences in English (7a) and Spanish (8a) and name visually presented words in both languages. For the English-Spanish bilingual condition, participants listened to sentences such as (7b) and then named a Spanish visual target. Participants in the Spanish-English condition listened to sentences such as (8b) and then named the visually presented target in English. All language conditions were either blocked (the predictable condition) or mixed (the unpredictable condition). To summarize, cross-language priming was obtained, but only when language presentation was blocked. That is, sentences (7b) and (8b) produced priming only when bilinguals were able to predict that a language switch would occur. When these bilingual sentences were presented in a mixed design, naming a related word in the second language, was no different than naming an unrelated target. However, when a delayed naming task was used, cross-language priming was obtained for both bilingual conditions in both blocked and mixed experimental conditions. However, sentences (7a, English sentence and English probe) and (8a, Spanish sentence and Spanish probe) exhibited priming regardless of whether they were presented blocked or mixed. In short, this pattern of results led Hern~indez et al. (1996) to conclude that "crosslanguage priming appears only when participants know what language to expect, when they have ample time to generate a response, or both" (p. 860). That is, switching from one language to the other takes time and access to a second language cannot occur
On-line Methods in Bilingual Spoken Language
15
unless the bilingual is in some type of "bilingual mode" (see also Hern~indez this volume). Two potential concerns with this task warrant further discussion. First, researchers choosing to use this task would do well to consider the point at which the visual target is presented during the spoken sentence. For example, notice that in (7a) above the sentence stops at the offset of the definite article the. This disruption and the importance of the definite article may give rise to what Grosjean and colleagues refer to as the "base-language effect" (Grosjean, 1988, 1997; L6wy & Grosjean, 2001; Li, 1996a). This effect refers to the bias toward the language of the sentence in which the listener expects or is primed for the next language item to be in the same language (Li, 1996a, p. 767). Bilinguals listening to sentence (7a) might immediately trigger or predict the correct word in English that is consistent with the preceding context and the grammatical structure of English. If true, when presented with sentence (7b), the bilingual must attempt to inhibit or develop a strategy to overcome the interference produced by the activation of the word that is consistent with the preceding language and contextual information. The general idea here is that the English word is more readily available, and to inhibit it would require extra time. If language presentation is blocked (the predictable condition) or response is delayed (delayed naming), there is sufficient time to bring under control this interference to successfully integrate the Spanish target word into the sentence (cf. Scarborough, Gerard, & Cortese, 1984). However, if no time is allowed to overcome this interference (the unpredictable condition), priming does not occur and the Spanish target word is not integrated and perhaps it is treated as a nonrelated word. In other words, the combination of the task processing requirements, the biasing context and the strong linguistic cues in effect mandates that the bilingual generate the lexical item most consistent with the preceding context and the linguistic structure of the sentence. The same reasoning applies to sentences (8a) and (8b). However, it could be argued that sentence (8b) would be more semantically felicitous at least for Spanish-English bilinguals who mix their languages during the communicative process. A second issue that applies to this paradigm and other on-line tasks concerns the "grammaticality" of the code-switched word. Linguistic evidence suggests that code switching follows a grammatical structure and that certain grammatical categories cannot be code-switched (Lederberg & Morales, 1985; Myers-Scotton, 1997). K. Pletsch de Garcia, (personal communication, December 5, 2000) suggests that care should be taken in constructing code-switched sentences because of the possibility of constructing unnatural linguistic groupings. Pletsch de Garcia argues that during codeswitching the natural tendency is not to break up linguistic categories such as noun phrases (e.g., the traffic) or infinitival phrases (e.g., to drive). It would be unlikely for a bilingual to code-switch the noun phrase the traffic into the trffico or the infinitive verb to drive into to manejar (cf. Lederberg & Morales, 1985). Inspection ofHern~indez et al's. stimuli suggests inconsistencies in relation to the position of the code-switched word and the preceding linguistic category. For example, in some instances the visually presented code-switched target is inserted following a word whose associate is highly related (e.g., traffic-LUZ), after a conjunction (e.g., when-TRAFICO) or breaking up an infinitival clause (e.g., to MANE JAR). Thus, the conclusion that cross-language
16
Roberto R. Heredia and Mark T. Stewart
priming is strategic and that it involves the use of expectations, deserves further consideration. Despite these concerns, the cross-modal naming task as implemented by Hem~ndez et al. (1996) has generally produced reliable cross-language priming effects similar to those reported in the bilingual associative priming literature (cf. Grainger & Beauvillain, 1988; see also, Keatley & de Gelder, 1992). Moreover, this task may prove useful in examining the effects of context on the comprehension of language in both bilingual (sentences 7b and 8b) and monolingual (sentences 7a and 8a; cf. Hem~ndez et al., in press) modes (for other possible manipulations such as visual degradation and speed naming, using this task, the reader is referred to the original paper).
The Gating Task The next task we will discuss is the gating task (Grosjean, 1980; 1988; Li, 1996a, 1996b; Li & Yip, 1998; Tyler, 1992). Participants in this task listen to a spoken word from left to right in segments of increasing duration, until the entire word has been presented. The first segment starts from the beginning of the word and has a duration of about 30-50 milliseconds (ms) and each successive gate may increase by about 30-50 ms and the last gate may correspond to the entire word. At each gate presentation, participants are required to identify or guess the most likely candidate based on the information being presented up to that particular gate. For example, in the Chinese sentence (9a) below, the first gate of the critical word FLIGHT may be Keoi ge FL. At this point, participants may generate such possible candidates as fiat, flood, and flop until they come up with the critical target or until they hear the entire word. These candidates are meaningful because they provide information into the underlying cognitive processes leading to the final identification of the critical target (Li, 1996a, p. 765). The general idea behind this task is to determine the amount (in milliseconds) or percentage of exposure needed to correctly identify the critical word. If a bilingual recognizes the critical target FLIGHT only after hearing the whole word, it can be said that 100% of acoustic information was needed to correctly identify the target. Research utilizing this task suggests that listeners recognize words very rapidly. Bilinguals and monolinguals need to hear about 200-250 ms of the beginning of a word in a constraining context and much more time in a neutral context in order to correctly identify it (Li 1996a; Marslen-Wilson, 1987). (9a) Keoi ge FLIGHTjin-ci (his/her FLIGHT is delayed) (9b) Keoi daap baan jin-ci ge FLIGHT (He/she boarded a delayed FLIGHT) Li (1996a) employed this task to examine the effects of three important psycholinguistic factors in the recognition of English targets embedded in Chinese sentences by Chinese-English bilinguals. Li's overall purpose was to replicate and extend earlier findings by Grosjean (1988). The first factor was a phonological variable concemed with the phonotactics or permissible sound sequences of both Chinese and
On-line Methods in Bilingual Spoken Language
17
English. For example, the English language allows both consonant-consonant (CC) and consonant-vowel (CV) clusters at the beginning of a word (e.g., BRide vs. Lion). Chinese, in contrast, allows CV clusters but lacks CC clusters. The objective of this manipulation was to determine the extent to which CC clusters, which are marked phonotactically as belonging to English, would be identified faster than CV clusters that are shared by both languages. In other words, because of its distinctiveness, lexical search of the target with the CC configuration would be directed specifically to the English lexicon without involving the Chinese lexicon. On the other hand, because both languages shared the CV cluster configuration, during lexical access, both lexicons would be searched to correctly identify the target word. The second factor concerned the phonetics or the manner in which the English critical target was pronounced. Li distinguished between "code-switchers" in which the critical target was pronounced using the appropriate English pronunciation, and "borrowers" in which the English target word was pronounced using the Chinese pronunciation. Code-switchers were predicted to be identified faster and to require less acoustic information because they are clearly English words. Borrowers would involve a search of both lexicons, and therefore would require more acoustic information during the identification process. The third factor examined the effects of context (prior context vs. neutral context) on the identification of code-switched words. To summarize, phonotactics made a difference in the amount of information needed to correctly identify the code-switched target. Contrary to the predictions, bilinguals needed more acoustic information to correctly identify English targets with CC than CV initial cluster configurations. More important, phonotactics interacted with phonetics. For the CC initial word configuration condition, borrowers needed more acoustic information of the critical target to correctly identify it. Code-switchers, on the other hand, needed significantly less information for identification. However, in the CV cluster configuration, both borrowers and code-switchers required the same amount of information to be correctly identified. Although context did not interact with the other two factors, the general trend was that critical targets in the prior context condition were recognized faster than targets in the neutral condition. In general, the results of this study are significant and underscore the importance of considering the influence of low-level linguistic information such as sound sequences in bilingual lexical access. Li's (1996a) results suggest that during lexical access, information that is shared by both Chinese and English (i.e., CV cluster configurations) is simultaneously activated and has priority over information that is not shared by the two languages (i.e., CC initial cluster configuration). Thus, it appears that the language preceding the code-switched target item determines which phonotactic configuration is to be activated (see also Grosjean's 1988). However, during the course of sentence processing, information that overlaps across languages has priority and is activated simultaneously. As a methodological tool, the gating paradigm is suitable to study ecologically valid issues that reflect real life situations such as the understanding of a "foreign accent," which is equivalent to what Grosjean (1988) refers to as "borrowing," and Li (1996a) as "borrowers." Another important aspect of this task is that it allows for the close examination of the bilingual's two languages and enables one to determine how each
18
Roberto R. Heredia and Mark T. Stewart
language's unique linguistic structure influences retrieval during the comprehension of code-switched information. The findings by Grosjean and colleagues are noteworthy when we consider that traditionally, these factors have been ignored in the bilingual literature. Moreover, research in the sentence-processing domain would benefit from future studies that systematically compare and contrast such tasks as the PTLD, the gating task, and the cross-modal naming technique described in the previous section. The first steps have been taken, and Li (1996a) has already shown remarkable similarities between the gating task and the cued shadowing task, to which we now turn.
The Cued Shadowing Task
The next task that we discuss is the cued shadowing task (Bates & Liu, 1996; Liu, Bates, Powell, & Wulfeck, 1997). It has been argued that this task is sensitive to lexical access, and has been used successfully in the monolingual literature to study the recognition of open- versus closed-class words (e.g., Herron & Bates, 1997), and code switching in bilingualism (Li, 1996a). Participants, in this task, are presented with an auditory sentence such as (9a). The target or code-switched word is recorded in a voice of the opposite sex from the voice in which the sentence was recorded. However, the code-switched word could be of the same voice as the rest of the sentence. The participant's objective is to repeat the target word aloud as quickly and as accurately as possible. Responses are recorded in milliseconds. Li (1996a) used this task to replicate and examine the generalizability of the gating task to other tasks. To summarize, the results of this task were remarkably similar to those obtained by the gating task. The cued shadowing technique proved to be sensitive to the effects of both phonotactic and phonetic factors. For example, bilinguals required significantly longer times to process English targets containing the CC cluster configuration than CV English targets. Again, this finding contrasted with Grosjean's (1988) results that found the opposite patterns. Although this task has replicated well-known psycholinguistic findings such as semantic priming and frequency effects, closer scrutiny is needed to determine whether the repeating or the shadowing of the code-switched word is performed without the activation of meaning (Bates & Liu, 1996). Moreover, researchers utilizing this task need to consider how the target word is to be presented. Will the target be recorded by the same or different gender voice? If recorded by the same voice, how are participants going to know which word from the sentence to repeat? In Li's (1996a) experiment, participants were told at the beginning of each block the specific location of the codeswitched word. For example, Chinese-English bilinguals were told that in certain blocks the code-switched word would always appear immediately after the phrase keoi g e _ in sentence (9a). Clearly, this issue calls for careful consideration. As an alternative to this potential methodological artifact, we propose a variant of this task in which regardless of voice shift, bilinguals are asked to listen for a target word that begins with a particular phoneme such as/f/for the target word in sentence (9a) above. Bilinguals can either press a computer key when the target sound is found, or repeat the word
On-line Methods in Bilingual Spoken Language
19
aloud (cf. Connine & Titone, 1996; see also Blank, 1980). Overall because the shadowing task does not require a high degree of metalinguistic knowledge, this task could be used to study language-processing issues with bilingual children, the elderly and with other bilingual populations that are unable to read and write in one or both of their two languages.
The Cross-Modal Lexical Priming (CMLP) Task The CMLP differs from the cross-modal naming task used by Hemfindez et al. (1996) in one critical way. During sentence presentation, the flow of the sentence is never interrupted, thus making it difficult for participants to engage in strategic processing (cf. Bates & Liu, 1996; Li, 1996a). This task has been widely used in monolingual studies and its sensitivity to semantic and associative relations, as well as contextual effects is well documented (e.g., Swinney, 1979; Swinney, Onifer, Prather, & Hirshkowitz, 1979; Tabossi, 1996; see also Ferreira & Anes, 1994). The procedure has been used repeatedly to demonstrate subject's abilities in resolving lexical ambiguities and multiple access during the comprehension of ambiguous words (e.g., Tabossi, 1996; but see Glucksberg, Kreuz, & Rho, 1986; Mckoon & Ratcliff, 1994; Prather, Swinney, 1988), syntactic related issues (e.g., Nicol & Swinney, 1989; Love & Swinney, 1996), figurative language (e.g., Blasko & Connine, 1993; Cacciari & Tabossi, 1988; Stewart & Heredia, in press; Titone & Connine, 1994) and stereotype processing (Heredia & Blumentritt, 1999,2001). Typically, participants are asked to listen to a sentence while watching a fixation point on a computer screen. At some point during presentation of the spoken stimulus (usually immediately following an ambiguous word), a test word replaces the fixation point and participants make a timed response to the item. The participants' task is to either make a lexical decision or name a critical target. Reaction times to test words and their matched controls are then compared to determine whether facilitatory priming exists. Li and Yip (1998) employed this task to examine the processing of Chinese-English homophones (words that share phonological similarities across languages, but whose meaning is different). Chinese-English bilinguals listened to sentences whose prior context was neutral or biased towards the meaning of a homophone embedded within a Chinese sentence. Homophones in the Chinese sentences were actual English words (e.g., lock) but pronounced in Chinese phonetics. The bilingual's objective was to name a visually presented English target that was the same word as the homophone (e.g., lock), and a Chinese target word that shared the similar phonological structure as the homophone in the sentence (e.g., lok). Additionally, bilinguals named two unrelated targets that shared no phonological similarities with the critical homophone in the sentence. The most significant result of Li and Yip's study was the interaction of homophony and language. Bilinguals were faster to name English targets (e.g., lock) than Chinese targets (e.g., lok). This facilitatory effect for the English visual target was surprising given that the auditory sentences were all in Chinese. Thus, it appeared as if the prior context was actually favoring the recognition of the English target and suppressing the
20
Roberto R. Heredia and Mark T. Stewart
recognition of the Chinese related target. However, this interpretation is only suggestive because context failed to interact with language and homophony. In fact, there was only a main effect of context, suggesting that prior context facilitated both language conditions equally. Critical to the results was the comparison between the unrelated controls and the related targets; however, this comparison was inappropriate because the unrelated targets were not appropriately controlled for frequency and word length relative to the related targets. Nonetheless, the possibility that semantic context can override language context is interesting and deserves further consideration. Overall, future work addressing this issue of bilingual lexical ambiguity and homophony would benefit from employing a more traditional priming paradigm. It would be possible to examine whether Chinese-English bilinguals activate one or both meanings of the Chinese-English homophone (e.g., lok vs. lock). For example, as bilinguals listen to the sentence and at the onset of the homophone, they name a target that is related to the English meaning of the homophone (e.g., key) and an appropriately controlled unrelated target (e.g., ice). If the English meaning of the homophone becomes activated, naming times for related targets would be faster (e.g., 600 ms) than response times for the unrelated controls (e.g., 740ms). In this case, one would say that there is a 140 ms priming effect. The logic would also apply for the other language condition. This task has also been used to examine the effects of context and lexical access across-languages (Heredia, 1998, 2000; Heredia & Stewart, 1998). Currently, we are employing the CMLP to address some of the assumptions of the revised hierarchical model of bilingual memory representation (Kroll & Stewart, 1994; cf. Hern~ndez this volume). This model suggests that bilingual lexical access is differentially affected by such factors as contextual and semantic information depending on whether bilinguals retrieve information from the first to the second language or vice versa. According to this model, access to the second-language from the first language (L l-L2) is sensitive to semantic or contextual information, whereas access to the first language from the second language (L2-L 1) is relatively insensitive to these factors (Kroll & Sholl, 1992; Kroll & Stewart, 1994). We believe that these assumptions can be addressed directly by systematically manipulating the contextual information prior to the critical prime within a sentence. We are studying this possibility in our ongoing research. For example, in one experiment, Spanish-English bilinguals listen to Spanish translations of contextually-unbiased sentences such as (10a) below. Notice that the information preceding the critical target (WAR) does bias the meaning of the critical prime. A second experiment involves sentences such as (10b). Notice that the preceding context of this sentence soldiers + combat is biased towards the meaning of the critical prime WAR. Participants in these two experiments are asked to name visually presented English or Spanish target words that are either related (peace, paz) or unrelated (light, luz) to the critical word. To be named targets are presented immediately after the critical prime denoted by subscript [* 1]. If the revised hierarchical model is correct and L l-L2 language directions are sensitive to contextual effects, cross-language priming should significantly increase from sentence (10a) to sentence (10b).
On-line Methods in Bilingual Spoken Language
21
(10a) It is difficult to admit that a WARt,~1sometimes brings more profits than losses (10b) Soldiers are trained for combat and WAR t*ll and that is why so much is invested in them Furthermore, using this task, our laboratory has been able to replicate the general observation that bilinguals in South Texas, a predominantly bilingual area, experience more interference from their second language (English) as they communicate in their first language (Spanish), and little or no interference from Spanish when they communicate in English. Our results using sentences such as (10a) in both Spanish and English show that bilinguals are generally faster in naming English target words while listening to Spanish sentences than to name Spanish words while listening to English sentences. That is, participants are faster at retrieving English words even when they are listening to Spanish sentences. When bilinguals listen to English sentences, as expected, English words are retrieved faster than Spanish words. More important is the finding that bilinguals are faster at naming English target words while listening to Spanish sentences than to name Spanish target words while listening to English sentences. Indeed, these results suggest a reliance on the second language rather than the first language. As argued by Heredia and Altarriba (2001), these results suggest that after a certain level of fluency and frequency of use is attained in a second language, a language shift occurs and the second language behaves as if it were the bilingual's first language. In other words, the second language becomes more readily accessible than the first language, and the bilingual comes to rely more on it. This language shift, as argued by Heredia and Altarriba, is due to frequency of language usage. Regardless of which language a bilingual leams first, the more active (dominant) language determines which lexicon is accessed faster (Heredia, 1997). Thus, unlike some theoretical accounts, bilingual lexical representation is not a static but a dynamic representational system in which the first language can fall in strength, while the second language becomes the dominant language. In short, the CMLP appears to be sensitive to measuring issues related to how bilinguals access information from their two language lexicons. More important, whether the critical prime (e.g., WAR) is in the same or different language, or it is recorded by the same or different voice is not an issue with this task because participants are responding to a visual target. However, empirical work should be carried out in which findings from other tasks (e.g., the PTLD and the cued shadowing technique) are systematically compared to the CMLP. In extending Li and Yip's (1998) original work, future work would benefit from using bilingualism as a tool to address issues related to multiple lexical access and ambiguity resolution. Cross-language homophones and homographs are the natural candidates to study these general psycholinguistic issues. Additional work on the examination of other psycholinguistic issues such as the comprehension and resolution of nonliteral language (e.g., Matlock & Heredia, this volume) will further our understanding of the interaction between the bilingual's two linguistic systems.
22
Roberto R. Heredia and Mark T. Stewart
The Auditory Moving-Window (AMW) The last task that we will discuss is the AMW. This task has been used to examine lexical frequency and syntactic complexity in monolinguals (e.g., Ferreira, Henderson, et al., 1996), the effects of prosody and language processing (e.g., Ferreira, Anes,& Horine, 1996), and most recently used to study the effects of aging and discourse processing (e.g., Titone, Prentice, & Wingfield, 2000), and hemispheric processing (e.g., Titone, Wingfield, Caplan, Waters, & Prentice, 2001). In this task, participants listen to sentences one or two words at a time and are required to press a button to receive successive segments as in sentence (11) (segments are depicted with the symbol "$"). Times between button presses or Interresponse times (IRT's) are recorded. Analysis can be performed on the IRT's or Difference times (DT's). DT's are computed by subtracting the IRT minus the duration of the segment during the digitizing of the word. For example, if the IRT for the target T E A C H E R in sentence (11) is 802 ms, and the time taken to record the target, (duration time), is 493 ms, then the DT would be 309 ms (Heredia, Stewart, & Cregut, 1997) (11) ,l,Erika ,l,estuvo,l, buscando a la,l,TEACHER,~ pero nunca,l, la encontr6,1,. (Translation: Erika was looking for the teacher but she never found her) Work underway in our laboratory employs the AMW task to investigate the effects of context and word frequency in the processing of code-switched sentences (e.g., Heredia et al., 1997). So far, our efforts have been successful and our results suggest interesting patterns depending on whether the target word is a code-switcher (pronouncing the target word with English phonetics) or a borrower (pronouncing the target word with Spanish pronunciation). For example, preliminary findings suggest that code-switchers are relatively unaffected by prior context. However, borrowers are actually processed faster under low-constraint than under high-constraint contextual conditions. Similarly, word frequency seems to affect borrowers the most. That is, high frequency borrowers seem to be processed faster than low frequency ones. In short, this task is an important altemative to some of the tasks discussed in this chapter, primarily because it is self-paced, and non invasive. That is, the AMW does not require the participant to produce a verbal or a decision response. This option is a valuable one, especially if the particular bilingual group of interest lacks academic training in one or both languages. However, researchers utilizing this task must assure that participants receive enough practice trials to familiarize themselves with the task. In addition to measuring the processing of a single target, like most of the tasks that we have reviewed, this task may be used to study the processing of linguistic units beyond the word level. For example, this task would be appropriate to study the comprehension of phrasal verbs such as turn on among bilinguals at different proficiency levels of the second language. Phrasal verbs represent a challenge to most speakers whose second language is English (for further details see, Matlock & Heredia, this volume). To illustrate, consider sentences (12a) and (12b).
On-line Methods in Bilingual Spoken Language
23
(12a) ,[,Mary,[, turned on,[,Colombo street,[, (12b) ,[,Mary,[, turned on,[, Colombo,[, In sentence (12a), turned on could be understood in terms of a literal interpretation in which it denotes some sort of physical motion and change in direction. Sentence (12b), on the other hand, may be interpreted in a nonliteral sense in which turned on may suggest that Colombo is sexually aroused by Mary (for further details, see Matlock & Heredia, this volume). Further experimental manipulations may involve biasing sentence (12a) with information congruent with the literal interpretation of the phrasal verb (e.g., Driving a new car). In this case, the preceding context would trigger the literal interpretation of the phrasal verb. The general idea in this possible experiment would be to measure the processing time for the phrasal verb under biasing and nonbiasing contextual conditions.
Final Thoughts In this chapter we have reviewed several of the experimental tasks used by researchers in bilingualism to examine spoken sentence comprehension. Our primary goal has been to describe each task and, where appropriate, to provide some insight into the usefulness of each of these procedures. We feel it is important that prospective users have a clear understanding of the potential strengths and weaknesses a particular paradigm may have. The final decision of which paradigm to use is left to the researcher and will be driven largely by the particulars of the empirical and theoretical issues under investigation. Moreover, there are technological constraints to consider; some tasks are more difficult to implement than others. Also, not all of these tasks are suitable for addressing the full range of empirical questions faced by researchers in bilingualism. Indeed, some are more focused than others and designed to examine very specific questions in greater detail. Regardless, our hope is that interested researchers will take from this chapter an understanding of the kinds of methodological issues and practices currently being developed for use in examining bilingual sentence comprehension, in the spoken domain.
References Altarriba, J. (1992). The representation of translation equivalents in bilingual memory. In R. J. Harris (Ed.), Cognitive processing in bilinguals (pp. 157-174). Amsterdam, Netherlands: Elsevier. Altarriba, J., Kroll, J. F., Sholl, A., & Rayner, K. (1996). The influence of lexical and conceptual constraints on reading mixed-language sentences: Evidence from eye fixations and naming times. Memory & Cognition, 24, 477-492. Amrhein, P. C. (1999). On the functional equivalence of monolinguals and bilinguals in "monolingual mode"" The bilingual anticipation effect in picture-word processing. Psychological Science, 10, 230-236.
24
Roberto R. Heredia and Mark T. Stewart
Bates, E., & Liu, H. (1996). Cued shadowing. Language and Cognitive Processes, 11, 577-581. Bates, E., Devescovi, A., & Wulfeck, B. (2001). Psycholinguistics: A crosslanguage perspective. Annual Review of Psychology, 52, 369-396. Blank, M. A. (1980). Measuring lexical access during sentence processing. Perception & Psychophysics, 28, 1-8. Blasko, D. G., & Connine, C. M. (1993). Effects of familiarity and aptness on metaphor processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 295-308. Connine, C. M., & Titone, C. (1996). Phoneme monitoring. Language and Cognitive Processes, 11, 635-645. Cacciari, C., & Tabossi, P. (1988). The comprehension of idioms. Journal of Memory and Language, 27, 668-683. de Groot, A. M. B., Dannenburg, L., & van Hell, J. G. (1994). Forward and backward word translation by bilinguals. Journal of Memory and Language, 33, 600629. Durguno~;lu, A. Y., & Roediger, H. L. (1987). Test differences in accessing bilingual memory. Journal of Memory and Language, 26, 377-391. Favreau, M., & Segalowitz, N. (1982). Second language reading in fluent bilinguals. Applied Psycholinguistics, 3, 329-341. Favreau, M., & Segalowitz, N. (1983). Automatic and controlled processes in reading a second language. Memory & Cognition, 11, 565-574. Ferreira, F., & Anes, M. (1994). Why study spoken language? In M. A. Gernsbacher (Ed.), Handbook of psycholinguistics (pp. 33-56). San Diego, CA: Academic Press. Ferreira, F., Anes, M. D., & Horine, M. D. (1996). Exploring the use of prosody during language comprehension using the auditory moving window technique. Journal of Psycholinguistic Research, 25, 273-290. Ferreira, F., Henderson, J. M., Anes, M. D., Weeks, P. A., & McFarlane, D. K. (1996). Effects of lexical frequency and syntactic complexity in spoken-language comprehension: Evidence from the auditory moving-window technique. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 324-335. Foss, D. (1970). Some effects of ambiguity upon sentence comprehension. Journal of Verbal Learning and Verbal Behavior, 9, 699-706. Glanzer, M., & Duarte, A. (1971). Repetition, between and within language in free recall. Journal of Verbal Learning and Verbal Behavior, 1O, 625-630. Grainger, J., & Beauvillain, C. (1988). Associative priming in bilinguals" Some limits of interlingual facilitation effects. Canadian Journal of Psychology, 42, 261-273. Grosjean, F. (1980). Spoken word recognition processes in the g~.ting paradigm. Perception & Psychophysics, 28, 267-283. Grosjean, F. (1988). Exploring the recognition of guest words in bilingual speech. Language and Cognitive Processes, 3, 233-274. Grosjean, F. (1996). Gating. Language and Cognitive Processes, 11, 597-604. Grosjean, F. (1997). Processing mixed language: Issues, findings and models. In A. M. B. de Groot & J. F. Kroll (Eds.), Tutorials in bilingualism: Psycholinguistic
On-line Methods in Bilingual Spoken Language
25
perspectives (pp. 225-254). Mahwah, NJ: Erlbaum. Grosjean, F., & Frauenfelder, U. H. (Eds.). (1996). Spoken word recognition [Special issue]. Language and Cognitive Processes, 11(3). Grosjean, F., & Soares, C. (1986). Processing mixed language: Some preliminary findings. In J. Vaid (Ed.), Language processing in bilinguals: Psycholinguistic and neuropsychological perspectives (pp. 145-179). Hillsdale, NJ: Erlbaum. Glucksberg, S., Kreuz, R. J., & Rho, S. H. (1986). Context can constrain lexical access: Implications for models of language comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12, 323-335. Guillelmon, D., & Grosjean, F. (2001). The gender marking effect in spoken word recognition: The case of bilinguals. Memory & Cognition, 29, 503-511. Heredia, R. R. (1997). Bilingual memory and hierarchical models: A case for language dominance. Current Directions in Psychological Science, 6, 34-39. Heredia, R. R. (1998, August). Cross-modal approaches to the investigation of bilingual spoken language comprehension. Paper presented at the meeting of the International Association for Cross-Cultural Psychology. Western Washington University, Bellingham, WA. Heredia, R. R. (2000, May). Bilingual lexical access and code-switching. Paper presented at the Fifth Conference on Applied Linguistics, Universidad de Las Am6ricas-Puebla, Mexico. Heredia, R. R., & Altarriba, J. (2001). Bilingual language mixing: Why do bilinguals code-switch? Current Directions in Psychological Science, 10, 164-168. Heredia, R. R., & Stewart, M .T. (1998, November). Bilingual on-line sentence processing.'A moment-to-moment processing approach. Poster presented at the 39 th annual meeting of the Psychonomic Society, Dallas, TX. Heredia, R. R., & Blumentritt, T. L. (1999, November). On-line processing of stereotypes during spoken language: A moment-to-moment processing approach. Poster presented at the 40 th annual meeting of the Psychonomic Society, Los Angeles, CA. Heredia, R. R., & Blumentritt, T. L. (2001). On-line processing of stereotypes during spoken language. Manuscript submitted for publication. Heredia, R. R., Stewart, M. T., & Cregut, I. (1997, November). Bilingual on-line sentence processing." Frequency and context effects in code switching. Poster presented at the 38 t" annual meeting of the Psychonomi'c Society, Philadelphia, PA. Hern~ndez, A. E., Bates, E., & ,/Wila, L. X. (1994). On-line sentence interpretation in Spanish-English bilinguals: What does it mean to be "in between"? Applied Psycholinguistics, 15, 417-446. Hern~ndez, A. E., Bates, E., &/~vila, L. X. (1996). Processing across the language boundary: A cross-modal priming study of Spanish-English bilinguals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 846-864. Hern~ndez, A. E., Fennema-Notestine, C., Udell, C., & Bates, E. (in press). Lexical and sentential priming in competition: Implications for two-stage theories of lexical access. Applied Psycholinguistics. Herron, D., & Bates, E. (1997). Sentential and acoustic factors in the recognition of open- and closed-class words. Journal of Memory and Language, 37, 217-239.
26
Roberto R. Heredia and Mark T. Stewart
Keatley, C., & Gelder, B. (1992). The bilingual primed lexical decision task: Crosslanguage priming disappears with speeded responses. European Journal of Cognitive Psychology, 4, 273-292. Kilborn, K. (1987). Sentence processing in a second language: Seeking a performance definition offluency. Unpublished doctoral dissertation, University of California, San Diego. Kilborn, K. (1989). Sentence processing in a second language: The timing of transfer. Language and Speech, 32, 1-23. Kolers, P. (1966). Reading and talking bilingually. American Journal of Psychology, 3, 357-376. Kroll, J. F., & Sholl, A. (1992). Lexical and conceptual memory in fluent and nonfluent bilinguals. In R. Harris (Ed.), Cognitive processing in bilinguals (pp. 157174). Amsterdam, Netherlands: Elsevier. Kroll, J. F., & Stewart, E. (1994). Category interference in translation and picture naming: Evidence for asymmetric connections between bilingual memory representations. Journal of Memory and Language, 33, 149-174. Lederberg, A. R., & Morales, C. (1985). Code switching by bilinguals: Evidence against a third grammar. Journal of Psycholinguistic Research, 14, 113-136. L6wy, N., & Grosjean, F. (2001). The computerized version of BIMOLA'A bilingual model oflexical access. Manuscript in preparation, University of Neuch~tel, Switzerland. Li, P. (1996a). Spoken word recognition of code-switched words by ChineseEnglish bilinguals. Journal of Memory and Language, 35, 757-774. Li, P. (1996b). The temporal structure of spoken sentence comprehension in Chinese. Perception and Psychophysics, 58, 571-586. Li, P., Bates, E., Liu, H., & MacWhinney, B. (1992). Cues as functional constraints on sentence processing in Chinese. In H-C. Chen & O. J. L. Tzeng. (Eds.), Language processing in Chinese (pp. 207-234). Amsterdam, Netherlands: Elsevier. Li, P., & Yip, M. C. (1998). Context effects and the processing of spoken homophones. Reading and Writing: An Interdisciplinary Journal, 10, 223-243. Liu, H., Bates, E., & Li, P. (1992). Sentence interpretation in bilingual speakers of English and Chinese. Applied Psycholinguistics, 13, 451-484. Liu, H., Bates, E., Powell, T., & Wulfeck, B. (1997). Single-word shadowing and the study of lexical access. Applied Psycholinguistics, 18, 157-180. Liu, H., Bates, E., & Li, P. (1992). Sentence interpretation in bilingual speakers of English and Chinese. Applied Psycholinguistics, 13, 451-484. Love, T., & Swinney, D. (1996). Coreference processing and levels of analysis in object-relative constructions: Demonstration of antecedent reactivation with the crossmodal priming paradigm. Journal of Psycholinguistic Research, 25, 5-24. Macnamara, J., & Kushnir, S. L. (1971 ). Linguistic independence ofbilinguals: The input switch. Journal of Verbal Learning and Verbal Behavior, 1O, 480-487. MacWhinney, B. (Ed.). (1987). Mechanisms of language acquisition. Hillsdale, NJ: Erlbaum. MacWhinney, B., & Bates, E. (Eds.). (1989). The cross-linguistic study ofsentence processing. New York, NY: Cambridge University Press.
On-line Methods in Bilingual Spoken Language
27
Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word-recognition.
Cognition, 25, 71-102 McDonald, J. L. (1986). The development of sentence comprehension strategies in English and Dutch. Journal of Experimental Child Psychology, 41, 317-335. McDonald, J. L. (1987a). Assigning linguistic roles: The influence of conflicting cues. Journal of Memory and Language, 26, 100-117. McDonald, J. L. (1987b). Sentence interpretation in bilingual speakers of English and Dutch. Applied Psycholinguistics, 8, 379-413. McDonald, J. L., & Heilenman, L. K. (1991 ). Determinants of cue strength in adult first and second language speakers of French. Applied Psycholinguistics, 12, 313-348. McDonald, J. L., & Heilenman, L. K. (1992). Changes in sentence processing as second language proficiency increases. In R. J. Harris (Ed.), Cognitive processing in bilinguals (pp. 325-336). Amsterdam, Netherlands: Elsevier. McKoon, G., & Ratcliff, R. (1994). Sentential context and on-line lexical decision tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 1239-1243. Myers-Scotton, C. (1997). Duelling languages: Grammatical structure in codeswitching. Oxford, England: Clarendon. Nicol, J., & Swinney, D. (1989). The role of structure in coreference assignment during sentence comprehension. Journal of Psycholinguistic Research, 18, 5-20. Prather, P. A., & Swinney, D. A. (1988). Lexical processing and ambiguity resolution: An autonomous process in an interactive box. In S. I. Small., G. W. Cottrell, & M. K. Tanenhaus (Eds.), Lexical ambiguity resolution: Perspectives from psycholinguistics, neuropsychology & artificial intelligence (pp. 289-310). San Mateo, CA: Morgan Kaufmann Publishers. Scarborough, D. L., Gerard, L., & Cortese, C. (1984). Independence of lexical access in bilingual word recognition. Journal of Verbal Learning and Verbal Behavior, 23, 84-99. Segalowitz, N. (1986). Skilled reading in the second language. In J. Vaid (Ed.),
Language processing in bilinguals: Psycholinguistic and neuropsychological perspectives (pp. 3-19). Hillsdale, NJ: Erlbaum. Sera, M. D., Berge, C. A. H., & del Castillo Pintado, J. (1994).Grammatical and conceptual forces in the attribution of gender by English and Spanish speakers. Cognitive Development, 9, 261-292. Soares, C., & Grosjean, F. (1984). Bilinguals in a monolingual and a speech model: The effect on lexical access. Memory & Cognition, 12, 380-386. Stewart, M.T., & Heredia, R. R. (2002). Moment-to-moment processing of reference metaphor during comprehension of spoken sentences. Journal of Experimental Psychology, 49, 1-11. Swinney, D. (1979). Lexical access during sentence comprehension: (Re)consideration of context effects. Journal of Verbal Learning and Verbal Behavior, 18, 645-660. Swinney, D., Onifer, W., Prather, P., & Hirshkowitz, M. (1979). Semantic facilitation across modalities in the processing of individual words and sentences. Memory & Cognition, 7, 159-165.
28
Roberto R. Heredia and Mark T. Stewart
Tabossi, P. (1996). Cross-modal semantic priming. Language & Cognitive Processes, 11, 569-576. Titone, D. A., & Connine, C. M. (1994). Comprehension of idiomatic expressions: Effects of predictability and literality. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 1126-1138. Titone, D., Prentice, K. J., & Wingfield, A. (2000). Resource allocation during spoken discourse processing: Effects of age and passage difficulty as revealed by selfpaced listening. Memory & Cognition, 28, 1029-1040. Titone, D., Wingfield, A., Caplan, D., Waters, G., & Prentice, K. (2001). Memory and encoding of spoken discourse following right hemisphere damage: Evidence from the auditory moving window (AMW) technique. Brain and Language, 77, 10-24. Tyler, L. K. (1992). Spoken language comprehension: An experimental approach to disordered and normal processing. Cambridge, MA: MIT Press. Vaid, J., & Pandit, R. (1991). Sentence interpretation in normal and aphasic Hindi speakers. Brain and Language, 41, 250-274.
Part II: Connectionist Models of Second Language Processing and Bilingualism
This Page Intentionally Left Blank
Bilingual SentenceProcessing- R.R. Herediaand J. Altarriba(Editors) 9 2002 ElsevierScienceB.V. All rights reserved.
Extending the Competition Model Brian MacWhinney Carnegie Mellon University
Abstract Current models of language learning tend to focus on the outcome of the learning process. As a result, they fail to fully describe the detailed operations of learning mechanisms for particular target structures. Current models also underestimate the extent to which learners rely on a variety of complementary learning mechanisms. By extending the Competition Model to deal with the role of at least nine important learning mechanisms, we can derive a fuller picture of the actual process of language learning in the individual learner. Current models of second language learning divide into two basic types: nativist and emergentist. Nativist models hold that second language learning of grammar, even in adults, proceeds in accord with the principles and parameters of Universal Grammar (Chomsky, 1965). Emergentist models, such as the Competition Model (MacWhinney, 1987; MacWhinney & Bates, 1989), view grammatical learning as based on an interaction between the learner, the input, and the context (MacWhinney, 2001 c). Although the opposition between these two types of models has helped to clarify many issues, it has tended to obscure others. In part, this is because both types of models have not committed themselves to providing complete accounts of the learning process. Nativist models are responsible for only showing that principles and parameters have some influence on the learning process. Once linguists secure evidence of these effects (Epstein, Flynn, & Martohardjono, 1996), additional aspects of the language learning process are treated as outside of the core of syntax and become the responsibility of the psychologists and applied linguists. Until now, the Competition Model has assumed a similar limited explanatory responsibility. It measures the learner's overall command of grammatical markings at various points during second language learning. It then attempts to show how changes in overall level of attainment can emerge from the interaction between mental characteristics of the learner and properties of the input (MacWhinney, 2001 a). However, this level of description does not tell us how the learner moves through the process of language learning step by step. Thus, neither the Competition Model nor the nativist analyses have yet assumed responsibility for characterizing the details of the process of second language learning. This absence of attention to detail is particularly surprising when we consider how rich the second language learning literature is with studies of specific error types (Corder, 1983), learner overgeneralizations (Selinker, Swain, & Dumas, 1975), and learning strategies (O'Malley & Chamot, 1990). Why has the attention to
32
Brian Mac Whinney
detail that is so obviously an integral part of this literature failed to make its impact on models that attempt to account for the overall process? We can trace this failure to at least five sources: 1. Nativist models concern themselves with a restricted core of syntactic patterns. As a result, many important components of language learning have remained outside of the limelight generated by the conflict between nativism and emergentism. 2. The standards of experimental psychology require that we test hypotheses against group data. By studying groups, we can measure overall attainment at different stages in second language learning. Unfortunately, this requirement makes it difficult to track the details of individual patterns of language learning. 3. Because linguistic analysis is not subject to the same requirements, studies based on error analysis have produced some of the best material we have on the details of second language learning. However, these details have not been articulated yet in terms of processing models. 4. Much work on second language learning has focused on a particular level of linguistic structure, such as lexicon, phonology, orthography, syntax, morphology, or pragmatics. Within each area, researchers have been able to articulate models for learning. However, natural language learning occurs simultaneously across all of these levels. This means that, if we want to understand individual steps in the process of language learning, we need to understand how learning integrates across levels. 5. Accounts of individual strategies in second language learning (O'Malley & Chamot, 1990) ground themselves on the theory of individual differences, rather than accounts of the application of actual processes. If we are going to make progress toward a unified understanding of the process of second language learning, we will need to devise ways of overcoming these methodological restrictions. Of the limitations noted above, the most constraining is the requirement that evidence on second language learning should come from group data. By looking at individual subjects, we can obtain a fuller understanding of the details of language learning (Ericsson & Simon, 1984; Newell & Simon, 1972) and the ways in which individuals apply different configurations of basic learning strategies. My basic thesis in this paper is that language learning depends on a fairly large set of learning mechanisms. The role that particular mechanisms play in acquiring different levels of linguistic structure is dependent on the shape of the target structure and the nature of the learner. Understanding the interaction between these mechanisms can allow us to formulate an extended and more accurate version of the Competition Model for second language learning.
Extending the Competition Model
33
What Learning Mechanisms Are Available for Language Learning?
Cognitive psychology is rich in proposals for learning mechanisms. Although it might seem at first that one could easily get lost in this enormous literature, the relevant cognitive mechanisms can be counted on the fingers of two hands. Specifically, we are most interested in this set: 1. 2. 3. 4. 5. 6. 7. 8. 9.
Pattern Detection Predictive Association Selective Attention Transfer Chunking Resonance Error Detection Mental Model Construction Input Strategies
We can describe each of these mechanisms using either neural networks or symbolic production systems. Of course, all cognitive processing derives from neural function. For low-level processes such as (1) to (5), psychologists tend to prefer neural network modeling. However, for higher-level processes such as (6) to (9), it is often practical to abstract away from the details of neuronal functioning and to think in terms of higher-level symbolic systems such as production systems (Anderson, 1993). For our current purposes, the contrast between neural networks and production systems is largely irrelevant. Our focus now is simply on the mechanisms themselves and how they can provide explanations regarding the details of second language learning. Pattern Detection
Pattern detection is the most basic of all learning mechanisms. In the area of language learning, it functions most obviously when we first begin to listen to words in a new language. At this point, our auditory cortex stores the overall prosody and shape of words and phrases in the new language. Recent work (Aslin, Saffran, & Newport, 1999) has shown that, even at the age of 6 months, infants have become sensitive to a wide range of prosodic regularities in their native language. Moreover, they are able to quickly record co-occurrence patterns of syllable strings in the input. In fact, their ability to store such patterns in working memory is nearly as good as that of adults. This mechanism plays a crucial role in the learning of sounds and words. Predictive Association
Once we have stored away a set of auditory patterns, we can use them to predict other patterns. In classical conditioning, we use a stimulus to predict a response. This prediction can be unconscious and implicit. In response learning, we track the ways in which our actions bring about changes in the world. Listening to the results
34
Brian Mac Whinney
of our sound productions involves this type of prediction. In other cases, prediction operates between some external form, such as a word, and an internal interpretation. For language learning, this latter type of predictive association is crucial. Words represent predictive relations between sounds and their meanings. The Competition Model places a great emphasis on this basic association between a form and its function. In comprehension, the learner maps forms onto functions. In production, learning involves mapping functions onto forms. In accord with current theorizing in Construction Grammar (Kay & Fillmore, 1999), we view language as a collection of form-function mappings, some of which are lexical and some grammatical. Theories of predictive association all provide a central role for reinforcement learning. Work on the power law of practice (Newell & Rosenbloom, 1981) emphasizes the role of reinforcement and the fact that the best predictor of level of learning is time on task. Lexical learning illustrates the importance of this effect. For example, we are likely to learn high frequency words such as want or go in a new language, before we learn words like supervise or eradicate. In fact, frequency effects appear everywhere in language (Bybee & Hopper, 2001), affecting all aspects of phonology, syntax, and pragmatics and determining the course of grammaticalization. The Competition Model deals with frequency in terms of the construct of cue reliability. For example, in English, we would say that the cue of preverbal positioning is an extremely reliable cue to the choice of a noun as the subject of a transitive verb. On the other hand, the inherent animacy of the noun is less reliable as a cue, since we can have sentences such as "The ball hit the boy on the shoulder." Reliability is the conditional probability that an interpretation X should be selected given the presence of a cue Y (i.e., p(XIY)). If this probability is high, then Y is a reliable cue to X. The prediction from this initial analysis is that forms with a high conditional probability should appear early in development, should be resistant to loss in aphasia, should transfer strongly in second language (L2) learning, and should be the strongest determinants of processing in adults. Most Competition Model experiments use a simple sentence interpretation procedure. Subjects listen to a sentence with two nouns and a verb and are asked to say who was the actor. In a few studies, the task involves direct-object identification (Sokolov, 1988, 1989), relative clause processing (MacWhinney & Pl6h, 1988), or pronominal assignment (MacDonald & MacWhinney, 1990; McDonald & MacWhinney, 1995), but usually the task is agent identification. Sometimes the sentences are well-formed grammatical sentences, such as the cat is chasing the duck. Sometimes they involve competitions between cues, as in the ungrammatical sentence *the duck the cat is chasing. Depending on the language involved, the cues varied in these studies include word order, subject-verb agreement, object-verb agreement, case-marking, prepositional case marking, stress, topicalization, animacy, omission, and pronominalization. These cues are varied in a standard orthogonalized ANOVA design with three or four sentences per cell to increase statistical reliability. The basic question is always the same: what is the relative order of cue strength in the given language and how do these cue strengths interact? These studies have yielded a remarkably consistent body of results. The most important finding is that the order of cue strength in adults always corresponds with
Extending the Competition Model
35
the order of cue reliability yielded by text counts in the language. In different languages, we find different cue dominance patterns. In English, the dominant cue for subject identification is preverbal positioning. For example, in the English sentence the eraser hits the cat, we assume that the eraser is the agent. However, a parallel sentence in Italian or Spanish would have the cat as the agent. In Spanish, the prepositional object marker "a" is a clear cue to the object and the subject is the noun that is not the object. An example of this is the sentence el toro mat6 al torero (The bull killed to-the bullfighter). No such prepositional cue exists in English. In German, case marking on the definite article is a powerful cue to the subject. In a sentence such as der Lehrer liebt die Witwe (The teacher loves the widow), the presence of the nominative masculine article der is a sure cue to identification of the subject. In Hungarian, the subject is the noun not marked by any suffix or postposition. In Russian, the subject often has a case suffix. In Arabic, the subject is the noun that agrees with the verb in number and gender and this cue is stronger than the case-marking cue. In French, Spanish, and Italian, when an object pronoun is present, it can help identify the noun that is not the subject. There are many alternative formulations of the concept of predictive association. Neural network theory treats this in terms of the theory of classifier learning. A classifier is a mental structure (expressed in vector weights) that has learned to properly predict the assigriment of a given form to a category. The predictive associations between forms and functions in the Competition Model can be modeled as classifier learning using neural networks (Elman, 1991). Perhaps the clearest demonstrations of the applicability of neural network models are in the area of morphological learning (MacWhinney, Leinbach, Taraban, & McDonald, 1989). For example, when learning the English past tense, neural networks extract different phonological regularities to produce successful generalizations such as bent and fell as the past tenses of bend and fall. This type of learning clearly plays a major role in both first and second language learning. Like the other mechanisms we will discuss, there must be a role for predictive association in the updated Competition Model, particularly in the domains of morphology and syntax. However, predictive association, by itself, is certainly not the whole of the story. Selective Attention
We can distinguish two different ways in which attention modulates learning. First, it can operate like the lens of a camera to suddenly expose us to a new stimulus. This type of overall increase in sensitivity to exposure has been called "flashbulb memory." It operates primarily under conditions of high emotional arousal. Flashbulb memory may play some role in language learning, but a second type of attentional selection is probably more central. This is selective attention to particular components of a larger stimulus. Selective attention has clear influences on learning new words or characters. For example, when I was leaming the Cantonese phrase for "pivoting your foot inward," I initially encoded it as kau geu, instead of the correct form kau geuk. This is because there is a tendency in Cantonese to reduce final/k/. However, the reduced final/k/is not totally absent and has an effect on the quality of the preceding vowel. At first, I did not attend to this additional component or cue. However, after my encoding for kau geu became
36
Brian Mac Whinney
automated, my attentional focusing was then freed up enough so that I could notice the presence of the final/k/. This expansion of selective attention during learning is a very general process. The classic version of the Competition Model assumed that the learner paid simultaneous attention to all available cues. However, McDonald and MacWhinney (1991) and Matessa and Anderson (2000) have shown that, during the initial phases of language learning in miniature linguistic system experiments, subjects tend to focus on only one cue at a time. For example, in an experiment with four cues based on word order, animacy, case marking, and tone, a learner would decide to focus attention on only one of these as the predictor of sentence interpretation. Later on, after having tracked the use of this first cue, the learner will add a second cue to the mix and begin to use the two in combination. As development proceeds, the learner pays attention to additional cues. Selective attention is also important in early word learning. Smith (1999) has shown how the modulation of attention can lead a child to group objects into cognitive categories. Merriman (1999) has shown that one can use the Competition Model to account for aspects of lexical learning such as mutual exclusivity. However, to do this, the model must incorporate stronger assumptions regarding attentional selection. For example, at first a child might not focus on the fact that a cup is likely to have a handle, whereas a glass does not. Over time, attentional focusing sharpens the competition of words for referents in this way. Transfer
Psychologists think of transfer as something that happens when an animal moves between separate, but highly related learning tasks. For example, a rat that has learned to turn right when seeing a yellow gate in a maze should be able to transfer this learning to a new maze, as long as the two mazes have a similar overall pattern. This means that transfer is based on both pattern detection and predictive association. Associationist models rely heavily on similarity detection to learn the classifiers that produce predictive association. Because human languages are similar in so many ways, models that rely on predictive association predict massive amounts of transfer. The emphasis on the pervasiveness of transfer is a fundamental component of the Competition Model for second language learning. One of our first investigations of second language processing effects (Bates & MacWhinney, 1981) examined the comprehension of English sentences by some of our academic colleagues. One subject--a native speaker of German--had lived in the United States for 30 years. Married to an American, he had published several important textbooks in experimental psychology, all written in English. Remarkably, we found that this subject processed simple English sentences using the cue strength hierarchy of German. This is to say that he used agreement and animacy cues whenever possible, largely ignoring word order when it competed with agreement and animacy. We now have evidence for this preservation of a syntactic "accent" in comprehension from over a dozen studies (Bates & MacWhinney, 1981; de Bot & van Montfort, 1988; Gass, 1987; Harrington, 1987; Kilborn, 1989; Kilborn & Cooreman, 1987;
Extending the Competition Model
37
Kilborn & Ito, 1989; Liu, Bates, & Li, 1992; MacWhinney et al., 1989; McDonald, 1987a, 1987b; McDonald & Heilenman, 1991; McDonald & MacWhinney, 1989).
Chunking Learning a second language involves developing an automatic level of access to words and syntactic patterns that can guarantee fluency and accuracy. It is crucial to realize that fluency requires more than just reinforcement learning. Reinforcement may work to solidify small, highly determinant sequences of actions, such as those found in short words. However, when the learner tries to combine words into larger units, the strength of the individual components alone cannot guarantee fluency. Instead, the learner must work to build up larger components by chunking (Anderson & Lebiere, 1998). For example, in Spanish, learners can chunk together the plan for buenos with the plan for dias to produce buenos dias. They can then combine this chunk with muy to produce muy buenos dias "very good morning." Chunking allows the learner to get around problems with Spanish noun pluralization, gender marking, and agreement that would otherwise have to be reasoned out in detail for each combination. Although the learner understands the meanings of the three words in this phrase, the unit can function as a chunk, thereby speeding production. Chunking is the primary mechanism supporting the growth in fluency that is so important in learning a second language. Chunking occurs when we store a series of actions or percepts as a single unit. During production, this occurs as we say similar phrases repeatedly. During comprehension, it occurs as we hear separate items together. In both cases, chunking is dependent on reinforcement, since each repeated exposure to a chunk strengthens its unity. At first, we access the individual components in a slow, halting fashion. However, after we chunk them together, access is smoother and quicker. Once this occurs, resources are freed up that can allow us to expand our selective attention, thereby picking up additional syntactic cues, lexical meanings, phonological distinctions, and orthographic details. Within the word, chunking involves the increased automatization of the transitions between syllables and segments. Expanding on an account provided by Grossberg (1978), Gupta and MacWhinney (1997) present a neurologically based, computational model of the process of automatization of word production. The basic idea underlying this model is that words are stored as a chain in which the production of beginning segments triggers production of later segments. Learning involves the strengthening of the links in this chain until the learner can articulate the whole word smoothly as a single chunk. We often use studies of resource competition to measure the effects of chunking. There have been several Competition Model studies of this type. Kilborn (1989) has shown that even fully competent bilinguals tend to process sentences in their L2 more slowly than monolinguals. Moreover, when monolinguals listen to sentences under conditions of white noise, their reaction times are identical to those of bilinguals not subjected to noise. Similarly Blackwell and Bates (1995) and Miyake, Carpenter, and Just (1994) have shown that, when subjected to conditions of noise, normals process sentences much like aphasics not subjected to noise. In general, we can assume that the task of having to process two or more languages concurrently or
38
Brian MacWhinney
sequentially puts some load on the system. However, we also know that these effects decrease as one achieves improved fluency in the second language. At first, processing in L2 is so resource intensive, that it is impossible to carry out secondary tasks. However, as L2 chunking advances, resources are freed up for smoother processing. The ability of fluent simultaneous translators to cope with two different languages in parallel illustrates the extent to which growths in fluency can overcome resource limitations. However, even for this nearly miraculous example of bilingual fluency, there are limits. Gerver (1974) and Seleskovitch (1976) have shown that, during simultaneous translation, further imposition of a load through noise can result in marked deterioration in translation quality. Resonance
The classic version of the Competition Model relied entirely on the mechanisms of predictive association and transfer discussed above. However, if we wish to extend the model to cover the finer details of language learning, we need to consider the role of additional processes. Of these, the most important is resonance. Resonance is a modem formulation of a process that psychologists earlier called "spreading activation" and "interactive activation." The psychology of the 1970s (Lindsay & Norman, 1977) placed a great emphasis on spreading activation as the primary method for activating cognitive structures. McClelland and Rumelhart (1981) formalized the notion of spreading activation in their interactive activation model of word naming. In that model, words were composed of letters and letters were composed of graphemic segments. Recognition of a word occurred when the bottom-up activation from the segments and the letters spread more activation to the correct lexical item than to its nearest competitors. Systems of this type are still extremely useful in modeling reading processes in bilinguals (van Heuven, Dijkstra, & Grainger, 1998). Interactive activation provides a powerful, simple way of understanding the details of cognitive processing. However, it has one crucial feature that made it less useful as a general account of cognition. This is the fact that there is no learning rule for an interactive activation network. It is necessary to adjust all of the weights and connections inside the model by hand in order to achieve correct empirical predictions. To address these limitations, researchers have turned to more adaptive systems that use the concept of resonance. Primary among these systems is the adaptive resonance theory (ART) model developed by Grossberg (1987) and colleagues. Resonance occurs when several concepts are co-activated and their mutual excitation leads to the formation of a resonant pattern. A network can often store a few such patterns without additional attentional intervention. However eventually the network will need to rely on additional resources in terms of both computational units and attentional input. In ART, attentional activation occurs primarily as a result of an error signal. The error signal arises when the network cannot properly recharacterize the input pattem. This notion of recharacterization of a pattern is possible in an ART network because the system uses its resonant pattern to regenerate the shape of the input. In this sense, the model expresses concepts such as the neuronal group of Edelman (1987) or the convergence zones of Damasio and Damasio (1994). In ART attentional units are triggered by the operation of
Extending the Competition Model
39
motivational and affective processes. For second language learning, this could express the impact of limbic systems on learning (Pulvermtiller & Schumann, 1994). Resonance can be triggered passively through attention to error. However, learners can also control attention strategically to form resonant connections between already existing items. Gentner and Markman (1997), Hofstadter (1997) and others have formulated models of analogical reasoning that have interesting implications for language acquisition models. Analogies can be helpful in working out the first examples of a pattern. For example, a child learning German may compare steh aufi "stand up!" with er muff aufstehen "He must get up." The child can see that the two sentences express the same activity, but that the verbal prefix is moved in one. Using this pattern as the basis for further resonant connections, the child can then begin to acquire a general understanding of verbal prefix placement in German. Error Detection
The contrast between implicit and explicit learning has received a great deal of attention in recent work on second language learning. This interest stems originally from the contrast between learning and acquisition that was at the heart of much theorizing in the 1970s (Krashen, 1994). More recently, neural network models have placed a new focus on the role of implicit learning (Reder, 1996). It is clear that much language learning occurs through low-level pattern detection, perceptual chunking, and reinforcement. However, it is also true that some learners benefit more than others from explicit correction and instruction (Carroll & Swain, 1993; Ellis, 1994; Schmidt, 1994; Tomasello & Herron, 1988). The bulk of this discussion has examined learning from the viewpoint of the instructor, since it considers whether the provision of explicit corrective feedback has a detectable impact on language learning (MacWhinney, 1993). We can also examine the issue of error detection from the viewpoint of the learner. Learners may detect errors during either comprehension or production. During comprehension, they may notice that they cannot understand some important word, or they may notice that some default understanding they have does not actually make sense. When this happens, they can then store away this word as one they need to learn (MacWhinney, 1978). During production, learners can detect errors when they compare their own productions against stored images of correct productions from native speakers. When this happens, they can tag the problematic items as ones that they need to refine through selective attention and rechunk. Resonance can play an important role in the resolution of errors. For example, I recently noted that I had wrongly coded the stress on the Spanish word abanico "fan" as on the second syllable, as in ab6nico, perhaps under the influence of words like pl6stico. To correct this error, I spent time both rehearsing the correct stress pattern a few times and then visualizing the word as spelled without the stress mark or with the stress on the second syllable, which is normally not written in Spanish orthography. I also tried to associate this pattern in my mind with the verb abanicar "fan" and even the first person singular of this verb that has the form abanico. Having rehearsed this form in these various ways and having established these
40
Brian Mac Whinney
resonant connections, the tendency to produce the only incorrect form was somewhat reduced, although it will take time to fully banish the traces of the incorrect pattern.
Mental Model Construction Mental model construction probably plays a minimal role in child language acquisition. However, it can be of some marginal importance in adult learning. Although we know that learners derive little benefit from long hours of formal grammatical instruction in a foreign language, we know that they can benefit from some attention to grammatical pattems. Typically, adult learners store away these patterns in terms of formal structures such as charts of conjugations or grids of pronominal declension. For example, when learning Cantonese verbs, I have found it useful to think in terms of the verb as a pair of syllables separated by an aspect marker. Visually, this looks something like A - B - A', where A and A' are the two parts of the Cantonese verb and B is the aspect marker. Thus, a verb could be something like yau sdui "swim water" and the past tense is yau j6 sdui with the aspect marker j6 inserted between yau and s~ui. I have found that mental models of this type also work well for discontinuous verbal prefixes in Hungarian and German. To take yet another example, I find that, when in doubt about affixation, it is often useful to quickly color a Hungarian word in my mind as being either highvowel or low-vowel in terms of the vowel harmony pattems of Hungarian. I then use this internalized color to activate one of the correct allomorphs of the Hungarian suffixes that attaches to this stem. Finally, when trying to remember which Hungarian locative case suffix to select from the basic set of nine, I still tend to activate the grid of three positions by three movements that I leamed in my introductory Hungarian textbook. Of course, we only fall back on these mental models as crutches when we forget a form or have some gap in our vocabulary. However, mental models can serve an important role in the first phases of adult language learning, and they remain available later if needed.
Input Strategies Just as children seldom worry about constructing mental models, they seldom have to worry about acquiring methods for maximizing linguistic input. Typically, they derive excellent input from their family and friends (Snow, 1995). For the second language learner, however, the situation is very different. Often second language learners spend only a few hours or less each day using the second language. Worse still, these hours are often interspersed with ongoing first language (L1) input, L1 usage, and frequent code switching. In the L2 classroom, both child and adult learners have little control over the interactional context. In this context, successful learning strategies amount to little more than paying attention to the teacher, taking good notes, and keeping up with assignments. Given the difficulty involved in securing good L2 input, the learner's goal has to be to maximize input, while occasionally getting a chance to produce utterances. If there is a language-learning laboratory, the student can use videotapes, audiotapes, and computerized lessons as further sources of input. Even in the most favorable
Extending the Competition Model
41
naturalistic contexts, the L2 learner often does not have access to the rich system of social support that provides high quality language input to the child (Locke, 1995; Snow, 1995). In extreme situations, the L2 learner is overtly excluded from close personal interactions. In such cases, it becomes difficult to engage in the type of rich ongoing use of language that can maximally support learning. To compensate for this, the learner can develop a system of autosupport to improve the outcome of language learning. Concrete autosupport strategies include listening to television, radio, and movies, rehearsing taped dialogs, practicing new lexical items, and direct study of grammatical theory. These activities allow the adult learner to remain in contact with the input in ways that promote the functioning of neuronal loops for rehearsal, memory, and learning. These input strategies can be profitably linked to targeted use of resonance. Good learners realize that words and patterns are best learned when they are meaningfully related to other words, phrases, and orthographic patterns. This means that the process of maximizing input must be linked to an active process of resonant construction that provides the learner with time to construct and elaborate these resonant connections. Both L1 and L2 learners rely on resonance as a core learning mechanism. The major difference in the two cases is that the L2 learner must link the processing of resonant connections to the maximization of good input. For the child, there is no need to maximize input, since adequate input is always available.
How Do These Mechanisms Apply Across Linguistic Levels? Phonology Phonological learning involves two very different processes. Auditory acquisition is primary and begins even before birth (Moon, Cooper, & Fifer, 1993). It relies on inherent properties of the mammalian ear (Moon et al., 1993) and early pattern detection through statistical learning. This same statistical learning mechanism is operative in children, adults, and cotton-top tamarins (Hauser, Newport, & Aslin, 2001). The major challenge facing the learner is not the acquisition of perceptual patterns, but the development of articulatory methods for reproducing these patterns (Menn & Stoel-Gammon, 1995). The coordination of motor mechanisms for speech output is a relatively late evolutionary emergence (MacWhinney, 2001b) and it is not surprising that it is relatively difficult skill for the child to control. However, by age 5, most children have achieved control over articulatory processes. For the adult L2 learner and the older child, the situation is much different. Their learning begins with massive transfer of L1 articulatory patterns to L2 (Flege & Davidian, 1984; Hancin-Bhatt, 1994). This transfer is at first successful in the sense that it allows for a reasonable level of communication. However, it is eventually counter-productive, since it embeds L1 phonology into the emergent L2 lexicon. In effect, the learner treats new words in L2 as if they were composed of strings of L 1 articulatory units. This method of learning leads to short term gains at the expense of long-term difficulties in correcting erroneous phonological transfer. Older children acquiring a second language can rely on their greater neuronal flexibility to
42
Brian Mac Whinney
quickly escape these negative transfer effects. In doing so, they are relying on the same types of adolescent motor abilities that allow adolescents to become proficient acrobats, gymnasts, dancers, and golfers. Adults have a reduced ability to rewire motor productions on this basic level. However, even the most difficult cases of negative transfer in adulthood can be corrected through careful training and rehearsal (Flege, Takagi, & Mann, 1995). To do this, adults must rely on resonance, selective attention, and learning strategies to reinvigorate a motor learning process that runs much more naturally in children and adolescents. Lexicon
The second language learner comes to the task of learning a new language with a well-organized system of meanings and concepts. Unlike the child, who must develop a complete understanding of the components of his culture, the adult can readily transfer meaning, particularly if the cultures involved are similar. As in the area of phonology, the adult can achieve rapid initial progress in lexicon building by simply transferring this LI conceptual world en masse to L2. Young bilinguals can also benefit from this conceptual transfer. In the case of the simultaneous bilingual, the child only has to acquire one set of meanings. Although some words will be more closely linked to one language than another, the overall set of conceptual structures used to break up the world is essentially unified. For children who are picking up their second language somewhat later, lexical transfer is still highly positive. The case of Helen Keller is a dramatic illustration of this effect. Helen had begun to learn language as an infant, when she suddenly lost her hearing and sight. Later, when she came to learn language, she already had a well-established conceptual world that was ready to acquire new forms. As a result, after learning her first word water, she went on in the first day to learn 30 new words. When learners first acquire a new L2 form, such as silla in Spanish, they treat this form as simply another way of saying chair. This means that initially the L2 system has no separate conceptual structure and that its formal structure relies on the structure of L1. Kroll and Sholl (1992) emphasize the extent to which L2 relies on L1 forms to access meaning, rather than accessing meaning directly. In this sense, we can say that L2 is parasitic on L1, because of the extensive amount of transfer from L 1 to L2. The learner's goal is to reduce this parasitism by building up L2 representations as a separate system. They do this by strengthening the direct linkage between new L2 forms and conceptual representations. Given the fact that connectionism predicts such massive transfer for L1 knowledge to L2, we might ask why we do not see more transfer error in second language lexical forms. There are three reasons for this. First, a great deal of transfer occurs smoothly and directly without producing error. Consider a word like "chair" in English. When the native English speaker begins to learn Spanish, it is easy to use the concept underlying "chair" to serve as the meaning for the new word "silla" in Spanish. The closer the conceptual, material, and linguistic worlds of the two languages, the more successful this sort of positive transfer will be. Transfer only works smoothly when there is close conceptual match. For example, Ijaz (1986) has
Extending the Competition Model
43
shown how difficult transfer can be for Korean learners of English in semantic domains involving transfer verbs, such as "take" or "put." Similarly, if the source language has a two-color system (Berlin & Kay, 1969), as in Dani, acquisition of an eight-color system, as in Hungarian, will be difficult. These effects underscore the extent to which L2 lexical items are parasitic on L 1 forms. 2. Second, learners quickly correct some types of incorrect transfer. For example, when a learner tries to form a cognate in Spanish for the English noun "soap" the result is "sopa" which means "soup." Similarly, an attempt to translate the English form "competence" into Spanish as competencia will run into problems, since competencia means "competition." Once an error like this is made, it will be quickly detected. Moreover, the learner will become increasingly sensitive to possible errors in cognate transfer. .
Third, error is minimized when two words in L1 map onto a single word in L2. For example, it is easy for an L1 Spanish speaker to map the meanings underlying "saber" and "conocer" (Stockwell, Bowen, & Martin, 1965) onto the L2 English form "know." Dropping the distinction between these forms requires little in the way of cognitive reorganization. It is difficult for the L 1 English speaker to acquire this new distinction when learning Spanish. In order to control this distinction correctly, the learner must restructure the concept underlying "know" into two new related structures. In the area of lexical learning, these cases should cause the greatest transfer-produced errors.
Most current models of lexical development assume that the lexical link between a sound and a meaning is just a simple connection. This was a basic assumption of the classic Logogen Model (Morton, 1970). The problem with this model is that it treats words as if they were single neurons with easily located mental addresses. As MacWhinne~r (1994) has noted, neurons do not have addresses and there is no easy way to locate a word in neural space. Instead, we need to think of words as distributed patterns that have a maximum in a more mental processing space determined by their component sounds and meanings. This means that establishing a link between sounds and meanings is a highly constructive process. Although it is true that learners can develop a "fast mapping" (Carey, 1978) between a new word and a concept, they are unlikely to remember the word later on unless they have also established additional supportive relations between the sound and the meaning. Learners can use sound symbolism to establish a resonant connection between a sound and a meaning. It is not necessary that this symbolism be in accord with any established pattern. Because each learner-finds some idiosyncratic patterns of associations, it has been difficult to demonstrate the use of mnemonic connections in group studies of lexical learning. However, we do know that constructive mnemonics provided by the experimenter (Atkinson, 1975) greatly facilitate learning. For example, when learning the German word Wasser, we can imagine the sound of water running out of a faucet and associate this sound with the /s/ of
44
Brian Mac Whinney
Wasser. For this word, we can also associate the sound of the German word to the sound of the English word water. At the same time, we can associate Wasser with collocations such as Wasser trinken which themselves resonate with Bier trinken and others. Together, these resonant associations between collocations, sounds, and other words help to link the German word Wasser into the developing German lexicon. It is likely that children also use these mechanisms to encode the relations between sounds and meanings. Children are less inhibited than are adults in their ability to create ad hoc symbolic links between sounds and meanings. The child learning German as an L 1 might associate the shimmering qualities of Wasser with a shimmering aspect of the sibilant; or the child might imagine the sound as plunging downward in tone in the way that water comes over a waterfall. The child may link the concept of Wasser tightly to a scene in which someone pours ein Glas Waser and then the association between the sound of Wasser and the image of the glass and the pouring are primary. For the first language learner, these resonant links are woven together with the entire nature of experience and the growing concept of the world. The adult second language learner tends to rely on r~/ther less imaginative and more structured resonant linkages. One important set of links available to the adult is orthography. When an L2 learner of German learns the word Wasser, it is easy to map the sounds of the word directly to the image of the letters. Because German has highly regular mappings from orthography to pronunciation, calling up the image of the spelling of Wasser is an extremely good way of activating its sound. When the L2 learner is illiterate or when the L2 orthography is unlike the L 1 orthography, this backup system for resonance will not be available. L2 leaming of Chinese by speakers of languages with Roman scripts illustrates this problem. In Mainland China, use of Chinese characters is often accompanied by Romanized pinyin spellings. This allows the L2 leamer a method for establishing resonant connections between new words, their pronunciation, and their representations in Chinese orthography. However, in Taiwan and Hong Kong, characters are seldom written out in pinyin in either books or public notices. As a result, learners cannot learn from these materials. In order to make use of resonant connections from orthography, leamers must then focus on the learning of the complex Chinese script. This learning itself requires a large investment in resonant associations, since the Chinese writing system is based largely on radical elements that have multiple resonant associations with the sounds and meanings of words. The ongoing articulation and elaboration of these resonant connections between elements of L2 helps the learner establish a firewall against ongoing interference effects from L1. Consider the case of the English word "table" and the Spanish word "mesa." When learning this new word, the leamer links it to related Spanish words. During this process, the learner links "mesa" more closely to Spanish phrases and meanings than to English phrases and meanings. The more these two synonymous nouns link into separate worlds and to other words in the same language, the stronger will be the firewall that prevents interference. This type of separation must be achieved not only on the lexical level, but also on the phonological, syntactic, and semantic levels. In effect, this work undoes the early parasitic association of concepts that the beginning second language learner used to bootstrap the first phases of learning. The result of this process is the tightening of
Extending the Competition Model
45
within-language links in contrast to between-language links. In this way, linguistic modularity emerges from the formation of resonant connections. However, the establishment of a non-parasitic L2 lexicon does not produce full insulation between languages, even in the most advanced bilinguals (Grainger & Dijkstra, 1992). The interactive nature of language processing continues to promote transfer and interference, even when the lexicons and grammars of both languages are fully acquired. For those bilinguals and multilinguals who acquire their languages simultaneously during childhood, separate lexicons and grammars are constructed directly and there is no need to go through a process of undoing the initial connections formed through transfer (De Houwer, 1995; Grosjean, 1982). However, even in simultaneous bilinguals, some transfer and interference occurs, due to the interactive nature of cognitive processing. Orthography The United States Foreign Service has developed a system of classifying languages in terms of their difficulty for L1 English speakers. In this system, most of the Indo-European languages of Europe are on the easiest level of difficulty. The second level includes Indo-European languages with structured grammars such as Hindi, German, or Lithuanian. It also includes some non-Indo-European languages such as Bahasa Indonesia. Level 3 includes non-Indo-European languages like Finnish or Hungarian. The next level includes languages with relatively regular non-Roman orthographies such as Arabic. The most difficult are the Asian languages with truly complex non-Roman scripts. This order of difficulty reflects, in large part, the extent to which problems with orthography can determine difficult in learning a language. One might think that this can be overcome by simply teaching the language in Romanized form. This is certainly of some value. However, because the culture itself does not use Romanized script, this method has clear limitations. As in the case of the acquisition of the lexicon, the learning of a complex new orthography, such as the Chinese-Japanese-Korean system of characters, must rely on extensive construction of resonant connections. The learner must learn to write the characters in order. Each character must be analyzed into its component radicals and the meanings of the radicals must be learned. Once this is done, the characters must then be associated back to newly learned words and further resonant connections must be established.
Syntax By learning words in larger chunks, the child acquires data that facilitates the acquisition of syntactic categories. For example, the German child learns ein guter Mann and this provides good clues for the gender of the noun and the whole system of agreement. Same of the use of the definite article in phrases such as "I saw the Grand Canyon flying to New York." All of these rote phrases with their associated clues form a resonant ensemble.
46
Brian MacWhinney
Transfer also affects the acquisition of grammar. In the Competition Model, early syntactic learning involves the setting of weights between grammatical forms (word order, morphology, stress, lexical subclassification) and grammatical roles (agent, theme, recipient, coreferent). The L2 syntactic system is initially parasitic on L1 in the same way that L2 phonology and lexicon are parasitic on L 1. This means that the initial setting of weights in the L2 network is based on the weights in L1. However, learning of L2 involves more than simple retuning of these weights. In some cases, the second language requires the learner to seek out entirely new conceptual or discourse distinctions that were ignored in the first language, but which are now obligatory grammaticaJ contrasts in the new language. A prime example of this type of restructuring might be the foreigner's attempts to pick up the category structure underlying the two major verbal conjugations of Hungarian. Every time a speaker of Hungarian uses a verb, he or she must decide whether it should be conjugated as transitive or intransitive. Making this choice is not a simple matter. The intransitive conjugation is used not only when the verb is intransitive, but also when the direct object is modified by an indefinite article or by no article at all. It is also used when it is in the first or second person, when the head of the relative clause is the object within the relative clause, when the direct object is quantified by words like each, no, and so on. For example, the intransitive conjugation is used when a Hungarian says John runs, John eats an apple, John eats your apple, and John eats no apple. On the other hand, the transitive conjugation is used when the object is definite, when it is modified by a third person possessive suffix, when it is possessed by a third person nominal phrase, and so on. Thus, the transitive or definite conjugation is used when the Hungarian wants to say, John eats the apple or John eats Bill's apple, whereas the intransitive is used to say John eats an apple. There are some 13 conditions which, taken together, control the choice between the transitive and intransitive conjugations (MacWhinney & Pl6h, 1997). There is no single principle that can group together these 13 conditions. Instead, transitivity, definiteness, and referential disambiguation all figure in as factors in making this choice. Not surprisingly, L2 learners of Hungarian have a terrible time marking this distinction; errors in choice of the conjugation of the verb are the surest syntactic cue that the learner is not a native Hungarian. In order to acquire this new category, the L2 learner begins by attempting to transfer from L 1. To some degree this can work. The learner attempts to identify the intransitive with the English intransitive. However, the fact that transitive sentences also take the intransitive if the objects are indefinite tends to block the simple application of this conceptual structure. In the end, the leamer must be resigned to picking up the pieces of this new category one by one and restructuring them together into a working system. Here is an area where attempts at formal linguistic analysis on the learner's part only make matters worse. If the leamer had proceeded like a Hungarian child (MacWhinney, 1974), he or she would have learned the conjugations by generalizing from a rich database of collocations and phrases. The adult needs to amplify this case-based approach to learning with a way of focusing on contrastive structures in which cues are competing. For the adult, such focusing on particularly difficult parts of a grammatical system will increase the efficiency of acquisition.
Extending the Competition Model
47
In many cases, the transfer of syntactic patterns from L1 to L2 is structurally correct, but pragmatically inaccurate. For example, Tr6vise (1986) observes that French speakers make excessive use of topicalization structures in English in the form of structures corresponding to left-dislocations, right-dislocations, and c'est que in French. Although these structures are all permissible in English, the actual conditions on their usage are far more restrictive than in French. Similarly, Seliger (1989) notes that Hebrew learners of English tend to systematically underuse the passive. He attributes this underusage to the relatively tighter, genre-dependent conditions on the use of the passive in Hebrew. In general, it is clear that simple transfer of an L1 structure to L2 is not sufficient to guarantee correct usage, since both underutilization and overutilization can occur until the full conditions governing the use of a construction in L2 are learned. Competition Model studies use the interpretations of simple transitive sentences to estimate the strength of cues in L2 learners. Because the experimental design places cues into competition orthogonally, we can use mathematical techniques to estimate the strength of each cue in terms of its ability to determine the shape of our experimental data. We find uniformly that the learning of sentence processing cues in a second language is a gradual process. The process begins with L2 cue weight settings that are close to those for L I. Over time, these settings change in the direction of the native speakers' settings for L2. The pattern of results found in this research is clearest in the data from McDonald's studies of English-Dutch and Dutch-English second language learning (McDonald, 1987b). Figure 1 shows the decline in the strength of the use of word order by English learners of Dutch over increased levels of competence. In this graph the monolingual cue usage pattem for English is given in the first column and the monolingual Dutch pattern is given on the right. Between these two patterns, we see a declining use of word order and an increasing use of case inflection across three increasing levels of learning of Dutch. In Figure 2, we see exactly the opposite pattern for Dutch learners of English. These results and others like them constitute strong support for the application of the Competition Model to second language learning. Figure 2 depicts a set of connections between forms and functions for the processing of cues to sentence interpretation. It is possible to convert this intuitive analysis of the subject-marking system into an actual running network model. Janice Johnson and I have done this by giving the various surface cues specified in Figure 1 as input to a "recurrent" network of the type developed by Elman (1990). This network is able to simulate the learning of Dutch and English. The model takes as input a corpus of over 1000 sentences in each language. This corpus contains grammatical sentences from a wide variety of sentence types, including relative clauses, simple transitives, imperatives, subordinates, pronominalizations, and grammatical word order variations. After training in either language as L1, the model is given ungrammatical Competition Model sentences as test materials. At this point, we find cue interpretation patterns characteristic of L1 learners. In order to model second language learning, we then continue with a period of training with sentences from both languages. We then see exactly the pattern of cue development reported in Figures 1 and 2 from McDonald (McDonald, 1987b, 1989).
48
Brian MacWhinney 100
.-.,...
Noun animacy
80
Case inflection
T
....
Word order
!=t I=1
t-
w 6o 8
Y
C
~>
40
e-
89 2o
English
E/D 1
E/D 2
E/D 3
Dutch
Figure 1. Changes in cue strength as English speakers learn Dutch (McDonald, 1987b). 100
L
80
"ID tO
8
60
t~ e-t~ L_ t~ >
40
Noun Animacy
t~ r-
Case Inflection Word Order
20 A
v
I
Dutch
,
9
I
DIE 1
9
I
DIE 2
,
I
DIE 3
,
English
Group
Figure 2. Changes in cue strength as Dutch speakers learn English (McDonald, 1987b).
Extending the Competition Model
49
Pragmatics The acquisition of pragmatic patterns is also heavily influenced by L1 transfer. When we first begin to use a second language, we may extend our L1 ideas about the proper form of greetings, questions, offers, promises, expectations, turn-taking, topic expansion, face-saving, honorifics, presuppositions, and implications. If the two cultures are relatively similar, much of this transfer will be successful. However, there will inevitably be some gaps. In many cases, the L2 learner will need to eventually reconstruct the entire system of pragmatic patterns in the way they were learned by the child acquiring L1. Much of this learning is based on specific phrases and forms. For example, the L1 learners' understanding of greetings is tightly linked to use of specific phrases such as Guten Morgen or byebye. Learning about how and when to use specific speech acts is linked to learning about forms such as could you? listen, and why not? Learning these forms in a concrete context is important for both L1 and L2 learners. However, pragmatics involves much more than simple speech act units or pairs. We also need to learn larger flames for narratives, argumentation, and polite chatting. By following the flow of perspectives and topics in conversations (MacWhinney, 1999), we can eventually internalize models of how discourse represents reality in both L1 and L2.
Morphosyntax Learning of the morphosyntax of a second language is very different from learning of the other five areas we have discussed. This is because, in morphosyntax, it is typically impossible to transfer from L 1 to L2. For example, an English learner of German cannot use the English noun gender system as a basis for learning the German noun gender system. This is because English does not have a fully elaborated noun gender system. Similarly, a Spanish learner of Chinese, cannot use L1 knowledge to acquire the system of noun classifers, because Spanish has no noun classifiers. Chinese learners of English cannot use their L1 forms to learn the English contrast between definite, indefinite, and zero articles. This is because Chinese makes no overt distinctions in this area, leaving the issue of definiteness to be marked in other ways, if at all. The fact that morphosyntax is not subject to transfer is a reflection of the general Competition Model dictum that "everything that can transfer will." In the areas of phonology, lexicon, orthography, syntax, and pragmatics, we see attempts to transfer. However, in morphology we see no transfer, because there is no basis i~or transfer. The exception here is between closely related languages. For example, when learning French, a Spanish speaker can profitably transfer L1 noun gender assignments. Although this will lead to some mistakes, the overall effect of this transfer is positive. Learning of morphosyntax is heavily reliant on predictive association. Neural network models of the learning of the English past tense, German declension, Italian agreement, Arabic numerals, and other morphosyntactic systems have shown how closely this learning can be modeled using neural network simulations. However, it would be a mistake to think that L2 learning is identical to L1 learning for morphosyntax, as we will see in the next section.
50
Brian Mac Whinney Utterance-Based and Word-Based Learning
There are many sources of individual differences in language learning. The age of learning of L2 is certainly the largest single factor. After a certain age, it becomes difficult to block the transfer of an L I accent. Other central variables are motivation, opportunity for exposure, working memory span, motoric ability, and the presence of any organic neural limitations. Among these many factors, one that is particularly interesting is the contrast between utterance-based and word-based learning. In the child language literature, children who prefer word-based learning are said to be analytic and nominal. Children who prefer utterance-based learning are said to be gestaltist and phrasal. This sharp distinction between the two learning styles shows itself most clearly during the first stages of L1 learning and then vanishes. Presumably, the distinction evaporates because the nominal children find that they need to pay some attention to whole phrases and the phrasal children come to deal with individual words and morphemes. L2 learners are also influenced by this dichotomy. Some learning methods emphasize analysis of words and phrases into their components and others emphasize learning of large units, sometimes through pattern drill. Because L2 learning is driven so heavily by transfer, it is virtually impossible for the L2 learner to avoid analytic, word-based learning. Textbooks, grammars, and dictionaries are heavily biased toward word-based learning, and it is difficult not to rely on these resources. If the L2 learner is given the same type of input as the L1 child, utterance-based learning is possible. However, L2 learners almost never have access to this type of input. As a result, they fall back on word-based learning. This reliance on word-based learning has several consequences. The first is positive, since we know that, in terms of time spent in language learning, adult L2 learners are more effective than younger L2 learners in vocabulary learning (Snow & Hoefnagel-Hohle, 1978). If adult learners make good use of resonant learning and orthographic support, they can acquire good control over the L2 lexicon. With careful attention to phonological contrasts, adults can also acquire basic phonological competence, although they may never be able to fully block penetration of L 1 accent. However, L2 learners who rely on word-based learning will have problems at the phrasal and syntactic levels. One way in which this problem will reflect itself is in the acquisition of morphosyntactic patterns. Consider the example of learning of German gender. The analytic adult learner tends to acquire German nouns as single words. At best, the learner may try to associate Mann with der by learning the phrase der Mann. Without creating some particular resonant relation between Mann and der, even this type of learning is not very successful. A phrasal learner, on the other hand, will pick up units such as ein guter Mann, fiir diesen Manner, mein Mann, and so on. From these stored phrases, which are themselves linked to specific interactional scenes with rich expressive content, the learner can then extract the relational patterns of the German gender system. L1 learners rely on phrasal learning as input to a neural network method for acquiring the predictive associations in the pattern. L2 learners cannot do this effectively without focusing their attention on these phrasal units. Fortunately, good teaching and learning practices can remedy this problem and facilitate this aspect of L2 learning.
Extending the Competition Model
51
A further consequence of the heavy L2 reliance on word learning is a reliance on a particular translations strategy that maps words, rather than phrases. For example, a beginning L2 learner of-Spanish may want to say "I want to go to the store." Instead of saying "Quiero ir al mercado," the student says, "Yo querer a ir a el mercado." Here is a breakdown of the word-by-word translation strategy: I want to go Yo querer a ir
to the store a el mercado
Learners quickly repair the worst parts of this type of word-by-word translation. For example, the L2 Spanish learner will soon learn that the Spanish infinitive incorporates the article "a" and this can therefore be omitted. Then the learner realizes that the subject pronoun can be dropped, since person is marked on the verb. Finally, the learner would contract "a" and "el" into "al". For languages of markedly different structure, modifications must be even more radical. For example, when learning to form questions in Cantonese, the learner must acquire distinctions between a set of four sentence final particles, some of which differ only in tone. In addition, wh-movement has to be suppressed in favor of in situ wh. The actual system of wh-words also differs conceptually in several ways. Then, for yes-no questions, the learner must substitute aux-movement with a form that sounds something like know not know? Once all of these modifications of the basic translation strategy have been achieved, the learner starts to produce questions in a Cantonese format. In view of the overall word-based nature of L2 learning, it is probably fruitless for an instructor to fight against the translation strategy in sentence production. Instead, the instructor can provide the learner with simple and evocative whole phrases that illustrate the ways in which the translation strategy can be corrected.
A Unified Account
The search for a unified theory of cognition has been a major goal in models of psychological processes. Some accounts have sought to produce this unified theory by relying primarily on a single cognitive mechanism, such as chunking, predictive association, or analogy. Dual-process accounts try to limit the number of mechanisms to two. These attempts to reduce all learning to one or two mechanisms are ambitious and interesting. However, in the end, they will fail, since it is clear that learners rely on more than one or two learning mechanisms. This should not be too surprising, since the environment provides us with many different types of learning challenges. Evolution has had hundreds of millions of years to work out alternative mechanisms for different learning challenges. Many of these mechanisms share core components for processing and storage, but their use of these components may vary in accord with the learning task. Recent hominid evolution and the development of human language and representational systems has further extended the basic set of learning mechanisms. This means that a unified account of language learning will need to provide a role for more than just one or two mechanisms.
52
Brian Mac Whinney
The empirical exploration of this extended version of the Competition Model will place increased weight on the study of individual differences in language learning. Students of first language learning have often noted major individual differences. However, these differences tend to vanish by about age 5, since children of this age become uniformly competent in their L1, despite the initial differences in reliance on particular mechanisms. Because L2 learning is more protracted, we can see the effects of individual differences more clearly. For example, we can see how some learners rely heavily on reformulation and error correction, whereas others rely primarily on the less observable processes of resonant formation of associations. The extended version of the Competition Model emphasizes the links between particular learning mechanisms and particular linguistic tasks. It also emphasizes linkages between learning mechanisms. For example, as we have seen, the influence of transfer in L2 learning has major secondary impact on the use of all the other learning mechanisms. To deal with transfer, adult learners must then make particular use of resonance and error detection. This means that our models of second language learning must be quite specific about the details of the learning process, since the process shifts from one mechanism to another across the course of learning. If we want to study these changes using group study methodology, we will have to focus more on specific attempts to promote use of particular learning strategies for particular target linguistic structures at particular points in development. Using data of this type, we can begin to sketch out a more dynamic view of language learning.
References
Anderson, J. (1993). Rules of the mind. Hillsdale, N J: Lawrence Erlbaum Associates. Anderson, J., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ: Erlbaum. Aslin, R. N., Saffran, J. RI,& Newport, E. L. (1999). Statistical learning in linguistic and nonlinguistic domains. In B. MacWhinney (Ed.), The emergence of language (pp. 359-380). Mahwah, NJ: Lawrence Erlbaum Associates. Atkinson, R. (1975). Mnernotechnics in second-language learning. American Psychologist, 30, 821-828. Bates, E., & MacWhinney, B. (1981). Second language acquisition from a functionalist perspective: Pragmatic, semantic and perceptual strategies. In H. Winitz (Ed.), Annals of the New York academy of sciences conference on native and foreign language acquisition (pp. 190-214). New York: New York Academy of Sciences. Berlin, B., & Kay, P. (1969). Basic color terms: Their universality and evolution. Berkeley: University of California Press. Blackwell, A., & Bates, E. (1995). Inducing agrammatic profiles in normals: Evidence for the selective vulnerability of morphology under cognitive resource limitation. Journal of Cognitive Neuroscience, 7, 228-257.
Extending the Competition Model
53
Bybee, J., & Hopper, P. (2001). Frequency and the emergence of linguistic structure. Amsterdam: John Benjamins. Carey; S. (1978). The child as word learner. In M. Halle, J. Bresnan, & G. Miller (Eds.), Linguistic theory and psychological reality (pp. 264-293). Cambridge, MA: MIT Press. Carroll, S., & Swain, M. (1993). Explicit and implicit feedback. Studies in Second Language Acquisition, 15, 357-386. Chomsky, N. (1965). Aspects of the theory of syntax. Cambridge, MA: MIT Press. Corder, S. P. (1983). A role for the mother tongue. In S. Gass & L. Selinker (Eds.), Language transfer in language learning. Rowley, MA: Newbury House. Damasio, A. R., & Damasio, H. (1994). Cortical systems for retrieval of concrete knowledge: The convergence zone framework. Cambridge, MA: MIT Press. de Bot, K., & van Montfort, R. (1988). 'Cue-validity' in het Nederlands als eerste en tweede taal. Interdisciplinair Tijdschrift voor Taal en Tekstwetenschap, 8, 111120. De Houwer, A. (1995). Bilingual language acquisition. In P. Fletcher & B. MacWhinney (Eds.), The handbook of child language (pp. 219-250). Oxford: Blackwell. Edelman, G. (1987). Neural Darwinism." The theory of neuronal group selection. New York: Basic Books. Ellis, R. (1994). A theory of instructed second language acquisition. In N. C. Ellis (Ed.), Implicit and explicit learning of language (pp. 79-114). San Diego: Academic. Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179-212. Elman, J. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195-225. Epstein, S., Flynn, S., & Martohardjono, G. (1996). Second language acquisition: Theoretical and experimental issues in contemporary research. Behavioral and Brain Sciences, 19, 677-758. Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cambridge: MIT Press. Flege, J., & Davidian, R. (1984). Transfer and developmental processes in adult foreign language speech production. Applied Psycholinguistics, 5, 323-347. Flege, J., Yakagi, J., & Mann, V. (1995). Japanese adults can learn to produce English "r" and "1" accurately. Language Learning, 39, 23-32. Gass, S. (1987). The resolution of conflicts among competing systems: A bidirectional perspective. Applied Psycholinguistics, 8, 329-350. Gentner, D., & Markman, A. (1997). Structure mapping in analogy and similarity. American Psychologist, 52, 45-56. Gerver, D. (1974). The effects of noise on the performance of simultaneous interpreters: Accuracy of performance. Acta Psychologica, 38, 159-167. Grainger, J., & Dijkstra, T. (1992). On the representation and use of language information in bilinguals. In R. J. Harris (Ed.), Cognitive processing in bilinguals. (pp. 207-220). Amsterdam: Elsevier Science.
54
Brian MacWhinney
Grosjean, F. (1982). Life with two languages: An introduction to bilingualism. Cambridge, MA: Harvard University Press. Grossberg, S. (1978). A theory of human memory: Self-organization and performance of sensory-motor codes, maps, and plans. Progress in Theoretical Biology, 5, 233-374. Grossberg, S. (1987). Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, 11, 23-63. Gupta, P., & MacWhinney, B. (1997). Vocabulary acquisition and verbal shortterm memory: Computational and neural bases. Brain and Language, 59, 267-333. Hancin-Bhatt, B. (1994). Segment transfer: A consequence of a dynamic system. Second Language Research, 10, 241-269. Harrington, M. (1987). Processing transfer: Language-specific strategies as a source of interlanguage variation. Applied Psycholinguistics, 8, 351-378. Hauser, M., Newport, E., & Aslin, R. (2001). Segmentation of the speech stream in a non-human primate: Statistical learning in cotton-top tamarins. Cognition, 78, B53-B64. Hofst~dter, D. (1997). Fluid concepts and creative analogies: Computer models of the fundamental mechanisms of thought. London: Allen Lane. Ijaz, H. (1986). Linguistic and cognitive determinants of lexical acquisition in a second language. Language Learning, 36, 401-451. Kay, P., & Fillmore, C. J. (1999). Grammatical constructions and linguistic generalization: The "what's X doing Y?" construction. Language, 75, 1-33. Kilborn, K. (1989). Sentence processing in a second language: The timing of transfer. Language and Speech, 32, 1-23. Kilborn, K., & Cooreman, A. (1987). Sentence interpretation strategies in adult Dutch-English bilinguals. Applied Psycholinguistics, 8, 415-431. Kilborn, K., & Ito, T. (1989). Sentence processing in Japanese-English and Dutch-English bilinguals. In B. MacWhinney & E. Bates (Eds.), The crosslinguistic study of sentence processing (pp. 257-291). New York: Cambridge University Press. Krashen, S. (1994). The Input Hypothesis and its rivals. In N. C. Ellis (Ed.), Implicit and explicit learning of languages (pp. 45-78). San Diego: Ac~idemic. Kroll, J., & Sholl, A. (1992). Lexical and conceptual memory in fluent and nonfluent bilinguals. In R. J. Harris (Ed.), Cognitive processing in bilinguals (pp. 191-206). Amsterdam: Elsevier Science. Lindsay, P., & Norman, D. (1977). Human information processing: An introduction to psychology. New York: Academic Press. Liu, H., Bates, E., & Li, P. (1992). Sentence interpretation in bilingual speakers of English and Chinese. Applied Psycholinguistics, 13, 451-484. Locke, J. L. (1995). Development of the capacity for spoken language. In P. Fletcher & B. MacWhinney (Eds.), The handbook of child language. Oxford: Basil Blackwell. MacDonald, M., & MacWhinney, B. (1990). Measuring inhibition and facilitation from pronouns. Journal of Memory and Language, 29, 469-492. MacWhinney, B. (1974). How Hungarian children learn to speak. Unpublished doctoral dissertation, University of California, Berkeley.
Extending the Competition Model
55
MacWhinney, B. (1978). The acquisition of morphophonology. Monographs of
the Society for Research in Child Development, 43, 1-123. MacWhinney, B. (1987). The Competition Model. In B. MacWhinney (Ed.),
Mechanisms of language acquisition (pp. 249-308). Hillsdale, NJ: Lawrence Erlbaum. MacWhinney, B. (1993). The (il)logical problem of language acquisition,
Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society (pp. 61-70). Hillsdale, NJ: Lawrence Erlbaum Associates. MacWhinney, B. (1994). The dinosaurs and the ring. In R. Corrigan, S. Lima, & M. Noonan (Eds.), The reality of linguistic rules (pp. 283-320). Amsterdam: John Benjamins. MacWhinney, B. (1999). The emergence of language from embodiment. In B. MacWhinney (Ed.), The emergence of language (pp. 213-256). Mahwah, NJ: Lawrence Erlbaum. MacWhinney, B. (200 l a). The Competition Model: The input, the context, and the brain. In P. Robinson (Ed.), Cognition and second language instruction. New York: Cambridge University Press. MacWhinney, B. (200 l b). The gradual evolution of language. In B. Malle & T. Giv6n (Eds.), The evolution of language. Philadelphia: Benjamins. MacWhinney, B. (2001c). Models of second language acquisition. In P. Robinson (Ed.), Cognition and second language instruction. New York: Cambridge University Press. MacWhinney, B., & Bates, E. (Eds.). (1989). The crosslinguistic study of sentence processing. New York: Cambridge University Press. MacWhinney, B., & Pl6h, C. (1988). The processing of restrictive relative clauses in Hungarian. Cognition, 29, 95-141. MacWhinney, B., & P16h, C. (1997). Double agreement: Role identification in Hungarian. Language and Cognitive Processes, 12, 67-102. MacWhinney, B. J., Leinbach, J., Taraban, R., & McDonald, J. L. (1989). Language learning: Cues or rules? Journal of Memory and Language, 28, 255-277. Matessa, M., & Anderson, J. (2000). Modeling focused learning in role assignment. Language and Cognitive Processes, 15, 263-292. McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception: Part 1. An account of the basic findings. Psychological Review, 88, 375-402. McDonald, J. L. (1987a). Assigning linguistic roles: The influence of conflicting cues. Journal of Memory and Language, 26, 100-117. McDonald, J. L. (1987b). Sentence interpretation in bilingual speakers of English and Dutch. Applied Psycholinguistics, 8, 379-414. McDonald, J. L. (1989). The acquisition of cue-category mappings. In B. MacWhinney & E. Bates (Eds.), The crosslinguistic study of language processing (pp. 375-396). New York: Cambridge University Press. McDonald, J. L., & Heilenman, K. (1991). Determinants of cue strength in adult first and second language speakers of French. Applied Psycholinguistics, 12, 313348. McDonald, J. L., & MacWhinney, B. (1989). Maximum likelihood models for sentence processing research. In B. MacWhinney & E. Bates (Eds.), The
56
Brian MacWhinney
crosslinguistic study of sentence processing (pp. 397-421). New York: Cambridge University Press. McDonald, J. L., & MacWhinney, B. (1991). Levels of learning: A microdevelopmental study of concept formation. Journal of Memory and Language, 30, 407-430. McDonald, J. L., & MacWhinney, B. J. (1995). The time course of anaphor resolution: Effects of implicit verb causality and gender. Journal of Memory and Language, 34, 543-566. Menn, L., & Stoel-Gammon, C. (1995). Phonological development. In P. Fletcher & B. MacWhinney (Eds.), The handbook of child language (pp. 335-360). Oxford: Blackwell. Merriman, W. (1999). Competition, attention, and young children's lexical processing. In B. MacWhinney (Ed.), The emergence of language (pp. 331-358). Mahwah, NJ: Lawrence Erlbaum. Miyake, A., Carpenter, P., & Just, M. (1994). A capacity approach to syntactic comprehension disorders: Making normal adults perform like aphasic patients. Cognitive Neuropsychology, 11, 671-717. Moon, C., Cooper, R. P., & Fifer, W. P. (1993). Two-day infants prefer their native language. Infant Behavior and Development, 16, 495-500. Morton, J. (1970). A functional model for memory. In D. A. Norman (Ed.), Models of human memory (pp. 203-248). New York: Academic Press. Newell, A., & Rosenbloom, P. S. (1981). Mechanisms of skill acquisition and the law of practice. In J. R. Anderson (Ed.), Cognitive skills and their acquisition (pp. 1-55). Hillsdale, NJ: Lawrence Erlbaum Associates. Newell, A., & Simon, H. (1972). Human problem solving. Englewood Cliffs, N.J.: Prentice-Hall. O'Malley, M., & Chamot, A. (1990). Learning strategies in second language acquisition. Cambridge: Cambridge University Press. Pulvermialler, F., & Schumann, J. (1994). Affect in second language learning. Language Learning, 44, 681-734. Reder, L. (Ed.). (1996). Implicit memory and metacognition. Mahwah, NJ: Lawrence Erlbaum Associates. Schmidt, R. (1994). Implicit learning and the cognitive unconscious: Of artificial grammars and SLA. In N. C. Ellis (Ed.), Implicit and explicit learning of languages (pp. 165-210). San Diego: Academic. Seleskovitch, D. (1976). Interpretation: A psychological approach to translating. In R. W. Brislin (Ed.), Translation: Application and research. New York: Gardner. Seliger, H. (1989). Semantic transfer constraints on the production of English passives by Hebrew-English bilinguals. In H. Dechert & M. Raupach (Eds.), Transfer in language production (pp. 21-34). Norwood: NJ: Ablex. Selinker, L., Swain, M., & Dumas, G. (1975). The interlanguage hypothesis extended to children. Language Learning, 25, 139-152. Smith, L. B. (1999). Children's noun learning: How general processes make specialized learning mechanisms. In B. MacWhinney (Ed.), The emergence of language (pp. 277-304). Mahwah, NJ: Lawrence Erlbaum Associates. Snow, C. (1995). Issues in the study of input: Finetuning, universality, individual and developmental differences, and necessary causes. In P. Fletcher & B.
Extending the Competition Model
57
MacWhinney (Eds.), The handbook of child language (pp. 180-193). Oxford: Blackwells. Snow, C., & Hoefnagel-Hohle, M. (1978). The critical period for language acquisition: Evidence from second language learning. Child Development, 49, 1114-1128.s
Sokolov, J. L. (1988). Cue validity in Hebrew sentence comprehension. Journal of Child Language, 15, 129-156. Sokolov, J. L. (1989). The development of role assignment in Hebrew. In B. MacWhinney & E. Bates (Eds.), The crosslinguistic study of sentence processing (pp. 158-184). New York: Cambridge. Stockwell, R., Bowen, J., & Martin, J. (1965). The grammatical structures of English and Spanish. Chicago: University of Chicago Press. Tomasello, M., & Herron, C. (1988). Down the Garden Path: Inducing and correcting overgeneralization errors in the foreign language Classroom. Applied Psycholinguistics, 9, 237-246. Tr6vise, A. (1986). Is it transferable, topicalization? In K. Kellerman & M. Sharwood Smith (Eds.), Crosslinguistic influence in second language acquisition (pp. 186-206). New York: Pergamon. van Heuven, W., Dijkstra, T., & Grainger, J. (1998). Orthographic neighborhood effect in bilingual word recognition. Journal of Memory and Language, 39, 458483.
This Page Intentionally Left Blank
Bilingual Sentence Processing- R.R. Heredia and J. Altarriba (Editors) 9 2002 Elsevier Science B.V. All rights reserved.
3
A Self-Organizing Connectionist Model of Bilingual Processing
Ping Li and Igor Farkas University of Richmond
Abstract Current connectionist models of bilingual language processing have been largely restricted to localist stationary models. To fully capture the dynamics of bilingual processing, we present SOMBIP, a self-organizing model of bilingual processing that has learning characteristics. SOMBIP consists of two interconnected self-organizing neural networks, coupled with a recurrent neural network that computes lexical co-occurrence constraints. Simulations with our model indicate that (1) the model can account for distinct patterns of the bilingual lexicon without the use of language nodes or language tags, (2) it can develop meaningful lexicalsemantic categories through self-organizing processes, (3) it can account for a variety of priming and interference effects based on associative pathways between phonology and semantics in the lexicon, and (4) it can explain lexical representation in bilinguals with different levels of proficiency and working memory capacity. These capabilities of our model are due to its design characteristics in that (a) it combines localist and distributed properties of processing, (b) it combines representation and learning, and (c) it combines lexicon and sentences in bilingual processing. Thus, SOMBIP serves as a new model of bilingual processing and provides a new perspective on connectionist bilingualism. It has the potential of explaining a wide variety of empirical and theoretical issues in bilingual research.
Introduction Connectionism, parallel distributed processing (PDP) models, or neural networks have significantly influenced research in the cognitive sciences in the last fifteen years. Language, as one of the most important human cognitive components, has received in-depth treatments since the beginning of connectionist research. The acquisition of the English past tense (Rumelhart & McClleland, 1986), the recognition of speech (McClleland & Elman, 1986), and the processing of sentences (McClleland & Kawamoto, 1986) are among the earliest domains of connectionist
60
Ping Li and Igor Farkas
research in the original PDP models. Unfortunately, connectionist models or modeling have had very limited impact on the field of bilingualism as a whole. To this date, there are only a handful of connectionist models that are implemented to account for the processing or representation of the bilingual mental lexicon (see our review below). This lack of interaction between connectionism and bilingualism is lamentable, and it gives us a good reason to pursue research that would fill this gap. In this chapter, we present a connectionist model, more specifically, a selforganizing neural network model of bilingual processing and representation. Let us begin by reviewing a few important lines of research in connectionist bilingualism. In particular, we will examine the Bilingual Interactive Activation (BIA) model (Grainger, 1993; Dijkstra & van Heuven, 1998; van Heuven, 2000), the Bilingual Model of Lexical Access (BIMOLA; Grosjean, 1988, 1997; L6wy & Grosjean, 2001), and the simple recurrent network (SRN) model of bilingual memory (French, 1998). The first two belong to the so-called "localist" models, while the last one belongs to "distributed" models. In localist models, a word or a concept is represented by a single, unitary processing node in the network, whereas in distributed models, information about a word or a concept is distributed across several or many different units of processing. The localist models, that is, BIA and BIMOLA, are based on earlier interactive models of word recognition, that is, the interactive activation model (IA) of McClelland and Rumelhart (1981) and the TRACE model of McClelland and Elman (1986), respectively. In the IA model, there are three levels of nodes, with ascending complexity: (1) features of a letter such as curves, straight lines, or crossbars, (2) individual letters, and (3) words. Information at all three levels can interact with each other during the word recognition process, which may flow both "bottom-up" (features to letters to words) and "top-down" (words to letters to features). Within levels, nodes compete for activation (thus inhibiting each other); across levels, nodes either inhibit or excite each other. According to IA, it is these inhibitory and excitatory connections that give rise to the appropriate activation of patterns that correspond to the perception of words. The BIA model is a straightforward extension of the IA model. It consists of four levels of nodes: features, letters, words, and languages. As in IA, there are connections between nodes at each level and between nodes across levels. All nodes at the word level are interconnected with mutual inhibition. Feature units activate appropriate letters, and letter units activate appropriate words in the appropriate language. B IA uses the same parameters to regulate interactions within and across levels as in the original IA model. What is special to the BIA model (apart from the incorporation of two lexicons) are the language nodes (one for English and one for Dutch). Language nodes in B IA function as an important mechanism for the selection or inhibition of words in one or the other language, given that the model argues for and implements the language-independent access hypothesis, according to which words from different languages are represented in an integrated lexicon and are simultaneously contacted during word recognition. Results from B IA
Connectionist Model of Bilingual Processing
61
simulations suggest that the model is able to account for empirical results that directly support the language-independent access hypothesis, while at the same time is compatible with data previously thought to support the language-selective access hypothesis (Dijkstra & van Heuven, 1998; van Heuven, 2000). While the BIA model focuses on visual word recognition, the BIMOLA model (Grosjean, 1988, 1997; L6wy & Grosjean, 2001) aims at handling the recognition of spoken words. It was partly inspired by the TRACE model (McClelland & Elman, 1986), an IA-based model for acoustic input. Like TRACE, it consists of three levels of nodes, corresponding to features, phonemes, and words. There is no separate level of language nodes in BIMOLA, unlike in BIA. The feature level is common to both languages, but the phoneme and word levels are organized in subsets according to languages (still in the same extended system). Features activate phonemes that, in turn, activate words. Connections (both excitatory and inhibitory) are unidirectional (ascending) between features and phonemes, but bidirectional between phonemes and words. Descending connections from top down (global language activation and higher linguistic information, especially the "bilingual speech mode"; Grosjean, 1997) serve to activate words that, in turn, can activate phonemes. Language activation (or selection) takes place through these descending connections but also through within-language connections at the phoneme and word levels. Compared to the BIA model, BIMOLA can account for language-specific activation without the use of language nodes--it is yet unclear whether language nodes or language tags are necessary components of bilingual processing (Li, 1998a; see also General Discussion). The BIMOLA model is currently being implemented and evaluated against empirical data (L6wy, 2001; L6wy & Grosjean,
2001). Both BIA and BIMOLA can be said to be permanent (or stationary) models, despite their differences in input (visual vs. acoustic) and architecture (with or without language nodes). They differ from some connectionist models with a learning mechanism. In BIA and BIMOLA, the representations are fixed and manually coded, and are designed to capture proficient adult bilingual speakers' mental lexicon (but are not designed to evolve). In contrast, connectionist learning models dynamically develop representations from computing statistical characteristics of the input data (sentences). Along this direction, French (1998) presented a distributed model of bilingual memory based on SRN, a simple recurrent network (Elman, 1990) that learns representations online through sentence processing. As in the original SRN, the model takes in one input word at a time from a continuous stream of sentences and its task is to predict the next word of the current input in the sentence. The input stream is a series of artificially generated sentences of the N-VN structure, with English and French sentences intermixed at the sentence level. Elman (1990) showed that distinct grammatical and semantic categories (e.g., nouns and verbs, animate and inanimate) can emerge in the SRN's hidden-unit representations once the network has learned the next-word prediction task with a reasonable size of sentences. This is because the prediction task involves detecting
62
Ping Li and Igor Farkas
the context in which the current input occurs. With the SRN's exposure to mixed bilingual input, French showed that not only distinct categories emerge within each language in the hidden-unit representations (as in Elman, 1990), but also there are distinct patterns of the two languages: words from the two languages are separated in space on a hierarchical cluster tree of the hidden-unit activations. Note that the network incorporates no mechanism (nodes or tags) to explicitly label words in the two languages. This model provides support to the hypothesis that bilingual memory is organized as a single distributed lexicon rather than two separately stored lexicons according to language. Thus, the model can display distinct behaviors of the two lexicons without invoking separate mechanisms for each language, unlike the BIA model that uses language nodes to separate the bilingual lexicon. The connectionist model of bilingual processing that we present in this paper has the same spirit of the SRN model, but it differs from the above models in three important respects. First, our model combines both learning and representational properties. It is a learning model in the sense that lexical representations of both languages can emerge from the statistical learning of the input speech. This property is similar to that of the SRN, but is based on our network in explicitly modeling lexical co-occurrences in the acquisition of word meanings (Li, 1999, 2000, in press; Farkas & Li, 2001, in press). On the other hand, our model also has some of the representational features of BIMOLA: lexical forms are encoded by articulatory features of the phonemes of words (see also Li & MacWhinney, 2001). The representational characteristics of the lexical forms and meanings can also become clearly discernible on a 2-dimensional topological space, given the self-organizing maps used in our model. In addition, given both word meanings and word forms in the model, learning can occur in the associative links between meanings and forms via Hebbian learning, a biologically plausible mechanism of co-occurrence learning (see later discussion). Second, our model displays both distributed and localist properties. Unlike SRN or other connectionist models that use back-propagation as the learning algorithm, our model employs principles of self-organization, a type of unsupervised learning (Kohonen, 1995). Although the inputs in our self-organizing network are in the form of distributed representations, the 2-D topological map of the network (the output) bears significant similarity to localist representations in that each unit on the map tends to represent one lexical item. The localized patterns on the map allow an "explicit" representation of the lexicon, instead of "implicit" representations as in the hidden-unit activations of an SRN (MacWhinney, 2000a). At the same time, each unit is surrounded by neighboring units on the map that can become coactivated, simulating a distributed lexicon in which similar words or word properties are grouped together (see details below). Third, our model relies on the use of realistic linguistic data as input to the network, in particular, child-directed parental speech. In the SRN model of bilingual memory (French, 1998) as well as many current connectionist models, researchers
Connectionist Model of Bilingual Processing
63
have relied on the use of highly simplified, artificially generated input. Although such inputs are easy to construct and to control for, they are often isolated from realistic language uses, and run the risk of being successful just because of the use of certain properties in the input (see Lachter & Bever, 1988 for an earlier criticism of connectionist representations). For example, French (1998) structured the input data in such a way that the input has a fixed probability of 0.001 of switching from one language to another. In other words, on average, the network will first have learned 1000 sentences in one language before learning any sentences in the other language. We suspect that this artificially determined probability is what causes the network to display disparate behaviors for the two languages. To overcome potential problems associated with this approach to network modeling, we thus rely on corpus-based linguistic data that closely approximate the reality of language use (see also Li, in press, for discussion). In what follows, we first present an overview of our model, SOMBIP, a connectionist self-organizing model of bilingual processing. We then discuss some preliminary simulation results from the model. We conclude by showing how our model can shed light on important issues in bilingual lexical and sentence processing.
The SOMBIP Model
Background The design characteristics of the SOMBIP model are based on our selforganizing neural network model of language acquisition by young children (Farkas & Li, 2001, in press; Li, 1999, 2000, in press). In recent years, self-organizing neural networks have become increasingly important for cognitive and perceptual studies (Hinton & Sejnowski, 1999). Although significant progress has been made with models based on back-propagation, there are known limitations associated with these models, including catastrophic forgetting (inability to remember old information with new learning), scalability (inability to handle realistic, large-scale problems), and above all, its error-driven learning process, a procedure which propagates error signals from the discrepancy between desired and actual outputs to adjust weights. Some of these problems become most transparent when considered in the context of language acquisition (see Li, in press). Consideration of these problems lead us to look for models that bear more biological and psychological plausibility in the context of language learning, and we turn to the self-organizing models. One of the most widely used self-organizing models is Kohonen's (1982, 1989, 1995) self-organizing map (SOM). SOM belongs to the class of "unsupervised" neural networks, because learning in these networks does not require the presence of a supervisor or an explicit teacher; learning is achieved by the system's self-
64
Ping Li and Igor Farkas
organization in response to the input. During learning, the self-organizing process extracts an efficient and compressed internal representation from a highdimensional input space and projects this new representation onto a 2-D topological structure (Kohonen, 1982, 1989, 1995). Several important properties of SOM and related features make it particularly well suited to the study of language. (1) Self-organization. Self-organization in these networks typically occurs in a two-dimensional topological map, where each unit (or node, or neuron) is a location on the map that can uniquely represent one or several input patterns. At the beginning of learning, an input pattern randomly activates a group of the many units on the map, according to how similar by chance the input pattern is to the weight vectors (codevectors) of the units. Once a unit becomes active in response to a given input, the weight vectors of the unit and its neighboring units are adjusted so that they become more similar to the input and will therefore respond to the same or similar inputs more strongly the next time. In this way, every time an input is presented, an area of units will become activated on the map (the so-called activity "bubbles"), and the maximally active units are taken to represent the input. Initially, activation occurs in large areas of the map, but gradually learning becomes more focused so that only the maximally responding unit or units are active. This process continues until all the inputs have found some maximally responding units. (2) Representation. As a result of this self-organizing process, the statistical structures implicit in the high-dimensional input space are represented as topological structures on the 2-D space. In this new representation, similar inputs will end up activating the same units in nearby regions, yielding activity bubbles that become clearly visible on the map. Such a self-organizing process and its representation have clear implications for language acquisition: the formation of activity bubbles may capture critical processes for the emergence of semantic categories in the acquisition of the lexicon. In particular, the network organizes information first in large areas of the map and gradually zeros in onto smaller areas; this zeroing-in is a process from diffuse to focused patterns, as a function of the network's continuous adaptation to the input structure. This process allows us to model the emergence of semantic categories as a gradual process of lexical development. It naturally explains many generalization errors reported in the acquisition literature (elg., substitutions of put for give or fall for drop as reported by Bowerman, 1978, 1982). It also explains language disorders that result from the breakdown of focused activation or the inability to form focused representations (Miikkulainen, 1997; Spitzer, 1999). (3) Hebbian learning. Hebbian learning is not an intrinsic property of SOM, but several SOMs can be connected via Hebbian learning, such as in the multiple feature-map model of Miikkulainen (1993, 1997). Hebbian learning is a wellestablished biologically plausible learning principle, according to which the associative strength between two neurons is increased if the neurons are both active at the same time (Hebb, 1949). The amount of increase may be proportional to the level of activation of the two neurons. In the multiple SOM model developed by
Connectionist Model of Bilingual Processing
65
Miikkulainen, all units on one map are initially connected to all units on the other map. As self-organization takes place, the associations become more focused, such that in the end only the maximally active units on the corresponding maps are associated. Hebbian learning combined with SOM has strong implications for language learning: it can account for the process of how the learner abstracts relationships between word forms, meanings, and grammatical morphology, on the basis of how often they co-occur and how strongly they are co-activated in the representation. Because of these properties, SOM (a) allows us to track the development of the lexicon clearly as an emergent property in the network's self-organization (from diffuse to focused patterns or from incomplete to complete associative links); (b) allows us to model one-to-many or many-to-many associations between forms and meanings in the development of the lexicon; and (c) provides us with a set of biologically plausible and computationally relevant principles to study bilingualism without relying on corrective feedback. It is ideally suited for our task also because the bilingual mental lexicon is constructed for the most part without supervision and undergoes continuous self-organization over time.
Architecture
Our SOMBIP model has been inspired by the multiple self-organizing featuremap model of Miikkulainen (1993, 1997). Miikkulainen proposed an integrated model of memory and natural language processing, in which multiple SOMs dedicated to different levels of information are connected. A sub-component of this model is DISLEX, a SOM model of the lexicon, in which different maps correspond to different linguistic information (orthography, phonology, or semantics) and are connected through associative links via Hebbian learning. Our model has also been inspired by the Hyperspace Analogue to Language (HAL) model (Burgess & Lund, 1997, 1999). In particular, we derive our meaning representations of the lexicon through a word co-occurrence detector (WCD), a mechanism similar to the principle of HAL computation. HAL attempts to capture meaning by reference to global lexical co-occurrences--how many words co-occur with the target word, and how often, in a moving window that runs through a large-scale language corpus (Usenet texts of 300 million words). A co-occurrence matrix for any number of words in a given window is derived, and weighted by the frequency of co-occurrences among words. Thus, a word is represented in HAL as a vector of the column and the row in the matrix that encodes the co-occurrence constraints in a high-dimensional space of language use.
66
Ping Li and Igor Farkas
word form ) self..organizatilon
SOM 1 lexical (phono) map Hebbian I ] leaming
~vord meaning] self-organization
SOM 2 semantic map
Figure l a. SOM1 and SOM2 that self-organize on word forms and word meanings, respectively. They are interconnected via associative pathways, trained by Hebbian .. learning.
SOM2 semantic map
Tm I reduced word repr I
Iq t
representation of word meaning
l qzt~
layer A (''
i""
(w
9 ord localist cod
. ) -~"~,~
one-to--one copy
layer B
* * CI
j ---
= CN
Figure lb. WCD (lower panel) that computes lexical co-occurrence statistics. Layers A and B have full connectivity via modifiable i and r links. Other, one-toone links serve to feed the unit activity from A to B with discrete single time-step delay. SOM2 receives distributed word representations extracted from the modifiable links. The representations are reduced to vectors of uniform length (100 dimensions) as inputs to SOM2 (see Farkas & Li, 2001, in press, for details).
Figure 1 presents a diagrammatic sketch of the different components of SOMBIP. Figure l a depicts the two SOMs used in our model and their interconnections. During learning, a lexical form (phonological input) activates a unit or a group of units on SOM1, and simultaneously, its word meaning (semantic input) activates a unit or a group of units on SOM2. Note that in our current
Connectionist Model of Bilingual Processing
67
simulations we have used only phonological input in SOM1 to simulate the bilingual spoken lexicon; one can easily use orthographic input in SOM1 to simulate the bilingual visual lexicon. SOM1 and SOM2 are connected via associative links, such that the activation on one map can cause an activity to form in the other map. If the direction of the associative activity is from phonology to semantics, comprehension is modeled; if it goes from semantics to phonology, production is modeled. The associative links are trained by Hebbian learning, and the strengths of the connections are adjusted according to the form-meaning pairings in the input, which leads to adaptive associations between the two SOMs. Figure l b depicts WCD (lower panel), a special recurrent neural network that leams the lexical co-occurrence constraints of words. The WCD reads through a stream of input sentences (one word at a time), and given a lexicon sized N, it computes the transitional probabilities of words in the lexicon (see Farkas & Li, in press, for details). Assume that at time t the current word is i, and is represented by a localist vector o - [o 1, ..., ON] in layer A. Previous word j is represented by the vector c -- [c! . . . . , CN] in layer B. The adaptable connections (! and r links) between layers A and B serve to approximate the transitional probabilities between successive words, and as such, they are trained by Hebbian learning with weight decay so that they become normalized. Specifically, the link l/j is updated to approximate p(jt-l[it) (i.e., the probability that the word j precedes the word/). At the same time, the link rji is updated to approximate p(it~t-1), that is, the probability that i follows j. Word i is characterized by a concatenation of vectors !i = [lil, ..., liN ], and r i = [rli . . . . , rNi ], where !i approximates the probability distribution of words preceding i (left context), and r i the probability distribution of words following i (right context). The concatenated vectors, qi = [li, ri], then serve as distributed word representations to SOM2 (upper panel in Figure l b). Because the dimensions of the vectors are determined by the size of the lexicon (2N for any given vector), the vectors, before they are read by SOM2, are also submitted to a dimension-reduction mapping mechanism, which reduces the vectors to lower, fixed dimensions (e.g., 100 units, see Farkas & Li, 2001). As SOM2 takes representations from the WCD vectors qi, SOM1 also takes as its input the phonological representations of words. To represent the phonology o f the bilingual lexicon, we have followed a syllable-based template coding originally developed by MacWhinney and Leinbach (1991) and recently by Li and MacWhinney (2001). This coding scheme has the advantage over traditional phonemic representations in that it can accurately capture the phonological similarities of multisyllabic words (most previous connectionist models have used only monosyllabic words as input). In this scheme, the phonology of a word is made up by combinations of syllables in a metrical grid, and the slots in each grid are made up by bundles of features that correspond to phonemes, C's (consonants) and V's (vowels).
68
Ping Li and Igor Farkas
To code our Chinese-English bilingual lexicon, we used 4 C-slots and 4 V-slots that allowed for the representation of words of one and two syllables, in the template of CVVCCVVC (each CVVC is a syllable; the first C represents the initial consonant, and the last C the final consonant). Thus, the Chinese (Cantonese) word jat (one) is represented in the slots as jaVtCVVC and the English word about is represented as C@VCbaUt. Each C or V is represented by a set of 5 feature units, and the feature values (scaled between 0 and 1) are determined according to the articulatory features outlined by Ladefoged (1982) for English and by the Hong Kong Linguistic Society (1997) for Chinese (the 5 articulatory features are: Sound, Place, Manner, Chromacity, Sonority). For example, the phoneme/i/is represented in both languages as [. 1 . 0 . 0 . 2 . 3 ] , indicating [vowel none none high front] for the 5 features. A separate set of 12 units are used to represent lexical tones in Cantonese (6 tones for each syllable), whereas these units are left empty for English. Thus, each word in the bilingual lexicon is represented by a vector of 52 units in the phonological representation (5 feature units for 8 phoneme-slots plus 12 tonal units). 1 Note that in neither the phonological nor the semantic representations described above is there a label or tag that tells which lexicon (English or Chinese) a given word should belong to. Learning in the two SOMs is standard (Kohonen, 1989). Every SOM unit k has an array of connections in the form of a codevector m k = [ink1, ..., mk,2N ], which learns to approximate the inputs (semantic or phonological vectors) in such a way that every SOM unit becomes "specialized" for a given word, and that the neighboring units will become specialists ("winners") to similar words. During learning, both neighborhood radius and learning rate decrease over time. Task and Procedure
Upon training of the network, a phonological input representation of a word is presented to SOM1, and simultaneously, the semantic representation of the same word is also presented to SOM2. By way of self-organization, SOM1 forms an activity in response to the phonological input, and SOM2 an activity in response to the semantic input. As the two SOMs receive input and continue to self-organize, they also simultaneously form associations through Hebbian learning for all the active units in the two maps that respond to the inputs. The network's task is to create an ordered layout for all the input words in the bilingual lexicon and be able to make the appropriate form-meaning associations. Because our SOMBIP handles a bilingual lexicon, translation equivalents in the two languages are associated with each other in the following way: if the phonology of an English word is presented to SOM1, the semantics of the English word and that of its cross-language translation equivalent in Chinese are also presented to SOM2. Similarly, if the semantics of an English word is presented to SOM2, the phonology of the English word and that of its translation equivalent in Chinese are also presented to SOM1. For example, the word boat and syun are associated by the
Connectionist Model of Bilingual Processing
69
phonology of boat or syun co-occurring with the semantics of boat and syun, and vice versa. This procedure works in the same way for words in both languages, ensuring that translation equivalents have a chance to be associated. Because of the difficulty in determining what words should be counted as translation equivalents (especially with regard to English and Chinese), this procedure applied only to the nouns and verbs in our simulations (i.e., disregarding adjectives, adverbs, pronouns, prepositions, etc.). Multiple translation equivalents (e.g., English tell and Chinese gong and waa) were also associated through co-occurrences in the network, although such cases were rare in our data. As discussed earlier, artificially generated input data are often problematic in matching up with realistic language use. In this study, we used a realistic bilingual data set, the Hong Kong Bilingual Corpus from the CHILDES database (Yip & Matthews, 2000; MacWhinney, 2000b). This corpus contains transcripts of conversations between a child and his native English-speaking father and native Cantonese-speaking mother. The recordings were made when the child was between ages 1 and 3. The parents followed the one parent-one language principle when addressing the child. The language between the parents was mainly Cantonese with a great deal of English mixed in, as is characteristic of the speech of Hong Kong middle class families. Despite the "one parent-one language" principle, the quantity of input from the two languages was not all balanced. On the whole, the child received more Cantonese than English input in his first three years. Because of the relatively young age at which the recordings were made, there was not enough productive speech from the child. However, there was plenty of parental speech as input. We therefore extracted all of the parental speech plus the speech of the student assistants who made the recordings during the home visits, forming the bilingual input speech corpus that we used for our simulations. These speech data also effectively allow us to simulate what the leaming system (human or network) receives in a concurrent bilingual environment and how the system can, on the basis of the input, develop lexical representations from sentence processing (we used a similar procedure in modeling first language acquisition; see Farkas & Li, 2001, in press; Li, in press). A total of 5,154 word types and 185,279 word tokens are found in our bilingual parental corpus, according to the freq (frequency count) output of the CLAN (Child Language Analysis) program (MacWhinney, 2000b). For our purposes we trained our model on the 400 most frequent word types in this corpus, which effectively covers 56% of the entire data. These 400 most frequent words happened to contain 184 Chinese words, and 216 English words. During training, SOM1 received the phonological representations of the 400 words and self-organized on them. The WCD network of our model received the words in the input sentences one at a time, and built semantic representations from the lexical co-occurrence statistics. These representations were submitted to our dimension-reduction mapping so that all vectors had a uniform length of 100 units, and subsequently sent to SOM2 for selforganizing learning. SOM1 and SOM2 were accordingly linked by associative
70
Ping Li and Igor Farkas
links. Note that training in SOM1 and SOM2 was simultaneous, and therefore learning in SOM1 and SOM2 did not begin until the WCD network and the dimension reduction had completed their job. In principle, however, SOM2 need not wait until WCD is done, given that it can self-organize on early stages of semantic representations as WCD learns online. This method has also been implemented in our model as an incremental learning process (see Farkas & Li, 2001).
Results and Discussion In this section, we report results from several simulations with SOMBIP, and our analyses will focus on the network's performance with respect to distinct behaviors in the two languages, the formation of lexical categories, the interlingual priming and interference effects, and effects of proficiency and resource limitation.
Language Separation Without Language Nodes As discussed earlier, an important difference between different connectionist models of bilingualism is whether the model explicitly includes a separate level of language nodes. The BIA model does, whereas the BIMOLA and the bilingual SRN models do not. In B IMOLA, the feature level is common to both languages, and the phoneme and word levels contain subsets of units for each language. The differences between these subsets, coupled with global language activation and higher linguistic information from top down, allow the system to separate the two languages. The SRN approach of French (1998) is more radical, in that it makes no a priori assumptions about the differences between the two languages in the bilingual lexicon, but simply lets the system learn the bilingual (artificial) sentence data. However, the way input was structured in the SRN was problematic, as we discussed earlier. In our model, there is no place for explicit marking of languages, as in the SRN model. We trained our model on a realistic parental input corpus with bilingual speech intermixed between Chinese and English. Figure 2 presents a sketch of the phonological and semantic organizations of the bilingual lexicon in the two SOMs, after the network has been trained on the 400 target words for 500 epochs (the WCD network was trained on the input sentences for 3 epochs, i.e., 555,837 word tokens). 2 As can be seen, our network clearly displays distinct clusters of lexical representations of Chinese from those of English, on both the form (SOM1) and the meaning level (SOM2). Note that because of the topological nature of the maps and the "bubble-filling" nature of SOM learning (Kohonen, 1995), the border between the two languages is not a straight line in either case. Because the sketch leaves out the details, cases where the English and Chinese representations are intermingled are not shown here (but see our discussion in interlingual interference). In general,
Connectionist Model o f Bilingual Processing
71
the intermingled cases are uncommon, and the overall separation of the two lexicons is clear on the 2-D maps.
i-:-.:~::.-.:~:i~~-..--..--i :.~.~..i :-.: :. ~.~:"--.- - ~:,~!:'..:.::..7-::4..:-...::..: .,-".".",",..",.".--',-,-,,,--"-...,..-.".,'."-",,"-'
~---.--i......:i~i!i!~!ii~i. :-z:!i:::::~i!?!~i~!;5!i!~!ii~ii! ~iii~ii~............~!!.i~i-!iiiiii~!~-i-!iIii~i!~i!iii~iiii!~ :~"~:~ !~i-:-:;:....;:-:~!i!~iii
ii:i:"-"~!iiii:i!;'iNiii!!!i?!ii!N~iiii?ii ii!iilNiliiiiiii!Niiiiii . .!i. i.!i.i.i. . . . .ii!iNi . . . . . .iiiiiii!!!! . . . .!~i!!~!i!iiii~i!!iiii!~~~ii!:;i!~ii~!!ilN.........................................................................................
i~i~: ~::~.~:~:~`;i :.:;i:~:::::i`::::::;:..::::i.::..:.~;:.`:~.::::. ::::;`:::::i~::::::~;;;Z:::.:::: ~....... ::::::~`~. ~!!~.~:-~',~:~".~ :.~ ......:!~::~-::~::-~ii~~%~.,.~ ~i "'".,.'i~Z~'s :: ...... ; ...... .?" " " : ..*""" ':" "" ~....... : ..~'.~i~'':':"'"~'""'-:"".~'""':r"~'v"!-'"": 9 :','.'~..-~:;'.-. .;.~::~::-~'. -'.'" ~..'..~ .!~;i.'..'-:~ '~..':.'.~.~: 9
...... ~............. !-...... i....... i"--.~-~ ............. ~.-; ...... i ..... ~............ i..--'.". " .~...... ~t.~ ............ :~i; :'~g~i
,~'"'i ........ "-'~"i-'-~! ': ....... ~'~-':~'-"-,~---i.'"~ '-~-
!:'-,'-+",~:! ...... ='-~':-~
. ........................... . . . . . . . ............ . . , . ............... . . . . .
!~~....'~;~;i :"~#~i :"~:~.~g f~;.~ i'~ i:~g~ ~gg~.~ggi g~i :~;:~":;:i:~gi!ti..:g~g~ :;::::i:;:~.~g~ ::~~'~~i..":~
...... ~i---'"'i'.,'~:~!..;;-:-",,;~;-~a:~;;!,~;;~:~-~
....... i~g~-i
~!-!-i;:~.:-i;i~i.'~:~.:-'.:::;!.~$;~.'i;i~Z-'."i~.;i~i ~
.".-.:......!
~i~.~;~
~:~
:~
:~i~;~":~;~;:~'~i~
~i~:;:~.~;~.~.~.~.~
iZ:~.)5. ~--?,e~!~!2[-?i2--12-2.2..}~?~ .?--.)...<-~N:-<--2{--i.?-~:2. ?. i..? .i:1!-.-72.1~2ii-~2,i... . :~ii:.-i--....)??~2:1!.?~.?...i..-.-2~ .?.i--i.i-.~; -,- .i
Figure 2. A sketch of the phonological and semantic organizations of the bilingual lexicon in SOM1 and SOM2 after the network has been trained on the representations for 500 epochs. The upper panel represents the lexical form map (SOM1), and the lower panel the semantic map (SOM2). Shaded areas indicate the English phonological and semantic representations, in contrast to the Chinese representations.
Our network received no labels or tags for which language a given word should belong to, yet the self-organizing process was able to separate the two lexicons. On the phonological level (SOM1), it is clear that English and Cantonese differ greatly, even though some vowels may share the same quality. In our phonological representation scheme, we have tried to adapt the template in such a way that it could fit both languages (CVVCCVVC), cutting some comers in English (e.g., eliminating the representation of initial consonant clusters). This adaptation does not seem to have much effect on the cross-language distinction. Overall, the syllabic templates, the phonemic inventory, and the lexical tones have given the learning
72
Ping Li and Igor Farkas
system a great deal of differences to explore for the identification of the two lexicons. With respect to semantics, SOM2 shows an even clearer separation between the two languages. Note that our network performed essentially a sentence processing task on the speech corpus, in which the WCD reads through the streams of sentences, computes the co-occurrence statistics for each of the 400 words, disregarding words not in the target lexicon (treating them as one unknown type). The words that precede and follow each of the 400 words in the lexicon form the contextual history for that given word, and it is this contextual history that captures the use and meaning of the word. Thus, the words in the two lexicons will have very different contextual histories, because words in English are typically associated with English words, and words in Chinese are typically associated with Chinese words. This is true even with a mixed bilingual corpus like ours, in which English and Chinese sentences are intermixed. From the learner's perspective, the input contains both English and Chinese sentences, but it is relatively consistent whether a given sentence is English or Chinese, because the intermixing occurs at the sentence level, but rarely within the sentence level. Although it is common for Hong Kong Cantonese speakers to code switch, the most common code-switching situation is one in which the speaker uses an English guest word in a Chinese sentence (Li, 1996a). Given that the code-switched guest words are often isolated instances and their associations with the lexicon of the borrowing language are relatively weak, our network did not get confused about the identity of the two lexicons. Overall, our simulation results match well with empirical studies of the bilingual lexicon that argue for the representation of language-specific (or distinct) lexicons, but in an integrated network (e.g., Kirsner, Smith, Lockhart, King, & Jain, 1984).
Lexical Categories in the Bilingual Lexicon Given that our model was able to clearly differentiate the two lexicons, does it show any ability in developing categories within each language? In other words, is our network able to organize the phonological and semantic features into structurally meaningful clusters? Our previous research shows that SOMs are good at deriving semantic categories of the lexicon, even in cases where linguists fail to identify the categories with clear labels (e.g., Whorf's cryptotypes; Li, 1999, in press). A further examination of Figure 2 reveals that the network developed a considerable amount of structure on its semantic organization. 3 Figure 3 presents the same results as in Figure 2, with illustrative labels on the lexical categories in SOM2. In Figure 3, we can see that the SOM2 treated the nouns and verbs distinctly, in both Chinese and English. The network also grouped other words together as clusters that share the same grammatical categories (e.g., English prepositions and pronouns occurred in the same neighborhood). More interestingly, within each of the categories (assuming there are enough instances), semantically similar words
Connectionist Model of Bilingual Processing
73
also occurred together. For example, in English, the state verbs (e.g., know, like, have, want) were grouped together (the darkest shaded area under the label "Verbs"), in contrast to other activity verbs; 4 in Chinese, words that are related to cooking/eating (within the verb category) were grouped together (e.g., sik 'eat', jam 'drink', cit 'cut', gaau 'gnaw'). One can also see a category of verbs indicating perceptual/mental activities in Chinese (marked as "Verbsp", including teng 'listen', tai 'look', gin 'see', seong 'think'). These effects of categorical emergence from statistical learning match up with results from our previous monolingual simulations (Farkas & Li, in press; Li, 1999, 2000, in press).
~-~,r
~i~;,;
i,~#n
.
.
.
.
I.~I;C~IL
~
l i~!i~}}i~}}}~.:~.~}~(~}~({.}}~!~}~!i.~E(.}!~ii~}}}}}}!}~(.~!~i~!~!}}~(~}~!~}}}~i~(~}~}~{!{i~i:~{i!.}!~}}~(~}~.~.~.~!.~!~!(~.~!~{~i!!~}}~i{~}}}~}}~!~:`.}}~(({.}}}}~{~}~{.}ii~.}~i!.}}}}}~.~{.}}!}}~((.if..~!.}}~.~!i.~.}}}}
I~;i---.~ ...... i ''':~ . ................. ~,.a~. ...... :.....: ~...~'.,~-~-....:=.::..-......'. ........ :..~....:. .:...:-.:....~:.,:-..:.~...::i.~;.~.i......!..::.-!~.~'~.~..,;~ ..... : ............
:. . . . . . . :. . . . . . . :. . . . . . . : " ". . . . . . . . . . . . . . . .
J ......
:
~ : :
:
:.~.,~:
"
"
~
-
:
-
:
....... ....:.~.:.~z.`Lq.:.~:a..~i.i.~...::~.i~.::i~:~i``..:.i~?~.~it~i*.~.t.:~.?:~7?.!.i!~176162 :
.:,~:,
t
~
;~:~::: : ~ :
::
~
;
,,:::,;
,,;
;
~
::~::
:t
: ~
:"~:i :.
:
~i~:::::~i~*~.~:.~.::::.i~:::::~[~i.~:?.$ii~!~i~].iik~i].tiii~;t!~!.]i..::~!!~.t!~.~ .!~!~:~t~i~(~.~:~i!:!~i:i:i~/:~/.:.i~.~.!i~t::::~!~:::~.~i~:i~:~!~t:~:~.~f::!~.~!~:~i~i~.~.t!:.?.!..:i!~.~i!it~:~it!.!?:ii~:~?.:~5::i~!.~ii!it..:r
I ~...... i~...Z..~:F.....;.~....:.~.:..:.5......................~T.~:..~:...;~...*.~..~.....ZZ7;..Z
. .S?.':?.'I ~.....~s163
i....... i....... ...... ~...... !...... {.............~............. i.-... ~a:~/ 'i~:!:::ti::b"];"i
"'.Y'.'.'.T:.-'FTS~:
.;-..... ~.----.~-.*......-..';.,.-:.~.---.:i...... [.-.--i ...... i-..--.~:,:~-:ir
-:;:i~;:--i~-~-::::~i-:~:;~:,.'!~o~-};{-:ti~t -.~-q*:~--ff~r
i::x:............:.::;;........~':~:............:......: 7........:~......... : ;:~-:::-:.:-:i:-~-~i::.::~~i~
i-*i
;
:Verb.sp
:
i
i i
i
:
:: i i:~
i~i!!!:!ii~i~!;~!~i!!ii!!!~!r
~i~!~!i!~!iiii~i!~i~!!i~!!i~
ii!iiir162
Figure 3. Emergence of lexical categories (e.g., nouns, verbs, and subcategories in Chinese and English) in SOM2.
Interlingual Priming and Interference In the empirical literature, there are heated debates on whether bilingual lexical representation is language-specific or language-independent, and whether the two lexicons are stored separately or together (see Smith, 1997, for a summary). These debates determine how researchers look at well-known effects of cross-language
74
Ping Li and Igor Farkas
priming and interference in empirical studies (e.g., de Groot & Nas, 1991; Kirsner et al., 1984). In a way, our above simulation results already indicated that the two lexicons can reside in an integrated network, but separately in space. Given that the two lexicons exhibit distinct behaviors in our network, and that there is no language-independent conceptual representation in our network (as is the case in many other bilingual models, see Conclusions), how can we account for priming effects or interference that show that the two lexicons do interact with each other? Recall that our procedure involved training the mapping between word forms and meanings in such a way that the mapping is not one-to-one, but many-to-many (for nouns and verbs, at least). Thus, words that are considered as translation equivalents are linked together through associative pathways that are trained by Hebbian learning. After training the network on the bilingual speech corpus for 500 epochs on the 400 words, we tested the network's performance on comprehension (from phonology to semantics) and production (from semantics to phonology). Probing into the network's associative pathways is laborious, which involves examining each of the 400 words on SOM 1 or SOM2 to see what words it can activate on the other map. Our analyses indicate that there are clear effects of priming and interference. To illustrate with examples, the activation of the following words on SOM2 (let~) led to the activation of words on SOMI (right) via the associative links (items in parentheses are translations of Chinese words): SOM2
SOMI
bear --) bear, there, fish, this jat (one) --)jat (one), loeng (two), soeng (think), saam (three), san (new) bibi (baby) --) bi4bil (baby), maami (mummy), baby, maybe, mummy sik (eat) --) it, eat, sik (eat), maai (buy) These are some representative examples that indicate semantic as well as phonological priming effects in production. For example, bear activates fish because fish was semantically related to bear and SOM2 placed it next to bear. There and this were activated because of their phonological similarity to bear and fish, respectively (SOM1 placed there next to bear, and this next to fish). The second example in Chinese (/at) can also be similarly explained. These two examples show only within-language priming effects--no translation equivalents exist in our simulated lexicon. The next two examples show cross-language priming effects: mummy became activated because of its semantic similarity to baby, which in turn was activated by the Chinese word bibi. In addition, maybe was activated because of its phonological similarity with baby on SOM1. The last example, sik 'eat', is a similarly interesting case. Its strongest response on SOM 1 was it and not eat, a phonologically similar word. Note that these priming effects represent possible interference in language production. Because of strong associative
Connectionist Model of Bilingual Processing
75
pathways, the speaker might have said it instead of sik or eat, especially in a strong bilingual speech mode (Grosjean, 1997). Similarly, our model was also able to show cross-language interference effects in comprehension. For example: SOMI SOM2 look -9 lok (fall), fall, look, tai (look), ceot (go out). boat ") boat, dog, car, syun (boat), sau (hand) waan (play) --) waanx (play), wan (look for), play, read, find, see sik (eat) --) eat, cut, sik (eat), maai (buy)
Again, cross-language priming resulted from both semantic and phonological similarities as a function of the co-activation of forms (SOM1) and meanings (SOM2). For example, because of the phonological similarity of look and lok on SOM1, look activated lok in SOM2, which in turn activated fall as a result of semantic similarity (lok means "fall"). Li (1996a) found that bilingual ChineseEnglish speakers do not need more time to process cross-language homophones like look/lok than do monolinguals in processing monolingual materials, when the homophone occurs in a sentence context; in other words, there was no switching cost associated with the recognition of code-switched English words. This finding contradicts the notion of a language switch or monitor mechanism that says that switching languages takes time (Macnamara, 1967; Obler & Albert, 1978; see also Green, 1998). Li suggested that a parallel activation mechanism might be involved: multiple candidates in the bilingual lexicon are activated, and sentence context plays a crucial role in affecting the final activation of the target word (see also Altarriba, Kroll, Sholl, & Rayner, 1996, for a similar argument). The results shown in this simulation are consistent with the view that multiple candidates may be simultaneously activated and selected due to effects of cross-language priming.
Effects of Proficiency and Individual Differences
To extend the scope of the SOMBIP model discussed above, we modeled the effects of proficiency and individual differences in two separate simulations using the same bilingual lexicon. Proficiency was modeled by having SOM2 self-organize on "novice" word representations--representations that come out of the WCD network's learning of a smaller sample of English sentences (92,640 word tokens, as compared with 555,837 tokens in the above), simulating a beginning Chinese learner of English. Figure 4 presents a snapshot of the network's representation with the same learning parameters as in Figures 2 and 3. As is illustrated in SOM2 of Figure 4, the semantic representations of the English words were squeezed into the upper right-hand corner, and occupy only a very small portion of the map. Although there was some rudimentary structure in these vectors (e.g., good and bad were nearby on the map, so were say and tell), the
76
Ping
Li
and
Igor
Farkas
semantic vectors from the WCD network were still fuzzy and did not represent the word meanings accurately; thus the SOM could not fully differentiate them and accordingly organize them into meaningful structures. This picture shows that for novice bilingual speakers, the bilingual mental lexicon is still largely monolingual, and words in the second language are largely clamped together. It is also consistent with the view that the bilingual lexicon is a dynamically developing system, one that changes as a function of which language becomes the dominant language (Heredia, 1997). As the bilingual speaker becomes more proficient and fine-tunes the semantic system (amongst other developments, e.g., an expanding size of the L2 lexicon), this "dominance" situation may shift to an L2 instead of L1 dominance (Heredia & Altarriba, 2001).
~i~i}i.}iii}i}}i.i.i~i~i~i.}i-iii:!.i.i.ii:}i:~'i.~.~.~.~i}!~!~}}}}~}~.~.i.~.}}}}}i~:}~i.}~i~}}~.~.}~5~.}}}~.}!.}~.~?~!!!~.~}~!}}~.~.~!~.' !il .:.!~!.~.}}~.}}i~}}~.!!!!~i~:~.}}}~.}}}?~}!.~.E~}}~}~.~.~.~:i!...-.!!~!~.~.~.~}~.}~.~.}~}}~})~.:!!~ ~}}~.}?~!~!~!~}}~..}~}~}}i!}.-..!:.~!~.}}~.}~}~.~.~?~!f?~f~f!~l~}~}~}~iii~i~i~ii~i~ i":
..:
~ . ..... .. i i ~'.~, 9. ~ i . ~..~.7. i . :!:~,~: ..i. :i,,~: ~
..;...i 9 .i;... ::.:............:...i..:....... ..i.:...-i:...:.... ::.i...::.!.:..::.....;. ::.:...:..::..i,::.:..i....:..iii i ! ' i ~
.~
~ ~"~
,
~-~. ~:,~: ~
~
i
~
~i,~,
~ii~i,~.,'.:.~i~,,~,~.~,~~!,
..... i ..... ~""~. . . . . .
! z ......
~!~'~;~F~!~;!:~::i~i!...`~:!.~:~.~;!.~i.i!:;~i;:~:i;~;..:i ::
...... ~...................... ,~t~. ............. ! ...... ~.............. i. ................... ~............. ~ , . . , ~ ! i ~ ? : . ' ~ i i : ~ i ~ i ~ T J ~ ' ~ : - ' ! ~ ' ~ ; ~ : . . . : : i ~ } ~ : ~ t~::~7~'~i~i~it&~,,~ff~i~?~g2ff~-":~i~.:~22:.':~i:!:2 :'~i~::.:~::~ii;~::~-;"2~:~-:::.::~"!?.~!'-:'..;-:~;~i ..................... ~-.'.. ~ ...... i ............. !....... i......~., ............................ "............. : , ~ : ! ~ : i : ! ~ i ~ ; ~:i~ ~:~:~:::!~.'.:~:!~:!~i~:~ ~ ~:~;~:!'.`~.~;~:~:.~i~;i`~!:;~?~:.~::~!:i~!~!~:~::::i~i~::i~ ii !,~ii~:..:~ !~: i:ii*ti~i-:~'.-':i ~ :i!
....~:.~........ -,,~ ........ ,~,,~: ........:,.~.~............. i...... .............: ~ . . . : ` ~ i ~ : ~ r ~ ; ~ i ! ~ ; ~ : : ~ : ~ ` : . : : : : ~ ` : ~ : ~ ! ~ ` 7 ~ : ~ ! ~ ` . ` ~ i ! i ~ ~ ~ ! i ~ z ~ i ~ ` ~ : ~ ; ~ ;..:;:.i..::...~.:::.::::::::i'.:.::::i~;::.:~:...:::.i..;..:i:..::::::::i~:~:.:~:.,.~::::..i.:.:.~.i::..'...:~~.~:~.;~...:.~.'::.':'~::::..i.::.:..~.~.~ ~:~2~.~.~::C~.~:~.~!~~~g~i!i~ii;!~
',.,.:............:....!.......i,,,,,!.......
'\~!;!.!~~~'.ii! ~~!.!i!~!~,~;~i~
~'~}~.~.~'. ~ : i!ii~zr
i..~i...............~,~....... ,: i:~...............i2 ............i.~.~.i..~.................i::.:,i.......,!i~i~N!i~ii;iii~iN~}}}~ii!iN!ii~:!iiii~}}iiiiii!i~!!iii}i!~i;~ili
7g'~.i............;...........~.':.~.~:!..i..~..<2..~.........~....;.....~.~.~.~.......i.....:..:..~.;..~.~`.~..................:.....!......:......;`.~:.~........:...2.........~...~.:..:.~i:;.~`~...............`~:..... .,,.:~=~,. ,,,u.~, ~ ~ , . ~ .
Itt:~MilNIi~
~1-~
..
.............. i............. i.~..~.:- ............. i.'.~.~ ............... ~:~,.ei..-...:.~,,} ............ ~...... i ...... i-...--:-.-,.,.i. ............ :~,..--i ...... i.-----}-.....i ............ ~-.---.i ...... i ...... ;~.~-: ................ ,':-.~:i...... ::........ ~ ~ " , , i , ~ i ! ~ ............................................. : ..... :~,,~. . . . . . . . . . . . . . . . . . . . . . . . :...... :. . . . . . . . . . . . . . . . . . . . . . . . . . . . ~. . . . . . . . . . . . . -..-...~,~:,~...~ ..... ~ . ~ ~ ~ ~ ~: :~: ~ :,',-~ ~ ~ , ~ i ...... ' ...... :....... : : ~ ......... ...... }...... i ...... :....... ::............. ~...... i ............ :....... i................... ~ : ~ i : ........ i ....i......i......~!!t!~i~J.~;ii;:~r162
'~ ~...... ~...... ~...... i...... ............. i " ' ~ i ~ ...... iTM " ~. , i
i~i~,~'~ ............ ~.......! " i ..........
~:.
~ ~:;;~.~:~ ~
!~~!ii!iiiiiiiii!~ l~iiii~ iiii~ii~i~i~i~ii ~ i i ~
~::r-~::~,
,~ ~ ~ ~
~..,,2"i!ii~iiii~,?~ii~',2i~,?~i~.,,,:-.--:-i~....::,-
..-.:..-....-~'~..i
~ ~:~ ~ ~!~ ~a:: ~.~:'.::z~ ::.;~ .~ ~ ; : ~ . . . . . ~!~::~ii~ii~ iiii~i!~i~i~i~?~:.~i~-~i~
~}i}}iiii!~i~i~i~~!~~!!~?}?i~i~}ii}i!i~}ii:,iilN~}'~i!ill ii!~-i}~'~ii :~}'~i}N}~:::
..... i...... - - ...... i .................. i ..... i .......... i ..... i .. : .... i ..... i ............ !.. ~ ....: .:...i .... i........... i.......i. : i ~ , ~ i ! ~ 2 ~ } : ~ ! ~ i ~ : , ~ i ~ : i ~ : : ~ ~ i i 2 ~ ~ ~ 2 i : i ~ ~ i ~ i ~ i i i
.
.
.
:-..-..! ..-...--..:.
i:~
.
:
:
i..... i.-..........}.i~.~:.i...: %-"i:-"i:~4::---
..... 2 . . . . ..... ~...... < ......... ~..2.. -: ...~ . ...i...... !.... : . . . .
': ................. :..... : : : ~ ........................ :,~i:.:~i'~-~a~:;~:::~"-:i
i ..... i , i ~ ! ~ ! i ~ i ! ~ i ~ ! i ' : . ~ i i ~ i ' ~ i ; ~ : ~ i ! ~ i ~ i ~ ~ : ~ i ~
!""i:""' i..., i.......~;2;.i.-....i-....!.. ......... r...... ;........ ~....-.! ...... !:~i:~!~i~:N~i:~~ ....... . . - . . i
;...2. .~,;~,~-......~...............~.~. ~...... ;. ....-.~.-.....f.......~:.:..: .::... ~;~.....:~.:....~..;...Z
..... i ~ " i
.~:.~i.~..`.~sNii.~.~ii~..`..i~!~iii~!iii1iii~i~i!!i~i~!!i~i.i!i~5!i~!~i~:.~
...:..:.:.;:... ";"-:i:-::-"-.i....:..!...... :.. ::::~ ...:..~...... ~.:....; ......~. 2 . : ;.::.. 2 . . . . :~............. :.._ .i... . ' . ' : i i - " -~
F i g u r e 4. Effects of proficiency in lexical representations. English semantic representations in SOM2 were based on the WCD network's learning of a limited amount of input (92,640 word tokens).
Finally, to simulate individual differences in the representation of the bilingual lexicon, we constructed a smaller-in-size but otherwise identical network as in previous simulations. In the simulations reported above, we used 2,500 units each (a 50x50 grid) in SOM1 and SOM2. In the new simulation, we used 1,600 units (a 40x40 grid). This reduction in map size is meant to simulate a bilingual processor
Connectionist Model of Bilingual Processing
77
that has limited resources (e.g., working memory capacity) at disposal. There is evidence that individual differences in language processing are often due to working memory capacity (Just & Carpenter, 1992). Our network that used a smaller map size might thus show disrupted representational structure or processing abnormalities. Analyses of this manipulation show that the network was able to display language separation, lexical categories, and priming and interference effects as in the bigger network. However, given the same amount of training (500 epochs), the network had difficulty in distinguishing between many words that are either phonologically or semantically similar. In other words, the same units on the SOM often represented two or more similar lexical items. Recognition rate dropped to 57% for SOM1 (as compared to 74% with the 50x50 map) and to 54% for SOM2 (as compared to 74% with the 50x50 maps). 5 As a result of the decrease in recognition fidelity, the network's associative links were much more diffuse than those in the larger networks. Thus, there were stronger within- and cross-language priming and interference effects as compared with the bigger network, some of which could not be explained easily. For example, activation of let on SOM1 caused bei (let), let, show, and, mouse, and red to become activated on SOM2; activation of go on SOM2 caused say, waa (say), waai (bad), hal (be), gong (say), hoei (go), and know to become activated on SOM1, and so on. These results indicate that the unusual patterns in bilingual processing may be due to diffuse bilingual representations, which might in turn be due to limited processing capacity. In summary, manipulations of size of the learning corpus and size of the network extend the capabilities of our model in accounting for a variety of bilingual phenomena.
General Discussion and Conclusions
In this chapter we provided an overview of connectionist bilingualism, and presented a sketch of SOMBIP, a self-organizing connectionist model of bilingual processing. A number of theoretical and architectural considerations of the model are discussed, and preliminary simulation results from the model are analyzed. Our SOMBIP model consists of two interconnected self-organizing neural networks, coupled with a recurrent neural network that computes lexical co-occurrence constraints. Simulations with our model indicate that (1) the model can account for distinct patterns of the bilingual lexicon without the use of language nodes or language tags, (2) it can develop meaningful lexical-semantic categories through self-organizing processes, (3) it can account for a variety of priming and interference effects based on associative pathways between phonology and semantics in the lexicon, and (4) it can explain lexical representation in bilinguals with different levels of proficiency and working memory capacity. These capabilities of our model are due to its design characteristics in that (a) it combines
78
Ping Li and Igor Farkas
localist and distributed properties of processing, (b) it combines representation (through self-organization) with learning (recurrent neural network WCD, Hebbian learning of associative pathways), and finally (c) it combines the lexicon and sentences in bilingual processing. Most previous connectionist bilingual models (e.g., BIA and BIMOLA) are stationary models. To fully capture the dynamics in bilingual language processing, a model needs to incorporate the above mentioned properties, especially the learning characteristics such as in SOM and WCD. Green (1998) proposed an inhibitory control (IC) model of bilingual lexical processing. Although the IC model is not a connectionist model--on the contrary, it has quite some symbolic AI flavor, for example, with goal-oriented decision boxes and control schemas--it attempts to integrate activation-based accounts (e.g., interactive activation mechanisms). According to Green, there are multiple levels of control, with each level associated with a specific schema, from high-level event scripts to low-level articulatory controls. The particular level at which the IC model operates is an intermediate level, the lemma level, whereby an inhibitory mechanism suppresses the activation of lemmas that are tagged as belonging to the language other than the intended one. Crucial to the functioning of this mechanism are the language tags, tags that are believed to be part of the conceptual system of the lexicon. Green likened the language tags to the language nodes in the B IA model, which function to reinforce lexical activation of the currently activated language, while simultaneously decreasing lexical activation in the other lexicon. Interestingly, in a brief note, Dijkstra and van Heuven (1998) mentioned that the B IA language nodes might be thought to represent abstractions of very different statistical regularities in the two languages due to lexical co-occurrence differences. This view differs from what Green assumed, but is completely consistent with our argument here (in SOMBIP, lexical co-occurrence statistics are explicitly incorporated through WCD). In a commentary on the IC model, Li (1998a) questioned the necessity of language tags. One piece of evidence against the notion of language tags in IC is that some recent studies show that there need not be a cost associated with language switching, especially in natural speech situations (Grosjean, 1988, 1997; Grosjean & Miller, 1994; Li, 1996a)--according to IC, language switching takes time, since the process of switching to another language involves the inhibition of previous language tags. In the IC model, the inhibition of a particular stimulus shuts down the activation of all other related stimuli in the same language from the top down. This assumption seems to contradict several activation-based accounts that the bilinguals' two languages may be always activated, though the strength of the activation differs in specific linguistic situations, depending on the frequency of the target words, the sentence context, the speaker's proficiency in the two languages, and the speech mode (Grosjean, 1988, 1997; Li, 1996a). Thus, Li (1998a) argued that it is unclear whether the language tags are ad hoc constructs or necessary components of bilingual processing, just as it is unclear whether language nodes are necessary. Li suggested that the seemingly separate representations of the two
Connectionist Model of Bilingual Processing
79
lexicons, and the related interlingual priming/interference effects, might arise as a result of lexical and grammatical learning in a self-organizing neural network, in which no distinct labels are given to items of the two or more languages. Specifically, words from both languages may exist in the same topological map, but over time the network can develop localized patterns of activity in learning the mappings between phonology, orthography, morphology, and semantics. These localized patterns of activity may then correspond to the learner's internalized, distinct representations of the two lexicons. In this study, we have implemented precisely such a model, the SOMBIP, which nicely accounts for the independent (yet interactive) representations of the bilingual lexicon. A supra-lexical level of language nodes or language tags is absent in SOMBIP, but the effects of the language nodes or tags are captured in the system. In response to Li's commentary, Green (1998) agreed that the bilingual lexicosemantic system is composed of self-organized networks, as a presumption of the subset hypothesis (Paradis, 1989) adopted by the IC model. But he argued that language tags are necessary and that even in self-organizing networks such as the one used by Miikkulainen (1997) there is still room for language tags. Interestingly, Miikkulainen (1993) did mention a multiple SOM network that functions exactly as Green would argue: each language has its own separate lexicon, stored on different maps. A high-level control network would determine which language modules are active, switching between conversations in two languages. Such an approach was intended to model, according to Miikkulainen, distinct language behaviors in the representation of the bilingual lexicon in the brain. But Miikkulainen also speculated on a different (in our view a more plausible) model, one that is very similar in spirit to SOMBIP. This model would represent all word forms from the multilingual lexicon in a single lexical map, and all semantic representations in a single semantic map. These two maps could be associated with each other via the associative links. Cross-language priming and interference ("slips between languages" in his terms) could be modeled by noisy semantic mapping (like the diffuse map in our limited-resource network) or by associative propagation. In this version of the model, there would still be a high-level control network, but the task of the control mechanism merely modulates language output such that the same sentence parser could work on different language materials. In SOMBIP, however, this control network was unnecessary as the WCD network works equally efficiently in both languages. The ability of our network to show distinct patterns of the bilingual lexicon without distinct mechanisms (e.g., separate language tags or nodes) is yet another case of the classical connectionist argument: modular linguistic behaviors that are different or dissociated need not arise from'dual-mechanisms or dual-routes in the representational or processing system. They can naturally emerge from learning the input characteristics within a single system, a system that can be operationalized in terms of statistical or connectionist principles. This argument was advanced by the original PDP studies of the English past-tense acquisition (Rumelhart &
80
Ping Li and Igor Farkas
McClelland, 1986), and has since been supported by research in many domains of language studies, including language acquisition, normal and impaired word reading, and language organization in the brain (e.g., Elman, Bates, Johnson, Karmiloff-Smith, Parisi, & Plunkett, 1996; Plaut, McClelland, Seidenberg, & Patterson, 1996; Miikkulainen, 1997; Small, Hart, Nguyen, & Gordon, 1995). As a final important note, our model departs from the tradition that has been adopted by many bilingual researchers (see Kroll & de Groot, 1997, for discussion) in that it does not assume there is an integrated level of meaning-concept or meaning and concept (the "conceptual level"). The mixture of meaning with concept has created much confusion, in both first language and second language acquisition research (see Li & Shirai, 2000). This confusion is further complicated by the fact that many psycholinguists adopt a language-specific lexical-semantic level, the lemma level (Levelt, 1989). To avoid these confusions, thus, in our model there is no language-independent "conceptual" representation. Our semantic representation, or more accurately, lexical representation that contains both syntactic and semantic information, comes out the WCD network's analyses of global lexical cooccurrence information. This level of representation matches more closely with what linguists call "meaning", rather than "denotata" (Lyons, 1977)--entities of concepts that many bilingual researchers use in their picture naming tasks. To conclude, the SOMBIP model outlined in this chapter serves as a new model of bilingual processing and provides a new perspective on connectionist bilingualism. It has the potential of explaining a wide variety of empirical and theoretical issues in bilingual research. So far, we have tested the SOMBIP model only on spoken (transcribed) bilingual Chinese-English data. How the model generalizes to other bilingual situations should be further tested. Chinese differs significantly from English and other Indo-European languages in phonological, syntactic, and lexical structures and these differences are bound to affect language processing (Li, 1996b, 1998b). When one examines bilingual pairs of languages that are more similar (e.g., Dutch and German), it is likely that the model will show more integrated phonological and semantic representations of the bilingual lexicon. Moreover, if the mixed bilingual data contain more intra-sentential code-switches (rather than inter-sentential as in our data), the network that receives such data as input might also produce more integrated semantic representations. In addition, the current SOMBIP design could easily be extended to simulate orthographic-semantic relationships, and be evaluated against empirical findings and results from the B IA model. Other interesting extensions of the model would include simulating impaired semantic, phonological, or orthographic structures by adding noise to the SOM maps and by damaging the associative pathways that connect word form and word meaning. Such modeling efforts could shed light on bilingual representation and processing in the brain (Fabbro, 1999).
Connectionist Model of Bilingual Processing
81
Author Notes
This research was supported by a grant from the National Science Foundation (#BCS-9975249) to the first author. We would like to thank Fran~;ois Grosjean and Nicolas L6wy for comments and suggestions on an earlier draft of the paper, and Kathy Reid and Clint Schlenker for their editorial assistance and data preparation. The simulations were run on a Pentium PC with Linux OS. We used the DISLEX code (Miikkulainen, 1999) to run the SOMI and SOM2 and their associative links, and the WCD code was written by the second author.
Notes
1 The current scheme represents a scaled-down version of the full tri-syllabic template representation of English (see Li & MacWhinney, 2001, for the full version). As such, it is limited in certain respects; for example, it is not able to represent consonant clusters as in string/strlN/, which would require additional Cslots (e.g., CCCVVC for one syllable). 2 It is impossible to show all the details of the original figure here because each of the 2-D maps contains 400 words spread on a 50x50 grid of 2,500 units. 3 The organizations of the phonological representations are mainly based on phonemic and template similarities, and are relatively less interesting for our purpose here. 4 The distinction between state and activity verbs has attracted significant debates concerning the nature and origin of this distinction in language acquisition. Some researchers propose that this distinction is innately given (Bickerton, 1981, 1984). We have argued that this distinction can arise from children's statistical learning of the semantic properties of verbs, in the same way as it would occur distinctly in our connectionist networks (Li & Shirai, 2000). 5 The recognition rate is calculated as the rate of correctly mapped words in the SOM over the total number of words in the lexicon. Whether a word is mapped correctly is determined by a function that calculates the nearest distance between the input and-the codevectors in the map. Note that the recognition rate of 74% is not too impressive at first glance. This is because the network learns a many-tomany mapping function for the noun and verb translation equivalents. If the network learns a one-to-one mapping, the recognition rate would go up to 93% and 97% for SOM1 and SOM2, respectively.
References
Altarriba, J., Kroll, J. F., Sholl, A., & Rayner, K. (1996). The influence of lexical and conceptual constraints on reading mixed-language sentences: Evidence from eye fixations and naming times. Memory & Cognition, 24, 477-492.
82
Ping Li and Igor Farkas
Bickerton, D. (1981). Roots of language. Ann Arbor, MI: Karoma. Bickerton, D. (1984). The language bioprogram hypothesis. Behavioral and Brain Sciences, 7, 173-188. Bowerman, M. (1978). The acquisition of word meaning: An investigation into some current conflicts. In N. Waterson & C. E. Snow (Eds.), Development of communication (pp. 263-287). New York: John Wiley and Sons. Bowerman, M. (1982). Reorganizational processes in lexical and syntactic development. In E. Wanner & L. R. Gleitman (Eds.), Language acquisition: The state of the art (pp. 319-346). Cambridge, UK: Cambridge University Press. Burgess, C., & Lund, K. (1997). Modelling parsing constraints with highdimensional context space. Language and Cognitive Processes, 12, 1-34. Burgess, C., & Lund, K. (1999). The dynamics of meaning in memory. In E. Dietrich & A. Markman (Eds.), Cognitive dynamics: Conceptual and representational change in humans and machines (pp. 17-56). Mahwah, NJ: Erlbaum. Dijkstra, T., & van Heuven, W. (1998). The BIA model and bilingual word recognition. In J. Grainger & A. M. Jacobs (Eds.), Localist connectionist approaches to human cognition (pp. 189-225). Mahwah, NJ: Erlbaum. Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179-211. Elman, J. L., Bates, E. A., Johnson, M. H., Karmiloff-Smith, A., Parisi, D., & PlunkeR, K. (1996). Rethinking Innateness: A connectionist perspective on development. Cambridge, MA: MIT Press. Fabbro, F. (1999). The neurolinguistics of bilingualism: An introduction. East Sussex, UK: Psychology Press. Farkas, I., & Li, P. (2001). Modeling the development of lexicon with a growing self-organ.izing map. Manuscript submitted for publication. Farkas, I., & Li, P. (in press). A self-organizing neural network model of the acquisition of word meaning. In E. Altmann, A. Cleeremans, C. Schunn, & W. Gray (Eds.), Proceedings of the 4th International Conference on Cognitive Modeling. French, R. M. (1998). A simple recurrent network model of bilingual memory. In M. A. Gemsbacher & S. J. Derry (Eds.), Proceedings of the 20th Annual Conference of the Cognitive Science Society (pp. 368-373). Mahwah, NJ: Erlbaum. Grainger, J. (1993). Visual word recognition in bilinguals. In R. Schreuder & B. Weltens (Eds.), The bilingual lexicon (pp. 11-26). Amsterdam/Philadelphia: John Benjamins. Green, C. D. (1998). Mental control of the bilingual lexico-semantic system. Bilingualism, I, 67-82. de Groot, A. M. B., & Nas, G. L. J. (1991). Lexical representation of cognates and noncognates in compound bilinguals. Journal of Memory and Language, 30, 90-123. Grosjean, F. (1988). Exploring the recognition of guest words in bilingual speech. Language and Cognitive Processes, 3, 233-274.
Connectionist Model of Bilingual Processing
83
Grosjean, F. (1997). Processing mixed languages" Issues, findings and models. In A. M. B. de Groot & J. F. Kroll (Eds.), Tutorials in bilingualism: Psycholinguistic perspectives (pp. 225-254). Mahwah, NJ: Erlbaum. Grosjean, F., & Miller, J. (1994). Going in and out of languages: An example of bilingual flexibility. Psychological Science, 5, 201-206. Hebb, D. (1949). The organization of behavior: A neuropsychological theory. New York: John Wiley. Heredia, R. R. (1997). Bilingual memory and hierarchical models: A case for language dominance. Current Directions in Psychological Science, 6, 34-39. Heredia, R. R., & Altarriba, J. (2001). Bilingual language mixing: Why do bilinguals code-switch? Current Directions in Psychological Science, 10, 164-168. Hinton, G. E., & Sejnowski, T. (1999). Unsupervised learning: Foundations of neural computation. Cambridge, MA: MIT Press. Hong Kong Linguistic Society. (1997). Yueyu pinyin zibiao (Cantonese romanization scheme). Hong Kong Linguistic Society. Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122-149. Kirsner, K., Smith, M. C., Lockhart, R. S., King, M-L., & Jain, M. (1984). The bilingual lexicon: Language-specific effects in an integrated network. Journal of Verbal Learning and Verbal Behavior, 23, 519-539. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59-69. Kohonen, T. (1989). Self-organization and associative memory (3rd ed.). Berlin: Springer-Verlag. Kohonen, T. (1995). Self-organizing maps. Heidelberg: Springer. Kroll, J. F., & de Groot, A. M. B. (1997). Lexical and conceptual memory in the bilingual: Mapping from meaning in two languages. In A. M. B. de Groot & J. F. Kroll (Eds.), Tutorials in bilingualism: Psycholinguistic perspectives (pp. 169-200). Mahwah, NJ: Erlbaum. Lachter, J., & Bever, T. (1988). The relation between linguistic structure and associative theories of language learning: A constructive critique of some connectionist learning models. Cognition, 28, 195-247. Ladefoged, P. (1982). A course in phonetics (2nd ed). San Diego, CA: Harcourt Brace. Levelt, W. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press. Ldwy, N. (2001). Computational psycholinguistics and spoken word recognition in the bilingual and the monolingual. Manuscript in preparation, University of Neuch~.tei, Switzerland. Ldwy, N., & Grosjean, F. (2001). The computerized version of BIMOLA: A bilingual model of lexical access. Manuscript in preparation, University of Neuch~tel, Switzerland.
84
Ping Li and Igor Farkas
Li, P. (1996a). Spoken word recognition of code-switched words by ChineseEnglish bilinguals. Journal of Memory and Language, 35, 757-774. Li, P. (1996b). The temporal structure of spoken sentence comprehension in Chinese. Perception and Psychophysics, 58, 571-586. Li, P. (1998a). Mental control, language tags, and language nodes in bilingual lexical processing. Bilingualism, 1, 92-93. Li, P. (1998b). Crosslinguistic variation and sentence processing: The case of Chinese. In D. Hillert (Ed.), Sentence processing: A crosslinguistic perspective (pp. 33-51). San Diego, CA: Academic Press. Li, P. (1999). Generalization, representation, and recovery in a self-organizing feature-map model of language acquisition. In M. Hahn & S. C. Stoness (Eds.), Proceedings of the 21st Annual Conference of the Cognitive Science Society (pp. 308-313). Mahwah, NJ: Erlbaum. Li, P. (2000). The acquisition of lexical and grammatical aspect in a selforganizing feature-map model. In L. Gleitman & A. Joshi (Eds.), Proceedings of the 22nd Annual Conference of the Cognitive Science Society (pp. 304-309). Mahwah, NJ: Erlbaum. Li, P. (in press). Language acquisition in a self-organizing neural network. In P. Quinlan (Ed.), Connectionism and developmental theory. Philadelphia/Briton: Psychology Press. Li, P., & MacWhinney, B. (2001). PatPho.'A phonological pattern generator for neural networks. Manuscript submitted for publication. Li, P., & Shirai, Y. (2000). The acquisition of lexical and grammatical aspect. Berlin/New York: Mouton de Gruyter. Lyons, J. (1977). Semantics (Vol. 2). Cambridge, UK: Cambridge University Press. Macnamara, J. (1967). The bilingual's linguistic performance: A psychological overview. Journal of Social Issues, 23, 59-77. MacWhinney, B., & Leinbach, J. (1991). Implementations are not conceptualiza-tions: Revising the verb learning model. Cognition, 40, 121-157. MacWhinney, B. (2000a). Lexicalist connectionism. In P. Broeder & J. M. Murre (Eds.), Models of language acquisition: Inductive and deductive approaches (pp. 9-32). Oxford, UK: Oxford University Press. MacWhinney, B. (2000b). The CHILDES project: Tools for analyzing talk (3rd ed). Mahwah, NJ: Erlbaum. McClelland, J. L., & Elman, J. (1986). Interactive processes in speech perception: The TRACE model. In J. L. McClelland, D. E. Rumelhart, & the PDP research group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 2, pp. 58-121). Cambridge, MA: MIT Press. McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In J. L. McClelland, D. E. Rumelhart, & the PDP research group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 2, pp. 272-325). Cambridge, MA: MIT Press.
Connectionist Model of Bilingual Processing
85
McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception. Part 1: An account of basic findings. Psychological Review, 88, 375-405. Miikkulainen, R. (1993). Subsymbolic natural language processing: An integrated model of scripts, lexicon, and memory. Cambridge, MA: MIT Press. Miikkulainen, R. (1997). Dyslexic and category-specific aphasic impairments in a self-organizing feature map model of the lexicon. Brain and Language, 59, 334366. Miikkulainen, R. (1999). The DISLEX simulator (new version) [on-line]. Available at ftp://ftp.cs.utexas.edu/pub/neural-nets/software/dislex.2.1.tar.Z. Obler, L., & Albert, M. (1978). A monitor system for bilingual language processing. In M. Paradis (Ed.), Aspects of bilingualism (pp. 105-113). Columbia, SC: Hornbeam Press. Paradis, M. (1989). Bilingual and polyglot aphasia. In F. Boiler & J. Grafman (Eds.), Handbook ofneuropsychology (Vol. 2, pp. 117-140). Amsterdam: Elsevier. Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996). Understanding normal and impaired word reading: Computational principles in quasi-regular domains. Psychological Review, 103, 56-115. Rumelhart, D., & McClelland, J. (1986). On learning the past tenses of English verbs. In J. McClelland, D. Rumelhart, & the PDP research group (Eds.), Parallel distributed processing." Explorations in the microstructure of cognition (Vol. 2, pp. 216-271). Cambridge, MA: MIT Press. Small, S. L., Hart, J., Nguyen, T., & Gordon, B. (1995). Distributed representations of semantic knowledge in the brain. Brain, 118, 441-453. Smith, M. C. (1997). How do bilinguals access lexical information?. In A. M. B. de Gro0t & J. F. Kroll (Eds.), Tutorials in bilingualism: Psycholinguistic perspectives (pp. 145-169). Mahwah, NJ: Erlbaum. Spitzer, M. (1999). The mind within the net. Cambridge, MA: MIT Press. van Heuven, W. J. B. (2000). Visual word recognition in monolingual and bilingual readers: Experiments and computational modeling. Unpublished doctoral dissertation, NICI, University of Nijmegen, Netherlands. Yip, V., & Matthews, S. (2000). Syntactic transfer in a Cantonese-English bilingual child. Bilingualism, 3, 193-208.
This Page Intentionally Left Blank
Part III: Memory Representation in Sentence Processing
This Page Intentionally Left Blank
Bilingual Sentence Processing- R.R. Herediaand J. Altarriba(Editors) 9 2002 Elsevier Science B.V. All rights reserved.
Cross-Language Facilitation, Repetition Blindness, and the Relation Between Language and Memory: Replications of Aitarriba and Soltano (1996) and Support for a New Theory Don MacKay University of California, Los Angeles Lori E. James University of Colorado Lise Abrams University of Florida, Gainesviile
Abstract
Using improved materials, procedures, analyses, and experimental design, the present study replicated and extended several interesting effects reported in Altarriba and Soltano (1996). In two experiments, 48 highly proficient Spanish-English bilinguals read and immediately recalled rapidly presented (RSVP) sentences and three-word lists resembling those of Altarriba and Soltano: each stimulus contained a target and pretarget word that were related in three ways: complete identity (e.g. ants-ants), semantic identity across languages (i.e., cross-language translation equivalents such as hormigas-ants), and unrelated within or across languages (e.g., tie-ants). Results indicated that with complete identity, repetition blindness occurred in both lists and sentences (i.e., participants always recalled targets better when unrepeated than when repeated within the same language). In contrast, with semantic identity across languages, repeated concepts were recalled better than unrepeated concepts in lists, but this cross-language facilitation effect only occurred for ambiguous words with many translation equivalents: Neither RB nor facilitation occurred for unambiguous words with a single translation equivalent. These results support a new theory of repetition blindness and carry implications for theories of the relation between language and memory for lists versus sentences. MacKay and Miller (1994) presented mixed-language sentences via Rapid Serial Visual Presentation (RSVP) to Spanish-English bilinguals for immediate recall and found that when target words (T) were repeated in the same language within a sentence such as 1 (below; translation:."They saw horses, but horses were forbidden to enter"), T recall decreased compared to unrepeated Ts as in 2,
90
Don G. MacKay, Lori E. James, and Lise Abrams
replicating an effect known as repetition blindness (RB). They also demonstrated reduced recall when the T was a translation equivalent of the prior "pretarget" word (PT) as in 3 (T and PT underlined), rather than unrelated as in 4, an effect they called "semantic blindness" (SB). 1. They saw caballos, but caballos were prohibito to enter. (RB repeated T condition) 2. They saw ovejas, but caballos were prohibito to enter. (RB unrepeated T condition) 3. They saw horses, but caballos were prohibito to enter. (SB repeated T condition) 4. They saw sheep, but caballos were prohibito to enter. (SB unrepeated T condition) However, Altarriba and Soltano (1996) noted that some of the mixed-language sentences in MacKay and Miller (1994) were ungrammatical, and using grammatical sentences and other seemingly minor procedural changes in their Experiment 1b, they found RB for within-language repetition (as in 1 vs. 2), but did not find SB for conceptual or cross-language repetition (as in 3 vs. 4). MacKay, Abrams, Pedroza, and Miller (1996) next conducted detailed analyses of the procedures, analyses, and experimental design in Altarriba and Soltano, and noted seven factors that may have worked to diminish SB in their results, such as long lags between PTs and Ts in the sentences, relatively slow presentation rates, use of a dependent measure that failed to exclude trials where the PT was undetected, and use of split-language sentences with a single, predictable language switch in each sentence. In an attempt to resolve the conflicting results of these earlier SB studies, the present study tested whether SB would occur using the procedures, analyses, and experimental design of MacKay and Miller but with fully grammatical sentences as in Altarriba and Soltano. A second goal of the present study was to replicate an interesting difference that Altarriba and Soltano (1996) reported for lists vs. sentences: With h translation equivalent (manejar) as PT, a T such as drive became easier to recall in lists but not in split-language sentences such as "Mike leamed to drive and empes6 a manejar al trabajo" (translation: "Mike learned to drive and began to drive to work"). To illustrate the list procedure in Altarriba and Soltano (Experiment 2), each contained three words as in 5-8 (PT and T underlined), preceded and followed by masking stimuli (symbol strings) that participants (proficient Spanish-English bilinguals) were instructed to ignore. The dependent variable was recall of the T (e.g., drive in 5-8) as a function of whether its PT was identical (as in 5), semantically identical (i.e., a translation equivalent, as in 7), or unrepeated (as in 6 and 8). The interesting finding was that when drive had a translation equivalent (manejar) as PT, participants exhibited a statistically reliable facilitation effect (relative to the unrepeated condition illustrated in 8), but exhibited the usual inhibitory effect (RB) with an identical PT (as in 5).
Repetition Blindness in Bilinguals 5. 6. 7. 8.
91
drive-(intervening word)-drive (RB repeated T condition) steer-(intervening word)-drive (RB unrepeated T condition) manejar-(intervening word)-drive (SB repeated T condition) guiar-(intervening word)-drive (SB unrepeated T condition)
The present study tested an account of this list-sentence difference developed in detail in MacKay et al. (1996). This account involves three postulates for which there is already considerable support: (a) that the mechanisms for storing and retrieving verbal materials in lists are inseparable from mechanisms that have evolved for producing, comprehending, and representing sentences (see e.g., MacKay & Abrams, 1996); (b) that proficient bilinguals represent the meaning of translation equivalents via a single lexical node (see e.g., MacKay, 1982); and (c) that RB and SB occur when the same node must be activated and then reactivated in quick succession under RSVP (see e.g., Abrams, Dyer, & MacKay, 1996). Given these assumptions, the cross-language equivalents had different effects for sentences vs. lists in Altarriba and Soltano (1996) because of a fact noted in MacKay and Bowman (1969)" that a word such as thick has many possible meanings and translation equivalents when presented in isolation, but only a single meaning and translation equivalent (grueso) within a sentence such as, "Mike likes comer queso grueso encima de thick bread." For example, taken in isolation, thick has at least four distinct meanings corresponding to the Spanish translation equivalents grueso, espeso, denso, and bruto, and under the theory, a single lexical node represents each of these meanings for proficient Spanish-English bilinguals. These four shared nodes are illustrated in Figure 1 (in rectangles), together with the phonological representations for thick, grueso, espeso, denso, and bruto (in dashed rectangles). As shown, the phonological representation for thick connects bottomup with all four lexical nodes, which in turn connect top-down with phonological representations corresponding to grueso, espeso, denso, and bruto. This means that the occurrence of thick as a pretarget in lists such as thick horses thick, and thick caballos grueso would strongly prime the lexical nodes for all four meanings of thick, even though only one becomes activated and perceived, say the one that represents the meaning "not watery" (espeso). However, the phonological representations for grueso, espeso, denso, and bruto will all be primed or readied for activation (see Figure 1), so that when grueso occurs in this list, the phonological and lexical nodes for grueso, being already primed, are easier than normal to activate, giving rise to the facilitation effect observed in Altarriba and Soltano. Experiment 1 directly tested an implication of this account of cross-language facilitation within lists. The implication is that only lexically ambiguous PTs with many Spanish-English equivalents should facilitate recall of cross-language Ts in lists (e.g., 7). The facilitation effect that Altarriba and Soltano observed should not be obtained for PTs with only a single meaning and only a single Spanish-English equivalent in split-language lists.
92
Don G. MacKay, Lori E. James, and Lise Abrams
Lexical Concept Nodes (shared in Spanish and English)
//
//
/
Q I|
ss S
ss| s S
J
s ~
oS J
~s~ ~
ss
s I
I
..~'
s
J
s
S
I s SS
s I fs
stupid
dense
not watery
thick
~.1" | I |
1! |. . . . . .
! ! ! ! ! ! ! ! ! I
"1
,THICK
I
English' Phonological Form
,, I ! !
,, !
Spanish Phonological Forms
(5 ,, I
L
GRUESO
i, "= I
A
I I
ESPESO
~' I I
' I '
DENSO
='
I _,
,'
BRUTO
I
I
,
I a
Figure 1. Connections in NST between the phonological form for thick, a sample of
its (shared Spanish-English) lexical conceptual nodes, and phonological forms for corresponding Spanish words (grueso, ~ , denso, and bruto). Initial presentation of thick primes the unshaded nodes connected with broken lines (but does not activate them; see MacKay, 1990, for the priming-activation distinction), and activates the nodes connected with solid lines, resulting in self-inhibition (solid shading).
Experiment 1" Effects of Within- vs. Between-Language Repetition in Lists Experiment 1 assessed the degree of RB and SB (if any) in RSVP lists using improved procedures noted in MacKay et al. (1996), but was otherwise similar to Altarriba and Soltano (1996, Experiment 2) except that half of the PTs were ambiguous and allowed many translation equivalents and half were unambiguous and allowed only one translation equivalent. We expected no facilitation effect for
Repetition Blindness in Bilinguals
93
cross-language Ts with unambiguous PTs but we expected to replicate the facilitation effect of Altarriba and Soltano for Ts with ambiguous PTs (e.g., 7).
Participants Participants were 48 college students (28 females, 20 males, mean age = 19.7, mean years of education = 14.1) who participated for course credit or payment. All were Spanish-English bilinguals who had approximately equal proficiency in Spanish and English as assessed via a background questionnaire and conversations in English and Spanish with a bilingual experimenter. The participants provided Likert-scale ratings in the background questionnaire that suggested only slightly more daily use of English than Spanish: On a 5-point scale (1 = mostly English, 3 = equal use of Spanish and English, and 5 = mostly Spanish), their mean rating was 2.44. Answers to a question on how often they switch languages mid-sentence in everyday conversation also suggested flexible proficiency in both languages: 77% of the participants indicated that they often or sometimes switch languages, while only 21% indicated that they rarely or never do so. Three other questions also suggested extensive proficiency in both languages. For one question, 35% of the participants reported speaking Spanish more often in everyday life, 25% reported speaking English more often, and 17% claimed to speak Spanish and English equally often. For another question, 85% of the participants indicated Spanish as their native language with English acquired during early childhood (mean age - 5 years old), and 15% indicated English as their native language. Finally, mean ratings on five-point scales assessing relative competence in English vs. Spanish (where 1 = English much better, 2 = English somewhat better, 3 = Equal, 4 = Spanish somewhat better, and 5 = Spanish much better) were 1.91 for knowledge of grammar, 2.00 for vocabulary knowledge, 2.22 for spoken fluency, and 2.62 for spoken comprehension, indicating somewhat greater competence for grammar, vocabulary, and spoken fluency in English but almost equal competence for spoken comprehension in English and Spanish.
Materials Materials were 40 filler and 72 experimental lists. Experimental lists are illustrated in Appendix A and contained three words in the order PT, intervening word, and T. Ts and PTs were either identical within a language (e.g. ants-ants: the within-language repeated T condition), semantically identical across languages (i.e., non-cognate translation equivalents such as hormigas-ants" the cross-language repeated T condition), or unrelated within or between languages (e.g., tie-ants: the unrepeated T condition). As in these examples, Ts were identical across these three basic conditions but counterbalanced across subjects. For unrepeated conditions, Ts and PTs were equal in mean number of letters, and across all conditions, Ts and PTs were either English or Spanish translation equivalents selected from DuboisCharlier, Pritchard, Senerth, and Sola (1987) by a proficient bilingual (native Spanish-speaker). Language of PT and T was crossed to yield two within-language
94
Don G. MacKay, Lori E. James, and Lise Abrams
combinations (English PT-English T and Spanish PT-Spanish T) and two betweenlanguage combinations (English PT-Spanish T and Spanish PT-English T). The eight conditions that resulted were counterbalanced across participants so that every PT-T pair appeared in every condition. For between-language trials (English PTSpanish T and Spanish PT-English T), the word intervening between PT and T was English for half the lists and Spanish for the remaining half. For unrepeated conditions, PT and T were always semantically and phonologically unrelated, and all three words were the same language in within-language lists. Finally, we manipulated ambiguity of the English words, yielding 16 conditions overall. Ambiguous words had four or more meanings listed in Webster's College Dictionary (1995), and three or more Spanish translation-equivalents in DuboisCharlier et al. (1987). Unambiguous words had one distinct meaning in Webster's and one primary Spanish translation-equivalent in Dubois-Charlier et al. (see Appendix A). Because word forms increase in frequency with ambiguity (i.e., how many meanings dictionaries list for the word form; Zipf, 1949), word frequency was higher for ambiguous than unambiguous words in the present materials (M = 177 vs. 59 per million in Francis & Ku(,era, 1982, SD = 116). Filler lists served to make word repetition, language, and number of words per list less predictable: Fillers were either two or four words long and contained words that were unrepeated and unrelated (semantically and phonologically), and were either in English (N = 10), Spanish (N = 10), or an unpredictable mix of English and Spanish (N = 20). Procedure
Participants completed an informed consent form, a demographic questionnaire, a digit span test, and the language background questionnaire. Following general instructions regarding RSVP procedures and mixed-language sentences, participants received 8 practice trials with instructions to read and immediately recall each word in its presented position in the list. They were cautioned not to translate but to recall each word in the language presented. Responses were scored on-line by the experimenter and verified later from a tape recording of the session. Each trial began with a 2000 millisecond (ms) warning signal: "Get ready for the next list." Next came seven 90 ms frames: a row of percent signs, a row of asterisks, the PT, the intervening word, the T, a row of dollar signs, and a row of percent signs. A row consisted of five symbols of a given type. A string of question marks signaled the end of the list and called for recall, remaining on the screen until the participant pressed any key to trigger the next trial.
Results
As instructed, participants almost never recalled a list item in the wrong language, and overall, 64% of words were correctly recalled (SD = 13%), precluding floor and ceiling effects. Two lists were discarded due to stimulus flaws
Repetition Blindness in Bilinguals
95
discovered post hoc (one list contained homophones, and one contained a nonword), giving 70 experimental trials per subject in our analyses. Trials were scored for correct inclusion of T in recall under three scoring criteria (lenient, strict, and Altarriba & Soltano, described shortly), and Figures 2a and 2b show mean correct T recall by condition using lenient scoring criteria. Lenient scoring first removed trials containing incorrect or missing PTs and then required correct recall of both PT and T in the correct order, but otherwise ignored recall of the intervening word and did not penalize intrusions. For example, PT and T in the list "sky (PT) red dog (T)" were scored as correct given recall of"sky dog," "sky raw dog," "sky girl red dog," or "boy sky red dog." If PT and T were both recalled but in the wrong order relative to the intervening word, only the word in correct list position was scored as correct, for example, dog (T) in "raw sky dog" and sky (PT) in "sky dog red" and neither PT nor T was scored as correct in "dog red sky," or "red dog sky." A 2 (repetition: repeated, unrepeated) x 2 (ambiguity: ambiguous, unambiguous) x 2 (combination: within-language, between-language) x 2 (target language: English, Spanish) repeated measures ANOVA yielded no main effect of ambiguity, but a main effect of repetition, F(1,42) = 73.47, MSe = 0.09, p < .001, with better recall of unrepeated than repeated Ts, a main effect of combination, F(1,42) = 56.49, MSe = 0.13, p < .001, with better recall of between- than within-language Ts, and a main effect of target language, F(1,42) = 22.11, MSe = 0.06, p < .001, with better recall of English than Spanish Ts. These effects were moderated by an interaction of repetition and combination, F(1,42)= 142.25, MSe = 0.08, p < .001, with RB for within-language repetition, t(47) = -13.40, p < .001, but facilitation for between-language repetition, t(47) = 3.34, p < .01, and a repetition by combination by ambiguity interaction, F(1,42) = 4.34, MSe = 0.07, p <. 05. For ambiguous targets, there was an interaction of repetition and combination, F(1,47) = 110.46, MSe = 0.04, p < .001, due to occurrence of RB for withinlanguage repetition, t(47)=-12.11, p < .001, and facilitation for between-language repetition, t(47) = 3.83, p < .001. For unambiguous targets, there was a repetition by combination interaction, F(1,47) = 64.13, MSe = 0.04, p < .001, due to RB for within-language repetition, t(47) = -11.54, p < .001, but no effect for betweenlanguage repetition, t(47) = 0.43, p =.67. Within-language RB was equivalent for ambiguous and unambiguous words (47% vs. 45%; see Figure 2a), while betweenlanguage facilitation was 14% for ambiguous words versus only 2% for unambiguous words (see Figure 2b). Strict scoring of correct T recall required that PT, the intervening word, and T all be recalled in the correct order. Strict scoring yielded essentially the same pattern of results as lenient scoring. For example, a 2 (repetition: repeated, unrepeated) x 2 (ambiguity: ambiguous, unambiguous) x 2 (combination: within-language, between-language) x 2 (target language: English, Spanish) repeated measures ANOVA yielded no main effect of ambiguity, but a main effect of repetition, F(1,29) = 44.85, MSe = 0.11, p < .001, with better recall of unrepeated than
96
Don G. MacKay, Lori E. James, and Lise Abrams
repeated Ts, a main effect of combination, F(1,29) = 18.91, MSe = 0.16, p < .001, with better recall of between- than within-language Ts, and a main effect of target language, F(1,29) = 10.05, MSe = 0.06, p < .01, with better recall of English than Spanish Ts. However, these effects were moderated by an interaction of repetition and combination, F(1,29) = 61.47, MSe = 0.11, p < .001, with RB for withinlanguage repetition but facilitation for between-language repetition.
90% 80%
~o%
59%
n~ 60%
57% !"II Repeated Targets
50%
~6 L-I,.,.
I I Unrepeated Targets
40% 30%
~ 20% 10% 0% Ambiguous
Unambiguous
Figure 2a. Mean correct recall by condition (under lenient scoring) for within-
language lists in Experiment 1.
90% 80% 70% 60%
61%
57% ==o,,_
40% z,,,,. ,.,__
t~
Repeated Targets Unrepeated Targets
g 50% 30%
~ 20% lO% 0% Ambiguous
Unambiguous
Figure 2b. Mean correct recall by condition (under lenient scoring) for between-
language lists in Experiment 1.
Repetition Blindness in Bilinguals
97
Altarriba and Soltano scoring analyzed T recall without first removing trials
containing incorrect PTs following Altarriba and Soltano (1996). This analysis yieldedthe same pattem of results as lenient scoring. Subsidiary Results Using lenient scoring to test for possible effects of Spanish vs. English Ts, recall of cross-language Ts was comparable for English and Spanish Ts overall (54% vs. 52%), as well as for unrepeated Ts in English vs. Spanish (51% vs. 47%) and repeated Ts in English vs. Spanish (58% vs. 57%), yielding a comparable facilitation effect for cross-language lists with English vs. Spanish targets (7% vs. 10%), t(47) = -0.52, p > .60. However, for within-language Ts, the level of target recall was much higher for English than Spanish Ts (44% vs. 26%), whereas recall was roughly proportional for unrepeated Ts in English vs. Spanish (70% vs. 42%) and for repeated Ts in English vs. Spanish (18% vs. 10%), yielding a larger RB effect for English than Spanish lists (52% vs. 32%), t(47) = 4.76, p < .001. We currently lack a plausible explanation for this difference. We also tested for possible effects of word frequency using lenient scoring, categorizing our English Ts as low frequency (LF) vs. high frequency (HF) via median split at 40.5 per million in Francis and Ku~era (1982; LF = 40 or fewer per million, HF = over 40 per million). A 2 (repetition: repeated, unrepeated) x 2 (combination: within-language, between-language) x 2 (frequency: LF, HF) repeated measures ANOVA on trials with English T yielded main effects of repetition, F(1,42) = 44.56, MSe = 0.10, p < .001, of combination, F(1,42) = 12.94, MSe = 0.12, p <. 01, and repetition by combination interactions for both HF targets, F(1,46) = 79.09, MSe = 0.07, p < .001, and LF targets, F(1,43) = 56.92, MSe = 0.05, p < .001, but no main effect of frequency and no interactions involving frequency at p < .05, despite the fact that frequency covaried with ambiguity.
Discussion Using procedures advocated in MacKay et al. (1996), present results replicated the basic findings of Altarriba and Soltano (1996, Experiment 2): the usual inhibitory effect (RB) for within-language repetition, but a reliable facilitation effect for between-language repetition. This between-language facilitation effect poses problems for token individuation theory in both its original and newer forms. The original token individuation theory (e.g., Kanwisher, 1987; Kanwisher & Potter, 1990) viewed token individuation to be purely visual or restricted to the processing of visual forms rather than orthographic, phonological, or semantic concepts, and cannot explain why translation equivalents become especially individuated, whereas exact repetitions fail to become individuated. Newer token individuation theories (e.g., Bavelier, 1994) likewise cannot explain the semantic facilitation effect because they predict that orthographic, phonologic, or lexical repetition will lead to inhibition, but not facilitation under RB conditions.
98
Don G. MacKay, Lori E. James, and Lise Abrams
However, the interactions of repetition and combination with ambiguity in Experiment 1 refined the Altarriba and Soltano (1996) cross-language facilitation effect: Between-language facilitation occurred for ambiguous targets with many translation equivalents, but not for unambiguous targets with only a single translation equivalent. These interactions were predicted and support the new theory of RB outlined in MacKay et al. (1996). The connections and processes for explaining cross-language facilitation in lists in this theory has already been discussed in the introduction, and the same connections and processes explain RB in rapidly presented word lists such as thick shift thick: Initial presentation of thick in such lists primes and activates the phonological nodes for thick, which strongly primes the lexical nodes for all four meanings of thick, and activates the most primed one, say the one corresponding to the meaning grueso in Figure 1. As a result, phonological nodes for thick and the lexical node for the grueso meaning may be undergoing the self-inhibitory process that follows activation (represented via solid circles in Figure 1) at the time when the third word in the list appears. Since the third word in this example is thick, RB is a likely outcome because the self-inhibited phonological nodes for thick will be harder to activate than when the initial word is phonologically, orthographically, and semantically unrelated to thick.
Experiment 2: Effects of Within- versus Between-Language Repetition in Sentences
Experiment 2 had the same participants as Experiment 1 and always immediately followed Experiment 1, but the stimuli were grammatical mixedlanguage sentences. We expected to replicate the reliable SB effect observed in MacKay and Miller (1994). Materials
Materials were 24 filler sentences and 24 mixed-language experimental sentences (shown in Appendix B). Experimental sentences were 9.4 words long on average (range 6-13 words) with 1.7 words on average intervening between PT and T. PTs and Ts were similar to those in Experiment 1 except that ambiguity was not manipulated, PTs and Ts never occupied first or last position in a sentence, English PT-Spanish T was the only between-language combination, and with the T omitted, 88% of the sentences became ungrammatical. Language switched within the sentences 1.95 times on average (SD = 0.37; range 1-3), and at unpredictable points across sentences. To further reduce predictability of word repetition and the occurrence and locus of language switches, filler sentences contained no repeated words and many were entirely in English (N = 7) or entirely in Spanish ( N - 7).
Repetition Blindness in Bilinguals
99
Procedure
Experimental and scoring procedures resembled Experiment 1 with four exceptions: there were 12 experimental conditions (because there was no Spanish PT-English T condition); presentation rate was 70 ms/word for half the sentences and 90 ms/word for the other half, with rates for particular sentences counterbalanced across participants; there were only 6 practice trials; and "Get ready for the next sentence" was the prompt that immediately preceded each sentence.
Results
Analyses discarded two experimental sentences due to flawed computer assignment to conditions (yielding 22 experimental trials per subject), and ignored the rare instances where participants recalled words in the wrong language (mean probability per participant was .019 per word). Overall, 51% of the words in a sentence were correctly recalled (SD = 12%), precluding floor and ceiling effects. Lenient scoring required correct recall of PT and T in the correct order, but otherwise ignored recall or misrecall of any other word in the sentences. To illustrate, for the sentence "I live at one(PT) three six(T) Elm Street," PT and T were scored as correct given recall as "I have one and six elms," or "I live at one four three six South Elm Street." Figure 3 shows mean correct T recall by condition (collapsed across rate) under lenient scoring. A 2 (repetition: repeated, unrepeated) by 2 (combination: within, between) by 2 (rate: 70 ms, 90 ms) repeated measures ANOVA revealed a main effect of repetition, F(1,25) = 7.94, MSe = 0.16, p < .01, with better recall of unrepeated than repeated Ts, and a main effect of combination, F(1,25) = 38.48, MSe = 0.19, p < .001, with better recall of between- than withinlanguage Ts. These effects were moderated by a marginally significant interaction of repetition and combination, F(1,25) - 3.71, MSe = 0.12, p < .07. However, rate neither yielded a main effect nor interacted with any other variable, which allowed us to co
