Media Borders, Multimodality and Intermediality

Media Borders, Multimodality and Intermediality

This page intentionally left blank

Media Borders, Multimodality and Intermediality Edited by

Lars Elleström Editorial Board: Jørgen Bruhn, Siglind Bruhn, Claus Clüver, Christina Ljungberg, Silvestra Mariniello, Jürgen E. Müller and Valerie Robillard

Introduction, selection and editorial matter © Lars Elleström 2010 Individual chapters © contributors 2010 All rights reserved. No reproduction, copy or transmission of this publication may be made without written permission. No portion of this publication may be reproduced, copied or transmitted save with written permission or in accordance with the provisions of the Copyright, Designs and Patents Act 1988, or under the terms of any licence permitting limited copying issued by the Copyright Licensing Agency, Saffron House, 6-10 Kirby Street, London EC1N 8TS. Any person who does any unauthorized act in relation to this publication may be liable to criminal prosecution and civil claims for damages. The authors have asserted their rights to be identified as the authors of this work in accordance with the Copyright, Designs and Patents Act 1988. First published 2010 by PALGRAVE MACMILLAN Palgrave Macmillan in the UK is an imprint of Macmillan Publishers Limited, registered in England, company number 785998, of Houndmills, Basingstoke, Hampshire RG21 6XS. Palgrave Macmillan in the US is a division of St Martin’s Press LLC, 175 Fifth Avenue, New York, NY 10010. Palgrave Macmillan is the global academic imprint of the above companies and has companies and representatives throughout the world. Palgrave® and Macmillan® are registered trademarks in the United States, the United Kingdom, Europe and other countries. ISBN-13: 978–0–230–23860–2 hardback This book is printed on paper suitable for recycling and made from fully managed and sustained forest sources. Logging, pulping and manufacturing processes are expected to conform to the environmental regulations of the country of origin. A catalogue record for this book is available from the British Library. A catalog record for this book is available from the Library of Congress. 10 9 8 7 6 5 4 3 2 1 19 18 17 16 15 14 13 12 11 10 Printed and bound in Great Britain by CPI Antony Rowe, Chippenham and Eastbourne

Contents List of Figures

vii

Acknowledgements

viii

Notes on Contributors

ix

Introduction Lars Elleström

1

Part I Media, Modalities and Modes 1 The Modalities of Media: A Model for Understanding Intermedial Relations Lars Elleström

11

Part II Media Borders of Qualified Media 2 Border Talks: The Problematic Status of Media Borders in the Current Debate about Intermediality Irina O. Rajewsky

51

3 Intermedial Topography and Metaphorical Interaction Axel Englund

69

4 Intermedial Strategies in Multimedia Art Christina Ljungberg

81

Part III Combinations and Integrations of Media 5 ‘Media’ before ‘Media’ were Invented: The Medieval Ballad and the Romanesque Church Sigurd Kværndrup 6 The Intermediality of Field Guides: Notes Towards a Theory Håkan Sandgren 7 Media on the Edge of Nothingness: Visual Apostrophes in Lettrism Sami Sjöberg v

99

111

124

vi Contents

Part IV Mediations and Transformations of Media 8 Penrose, ‘Seeing is Believing’: Intentionality, Mediation and Comprehension in the Arts Siglind Bruhn 9 Beyond Definition: A Pragmatic Approach to Intermediality Valerie Robillard 10 Translating Sounds: Intermedial Exchanges in Amy Lowell’s ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’ Regina Schober 11 ‘Transgenic Art’: The Biopoetry of Eduardo Kac Claus Clüver

137 150

163 175

12 Photo/graphic Traces in Dubravka Ugreši´c’s The Museum of Unconditional Surrender Katalin Sándor

187

13 The Dance of Intermediality: Attempt at a Semiotic Approach of Medium Specificity and Intermediality in Film Hajnal Király

199

14 Media in the Cinematic Imagination: Ekphrasis and the Poetics of the In-Between in Jean-Luc Godard’s Cinema Ágnes Peth˝ o

211

Part V The Borders of Media Borders 15 Heteromediality Jørgen Bruhn

225

16 Intermediality Revisited: Some Reflections about Basic Principles of this Axe de pertinence Jürgen E. Müller

237

Index

253

List of Figures 1 The modalities and modes of media 2 The first measures of Maurice Ravel’s piano piece Gaspard de la nuit 3 Lucia Leão, Hermenetka (2005–2007): ‘What is the Mediterranean for you?’ 4 The ballad dance sublime; the church of Ørslev, Denmark, ca. 1300 5 An excerpt of Alain Satié’s ‘Sur le pont d’Avignon’ 6 The ‘musical icon’ of John Tavener’s Mary of Egypt 7 John Tavener, Mary of Egypt: schematic representation of the hand bells of ‘unearthly stillness’ 8 Messiaen’s favourite vertically-symmetric chord: the generation of its spatial pattern and its distribution on a keyboard 9 The differential model 10 Calum Colvin, Fragments of Ancient Poetry IV 11 Eduardo Kac, Genesis, 1999. Transgenic work with artist-created bacteria, ultraviolet light, Internet, video (detail) 12 Greimas’s semiotic square of cognitive modalities, from A. Greimas (1987) 13 Frames from Satan’s Tango by Béla Tarr (1994) 14 Fernsehstube, hall and audience 15 A ‘married couple’ in First and Second Life

vii

36 73 92 100 126 140 141

147 152 155 177 206 207 238 249

Acknowledgements The editor and publishers wish to thank the following for permission to reproduce copyright material: S. Fischer Verlag for Paul Celan, ‘Anabasis’, from Paul Celan, Die c 1963 S. Fischer Verlag GmbH, Frankfurt am Main Niemandsrose,  Calum Colvin for his artwork Fragment IV from the series Ossian, Fragments of Ancient Poetry (2002) Eduardo Kac for his artwork Genesis (1999) Lucia Leão for her internet art work Hermenetka (2005–2007) Alain Satié for his poem ‘Sur le pont d’Avignon’ (1970) The journal Amerikastudien for parts of Regina Schober, ‘Amy Lowell’s Peasant Dance’, Amerikastudien 53(2) (2008) Every effort has been made to trace rights holders, but if any have been inadvertently overlooked the publishers would be pleased to make the necessary arrangements at the first opportunity. Unless otherwise noted, all translations are by the authors of the essays.

viii

Notes on Contributors Jørgen Bruhn is Assistant Professor in the Department of Comparative Literature, Linnaeus University, Sweden. He has published books and articles on Marcel Proust, M. M. Bakhtin, medieval literature, the history of the novel and the theory and practice of intermedial studies. Siglind Bruhn is a musicologist, concert pianist and interdisciplinary scholar. A full-time researcher at the University of Michigan’s Institute for the Humanities and a former guest researcher at the Sorbonne’s Institute for the Aesthetics of Contemporary Arts, she has authored thirty booklength monographs, primarily in the field of twentieth-century music and its relationship with literature, art and religion (her most recent publications in English include a book trilogy on Olivier Messiaen’s musico-symbolic language). She is a co-editor of the Pendragon Press book series Interplay: Music in Interdisciplinary Dialogue. In 2001 she was elected to the European Academy of Arts and Sciences; in 2008 she received an honorary doctorate from Växjö University, Sweden. Claus Clüver is Professor Emeritus of Comparative Literature, Indiana University. He has also taught at New York University and in Germany, Sweden, Denmark and repeatedly in Portugal and Brazil. His publications include over 30 essays on intermediality and interarts studies, especially on concrete and visual poetry, intersemiotic transposition, ekphrasis and representation in the arts. He is co-editor of The Pictured Word (1998), Signs of Change: Transformations of Christian Traditions and their Representation in the Arts, 1000–2000 (2004), Orientations: Space / Time / Image / Word (2005) and Intermidialidade (Belo Horizonte, Brazil, 2006). In 2003 he received an honorary doctorate from Lund University, Sweden. Lars Elleström is Professor of Comparative Literature, Linnaeus University, Sweden. He organizes the Forum for Intermedial Studies, Linnaeus University, and chairs the board of the Nordic Society for Intermedial Studies. Elleström has written and edited several books, including Divine Madness: On Interpreting Literature, Music, and the Visual Arts Ironically (Bucknell University Press, 2002). He has also published numerous articles on poetry, intermediality, gender and irony. Axel Englund is a Doctoral Candidate in Comparative Literature at Stockholm University, where he also teaches metrics and modernist exile literature. He has previously taught music theory and computer notation ix

x

Notes on Contributors

at Lund University. Englund holds degrees in Musicology (BA, Lund University), Comparative Literature (MA, Lund University) and Composition (MFA, Lund University), and in 2009 he held a visiting scholarship at Columbia University. He is currently working on a dissertation dealing with music in and around the poetry of Paul Celan and has recently published articles on this topic in Seminar: A Journal of Germanic Studies and Perspectives of New Music. Hajnal Király is a freelance film critic, presently translator at the Cardiff (UK) office of Thomson Reuters news agency. Her PhD dissertation deals with alternative discourses of the film–literature relationship, her wider research interests being medium theory, intermediality and the issues of a suitable approach to painting–film–literature interaction. Her publications have appeared in English and in Hungarian, several of them in essay collections on intermediality edited by Ágnes Peth˝ o. Additionally, she has published articles in film periodicals in both Romania and Hungary. Sigurd Kværndrup was educated in Danish and theology at the University of Copenhagen where he taught history of Danish literature from 1969 to 2006. Now he is working as lecturer at the gymnasium in Nyköbing Falster and as researcher at the Linnaeus University, Sweden. He was head of Skaelskoer Folk High School between 1984 and 1990. In 1996 he earned his PhD at the University of Southern Denmark, Odense, with the dissertation, ‘Twelve Principles in Saxo: An Analysis of Gesta Danorum’, printed in Copenhagen 1999, and he received the degree of D Phil with the thesis, The East-Nordic Ballad: Oral Theory and Textual Analysis, Copenhagen 2006. Kværndrup was co-editor of the Anthology of Northern Literature 1–11 (1972–1983) and editor of The History of Danish Literature 1–9 (1984–1985). Christina Ljungberg is Adjunct Professor of English and American Literature at the University of Zurich. Her research interests focus on visuality and narrativity, intermediality, and iconicity. She is the author and editor of To Fit, to Join, and to Make, The Crisis of Representation: Semiotic Foundations and Manifestations in Culture and the Media (with W. Nöth) and Insistent Images (with E. Tabakowska and O. Fischer). She has just completed Redefining Literary Semiotics (with J. D. Johansen and H. Veivo) and Signergy (with M. Beukes, J. Conradie and O. Fischer) and is currently working on her book Creative Dynamics: Diagrammatic Strategies in Narrative. Jürgen E. Müller is Professor and Chair of the Media Department at the University of Bayreuth, heading Media Lab and Campus TV projects. He has a background in literary studies, sociology, linguistics and French studies, and his research spans a number of areas including the history and theory of modern media, intermediality, media culture, French cinema and reception

Notes on Contributors

xi

theory. Among his publications, the book Intermedialität. Formen moderner kultureller Kommunikation (1996) can be noted. He is one of the founders of the CRI (International Center for Research on Intermediality). In 2005 he organized, in cooperation with the CRI, the international congress ‘Media Encounters’ at the University of Bayreuth. Ágnes Peth˝ o is Associate Professor at the Babe¸s-Bolyai University and the Sapientia Hungarian University of Transylvania in Cluj-Napoca, Romania. Her research interests include the relationship of cinema with the other arts, cinematic intermediality and self-reflexivity. She is the editor of the volume Words and Images on the Screen: Language, Literature, Moving Pictures, published by Cambridge Scholars Publishing, 2008. Her other major publications in Hungarian include Múzsák tükre. Az intermedialitás és az önreflexió poétikája a filmben [Mirror of the Muses: The Poetics of Intermediality and SelfReflexivity in Cinema, 2003] and the edited volumes of essays Képátvitelek. Tanulmányok az intermedialitás tárgyköréb˝ ol [Image Transfers: Essays on Intermediality, 2002], Köztes képek. A filmelbeszélés színterei [Images In-Between: Scenes of Filmic Narration, 2003] and Film. Kép. Nyelv [Film. Image. Language, 2007]. Irina O. Rajewsky is Associate Professor of Italian and French Literature at the Freie Universität Berlin, associated with the Institute for Romance Languages and Literature and with the Friedrich Schlegel Graduate School of Literary Studies. Her main areas of research are inter and transmediality, literary theory (in particular narratology, intertextuality, fictionality, metaphenomena) and performativity. Currently she is conducting a research project on ‘Mediality – Transmediality – Narration: Perspectives of a Transgeneric and Transmedial Narratology (Film, Theatre, Literature)’. Her publications include Intermedialität (Tübingen, 2002) and Intermediales Erzählen in der italienischen Literatur der Postmoderne (Tübingen, 2003); she is also co-editor of Im Zeichen der Fiktion. Aspekte fiktionaler Rede aus historischer und systematischer Sicht (Stuttgart, 2008). Valerie Robillard is Associate Professor in the Department of English Language and Culture at the University of Groningen. She has published numerous articles in the area of Interarts Studies and co-edited Pictures Into Words: Theoretical and Descriptive Approaches to Ekphrasis (Amsterdam, VU Press, 1998). Her current research interests include intermediality in the performing arts and in fashion and she is closely involved with the didactic aspects of word and image studies in both academic and public contexts. Håkan Sandgren is Senior Lecturer in Swedish and Comparative Literature at Kristianstad University College, Sweden, where he teaches literature and film and holds a position as Head of Teacher Staff. He is the author of

xii Notes on Contributors

Landskap på jorden och i drömmen. Studier i Folke Isakssons lyrik [Landscapes on Earth and in Dreams: Studies in Folke Isaksson’s Poetry] (Lund University Press, 1999), and was the first literary scholar to introduce ecocriticism and green studies in Sweden. Sandgren is currently working on a book-length study of Swedish ecopoetry in the twentieth century. Katalin Sándor is currently Assistant Lecturer at the Faculty of Letters of Babe¸s–Bolyai University, Cluj-Napoca, Romania. Her research interests include theories of intermediality, visual poetry, ekphrasis, photography in literature and visual culture. She has published her studies in various conference volumes and periodicals (Iskolakultúra, Magyar M˝ uhely, Nyelvés Irodalomtudományi Közlemények, Studia Universitatis Babe¸s–Bolyai–European Studies, Korunk, A Hét). Regina Schober has completed her PhD thesis on intermediality in Amy Lowell’s poetry at Leibniz University, Hanover, in 2009. Her research interests include intermedial relations between music and literature and American modernist poetry. Sami Sjöberg is a researcher at the University of Helsinki in the field of comparative literature, currently working in the research project ‘Literature, Transcendence, Avant-Garde’, funded by the Academy of Finland. His main research subject is the avant-garde movement Lettrism, on which he has published numerous articles in Finnish literary journals and textbooks. His areas of expertise are the contemporary philosophical and literary manifestations of medieval cabbalism, visual poetry and the concept of nothingness.

Introduction Lars Elleström

Let us take a look at something. Let us see what can be found in something that could be anything, for instance, something like an X. What is it? Is it a sign? Yes, at least it can be a sign. The moment we decide that it means something, it becomes a sign. The easiest way to make the X mean something is to put it in some sort of context, for instance, a conventional sign system: the Greek or the Latin alphabets, the sign systems of mathematics or logic, or some other more or less settled scheme. We know what to do with X in the word ‘mix’ and we know that X, in an equation such as ‘3 + X = 5’, means the unknown number. There are very many alphabets, systems and schemes, however, that can make signs ‘readable’ and, furthermore, signs often become signs not in determined sign systems but in contexts that are tied only marginally or indirectly to the prevailing conventions. Perhaps it is not even a very good idea to say that all signs are read, since the acts of interpretation are much diversified. If we stubbornly persist in ‘taking a look’ at the X, we have to admit that even a simple little sign such as this one leads a rather complicated double or triple life. Actually, both well-defined and temporary signs, as well as sign systems and specific media productions and works of art, can be said to share a lot of properties with and, hence, also overlap with other signs, systems and types. The general characteristics of X, shared by innumerable other signs, can be summarized as follows. It has a material interface. The reader of this Introduction is probably confronted with some sort of flat surface: a paper, hopefully in a book, or a computer screen. This kind of material interface allows for a variety of appearing content. You can put almost anything in a book, as long as it is flat and static, and you can put almost anything on a screen, including things that move, as long as it is flat. This flat surface is, first and foremost, seen by the reader, spectator or interpreter. Of course, it is not forbidden to use the other senses as well when interacting with the displayed content of screens and pages, but in this case nothing very significant would be added by way of, for instance, trying to taste the X that can be seen in this 1

2

Introduction

sentence. Nevertheless, many signs that are part of conventional systems have sounds attached to them. When reading this English text, most people form the sound ‘eks’, silently or aloud, when seeing the mark X. This sound is, however, a secondary aspect since it is not part of the material interface but a result of the interpretative interaction with it. In a Greek context, the appearance of the visually identical symbol X would generally lead to the sound ‘chi’ (pronounced as the German ich backwards). Spatiotemporally, this text is static since it remains the same once it has been mediated into a printed or electronically fixed product. It takes time to read it, however, and the alphabetic form invites the reader to decode it according to certain rules including, among other things, standards for sequential reading. On the level of what it represents, this text also creates temporal relations. As regards my writing about primary and secondary sensorial experiences, for example, it is implied that in the described, conceptual ‘world’ that the linguistic expressions refer to, the visual comes first and the aural at a later stage in the interpretative process. There is thus some sort of virtual time in the description. The manner in which the equation ‘3 + X = 5’ is outlined also implies a temporal sequence mirrored by the reading order: first Linda catches three fishes and then an unknown number of fishes, and when she returns at the end of the day she has five fishes. The X, as a written sign, is obviously spatial, both in itself and as part of conventional sign systems or unfixed context. As a visual sign, it stretches out in four directions with a junction in the middle and it is generally conceived as a combination of two intersecting lines. Its spatiality is normally, and certainly in the context of this Introduction, two-dimensional, but also three-dimensional objects, such as Ronald Bladen’s minimalistic but nevertheless very large aluminium sculpture The X, can be seen as parts of the wide sign type of X. X-like figures that we see in paintings, drawings and photographs can acquire virtual, three-dimensional spatiality, but the X in Bladen’s sculpture, as well as the letters of many alphabetical toy bricks, is three-dimensional in itself. It must also be recognized that there is an important aspect of spatiality involved in the activity of ‘taking a look’ at X. To take a look at X means both actually to process the information perceived by our photosensitive receptors and to form cognitive, spatial structures when considering the many aspects of what X can be understood to be and how this being can be understood. To ‘see’ is to form mental structures in cognitive space. The X actually means nothing, however, until it is interpreted as some kind of sign. When reading about ‘mixed vegetables’ in a cookbook most people automatically and rationally see the little X as a conventional sign that is part of a word with a specific meaning: blended, composed of different elements and so forth. If, in the same cookbook, the word ‘mixed’ is shaped by various vegetables with a stalk of asparagus and a stick of celery forming the X, there is an iconic element attached to the conventional

sign: the word ‘mixed’ is no longer arbitrarily connected to its meaning since it actually also resembles what it represents: mixed vegetables. If the caption of an image of mixed vegetables forming the words ‘mixed vegetables’ says that ‘mixed vegetables are good for your health’ we assume that the text, because of its closeness to the image, can be read as a part of what the image expresses. Moreover, we can assume also that vegetables that are not depicted, but are nevertheless closely related, are actually as healthy as the ones depicted, or we can assume that the ones that occur more frequently are actually healthier than the others. All in all, the interpretative acts that form the meaning of this page in the cookbook, that is, the interpretative acts that actually put the sign-functions into action, or even create the signs, can be said to be based on convention, resemblance and contiguity. We must now remember that the X is merely an example. It is something that could be anything. Yet, considering its conventional attributes and significance, it is a funny little sign. When we ponder on, say, Ronald Bladen’s X or the many visually striking Xs in popular culture (graffiti, record sleeves, movie posters and so forth), a whole range of potential meanings pops up: the X can represent meetings and mixes, but it also implies erasure and the forbidden. Nonetheless, X is also the sign of presence. It can mean that something is valid or present and it is the only alphabetic letter that may also be written analphabetically? Even if you cannot ‘read’ the X, you can see its iconic potential. The X always carries visual aspects that may be activated as iconic significance as soon as the interpreter finds an opportunity. When there is a point in it, the X looks like a pair of crossed fingers, branches or vegetables, and since the wings of the windmill definitely look like an X, the X also looks like the wings of a windmill. Ian Hamilton Finlay is said to have written a poem called ‘The Windmill’s Song’ that reads like this: ‘X’. The poem certainly represents the same kind of windmills that can be seen in, say, many paintings by Jan Brueghel, both the elder and the younger. It would thus be easy to put the wings into the sequence of conventionally formed letters. Don Quijote, as we know, had windmills on his mind, and if we spell his name in the original way, and add a slight emphasis by way of capitalizing the crucial letter, we suddenly have something that might be called a concrete poem: ‘Don QuiXote’. In this version of his name, we can actually see, iconically, that he is full of windmills. The X, whether seen as a letter, a conventional, non-alphabetical sign, a poem, an image, a sculpture, a part of a linguistic sequence, a graphic mark on a surface, a sound, a part of a work of art or a media production, or a sign used in completion of the income-tax return, is obviously part of a context where all ‘texts’ and ‘systems’, in the widest senses of the words, overlap. It would be very wrong to say that all systems are the same; that there is no difference at all between the way meaning is produced when we find the X in an exhibition hall and in a form respectively, but if we are to talk about borders we are better off talking about border zones rather than

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 3

Introduction

strictly demarcated borders. Hence, I would say that all kinds of sign systems and also specific media productions and works of art must be seen as parts of a very wide field including not least the material, sensorial, spatiotemporal and semiotic aspects. In my essay in this volume, I call these the four ‘modalities’ of media. By investigating the modalities one clearly sees that all forms of art, media, languages, communication and messages have some characteristics in common which make it possible for something like the X to hover between different systems and simultaneously be part of various frameworks without losing its relative stability. For various reasons it is convenient to talk about ‘media’ when discussing these issues. The notion of a medium is fairly inclusive and offers a way to bring together scholarly efforts within a considerable number of disciplines. The phenomenon whereby the properties of all media partly intersect and the study of this same phenomenon are called intermediality. As might be expected, however, the term intermediality has been used in many ways that are not always entirely compatible. Sometimes the study of intermediality is seen as the study of a group of media and art forms that fall outside the borders of the ‘normal’ or ‘established’ media. In this collection of essays, however, most of the writers, at least most of the time, see intermediality as the precondition for all mediality. Even so, it may occasionally be appropriate to highlight the fact that some media may be perceived as more border-crossing than others. Our small survey of the X has hopefully demonstrated that media actually cross each other rather than border each other. Presumably we will never stop talking about media borders, but in this volume they will be partly crossed out. Media Borders, Multimodality and Intermediality is a collection of essays that have found their final forms in dialog with each other. The initial question posed was simply ‘what is intermediality?’ To answer this question, one must also ask what a medium is and where we find the ‘gaps’ that intermediality bridges. Clearly, the supposedly crossed borders must be described before one can proceed to the ‘inter’ of intermediality. We also set out to investigate how the two notions of intermediality and multimodality, rarely discussed together, are related. The aim was to facilitate communication and theoretical cross-fertilization over the borders between the aesthetic disciplines, media and communication studies, linguistics, and so forth. That turned out to be a lucky strike, I would say, but the edges of the conceptual tools had to be sharpened in order to work properly. Further questions that we all had in common from the beginning, and that are clearly visible in many of the essays, concerned the relations between notions such as technical medium, medium, art form and genre. We wanted to investigate what happens, from a historical and social point of view, when new media and art forms emerge and are delimited and to determine the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

4

theoretical and practical implications of describing something as intermedial or multimodal. The essays thus have common starting-points and they have been influenced by each other. We have sought to avoid confusing conflicts between terminologies and the research angles are relatively compatible, but there is no absolute harmony between the essays. They are written by individual scholars who rarely agree on the best way to deal with tricky queries. They are, however, focused on a coherent range of questions and topics that are believed to be important. Of course, the titles of the volume’s sections only partly mirror the content of the essays, but they might give a hint of the main focus. The first essay, ‘The Modalities of Media: A Model for Understanding Intermedial Relations’, is my own attempt to build a theoretical framework that explains how all media are related to each other: what they have in common and in which ways they differ. The key to this understanding is to be found in the four modalities that were briefly mentioned earlier in this Introduction. I also find it imperative to distinguish between three aspects of the notion of medium. Basic media are simply defined by their modal properties whereas qualified media are also characterized by historical, cultural, social, aesthetic and communicative facets. Technical media are any objects, or bodies, that ‘realize’, ‘mediate’ or ‘display’ basic and qualified media. Part II, ‘Media Borders of Qualified Media’, contains three essays. Irina O. Rajewsky’s ‘Border Talks: The Problematic Status of Media Borders in the Current Debate about Intermediality’ discusses the many critical approaches that make use of the notion of intermediality. Rajewsky focuses on the assumption of tangible borders between individual media and the recent questioning of precisely this fundamental premise of discernible media borders. Yet, Rajewsky argues, media borders and medial specificities are of crucial importance for art practice, and she specifies how media can be conceived as distinct and how they actually come into play in concrete intermedial practices. In ‘Intermedial Topography and Metaphorical Interaction’, Axel Englund also highlights the tendency to think of arts and media in terms of geographic areas delineated by definable borders, and consequently of intermedial studies as a kind of topographical description. Focusing on relations between music and literature, his essay points to some of the implications of this topographical model, and contrasts them with another way of conceptualizing intermedial relations, namely as metaphorical phenomena. Englund argues that many musico-literary artefacts can be successfully read as a metaphorical interaction between their musical and verbal elements. Christina Ljungberg’s semiotically-oriented essay ‘Intermedial Strategies in Multimedia Art’ proposes that intermediality either concerns the transgression of the borders between conventionally distinct media of communication or the iconic enactment of one medium within another. Ljungberg argues that both of these instances of intermediality are

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 5

Introduction

highly performative, as we are confronted with hybrid forms that generate something new and unique, that they are strongly self-reflexive, since they focus attention both on their own mode of production and on their own semiotic character, and that they constitute a highly effective communication strategy, as they give receivers access to different levels of meaning. Part III, ‘Combinations and Integrations of Media’, includes three essays that mainly draw attention to media as multimodal products of combined and integrated modalities and other media aspects. In ‘ “Media” before “Media” Were Invented: The Medieval Ballad and the Romanesque Church’, Sigurd Kværndrup approaches the medieval ballad as an intermedial art form and proposes a new theory of one aspect of its origin. Kværndrup builds on Marshall McLuhan’s theories about the development of ‘mass media’ during the Middle Ages and relates these to Romanesque church-building. Håkan Sandgren’s ‘The Intermediality of Field Guides: Notes Towards a Theory’ is an essay informed by ecocriticism that investigates the field guide to birds, a hitherto unexplored area for the scholar of intermediality. In the field guide, descriptive prose is combined with images, maps and transcriptions of birdsong. A particular form of transcription offers examples of how the author tries to transcribe bird song to text, an endeavour that sometimes approaches the domain of concrete poetry. In Sami Sjöberg’s ‘Media on the Edge of Nothingness: Visual Apostrophes in Lettrism’, the author is interested in how the Lettrists set out to challenge the limits of media and representation. The techniques applied by this avant-garde group resulted in works blending poetry, narrative fragments and imaginary signs. Sjöberg proposes two new concepts for analysis: the ‘visual apostrophe’ locates omissions in the text, where invented signs either replace the omitted part of the text or supplement the work in an ambiguous relation to language, and ‘meontologization’ refers to the dynamics of signification and nothingness engaged in the artwork. The volume’s fourth and most extensive part is called ‘Mediations and Transformations of Media’ and it includes seven essays that deal with the large and multifaceted field of media transformations. Beginning with the model of the much-discussed relation between creator, work and beholder, Siglind Bruhn suggests a triangular concept of expressive intent in her essay ‘Penrose, “Seeing is Believing”: Intentionality, Mediation, and Comprehension in the Arts’. Regarding the rarely considered discrepancy between the human mind’s limited capacity to grasp the various media equally and the near-unlimited scope of creative options, Bruhn draws attention to areas where the chain of mediation is less than straightforward in cases of both extreme and ostensibly quite accessible messages. Finally, she presents an unusual three-tiered ‘transmedialization’ that proceeds from a visual through a verbal and on to a musical representation. In ‘Beyond Definition: A Pragmatic Approach to Intermediality’, Valerie Robillard argues that definitions, although essential in laying out common terms of discourse, do not

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

6

fully contribute to our understanding or articulation of the various types and degrees of medial interaction. The purpose of her essay is to demonstrate the need to employ ontological systems in the delineation of medial types, systems that have proven indispensable to other disciplines (such as the natural sciences and linguistics) in determining the relative positions of concepts and categories with respect to one another. Robillard proposes the use of a pragmatic intertextual model to delineate and differentiate types of medial interaction and to demonstrate the difference between the textual aspects of intermediality (the ‘message’) and the medial aspects (‘materials’). Regina Schober’s essay ‘Translating Sounds: Intermedial Exchanges in Amy Lowell’s “Stravinsky’s Three Pieces ‘Grotesques’, for String Quartet” ’ calls attention to notions such as ‘transcription’ and ‘intermedial translation’ that imply dynamic processes of movement and transfer between medial borders. More specifically, Schober discusses a verbal representation of a musical work by American modernist poet Amy Lowell as a paradigmatic example of an experimental transformation of one medium into another. Following Roman Jakobson’s term ‘intersemiotic translation’ as well as recent attempts to broaden the terms ‘translation’ and ‘transcription’, she comprehends Lowell’s poem as both a literary and a cultural translation of Stravinsky’s Three Pieces for String Quartet from 1914. In ‘ “Transgenic Art”: The Biopoetry of Eduardo Kac’, Claus Clüver investigates a very different sort of media transformation resulting in poetry: the ‘transgenic’ artwork Genesis (1999) by the Brazilian poet and artist Eduardo Kac, who has radically explored the possibilities of contemporary media technology for art-making. Genesis was based on an artificial gene created from a phrase from the biblical Genesis represented in Morse code that was then converted into a DNA sequence according to a special code, mutated and eventually ‘retranslated’. In his essay, Clüver sets out to explore the functions of the work, the demands it makes on the receiver and its implications for the discourse on media and intermediality and for the notion of ‘poetry’. In Katalin Sándor’s essay, ‘Photo/graphic Traces in Dubravka Ugreši´c’s The Museum of Unconditional Surrender’, Ugreši´c’s novel is interpreted as a text very much concerned with, shaped by and disrupted into (textual) photographs, albums, museums, archives, cards, collections and memories. Sándor focuses mainly on those passages which deal with photographic representation and practices, as well as with a specific photograph incorporated both visually and textually into the exile-narrative. She looks for the medial traces of photography and explains how the photographic medium becomes a kind of apparatus for the textual process of remembrance. In ‘The Dance of Intermediality: Attempt at a Semiotic Approach of Medium Specificity and Intermediality in Film’, Hajnal Király also analyses a specific work of art: Hungarian director Béla Tarr’s Satan’s Tango, a seven-and-a-half-hour movie merging two cinematographic trends beginning with the 1990s that have raised the problematic notions of medium specificity and intermediality in films: the so-called writer’s movies and the contemplative, extremely

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 7

Introduction

slow-paced movies, mostly coming from the Far East, defying all the complex narratorial accomplishments of the film medium. Király argues that the first kind of movie systematically overturns the strict delimitation between literature and film, the idea of conceptuality of the first and visuality of the latter, and that the second kind of movie continuously turns the running time of the narrative into an almost static, ‘plastic’ visual work of art: a picture. In an essay by Ágnes Peth˝ o, ‘Media in the Cinematic Imagination: Ekphrasis and the Poetics of the In-Between in Jean-Luc Godard’s Cinema’, the work of Godard, another film director, is the centre of attention. Peth˝ o states that his films have long been associated with the idea of cinematic intermediality and that both his fiction films and his cinematic essays can be considered a sort of direct theory, or ‘archeology’ of cinema as a medium. The essay attempts an application of the notion of ekphrasis, traditionally understood as verbal descriptions of images, to the medium of cinema. A few major conditions for the perception of cinematic ekphrasis are outlined and, from the variety of intermedial relations in Godard’s films that can be called ekphrastic, four types are charted and exemplified. The fifth and last part of the volume is called ‘The Borders of Media Borders’. The title is an attempt to capture the more general perspectives introduced by the section’s two essays. In his contribution, ‘Heteromediality’, Jørgen Bruhn proposes a widened concept of intermediality constructed on the principle of multimodality. He suggests we begin considering all cultural texts as mixed media and that we start understanding the particular constellations of modalities as tensional relationships in the sense that the mixing of modalities, and consequently of media, always refers to a wider historical and ideological context. An adoption of this heteromedial viewpoint will, he argues, transfer studies of intermediality to the very centre of future humanistic studies. In ‘Intermediality Revisited: Some Reflections about Basic Principles of this Axe de pertinence’, Jürgen E. Müller asserts that the research axis of intermediality actually keeps numerous scholars at universities and research centres all over the globe busy. The variety of aspects of the concept of intermediality makes it very difficult or almost impossible to present some sort of general overview with regard to all options, Müller argues, without opening a sort of academic bookkeeper discourse on different terminological, theoretical, methodological and historical items. Instead, he develops some ‘aphorisms’ on the state of affairs of intermedial studies and some perspectives for a historical approach. Müller is concerned with the question of when a ‘new’ medium becomes a ‘new’ medium and, aided by the test case of television, he discusses intermediality as a process. Furthermore, the concept of intermediality is compared with intertextuality, interartiality and hybridity. Finally Müller asks what the substance of intermedial studies actually is. Hopefully, not only Müller but also the volume as a whole provides a good answer to that question.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

8

Part I

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Media, Modalities and Modes

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

This page intentionally left blank

1 Lars Elleström

What is the problem? Scholars have been debating the interrelations of the arts for centuries. Now, in the age of electronic and digital media, the focus of the argumentation has somewhat shifted to the intermedial relations between various arts and media. One important move has been to acknowledge fully the materiality of the arts: like other media, they are dependent on mediating substances. For this reason, there is a point in not isolating the arts as something ethereal but rather in seeing them as aesthetically developed forms of media. Still, most of the issues discussed within the interart paradigm are also highly relevant to intermedial studies. One such classical locus of the interart debate concerns the relation between the arts of time (music, literature, film) and the arts of space (the visual arts). In the eighteenth century, Gotthold Ephraim Lessing argued famously in Laocoön that there are, or rather should be, clear differences between poetry and painting,1 but for the moment there is a tendency rather to deconstruct the dissimilarities of various arts and media. W. J. T. Mitchell is perhaps the most influential contemporary critic of attempts to find clear boundaries between arts and media. Many important distinctions have thus been made, and then successfully erased; much taxonomy has been construed, and then torn down, and this process has led to many valuable insights – Is that not enough? What is the problem? The problem is that intermediality has tended to be discussed without clarification of what a medium actually is. Without a more precise understanding of what a medium is, one cannot expect to comprehend what intermediality is. This is not only a terminological problem. On the contrary, the understanding of what a medium is and what intermedial relations actually consist of has vital implications for each and every inquiry in old and new fields of study concerning the arts and media: ekphrasis, cinema, illustration, visual poetry, remediation, adaptation, multimedia and so on. I find it as unsatisfying to continue talking about ‘writing’, ‘film’, ‘performance’, ‘music’ and ‘television’ as if they were like different persons that 11 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

The Modalities of Media: A Model for Understanding Intermedial Relations

Media, Modalities and Modes

can be married and divorced2 as to find repose in a belief that all media are always fundamentally blended in a hermaphroditical way. The crucial ‘inter’ of intermediality is a bridge, but what does it bridge over? If all media were fundamentally different, it would be hard to find any interrelations at all; if they were fundamentally similar, it would be equally hard to find something that is not already interrelated. Media, however, are both different and similar, and intermediality must be understood as a bridge between medial differences that is founded on medial similarities. The most important aim of this essay is to present a theoretical framework that explains and describes how media are related to each other: what they have in common, in what ways they differ and how these differences are bridged over by intermediality. In order to accomplish this, it must be understood that the concept of medium generally includes several types or levels of mediality that have to be correlated with each other. ‘Medium’, of course, is a term widely employed and it would be pointless to try to find a straightforward definition that covers all the various notions that lurk behind the different uses of the word. Dissimilar notions of medium and mediality are at work within different fields of research and there is no reason to interfere with these notions as long as they fulfil their specific tasks. Instead, I will circumscribe a concept that is applicable to the issue of intermediality. Since intermediality will be understood as a general condition for understanding communicative and aesthetic mechanisms, events and devices, rather than a peripheral exception to ‘regular’ mediality, such a concept must actually include most of the media notions circulating in the academic world. Hence, I will not produce a two-line definition of ‘medium’. I find such definitions counterproductive when it comes to complex concepts and any clear-cut definition of medium can only capture fragments of the whole conceptual web. Instead, I will try to form a model that preserves the term medium and yet qualifies its use in relation to the different aspects of the conceptual web of mediality. As a term, ‘medium’ should thus be divided into subcategories to cover the many interrelated aspects of the multifaceted concept of medium and mediality. As my arguments unfold, I will distinguish between ‘basic media’, ‘qualified media’ and ‘technical media’. Basic and qualified media are abstract categories that help us understand how media types are formed by very different sorts of qualities, whereas technical media are the very tangible devices needed to materialize instances of media types. Consequently, when talking about a medium without specifications, the term can refer to both a media category and a specific media realization. Evidently, it is important to note that qualified, basic and technical media are not three separate types of media. Instead, they are three complementary, theoretical aspects of what constitutes media and mediality. The wide concept of medium that will be presented here thus comprises several intimately related yet divergent notions that will be terminologically distinguished.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

12

I believe that intermediality cannot fully be understood without grasping the fundamental conditions of every single medium and these conditions constitute a complex network of both tangible qualities of media and various perceptual and interpretive operations performed by the recipients of media. For my purpose, media definitions that deal only with the physical aspects of mediality are too narrow, as are media definitions that strongly emphasize the social construction of media conceptions. Instead, I will emphasize the critical meeting of the material, the perceptual and the social. Media of production and storage are not really relevant for the forthcoming discussion and although I recognize the relevance of the aspect of communication in its widest sense, my aim is not to discuss intermediality within the framework of communication models. Instead, I want to treat mediality from a hermeneutical point of view. I bracket much of the conditions of media production and focus on the perception, conception and interpretation of media as material interfaces situated in social, historical, communicative and aesthetic circumstances. The material of my theoretical framework consists of the notions of modality and mode. Intermedial studies have their historical roots in aesthetics, philosophy, semiotics, comparative literature, media studies and, of course, interart studies.3 During the last few decades, however, the notion of multimodality has also gained ground, while the roots of this new plant have grown in different soils; social semiotics, education, medicine and language and communication studies. There are seldom cross-references between the two research fields of intermedial and multimodal studies and the notions of intermediality and multimodality are surprisingly seldom related to each other.4 Also, in qualified texts of recent date, it is far from clear how ‘intermedial’, ‘multimodal’, ‘intermodal’ and ‘multimedial’ are related.5 Since it is a waste of intellectual energy to develop two closely related research fields separately, it is a matter of priority to straighten things out as far as core concepts and basic terminology are concerned.

What is a medium? Medium means ‘middle’, ‘interval’, ‘interspace’ and so on. The standard definition found in dictionaries stresses that a medium is a channel for the mediation of information and entertainment. Art might be seen as a complex blend of information and entertainment (Horace’s utile dulci) so it should be fully possible to include the art forms among other media. As we know, however, the term ‘medium’ is used in many related but different ways and it is also applied in contexts that are not relevant here. According to Marshall McLuhan’s influential ideas, media are the ‘extensions of man’ and he suggestively argues that not only the spoken word, the photograph, comics, the typewriter and television are media, but also are money, wheels and axes.6 Within the framework of McLuhan’s own sociological theory, this

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 13

Media, Modalities and Modes

notion works rather well, but in order to take the step from ‘medium’ to ‘intermediality’, more accuracy is needed. The term ‘modality’ is related to ‘mode’ and these terms are also widely used in different fields. A ‘mode’ is a way to be or to do things. In the context of media studies and linguistics, ‘multimodality’ sometimes refers to the combination of, say, text, image and sound, and sometimes to the combination of sense faculties; the auditory, the visual, the tactile and so forth.7 In the work of Gunther Kress and Theo van Leeuwen, a mode is understood as any semiotic resource, in a very broad sense, that produces meaning in a social context; the verbal, the visual, language, image, music, sound, gesture, narrative, colour, taste, speech, touch, plastic and so on. This approach to multimodality has its pragmatic advantages but it produces a rather indistinct set of modes that are very hard to compare since they overlap in many ways that are in dire need of further theoretical discussion.8 It is no wonder, then, that the discourses on media and modalities tend to be either separated or mixed up. Why bother to combine, or to keep apart, notions that seem to be fuzzy in rather similar ways? A medium is a channel, one might say, and of course there are many media, that is, modes of mediating information and entertainment. In ordinary situations, this language use is rather unproblematic. If one wants to understand the complexity of individual media in a more precise way, however, I think it is wise to differentiate between medium/intermediality and mode/multimodality. As far as I can tell, there is nothing in the etymology of the words ‘medium’ and ‘mode’, or in the established conceptual uses of them, that clearly determines how they should be related to each other, so here I will see it as my task to raise a theoretical construction and propose how to use the central terms in relation to each other. Earlier efforts to describe the relations between different media and art forms as a rule start off with conceptual units such as image, music, text, film, verbal media or visual media, presuming that it is appropriate to compare these entities. The complexity resulting from such comparisons is often slightly confusing, I would say, because of two limitations. The first problem is that the units compared are often treated as fundamentally different media with little or nothing in common. Thus, every intermedial relation seems to be more or less an anomaly where the supposedly essentially different characteristics of allegedly separate media are presumed to be more or less transformed, combined or blended in a unique way. Mitchell has successfully criticized this way of thinking by pointing to the way various important traits are in fact shared by art forms that are generally seen as opposites, yet Mitchell’s discourse is also paradoxically but profoundly trapped in the tradition of treating art forms as separate entities. In spite of the efforts to erase most of the differences between poetry and painting, he anthropomorphizes the two art forms and emphasizes the ‘struggle’ between them, which makes it difficult to grasp the exact nature of the similarities of media as conceived

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

14

by Mitchell.9 Media are both similar and different and one cannot compare media without clarifying which aspects are relevant to the comparison and exactly how these aspects are related to each other.10 The second problem with many comparisons between conceptual units such as ‘dance’ and ‘literature’ is that the materiality of media is generally not distinguished from the perception of media. This is understandable since it is, in practice, impossible to separate the two. For human beings, nothing exists outside perception. Nevertheless, it is crucial to discriminate theoretically between the material and the perception of the material if one wants to understand how media can be related to each other. One must be able to determine to what extent certain qualities belong to the material aspects of a medium and to what extent they are part of the perception. This is a slippery business, no doubt, but one must acknowledge that, for instance, the quality of ‘time’ in a movie is not the same as the ‘time’ that is necessary to contemplate a still photograph, and that ‘time’ can be said to be present in many forms in one and the same medium. If one avoids taking notice of this intricacy, one is left with a featureless mass of only seemingly identical media that cannot be compared properly. I therefore consider it a matter of urgency to put forward a model that starts at the other end, so to speak: not with the units of established media forms, or with efforts to distinguish between specific types of intermedial relations between these recognized media, but with the basic categories of features, qualities and aspects of all media. My point of departure will be what I call the modalities of media. The modalities are the essential cornerstones of all media without which mediality cannot be comprehended and together they build a medial complex integrating materiality, perception and cognition. Separately, these modalities constitute complex fields of research and they are not related to the established media types in any definite or definitive way; however, I believe that they are indispensable in all efforts to describe the character of every single medial expression. They are all very familiar although their interrelations have not been systematically accounted for. I call them the material modality, the sensorial modality, the spatiotemporal modality and the semiotic modality, and they are to be found on a scale ranging from the tangible to the perceptual and the conceptual. Media and art forms are constantly being described and defined on the basis of one or more of these modalities.11 The categories of materiality, time and space, the visual and the auditory, and natural and conventional signs, have been reshaped over and over again, but they tend to be mixed up in fundamental ways. Hence, in insightful essays, such as Jiˇrí Veltruský’s ‘Comparative Semiotics of Art’, it remains unclear what the ‘material’ of an art form is.12 According to Veltruský, materials can be divided into the ‘auditory and visual’; the material of music is said to be ‘tones’ and the material of literature is said to be ‘language’. Furthermore, the material of literature is supposed to oscillate ‘between materiality and immateriality’.13 Although

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 15

Media, Modalities and Modes

this categorization is fairly representative, it is not at all illuminating. The category of ‘material’ is fundamentally untenable since it includes aspects of the arts that cannot be treated as equals; tones, language and even the immaterial. ‘Tones’ must be seen as related primarily to the sensorial modality whereas ‘language’ must be understood in semiotic terms; however, language actually also consists of some sorts of ‘tones’. What the ‘immaterial’ material is, I do not know. Perhaps the most common mistake in intermedial comparisons is to confuse the notions of ‘visual’ and ‘iconic’: the visual is about using a specific sense, as will be discussed later, whereas the iconic is semiosis based on similarity (that only sometimes can be seen). I thus propose that we distinguish between the four modalities mentioned above to enable a clearer view of how media are constituted by both the physical realities and the cognitive functions of human beings. I want to stress that all media, as I understand the concept, are necessarily realized in the form of all four modalities; hence, it is not enough to consider only one or a few of them if one wants to grasp the character of a particular medium. In this respect, there is a fundamental difference between my approach and the systematic, often hierarchic, but necessarily simplistic classifications and divisions of the arts that were put forward from the eighteenth century and well into the twentieth century.14 The proposed model can be used to highlight both crucial divergences and fundamental parallels between all sorts and variants of media forms, which gives a firm ground for understanding, describing and interpreting the most elementary intermedial relations. Of course, the complexity of the innumerable intermedial relations that can be derived from the four modalities, not least from the semiotic modality, can only be hinted at. When I speak of modalities henceforth, I mean these four necessary categories in the area of the medium ranging from the material to the mental, and when I speak of modes, I mean the variants of the modalities as described below. Entities such as ‘text’, ‘music’, ‘gesture’ or ‘image’ are not seen as modalities or modes. The modalities are obviously interrelated and dependent on each other in many ways, but nevertheless they can be rather clearly separated theoretically. Also, the modes are entangled with each other in many different ways, depending on the character of the medium. Before discussing the four modalities, a preliminary distinction must be made. All media need technical media to be realized. Our knowledge of the outer world is always limited by and dependent on our senses but, unless one gives oneself up to solipsism, one must assume that all media have a material ground. The notion of a technical medium will be discussed and defined later in this essay, since a more delineated explanation of what a technical medium is requires an understanding of the four modalities; here, it must suffice to say that a technical medium is not the same as the material modality. The modes of the material modality, like the modes of the three other modalities, must be understood as latent properties of media, whereas

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

16

Lars Elleström 17

the technical medium is the actual material medium, the ‘form’, that realizes and manifests the latent properties of media, the ‘content’.

The order in which the four modalities will be presented is not arbitrary. I would not say that the order from the material, the sensorial and the spatiotemporal to the semiotic is ‘temporal’ or ‘hierarchical’ in any clear-cut way, but I do suggest that there is a point in starting with the material aspect since this is what would exist even if all living creatures were to be wiped out from the surface of our planet. The sensorial is the next stage since it is a prerequisite for the more ‘advanced’ spatiotemporal and semiotic modalities. Without sensory impressions there cannot be any conceptions of time, space or meaning. The semiotic modality is the ‘last’ modality since it can be said to include, or at least be based on, the other three. It is hence also the most complex modality. The material modality can thus be defined as the latent corporeal interface of the medium. The material interface of television programs and motion pictures, for instance, consists of a more or less flat surface of changing images (in a wide sense of the notion) combined with sound waves. The interface of most kinds of written text also consists of a flat surface, but the appearance of the surface is not changing. The interface of music and radio theatre consists of sound waves. Regular theatre, on the other hand, must be understood as a combination of several interfaces: sound waves, surfaces that are both flat and not flat and that have both a changing and static character, and also the very specific corporeal interface of human bodies. The interface of sculptures normally consists of extended, generally solid materiality. The materialities of media can differ in many ways that cannot always be clearly separated, of course, but I think it is proper to make an approximate distinction between three modes of the material modality: human bodies, other materiality of a demarcated character such as flat surfaces and three-dimensional objects, and material manifestations of a less clearly demarcated character such as sound waves and different sorts of laser or light projections. The sensorial modality is the physical and mental acts of perceiving the present interface of the medium through the sense faculties. Media cannot be realized: that is, cannot mediate, unless they are grasped by one or more of our senses. Usually, we talk about the five senses of humans, which may here be described as the five main modes of the sensorial modality: seeing, hearing, feeling, tasting and smelling. Still, the issue is, as usual, more complex. At least three levels of the sensorial must be discerned. The first level is sense-data that originate from objects, phenomena and occurrences but that can never be captured in isolation without a perceiving and interpreting agent. Often, but far from always, sense-data tend to cause inter-subjective

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

The four modalities of media

Media, Modalities and Modes

sensations. The sense-data of media come from the realized material interface. The second level of the sensorial consists of our receptors: cells that when stimulated cause nerve impulses that are transferred to a nervous system. The third level is the sensation, meaning the experienced effect of the stimulation. All our sensations consist of integrated experiences of the way a variety of receptors perceive and interpret an array of sense-data. The exact nature of sense-data and sensations, and the relation between them, is very much disputed, whereas the physical receptors have been studied and described in detail.15 Exteroceptors register changes in the external environment, interoceptors are sensible to the internal conditions and proprioceptors give us information regarding length and tension in muscle fibres and sinews. Our five senses are thus actually, to be more precise, the five sense organs that register changes in the exterior environment: eyes, ears, olfactory organ, gustatory organ and skin.16 For the moment we witness an increased interest in the interoceptors and the proprioceptors but most media are still primarily understood as exterior channels of information. Chiefly sight and hearing, the two cognitively most advanced faculties, deserve our attention in the context of media and arts, but not exclusively. Music and speech are first and foremost heard, but there is a clear physical link between exterior hearing and inner balance that cannot be ignored. A sculpture is mainly seen, but it is impossible to grasp its entity without moving and hence also involving the inner senses. Even if one does not actually touch its surface one sees and indirectly feels its tactile qualities. The reactivation of memories of sensorial experiences plays a certain part in the perception of media. Reading a text, for instance, often involves the creation and recollections of visual experiences that are very remote from the way the alphabetic letters look, and it also involves an inner hearing of the sounds of the words. New sensations are thus frequently a complex web of perceived and conceived sense-data combined with retrieved sensations. Sense-data cannot be grasped, cannot be conceived as sensation, unless they are given some sort of form, Gestalt, in the act of perception. The spatiotemporal modality of media covers the structuring of the sensorial perception of sense-data of the material interface into experiences and conceptions of space and time. Media, like all objects and phenomena, receive their multilayered spatiotemporal qualities in the act of perception and interpretation; thus, the spatiotemporal cannot be identified with the properties covered by the material modality, although there is certainly a strong link between these two modalities. I basically adhere to Kant’s idea that space and time are a priori sensory intuitions ‘that must precede all empirical intuition (i.e., the perception of actual objects)’.17 Thus, because of cognitive conditions, all media necessarily in some respect receive both spatial and temporal qualities. Furthermore, the principles of physics teach that the spatiotemporal relationship is indeed very complex: time and space interact not only on the level of perception but as physical phenomena as such, but we

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

18

do not have to bother about that when it comes to media modalities. In this context, it suffices to state that all media have aspects of the two basic modes of space and time which must theoretically be kept apart in some respects and brought together in other respects. The closer we come to the sense-data, the more time and space seem to be able to be considered separately and the more they can be said to be part of the material modality; the closer we come to the sensations, the more the very distinction between space and time loses its relevance. This critical difference is often overlooked, which has led to some confusion in the discussion of intermediality. Spatiotemporal perception can be said to consist of four dimensions; width, height, depth and time. The corporeal interface of a photograph has only two dimensions; width and height. A sculpture has three material dimensions, all of them spatial; width, height and depth. A dance has four dimensions; width, height, depth and time. Every dance performance has a beginning, an extension and an end situated in the dimension of time, while a photograph, as long as it exists, simply exists. If one closes one’s eyes in the middle of a dance performance, something is missed and the spatiotemporal form cannot be grasped in its entirety. If one closes one’s eyes while watching a photograph, nothing is missed and the spatial form remains intact. In this respect, considering the material modality through the spatiotemporal modality, there are very distinct and certainly relevant spatiotemporal differences between media. Hence, media that lack the fourth dimension, time, can be said to be static, considered as material objects: their sense-data remain the same. For media that do incorporate the dimension of time in their physical manifestation, meaning that their sense-data change, some further distinctions can be made. Motion pictures and recorded music, for instance, have fixed sequentiality. Hypertexts and much music accompanying computer games can be said to have partially fixed sequentiality. Mobile sculptures, truly improvised music and a performance broadcast live on television have (at least potentially) non-fixed sequentiality. There are certainly no definite borders between these categories, and for some media one must also consider the semiotic modality in order to understand the spatiotemporal nature of the medium. Listening to a recorded poem is like listening to recorded music: the interface of the medium must be said to have fixed sequentiality. Listening to a poem being read live is to perceive a medium hovering between the fixed sequence and the non-fixed sequence. Reading a printed poem is to perceive a medium with a clearly spatial material interface, but as soon as the conventional semiotic aspect of language is considered, the perception also incorporates temporality and fixed sequentiality (for most standard poems) or at least partly fixed sequentiality (for poems lacking clearly distinguishable lines). However, this kind of sequentiality, being attributed not to the material interface but to the realization of sequential sign systems, has a less definite character.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 19

Media, Modalities and Modes

The most basic form of spatiality is hence the manifestation of the material modality in terms of physical width, height and depth, but that is far from the whole story since our cognition to a large extent works in terms of spatiality. Also, abstract concepts and experiences of time have spatial characteristics. Thinking in terms of spatiality is a fundamental trait of the human mind that has a significant effect on the way we perceive and describe media. Experiences and interpretations of, for instance, narratives and music are also conceived of as spatial relations and patterns.18 Some such conceptions are closely connected to certain types of primarily visual sense-data. The notion of virtual space covers the effects of media that are not three-dimensionally spatial on the level of the material interface but that nevertheless receive a spatial character of depth in the perception and interpretation. Paintings and photographs actually have only two dimensions, width and height, but often, by means of resemblance of certain visual qualities in the perceived world they give the illusion of a third, depth, which creates a virtual space in the mind of the beholder. The interface of a movie, correspondingly, has three dimensions: width, height and (fixed sequential) time, but usually an illusion of depth is created. The virtual space created by a computer is undoubtedly slightly different, since we can choose to a certain extent how to move within it, but it nevertheless consists of width, height and (partly fixed sequential) time, together also creating the illusion of depth. Indeed, verbal narratives also create various sorts of virtual spatiality in the mind of the listener or reader – not only abstract, conceptual spatiality but virtual worlds within which the reader can navigate.19 Consequently, at least three levels of spatiality in media can be discerned: space as a trait of the interface of the medium (the material modality considered through the spatiotemporal modality), space as a fundamental aspect of all cognition and space as an interpretive aspect of what the medium represents (virtual space). Temporality in media can be understood in a similar way. The most fundamental form of time consists of the way the medium’s material modality is manifested through its sense-data. Some media have corporeal interfaces that are simply not temporal. Yet, it is important to note that all media are obviously realized in time: all perception and interpretation of media and what they mediate are necessarily inscribed in time, which complicates the modal relations between time and space. Also, media that are not basically temporal become situated in time as soon as they catch our attention, which of course has implications for our conception and interpretation of such media. As a counterpart to virtual space, the notion of virtual time might furthermore be introduced. Some specific media have spatial characteristics that encourage the interpretation of the spatial in terms of time passing. To some extent, there are conventions that make us look at pictures, in a comic strip for instance, in a certain temporal order. However, this is not a case of virtual time but rather an instance of pictorial sequentiality produced by

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

20

merging conventions of decoding symbolic and iconic signs. Virtual time is rather characterized by the capacity of individual pictures to depict not only one static moment but a series of occurrences.20 Interpretations of still images of what we, on iconic grounds, take to be moving objects or creatures always include an interpretation of where the object or creature was ‘before’ and ‘after’ the frozen time in the image.21 Some still images, for instance photographs, may have qualities in the image, such as objects represented with blurred contours or stretched and transparent objects, that we take to be indexical depictions of objects moving in space and time.22 These ‘illusions’ of partly fixed sequential time might be called virtual time, which is the case also for all sorts of time represented by verbal narration. In short, virtual space and virtual time can be said to be manifest in the perception and interpretation of a medium when what is taken to be the represented spatiotemporal state is not the same as the spatiotemporal state of the representing material modality considered through the spatiotemporal modality.23 Again, we have at least three levels of temporality in media: time as a trait of the interface of the medium (the material modality considered through the spatiotemporal modality), time as a necessary condition of all perception and time as an interpretive aspect of what the medium represents (virtual time).24 Consequently, there are certainly fundamental differences between media when it comes to time and space. If one does not acknowledge these differences, one cannot understand the complexity of interpreting media in terms of clashes, fusions and mutual exchanges between the categories of time and space. The difference between media with various forms of spatiotemporal interfaces is never dissolved, of course, but it is certainly crucial to note the tension created in a medium lacking, for instance, temporal qualities in the interface, and yet provoking temporal aspects in the perception and interpretation. So far nothing has been said about meaning, which I think primarily belongs to the semiotic modality. Since the world is meaningless in itself, meaning must be understood as the product of a perceiving and conceiving subject situated in social circumstances. All meaning is the result of an interpreting mind attributing significance to states of affairs, actions, occurrences and artefacts. In its widest sense, semiotics is a theoretical field aiming at understanding how the processes of signification work. For me, the most prolific endeavours of semiotics are those bordering on hermeneutics, such as the pragmatic sign discussions of Charles Sanders Peirce. Following Peirce, meaning can be described as the result of sign functions, and although there are no signs until some interpreter has attributed significance to them, one can distinguish between different sorts of signs, or sign functions. The material interfaces of media have no meaning in themselves, of course, but the process of interpretation already begins in the act of perception. Conception and cognition do not come after perception; rather, all our sensations are the results of an interpreting, meaning-seeking mind.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 21

Media, Modalities and Modes

The moment we become aware of a visual sensation, for instance, the sensation is already meaningful at a basic level. Seeing a dancer is to become aware of a visual sensation of a body being inscribed in a spatiotemporal continuum. The sensation may also include apprehended similarities with other phenomena in the world and gestures that we recognize from other performances. The dancer may wave her arms like a bird, jump like a frog and then bow. What we take to be imitations of animals may be described as iconic sign functions, whereas the bow is primarily a conventional sign denoting ‘the end’. The semiotic modality thus involves the creation of meaning in the spatiotemporally conceived medium by way of different sorts of thinking and sign interpretation. The creation of meaning already starts in the unconscious apprehension and arrangement of sense-data perceived by the receptors and it continues in the conscious act of finding relevant connections within the spatiotemporal structure of the medium and between the medium and the surrounding world. There are two different but complementary ways of thinking: on the one hand, some cognitive functions are mainly directed by propositional representations, while other cognitive functions mainly rely on pictorial representations.25 Brain research has shown that to a great extent the two ways of thinking can be located in the two cerebral hemispheres. We think both in an abstract way and in a concrete (visual and spatial) way. These interrelated but nevertheless different ways of cognition are deeply correlated, I would say, with the semiotic categories. Earlier, it was common to distinguish between conventional or arbitrary signs and natural signs. Peirce’s most important trichotomy – symbol, index and icon – has the advantage of avoiding the slightly misleading idea that some signs exist ‘in nature’, but obviously the symbol is a conventional sign, as Peirce states, and the index and the icon are in a way natural signs. The indexical sign function is based on cause and closeness, while iconicity is based on similarity: capacities that are part of the outer world as it is perceived and conceived by us.26 In semiotic terms, thinking based on propositional representations can be described as meaning created by conventional, symbolic sign functions, whereas thinking based on pictorial representations can be described as meaning created by indexical and iconic sign functions. The indexical and the iconic sign functions are deeply related to the way the mind conceives sense-data as spatiotemporal structures, which is why especially this kind of meaning is the result of interpretation also on the subliminal level. The spatiotemporal structures conceived by our mind are ‘designed’ to be meaningful – not in a propositional way, but in a pictorial way. I thus propose that convention (symbolic signs), resemblance (iconic signs) and contiguity (indexical signs) should be seen as the three main modes of the semiotic modality. According to Peirce, who stresses that the determinate aspect of all signs are ‘in the mind’ of the interpreter, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

22

three modes of signification are always mixed, but often one of them can be said to dominate.27 In most written texts, the symbolic sign functions of the letters and words dominate the signification process. Instrumental music and all kinds of visual images (for instance, drawings, figures, tables and photographs) are generally dominated by iconic signs, although photos also have an important indexical character. The iconic qualities of music and images differ, of course, since the musical signs are auditory signs that mainly refer to motions, emotions, bodily experiences and cognitive structures, while the visual signs of images largely refer to other visual entities, but all of these sign functions are based on resemblance.28 The semiotic character of all media is exceedingly complex but there is no doubt about the basic semiotic differences between, for instance, a written text and a moving image. These semiotic modes, together with the spatiotemporal, the sensorial and the material modes, form the specific character of every medium. Let us briefly and rudimentarily examine a few examples. Traditional sculpture has a three-dimensional, solid and static material interface. It is primarily perceived visually but it also has tactile qualities. Generally, the iconic sign function dominates. An animated movie consists of a fixed sequence of moving images and sounds. Its corporeal interface is a flat surface with visual qualities combined with sound waves, and the combination of twodimensional images and sound often creates an effect of virtual space. The images are first and foremost iconic and they lack the specific indexical character of images produced by ordinary movie cameras. The sound generally consists of voices, sound effects and music: the musical sounds, but often also much of the voice qualities, are very much iconic, while the parts of the voices that can be discerned as language are mainly decoded as conventional signs. Printed poetry has a solid, two-dimensional material interface, or a sequential combination of such interfaces (if realized in the technical medium of a book). It is perceived by the eyes, but also when read silently it becomes apparent that it also has latent auditory qualities in the conventional system of signification called language. Most poetry gains its meaning through these conventional signs, but there may also be substantial portions of iconicity in both the visual form of the text and the silent, inner sound experiences produced by the mind. In terms of spatiotemporality, printed poetry is essentially spatial. Very rarely, virtual space is perceived as a result of illusive depth in the two-dimensional visual appearance of the poem, whereas virtual space in the sense of illusionary worlds is often created. Printed poems that are dominated by readable words, rather than, for instance, clusters of letters, are indirectly (partly) sequential since the conventional signs (partly) determine the temporal realization of the written language. As one can see from these few examples, the modes of different media clearly differ and the modalities always interact in more or less complex

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 23

Media, Modalities and Modes

ways. Since the modalities cannot be seen as isolated entities, the proposed model offers no simple, mechanical way of checking off the modes of the modalities, one after another, but it suggests a method of investigating minutely the features of various media and how they may be interpreted. The model roughly supports ideas about media always containing other media (McLuhan29 ), or media always being mixed media (Mitchell30 ), but it also accounts for, in some detail, what it can mean to say that media are always entangled in each other, and in which respects, in fact, media are not contained by or mixed with other media: it is about media necessarily sharing the four basic modalities, but having the shifting modes of the modalities only more or less in common; it is also about seemingly having modes in common when in fact many media features come into existence on different levels, ranging from the material interface to the perception and interpretation of the medium. There are thus media similarities and media dissimilarities. All media are mixed in different ways. Every medium consists of a fusion of modes that are partly, and in different degrees of palpability, shared by other media. Every medium has the capacity of mediating only certain aspects of the total reality. Since the world, or rather our perception and conception of the world, is utterly multimodal, all media are more or less multimodal on the level of at least some of the four modalities, meaning that they in some respect include, for instance, both the visual and the auditory mode, both the iconic and the symbolic mode, or both the spatial and the temporal mode (materially or virtually). I think it is fair to say that all media are multimodal as far as the spatiotemporal and the semiotic modalities are concerned, whereas some media, such as computer games and theatre, are multimodal on the level of all four modalities.

The two qualifying aspects of media The four modalities are thus necessary aspects of all conceivable media, but it is not always sufficient to consider only the modes of the modalities to reach a proper understanding of how media are actually realized and understood. A deeper understanding of individual media realizations, their infinitely many qualities and their way of taking part in a world of constant change, requires additional perspectives. There are at least two other aspects involved in media constructions and media definitions. These aspects complement the modalities, but they are also to some extent involved in the character of the modes. I propose they be called qualifying aspects of media. The first of these two qualifying aspects is the origin, delimitation and use of media in specific historical, cultural and social circumstances. This may be called the contextual qualifying aspect. Modal combinations and blends can be performed in very many ways and often there is no manner of automatically deciding, on the basis of the modal properties, where the limits of a medium

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

24

are to be found. That can be determined only by way of investigating historically determined practices, discourses and conventions. We tend to talk about a medium as something that begins to be used in a certain way, or gains certain qualities, at a certain time and in a certain cultural and social context.31 ‘Visual art’, ‘Morse messages’, ‘sign language’ and ‘e-mail’ are not eternal media although they may be neatly described as far as the modal properties are concerned – they appear and (perhaps eventually) disappear and they are intelligible only in certain cultural and social contexts. Sometimes it is a more or less radical change on the material and technical level, such as the invention of a new printing technique or a new technological device, that triggers the genesis of what is taken to be new media. Sometimes it is rather old techniques that are seen as new media when adopted in new contexts, as when photographs are exhibited at galleries and museums or when letters are used to perform ‘mail art’. The second of the two qualifying aspects that define media includes aesthetic and communicative characteristics. This may be called the operational qualifying aspect.32 There is a strong tendency towards treating a medium as a medium, or an art form as one form of art, only when certain qualitative aspects can be identified. Such aspects are, of course, not eternally inscribed but formed by conventions.33 In fact, Lessing’s notorious assertions concerning the rigorous difference between poetry and painting are clearly normative and deal with qualifying aspects of the arts of time and space. Lessing’s claims regarding very distinct differences between the temporal art of poetry representing action and the spatial art of painting representing objects do not really concern the basic, modal aspects of media. He recognizes important semiotic differences between the arts, of course, but constantly demonstrates not least how (allegedly bad) poetry can represent objects. Poetry, however, should not be as ‘speech and its signs in general’ he claims.34 According to Lessing, then, the restrictions concerning spatiality and temporality in poetry and painting, respectively, are primarily a question of qualifying aspects. Another example of how the operational qualifying aspect works would be ‘cinema’ which, it has been argued, did not become ‘cinema’ the day the technique was invented.35 Cinema, like other new media, borrowed aesthetic and communicative characteristics belonging to old media, and although the first films also had distinct communicative and aesthetic characteristics, of course, it took a while before the many qualifying characteristics of the mediated content developed into recognizable media forms. Eventually, there came to be two notions attached to the same term: cinema as a set of techniques and cinema as a multifaceted qualified medium developed within the frames of, but not determined by, the technical aspects.36 Music, on the other hand, can be mediated by a variety of technical media, but most people would not include simply any kind of sound in the notion of music. Music, as an art form, a qualified medium, must be produced within assured

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 25

Media, Modalities and Modes

communicative circumstances and fulfil certain conventional aesthetic criteria to be accepted as music. These circumstances and criteria vary, no doubt, but if they were to be annihilated, few people would find it meaningful still to talk about the medium of music. Dance is also a qualified medium governed by aesthetic standards, yet this art form is closely related to gesture, which might be seen as a qualified medium of another kind. The primary modes involved in both dance and gesture are the body, visuality and spatiotemporality. As far as the semiotic modality is concerned, both dance and gesture include iconic and symbolic signs. I presume there is a tendency towards more indexical sign-functions in gesture but the main difference between dance and gesture is to be found in the operational qualifying aspect: dance adheres to conventions of aesthetic expression whereas gesture is primarily part of communicative situations. These two qualifying aspects often interact, of course (and I guess it would be feasible to split them into three, four or even more specific aspects). As emphasized by Jürgen E. Müller, the aesthetic and the communicative features of a medium often arise, or become gradually accepted, or disappear, at a certain moment in history and in certain socio-cultural circumstances.37 The relativity in many definitions of particular media is thus strongly related to the relativity of defining genres and subgenres of media. A genre cannot be circumscribed as an abstract entity without considering how both ‘form’ and ‘content’ are related to both aesthetic and social changes and sometimes it is an open question whether a new aesthetic or communicative practice should be called a medium or a genre.38 The two qualifying aspects thus cannot be left out when trying to delineate the contours of a medium. A painting consists of paint on a twodimensional (or weakly three-dimensional) surface that can be seen (and to a lesser degree felt and smelled). Generally, the iconic signs dominate. The iconic signs, together with conventions for representation, very often make us perceive virtual space in the depiction. In order to be counted as a painting instead of only paint spread around, however, the picture must be produced and presented within generally accepted social and artistic frames and it should have some aesthetic qualities. None of these qualifying aspects are truly stable, though. Like all art forms and other qualified media, the nature of ‘painting’ can only be circumscribed ad hoc. The modalities of the shifting notions of painting are rather stable, however, and provide a useable starting point for discussing the limits of the medium. If the material surface of an alleged painting is strongly three-dimensional, it can consequently be argued straightforwardly, on the basis of conventional genre and media borders, that it in fact should be seen as a sculpture due to its material modality. Of course, this ‘redefinition’ leaning on modality properties may have an impact on the way the painting or sculpture is conceived when taking into account the qualifying aspects of media definitions.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

26

All of the four modalities, and as a rule also the two qualifying aspects, must hence be considered when attempting to find the core of one medium or another – if there is one. However, I think there is a lot to gain in acknowledging not only the existence of modalities and qualifying aspects but also their different natures. There is no point in comparing different media if the media in question are described or defined on the basis of only a selection of modalities and qualifying aspects that are not properly related to each other. There is a point in defining ‘music’ as a medium since it can be delimited rather unambiguously by way of the four modalities and the two qualifying aspects, notwithstanding the open character of the aesthetic qualities. ‘Literature’ and ‘alphabetic text’ are not media as such though, I would say, since there is a distinct and extensive modal difference between the material, sensorial and spatiotemporal modalities of visual text and auditory text. ‘Visual text’ and ‘visual literature’ (based on printed or otherwise inscribed signs), however, might be seen as media, since they are both categories that include fairly similar medial objects (if ‘visual text’ is understood to be a written sequence of linguistic signs on a spatial surface). On the other hand, there is a difference between the media ‘visual text’ and ‘visual literature’: visual literature is heavily dependent on the two qualifying aspects while visual text is a sort of medium that can largely be defined by way of only the four modalities. Media that are mainly identified by their modal appearances I propose to call basic media. Art forms and other cultural media types always rely strongly on the two qualifying aspects and hence can be called qualified media. The distinction between basic media and qualified media is not absolute and, since the modes of the modalities are not easily isolated entities, there is no definite set of basic media, I think. However, if we define ‘text’ as any conventional sign-system, media such as ‘auditory text’, ‘tactile text’, ‘still image’, ‘moving image’, ‘iconic body performance’ and ‘organized nonverbal sound’ would be examples of what can be seen as basic media. ‘Visual text’, however, should be seen as a cluster of basic media that differ depending on whether they are produced by material signs or body movements, whether they are fixed in space or inscribed in a temporal flow and perhaps also whether they consist of singular sign units or sequences of signs.39 Apart from being defined by the two qualifying aspects, qualified media can consist of both single basic media, for instance documentary photography being based on still images, and combinations of basic media, for instance motion pictures being primarily based on moving images, auditory text and non-verbal sounds.

What is intermediality? It has been argued, for good reason, that intermediality is a result of constructed media borders being trespassed; indeed, there are no media

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 27

Media, Modalities and Modes

borders given by nature, but we need borders to talk about intermediality. Werner Wolf emphasizes that media borders are created by conventions and Christina Ljungberg stresses the performative aspect of border crossings.40 Intermediality would thus be something that sometimes ‘happens’; an effect of unconventional ways of performing medial works. Media borders are of at least two kinds, however: media differ partly because of modal dissimilarities and partly because of divergences concerning the qualifying aspects of media and the conventionality of media borders is mainly a facet of the qualifying aspects.41 Intermedial relations between basic media such as ‘moving image’ and ‘still image’ can thus be relatively clearly described within the framework of the four modalities, whereas intermedial relations between qualified media such as ‘auditory literature’ and ‘music’ to a great extent also rely on the two qualifying aspects. In the first case, the border between the two basic media of ‘moving image’ and ‘still image’ is mainly to be found in the spatiotemporal modality, since still images are spatial whereas moving images are both spatial and temporal. In the second case, the border between ‘auditory literature’ and ‘music’ is partly of a modal character, considering that all literature is primarily (but not exclusively) symbolic and music is primarily (but not exclusively) iconic, and partly of a qualified character, since the boundaries between what is counted as literature and music are also largely dependent on cultural and aesthetic conventions. A ‘normal’ reading of a poem is generally seen as literature, whereas a singing performance of the same poem counts as music – and there are many performance variants in between the literary and the musical that cannot be classified as either literature or music in a clear-cut way since there is no definite border to be crossed. Sometimes it is rather a question of whether the poem is being performed in a ‘poetry reading’ or a ‘concert’. This cultural and aesthetic ambiguity of the difference between auditory literature and music is clearly linked to the semiotic modality, however. Also, a rather neutral reading of a poem has some iconic potential, and what is taken to be the increasing ‘musicality’ of a more varied, rhythmic and melodic reading is in fact strongly linked to an increase of the iconic sign function. Both kinds of media borders, the modal and the qualified, can be crossed in two rather dissimilar ways. I think it is appropriate to distinguish between, on the one hand, combination and integration of (basic or qualified) media and, on the other hand, mediation and transformation of (basic or qualified) media.42 Theatre, for instance, normally combines and integrates, to varying degrees, basic media such as auditory text, still image and body performance. The aesthetic aspects of these combinations and integrations of basic media are part of how theatre is understood and defined as a qualified medium. Each basic medium has its own modal characteristics and when combined and integrated according to certain qualitative conventions the result is what we call ‘theatre’, consisting of different kinds of material

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

28

interfaces, appealing to both the eye and the ear, being both profoundly spatial and temporal, producing meaning by way of all kinds of signs and, certainly, being circumscribed by way of historical and cultural conventions and aesthetic standards. Theatre may thus be said to be a qualified medium that is very much multimodal and also, in a way, very much intermedial since it combines and integrates a range of basic and qualified media.43 The pop song, to take another example, is a qualified medium that includes the two basic media ‘auditory text’ and ‘organized non-verbal sound’. The consequences of the combination and integration of these two basic media are not as far-reaching as the combination of several basic media in theatre. Auditory text and organized non-verbal sound have the same material interface: sound waves that are taken in by the organs of hearing. Their way of being fundamentally temporal, but also to a certain degree spatial, is similar. The difference between auditory text and organized non-verbal sound is clearly to be found in the semiotic modality: the process of signification in auditory texts is mainly a question of decoding conventional signs, whereas the meaning of the organized non-verbal sound first and foremost is a result of interpreting the sounds in terms of resemblance and contiguity. An unqualified combination and integration of these two basic media is not enough to produce a pop song, however. Normally, both the auditory text and the non-verbal sound need to have certain qualities that confer on them not only the value of ‘lyrics’ and ‘music’ but also of ‘pop lyrics’ and ‘pop music’. The qualities of qualified media become even more qualified, so to speak, when aspects of genre are involved; a genre might therefore be called a sub-medium. Indeed, we usually deem that the lyrics produced by the singer are in themselves music, as is the sound produced by the mechanical and electronic instruments. The integration of the two basic media in a pop song is consequently in effect very deep, since the two media are more or less identical when it comes to three of the four modalities, and concerning the fourth modality, the semiotic, it is perfectly normal to integrate the symbolic and the iconic sign-processes in the interpretation of both literature and music. Texts are generally more symbolic and music is generally more iconic, but the combination and integration of words and music stimulates the interpreter to find iconic aspects in the text and to realize the conventional facets of the music. Whether it is relevant to talk about the combination and integration of media is thus a question of degree: media that share no or few modes, such as music and visual literature, can only be combined or weakly integrated,44 whereas media that have many modes in common may be deeply integrated. In fact, one may certainly say that media consisting of many different modes in a way are ‘integrated’ or even ‘mixed’ already as ‘isolated’ media, as Mitchell emphasizes.45 However, it is imperative to note that every medium is modally ‘mixed’ in a way that is more or less unique, allowing

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 29

Media, Modalities and Modes

different kinds of intermedial mixtures with other media consisting of dissimilar modal combinations.46 Similarly, whether it is reasonable to talk about the mediation or transformation of media is a question of grade. In order to understand this properly, the notion of technical medium, which has already been used tentatively, must be discussed further. I define a technical medium as any object, physical phenomenon or body that mediates, in the sense that it ‘realizes’ and ‘displays’ basic and qualified media.47 In this sense, paper is a technical medium since it can mediate written words, whereas a pen, which can only produce and not display written words, is not a technical medium. A guitar, however, which can both produce and at the same time realize musical sound, can also be said to be a technical medium if one considers especially its sound-realizing aspects.48 Basic and qualified media can exist only as ideas without technical media. A technical medium can thus be described as realizing ‘form’ while basic and qualified media are latent ‘content’. The crucial connection between the ‘form’ and the ‘content’ of media is found in the relation between the technical medium and the material modality: the material modality of a medium consists of a latent corporeal interface that can be realized in actual manifestations by technical media. Like all form–content relations, the relation between technical media and the material modality is very tight: the theoretical distinction can and must be made, but in practice the two cannot be separated. For instance, the material modality of sculpture consists of (an idea of) extended, generally solid materiality that can be realized by technical media such as bronze, stone or plaster. As an abstract notion, sculpture is not connected to specific technical media. Actual sculptures, however, are always necessarily realized by particular technical media, for instance, metal or plastics. Accordingly, when talking about media, many aspects are involved: ‘a medium’ may mean both a basic or qualified medium with latent qualities and a particular realization of a basic or qualified medium in a specific technical medium. We generally say that both ‘sculpture’ and ‘a sculpture’ are media, although it would perhaps be more lucid to say that the latter is an instance of a medium or a ‘medial configuration’ in the phrasing of Irina Rajewsky.49 Hence, intermediality is both about abstract relations between basic and qualified media and about connections between and features of specific works, performances and media products. Every technical medium can be identified according to the range of basic media it has the capacity of mediating: that is, which modal variants of the four modalities it can mediate. The defining features of a technical medium are its capacity to realize specific material interfaces and the perceiver’s capacity to interact with these interfaces and with other users of the medium, whereas the more or less hidden technical properties of the technical medium (the means of production and storage in a wide sense) are of subordinate interest as far as this proposed conception is concerned.50

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

30

Mass media should be understood as a kind of technical media that have the capacity of permitting, say, ‘simultaneous participation of many people in some significant pattern of their own corporate lives’, as McLuhan suggests.51 The television set is an illuminating case in point: it is a technical medium able to mediate a range of basic media, primarily ‘moving images’, ‘auditory texts’ and ‘organized non-verbal sound’, but also ‘still images’ and various sorts of ‘visual texts’. There is a decisive difference between television screens being able to transmit images in colour or in black-and-white only, since the latter sort limits the range of potential interfaces. In contrast, the procedural difference between analogue and digital technologies has no importance in itself when focusing on how the senses meet the material impact. The computer, another technical medium, can mediate the same basic media as the television set. Furthermore, it provides the opportunity to interact with the material interfaces and to communicate with other computer users. The orchestra is a technical medium that realizes ‘organized non-verbal sound’. The singer is a technical medium, being able to mediate both ‘auditory texts’ and ‘organized non-verbal sound’, and certainly also ‘body performance’. Stereo equipment is a rather ‘pure’ technical medium that mediates, without having the capacity to produce, ‘auditory texts’ and ‘organized non-verbal sound’. Some technical media combine the human and the non-bodily materiality: a man playing a Jew’s harp might be seen as a cyborg able to mediate unusual fusions of ‘auditory texts’ and ‘organized non-verbal sound’. Every technical medium, accordingly, can fully mediate certain basic and qualified media but only partly mediate other media. Basic and qualified media can hence be mediated more or less completely and successfully by different technical media. A theatre performance can only be realized by a combination of technical media such as, for instance, human bodies, an orchestra and properties. A television set, which mediates a feature film very well (except for the size of the screen), is only capable of partly mediating a theatre performance: the complex corporeal interface of the theatre appealing to many senses is reduced to a flat screen and a concentrated source of sounds and the true visual three-dimensional spatiality is transformed to virtual spatiality. A solo dance is mediated quite well by a television set, not very well by still photographs and only in a radically altered form by a radio – all depending on the shifting modal capacities of the technical media. Of course, qualified media can be mediated many times by a row of technical media, which might be called remediation.52 Some terms, we must remember, hover in a slightly confusing way between denoting technical and qualified media. We have already noted that ‘cinema’ did not become cinema in a qualified way the day the technology was invented. The term ‘photography’, which I recently used to denote rather vaguely a technical medium, is also the name of a qualified medium which has in fact been mediated by various technical media through history.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 31

Media, Modalities and Modes

Cameras are technical devices of production (with the capacity to register light chemically or physically) which can be said to be attached, more or less distantly, to technical media with shifting properties, for instance, silverplated sheet copper, photographic paper or a screen (a computer screen or a display on the camera itself). Certain technical media can mediate basic or qualified media that may represent other technical media, which is the case, for instance, when we see a book or a dancing body on the television screen, but I would not say that a technical medium as such can be mediated by another technical medium. The technical medium of a body can be represented on the television screen, but it is rather the qualified medium of dance that is being mediated. Similarly, seeing a representation of a book on the screen has very little in common with interacting with a real book since the technical medium book is not mediated. However, the basic media that a book can mediate – certain visual texts and still images – can also be mediated very well by the television screen. Seeing parts of a book in a television program may thus be described as seeing a representation of the technical medium ‘book’ mediating certain basic media which are actually also being mediated by the technical medium ‘television set’. To put it more straightforwardly, the technical medium ‘television set’ mediates the qualified medium ‘television program’ that represents the technical medium ‘book’ that mediates the basic medium (visual, verbal, static) ‘text’. If one brackets a few links in the chain one can also say, correctly, that the technical medium ‘television set’ mediates the basic medium (visual, verbal, static) ‘text’. It is thus important to realize that mediation and representation are closely associated and yet distinct. Mediation is a relation between technical media and basic or qualified media whereas representation (in this context) is a relation between basic or qualified media and what they signify (which may be almost anything, including technical media and other qualified media). The issue of representation thus belongs to the semiotic modality, which is only one of the many aspects of media and mediation. Sometimes, however, when the process of mediation is very smooth; that is, when the material, the sensorial and the spatiotemporal modalities do not cause any friction in the mediating procedure, representation and mediation seem to come very close: a photograph of a landscape painting is definitely a question of mediation and when asked what the photograph represents one is inclined to say ‘a landscape’ whereas it actually represents ‘a painting’: the photograph mediates a painting that represents a landscape. To be even more detailed: the technical photographic medium, for instance, photographic paper, mediates the qualified medium of photography that represents the technical medium of a coloured surface that mediates the qualified medium of, say, oil painting that represents a landscape. No doubt it is easier, and often sufficient, to simply say that the photographic image

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

32

represents a landscape, but when wrestling with intermedial issues there are no short cuts!53 The relation between technical media and basic media is thus a question of technical media being able or not being able to mediate certain modes of the modalities. Consequently, an important facet of the relation between technical media and qualified media is about technical media also being fit to realize the qualifying aspect of aesthetic and communicative characteristics. As I stated earlier, all qualified media are characterized by their origin, delimitation and use in specific historical, cultural and social circumstances (the contextual qualifying aspect). Since the existence of specific technical media is an essential facet of every historical moment and cultural space, all qualified media (qualified ideas of mediality) are more or less strongly determined by specific technical media (realizations of mediality). Some qualified media are actually fundamentally linked to irreplaceable technical media. Hence, technical media inevitably also play a crucial part in the forming of the characteristic aesthetic and communicative qualities of qualified media (the operational qualifying aspect). Oil painting can be described as a qualified medium characterized not only by certain modes but also by unique aesthetic qualities linked to the technical medium of oil colour, which was invented and developed at a certain time and in a certain cultural context. Similarly, qualified media types such as computer games are inconceivable without the resource of recently invented technical media that allow advanced interaction with the displayed interface. When the mediation of basic and qualified media through technical media is restricted by the modal capacities of the technical media, or when the technical media allow of modal expansion, that is, when the mediation brings about more or less radical modal changes, it may rather be described as transformation. A solo dance being mediated by a radio is drastically transformed by the mediation. I can think of two ways to perform such mediation: either the dance is transformed to auditory text or to organized non-verbal sound. In both cases, the human body of the dancer, which is normally considered to be a vital part of dance, is substituted by sound waves and the visual mode is substituted by the auditory mode. In the case of a dance being transformed to an auditory text, the spatiotemporal furthermore is reduced to a primarily temporal mode, and, perhaps most importantly, the iconic mode is transformed to a symbolic mode. The qualified medium photography being mediated by a book is another example of potentially radical mediation. Again, I can think of at least two ways of mediating photography by way of the technical medium of a book. If reproduced as an image, the book is an ideal technical medium for mediating a photograph, and certainly other still images as well, presenting virtually no modal limitations at all compared with how photography is usually presented. In fact, the book must be seen as one of the technical

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 33

Media, Modalities and Modes

media that originally determined the qualifying aspects of photography. If mediated as a verbal description, however, the book offers the possibility of a smooth mediation of visual texts. However, the visual text is in itself a radical transformation of the photography: the two-dimensional spatiality (involving virtual depth) of the primarily iconic and indexical still image has been transformed to two-dimensional spatiality that, because of the mainly symbolic character of the visual text, also involves the temporal aspect of fixed sequentiality. It is hence not necessarily the technical medium that ‘forces’ the transformation. Ekphrasis, for instance, is part of the general habit of transforming basic and qualified media to other basic and qualified media, which is sometimes a result of the modal constraints of technical media (as when a football match is covered on the radio) and sometimes a result of aesthetic or communicative choices to take advantage of modal possibilities (as when a verbal narrative is transformed to a symphonic poem or when a movie is transformed to a computer game). The classical ekphrasis, a poem describing a painting, is characterized by a certain kind of medial transformation: the factual space and the virtual time of the painting’s visual iconicity are being transformed to the virtual space of the poem’s sequentially arranged symbolic signs.54 Obviously, there are very many kinds of intermedial transformations.55 Sometimes these involve fairly clear and complete relations between specific art works or media productions, as when a particular newspaper article is clearly recognizable in its Internet version (although with fewer words and added animations and hyperlinks) or when a specific novel can be identified as the source of a movie (although the narrative has been abridged and visual and iconic qualities have been added). Sometimes it is rather a question of less definitive and fragmentary media traits that travel between modes and media types, as when musical form is traced in a short story or when visual traits associated with comic strips can be said to have found their way to the moving images of motion pictures. In her book Intermedialität, Irina O. Rajewsky operates, on different levels of distinction, with three notions that I find relevant when discussing media transformation: transmediality (Transmedialität, phenomena that are not media specific, such as parody), medial transposition (Medienwechsel, media transformations such as adaptation) and intermedial references (intermediale Bezüge, for instance narrativization of music or musicalization of fiction).56 These distinctions are valuable as long as one does not force them. In practice, however, it is not always clear when a medium is actually a distinct transformation of another medium, exactly when some of the indistinct media borders have been transgressed, or which traits are to be considered as belonging to the one medium or the other. Intermedial transformations can only partly be described as a fixed set of media relations. Ultimately, it is a question of hermeneutics: when finding traces of another medium, whether

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

34

it is a basic medium or a specific qualified medium, it sometimes makes sense to say that the initial medium is very much recognizable and that it has been transformed to the other medium. Sometimes, if the connection seems more fragile, one might prefer to say that the one medium simply refers to the other. The only method of deciding whether it is a case of ‘strong’ transformation or ‘weak’ reference is to interpret. Actually, it even makes sense also to talk about the transformation of fictive media. There is no fundamental difference between the descriptions of, for example, an existing photo and a non-existent photo. At times, the most important issue is not at all to determine the transformational direction and specific relation between two specific media instances but rather to compare traits, structures and forms of meaning that are to be found in many qualified media, within a specific historical context or crossing historical and social boundaries, where they ‘circulate’ without being definitely linked to the one or the other medium. Such transmedial phenomena are best captured by transmedial notions and concepts.57

What is the conclusion? The starting point of this essay was the simple idea that if the notion of medium is not specified, the notion of intermediality cannot be understood properly. Media must be understood to be both similar and different and the notion of multimodality can be used to describe in a rather strict way what the many similarities and differences are. There are four modalities that underlie all conceivable media but each modality encloses several modes that vary between media. The modes of the modalities are not always easily detectable properties; rather, they are to be found on a scale from the material to the perceptual and the conceptual. Understanding the modal qualities of media is not a question of simple observation; it also includes cognition and interpretation. The model presented for understanding intermedial relations is a bottomup model. Instead of beginning with a small selection of established media and their interrelations, which is the usual method, it starts with the modalities and modes that are shared by all media. The relations between the four modalities and the modes that are most easy to track down are illustrated in Figure 1. Apart from the modalities, two qualifying aspects must be considered in order to understand the notion of medium. The contextual qualifying aspect is the origin, delimitation and use of media in specific historical, cultural and social circumstances. The operational qualifying aspect is the aesthetic and communicative characteristics of media. What I propose to call basic media, are defined by the four modalities whereas qualified media are defined by the four modalities and the two qualifying aspects. All qualified media are based on one or more basic media.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 35

Media, Modalities and Modes

Modality

What the modality is

The most important modes of the modality

Material modality

The latent corporeal interface of the medium; where the senses meet the material impact

 human bodies  other demarcated materiality  not demarcated materiality

Sensorial modality The physical and mental acts of perceiving the interface of the medium through the sense faculties

    

Spatiotemporal modality

The structuring of the sensorial perception of the material interface into experiences and conceptions of space and time

 space manifested in the material interface  cognitive space (always present)  virtual space  time manifested in the material interface  perceptual time (always present)  virtual time

Semiotic modality

 convention (symbolic signs) The creation of meaning in the  resemblance (iconic signs) spatiotemporally conceived medium by way of different sorts  contiguity (indexical signs) of thinking and sign interpretation

seeing hearing feeling tasting smelling

Figure 1 The modalities and modes of media

Thus, intermedial relations can be found both between basic media and between qualified media. Intermedial relations consequently include both modal relations and qualified relations founded on conventions and a range of historically changing circumstances. Furthermore, intermediality is about both basic and qualified media as such and about specific works and performances. Intermedial relations have been categorized in many intricate systems but for the purpose of this essay I find it sufficient to differentiate between two main types, each holding a variety of merging variants: on the one hand, combination and integration of media and, on the other hand, mediation and transformation of media. Depending on their modal character and to a certain extent also on their qualifying properties, media can be both rather loosely combined and intimately integrated. Media consisting of many modes are in a way ‘integrated’ in themselves. Basic and qualified media must be understood as abstractions that need technical media to be materially realized. The material modality is the latent corporeal interface of a media type that can be realized in actual manifestations by technical media. The relation between technical media and basic media is consequently very much a question of technical media being able 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

36

to or not being able to mediate certain modes, and all technical media must be defined in relation to the range of basic media they have the capacity of mediating. When mediation involves more or less radical modal changes, it is feasible to say that it involves transformation. Doubtless, there are other kinds of intermedial relations that have not found their way into my model but might be compatible with it. For instance, the relations between various technical media, regardless of their mediated ‘content’, may well be described partly in terms of modal differences. Needless to say, the many abstract examples of media characteristics and intermedial relations that I have briefly discussed have forced some rather brusque simplifications, but my point has been not so much to exactly circumscribe certain media characteristics and specific intermedial relations as to provide a model and a rudimental method for such business. What is a medium? The confusion around this question, and the incompatibility of many of the suggested answers, is largely caused, I think, by the shifting approaches of different scholars and research traditions. Technical aspects as well as modal and qualifying aspects have been emphasized in diverse and often exclusive ways in the efforts to find narrow and hence efficiently operable definitions of the notion of medium.58 One alternative has been to lean on conceptions of media that are open-ended and mind-triggering but difficult to handle in the context of intermediality. The advantage of rather seeing a medium as a complex of interrelated facets – the technical, the modal and the qualifying aspects – is that such a notion sets certain limits while at the same time it incorporates most of the actual comprehensions of mediality. What is multimodality? To say that a medium is multimodal if it combines, for instance, solid materiality, visuality, spatiality and iconicity, is a truism since there simply are no media that are not being realized by at least one mode of each modality. Multimodality in a more qualified sense must hence mean that a medium includes many modes within the same modality. However, all media are at least slightly multimodal as far as the spatiotemporal and the semiotic modalities are concerned, whereas some media are multimodal on the level of all four modalities. It can thus be argued that multimodality is very much about really observing and emphasizing the very common and perfectly normal multimodal characteristics of media. Certainly, multimodality is a very general phenomenon that may also be studied outside the context of media. What is then intermediality? The many possible intermedial relations within and between media have been discussed in some detail so far and it has become clear, I think, that intermediality is a notion that cannot be understood without the notions of modality, mode and multimodality. Intermediality might be described as ‘intermodal relations in media’ or ‘media intermultimodality’. I do not expect these terrible terms to win general praise but I think there is a point in seeing intermediality as a complex set of relations between media that are always more or less multimodal.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 37

Media, Modalities and Modes

I have hesitated to describe certain kinds of media as intermedial ‘in themselves’. Many media, if not all, are indeed multimodal ‘in themselves’, but when also considering the qualifying aspects of media, things become more complex. Media characterized by strong multimodality may be said to be intermedial in themselves in the sense that certain modal ‘borders’ are crossed. However, theatre and computer games, two examples of strongly multimodal media, are conventionally understood and rather well defined as qualified media, so in that sense they are coherent media rather than examples of pronounced intermedial crossings of conventional borders, although they may be said to fuse a multitude of qualified media that also exist in their own right: music, for instance. It is thus necessary to acknowledge that qualified media are conventionally circumscribed, but one must also realize that the circumscriptions of qualified media in themselves create complex networks of conventional media borders. All qualified media overlap, and some conventional media are totally engulfed by other conventional media. Consequently, one is actually not much helped by the notion that intermediality is the crossing of conventional media borders. The point is that both multimodality and intermediality are to be found everywhere. One can thus say that everything is intermedial and multimodal, which is definitely true in a way, but that might come dangerously close to saying that nothing is intermedial or multimodal. Given the fact that qualified media are changing entities, I still find it most profitable to hold on to the idea that all ‘mediality’ involve ‘intermediality’. I do not believe that it is necessary or even possible to circumscribe a specific corpus of intermedial works or a set of fixed relations between media, although I find many of the scholarly systems of intermedial ‘works’ and ‘relations’ very valuable.59 Of course, it is essential to discuss which media, and which relations between media, might be of specific intermedial relevance, but I think it is pointless to try to establish clear borders – they are bound to be crossed. Who would, today, dream of deciding the exact delimitations of ‘art’ or even ‘visual art’ in order to fix the area of investigation for art historians? Who would want the objects of ‘popular culture’ to be narrowly defined in order to select suitable objects for, say, cultural studies to interpret? The nature of intermedial relations, as they have been described here, is thus only seemingly exact and one must realize that they can be pinned down only to a certain extent. Intermedial analysis cannot live without her twin sister intermedial interpretation. Intermediality is thus certainly about specific intermedial relations but it is also, and perhaps primarily, I would say, about studying all kinds of media with a high level of awareness of the modalities of media and the crucial modal differences and similarities of media. What makes intermedial studies important is that they offer insights into the medial nature of all media, not only a selection of peripheral media.60 The objects of intermedial studies may well be, for instance, media that have been categorized as ‘intermedial’ in themselves but they may also be what have been taken to be ‘normal’ media.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

38

Lars Elleström 39

Notes 1. G. E. Lessing (1984) Laokoön: An Essay on the Limits of Painting and Poetry, translated, with an introduction and notes, by E. A. McCormick (Baltimore MD and London: Johns Hopkins University Press). Lessing states that the poet can treat ‘two kinds of beings and actions, visible and invisible’, whereas in painting ‘everything is visible’ (pp. 66, 76). It is certainly questionable to propose that painting cannot deal with the ‘invisible’, but what is by far the most important for Lessing is to be normative rather than descriptive: the good poet should not deal with the visible unless it is inscribed in time in the form of action, whereas the good painter should not deal with action at all, but only with visible objects that are not inscribed in time. Actually, he once states that ‘signs existing in space can express only objects whose wholes or parts coexist, while signs that follow one another can express only objects whose wholes or parts are consecutive’, which sounds very definite, but his conclusion deals with the ‘true subjects’ of poetry and painting (p. 78) and in the rest of the essay he constantly refers to examples of, for instance, poetry describing static objects, which is of course fully possible but not, according to Lessing, recommendable (p. 85). A philosopher such as Susanne K. Langer is much more consistent when it comes to upholding tenable borders between the arts. See S. K. Langer (1957) ‘Deceptive Analogies: Specious and Real Relationships among the Arts’, Problems of Art: Ten Philosophical Lectures (New York: Charles Scribner’s Sons), pp. 75–89. 2. Cf., for instance, Jörg Helbig’s recent taxonomy of intra, inter, trans and multimedial relations in J. Helbig (2008) ‘Intermedialität – eine spezifische Form des Medienkontakts oder globaler Oberbegriff? Neue Überlegungen zur Systematik intersemiotischer Beziehungen’ in J. E. Müller (ed.) Media Encounters and Media Theories (Münster: Nodus Publikationen). It works quite well as a very rough model and it is representative of the interart tradition where the media are very much seen as more or less separate entities. Its value is nevertheless severely reduced because of the idea that media can be understood as ‘distinct sign systems’ (p. 83) with fixed ‘medial borders’ (p. 79), each medium having its ‘medial surface’ (p. 85). Cf. also Axel Englund’s critical discussion, in this volume, of ‘topographic’ ways of defining and delimiting media. 3. See the very comprehensive overview of the development of the research field intermedial studies in C. Clüver (2007) ‘Intermediality and Interarts Studies’ in J. Arvidson, M. Askander, J. Bruhn and H. Führer (eds) Changing Borders: Contemporary Positions in Intermediality (Lund: Intermedia Studies Press), pp. 19–37. Also Irina Rajewsky’s overview of the field, from the point of view of comparative literature and media studies, is valuable: I. O. Rajewsky (2008) ‘Intermedialität und remediation: Überlegungen zu einigen Problemfeldern der jüngeren Intermedialitätsforschung’ in J. Paech and J. Schröter (eds) Intermedialität Analog/Digital: Theorien – Methoden – Analysen (Munich: Wilhelm Fink), pp. 47–60. Many of the studies of ‘intermediality’ are, in fact, considering both the theoretical frameworks and the objects of research, rather studies of ‘interartiality’, for instance

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

The outcome of the studies does not so much depend on the objects of study as on the way the studies are performed. Some studies of pattern poems or newspapers may totally lack intermedial relevance, whereas other studies of written prose texts or photographs may be bursting with intermedial aspects.

4.

5.

6. 7.

8.

Media, Modalities and Modes T. Eichner and U. Bleckmann (eds) (1994) Intermedialität: Vom Bild zum Text (Bielefeld: Aisthesis Verlag) and W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History of Intermediality (Amsterdam and Atlanta, GA: Rodopi). Mikko Lehtonen, however, arguing eloquently for the introduction of the perspective of intermediality in cultural studies, has published an essay in a journal of media and communication studies where the notions of multimodality and intermediality are combined: M. Lehtonen (2001) ‘On No Man’s Land: Theses on Intermediality’, Nordicom Review 22, 71–83. Lehtonen is not aware of the research tradition of intermediality described by Clüver and hence he more or less has to invent the topic. Although he uses the notions in a different way from that proposed in this essay, the relation between multimodality and intermediality is accurately described: ‘multimodality always characterizes one medium at a time. Intermediality, again, is about the relationships between multimodal media’ (p. 75). Lehtonen is not the only one who has invented intermediality anew. See, for instance, L. M. Semali and A. W. Pailliotet (eds) (1999) Intermediality: The Teachers’ Handbook of Critical Media Literacy (Boulder CO and Oxford: Westview Press), where media literacy is discussed in terms of intermediality without knowledge of the existing research fields of intermediality and multimodality. See S. Moser (2007) ‘Iconicity in Multimedia Performance: Laurie Anderson’s White Lily’ in E. Tabakowska, C. Ljungberg and O. Fischer (eds) Insistent Images, in Iconicity in Language and Literature 5 (Amsterdam: John Benjamins), p. 323. In another essay, S. Moser (2007) ‘Media Modes of Poetic Reception: Reading Lyrics Versus Listening to Songs’, Poetics 35, 277–300, the author uses the terms ‘modality’ and ‘mode’ in a perfectly comprehensible but not systematic way: ‘modalities of poetic language (print/song) and corresponding modes of reception (reading/listening)’ (p. 277); ‘Songs are a multisensorial mode of linguistic communication’ (p. 278); ‘lyrics occur in different media modalities, namely oral . . . printed . . . and audiovisual’ (p. 278) and so forth. The ‘intermedia practice’ of popular songs is said to enact and embody ‘the interplay and integration of oral, literate and audiovisual modes of linguistic communication’ (p. 283). M. McLuhan (1994) Understanding Media: The Extensions of Man, Introduction by Lewis H. Lapham (Cambridge MA and London: MIT Press), p. 24. Cf. B. Granström, D. House and I. Karlsson (eds) (2002) Multimodality in Language and Speech Systems (Dordrecht, Boston MA and London: Kluwer Academic Publishers). In the ‘Introduction’ by the editors, it is stated that multimodality is, in essence, ‘the use of two or more of the five senses for the exchange of information’ (p. 1). In many of the essays in the same volume, however, modalities are also understood as gesture, speech, writing and so forth. In one of them, N. O. Bernsen (2002) ‘Multimodality in Language and Speech Systems: From Theory to Design Support Tool’, a medium is circumscribed as ‘the physical realisation of some presentation of information at the interface between human and system’, meaning in effect that media are defined by the ‘sensory modalities’ (p. 94). G. Kress and T. van Leeuwen (2001) Multimodal Discourse: The Modes and Media of Contemporary Communication (London: Hodder Arnold), pp. vii, 3, 20, 22, 25, 28, 67, 80; G. Kress and T. van Leeuwen (2006) Reading Images: The Grammar of Visual Design, 2nd edn (London and New York: Routledge), pp. 46, 113, 177, 214. In spite of claims of systematic analysis, the fundamental notion of multimodality is still circumscribed rather haphazardly by researchers following Kress and van Leeuwen, for instance J. A. Bateman (2008) Multimodality and Genre: A Foundation

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

40

for the Systemic Analysis of Multimodal Documents (Basingstoke and New York: Palgrave Macmillan), who takes modes such as text, image, diagram, the visual, the spatial and the verbal to contribute to multimodality (pp. 1, 7), although these overlapping modes are never clearly related to each other as far as semiotic, sensorial or spatiotemporal aspects are concerned. 9. In W. J. T. Mitchell (1986) Iconology: Image, Text, Ideology (Chicago and London: University of Chicago Press), his fundamental and very traditional dichotomies are text/image, verbal/pictorial and poetry/painting. This is understandable in the light of the historical tradition that he wrestles with, but these dichotomies must be overcome in order to grasp fully the similarities of media. By way of constantly reinforcing these dichotomies through the figure of paragone (the ‘battle’ of the arts), which is of course necessary to achieve the historical understanding of culturally constructed differences between poetry and painting (which seems to be Mitchell’s most important aim), he thus in a way reinstates the differences that he simultaneously deconstructs on the ahistorical level. In M. Bal (1991) Reading Rembrandt: Beyond the Word-Image Opposition (Cambridge: Cambridge University Press), the author convincingly and brilliantly demonstrates that ‘word’ and ‘image’ are interrelated and integrated in complex ways, but she continues to operate with the dichotomy ‘verbal/visual’ in spite of the modal incommensurability of the two notions (the verbal and the visual are not to be understood as media-specific characteristics, though). 10. Cf. the detailed and often enlightening comparison of literature and music in W. Wolf (1999) The Musicalization of Fiction, and the excellent comparison of prose fiction and fiction film in C. Johansson (2008) Mimetiskt syskonskap: En representationsteoretisk undersökning av relationen fiktionsprosa-fiktionsfilm [Mimetic Sisterhood: A Representation Theoretical Study of the Interrelations of Prose Fiction and Fiction Film] (Stockholm: Acta Universitatis Stockholmiensis). 11. An important early thinker who saw things more clearly than most was Moses Mendelssohn. In ‘On the Main Principles of the Fine Arts and Sciences’ [Über die Hauptgrundsätze der schönen Künste und Wissenschaften, 1757], translated by D. O. Dahlstrom, in D. O. Dahlstrom (ed.) (1997) Philosophical Writings (Cambridge: Cambridge University Press), Mendelssohn builds a typology with the aid of distinctions such as ‘natural’ versus ‘arbitrary’ signs, ‘the sense of hearing’ versus ‘the sense of sight’ and signs that are represented ‘successively’ versus ‘alongside one another’ (pp. 177–9). The typology is sketchy but instructive since Mendelssohn clearly realizes that the borders of the arts ‘often blur into one another’ (p. 181). In modern times, Wendy Steiner has provided one of the most nuanced and constructive accounts of many of the problems connected to the spatiotemporal and semiotic aspects of interart comparison where she manages to avoid most of the common pitfalls: W. Steiner (1982) The Colors of Rhetoric: Problems in the Relation between Modern Literature and Painting (Chicago and London: Chicago University Press). In W. J. T. Mitchell (1987) ‘Going Too Far with the Sister Arts’ in J. A. W. Heffernan (ed.) Space, Time, Image, Sign: Essays on Literature and the Visual Arts (New York: Peter Lang), the author discusses ‘four basic ways in which we theoretically differentiate texts from images’. Three of these ways are ‘perceptual mode (eye versus ear)’, ‘conceptual mode (space versus time)’ and ‘semiotic medium (natural versus conventional signs)’. He argues that ‘there is no essential difference between poetry and painting, no difference, that is, given for all time by the inherent natures of the media, the objects they represent or the laws of the human mind’ (pp. 2–3). Mitchell demonstrates very

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 41

12. 13. 14.

15.

16.

17.

18.

Media, Modalities and Modes well that ideological considerations often permeate much of the rhetoric concerning medial differences, and that there are important similarities, but he does not really suggest that there are no differences. Although it is important not to exaggerate the differences between media, I would say that it is fully possible ‘to give a theoretical account of these differences’ (p. 2), essential or not, which Mitchell sincerely doubts. J. Veltruský (1981) ‘Comparative Semiotics of Art’ in W. Steiner (ed.) Image and Code (Ann Arbor: Michigan Studies in the Humanities). Ibid., p. 110. See the enlightening chapter, ‘Philosophical Classifications of the Arts’ in T. Munro (1969) The Arts and Their Interrelations, revised and enlarged edition (Cleveland OH and London: Press of Case Western Reserve University), pp. 157–208. Living creatures have photosensitive, chemosensitive, mechanosensitive, electrosensitive and thermosensitive receptors. The photoreceptors of human beings are found in the eyes, while other creatures have them in the skin. Our chemoreceptors are located in the organs of taste and smell, but also in certain blood vessels. Mechanoreceptors that register changes of position and pressure are in the organs of balance and hearing, and in the skin we also have mechanoreceptors that register touch, pressure and vibrations. Similarly, electro and thermoreceptors are located at various places in the body. These sense organs do not, however, simply consist of five different kinds of receptors. Both the olfactory and the gustatory organs consist of chemoreceptors, and the skin consists of both mechanoreceptors and thermoreceptors. Moreover, the sense organs and the different kinds of receptors do not work in isolation. Strong sound, for instance, can both be heard and felt by the whole body, although human beings do not have the very sensitive external sense organ of fishes, the side line, which registers all kinds of sounds, vibrations and movements in the fish’s environment. I. Kant (1997) ‘Prolegomena to any future Metaphysics that will be able to come forward as Science’, with selections from the ‘Critique of Pure Reason’, translated and edited by G. Hatfield (Cambridge: Cambridge University Press), Prolegomena p. 35, § 10. Cf. T. Munro (1969) The Arts and Their Interrelations, pp. 362–3, 399–406. The notion of ‘spatial thinking’, inspired by R. Arnheim (1969) Visual Thinking (Berkeley and Los Angeles CA: University of California Press) is discussed in L. Elleström (2002) Divine Madness: On Interpreting Literature, Music, and the Visual Arts Ironically (Lewisburg PA and London: Bucknell University Press), pp. 184–93, 219–24 and in L. Elleström (forthcoming) ‘Iconicity as Meaning Miming Meaning, and Meaning Miming Form’ in M. Beukes, J. Conradie, O. Fischer and C. Ljungberg (eds) Signergy, in Iconicity in Language and Literature 7 (Amsterdam: John Benjamins). For spatiality in music, see R. P. Morgan (1980) ‘Musical Time/Musical Space’ in W. J. T. Mitchell (ed.) The Language of Images (Chicago and London: University of Chicago Press), pp. 259–70. For spatiality in literature, see W. J. T. Mitchell (1980) ‘Spatial Form in Literature: Toward a General Theory’ in the same volume, pp. 281–6. In this inspiring essay, Mitchell clearly sees the complexities of the spatiotemporal and he is eager to make the important point that space and time are closely interrelated. However, he makes no distinction between, for instance, ‘spatial forms’ and the ‘experience’ of spatial forms, which somewhat reduces the compass of his arguments.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

42

19. Cf. J. Frank (1991) ‘Spatial Form in Modern Literature’ [1945] in The Idea of Spatial Form (New Brunswick and London: Rutgers University Press), pp. 5–66. The notion of virtual worlds has since then been extensively explored within the field of cognitive poetics. 20. See the enlightening discussions in J. A. W. Heffernan (1987) ‘The Temporalization of Space in Wordsworth, Turner, and Constable’ in J. A. W. Heffernan (ed.) Space, Time, Image, Sign: Essays on Literature and the Visual Arts (New York: Peter Lang), pp. 64–5. 21. Lessing acknowledges that there are represented bodies in painting ‘which, by their position, permit us to conjecture an action’: G. E. Lessing (1984) Laokoön, p. 77. He actually clearly states (which seems to be at odds with his earlier one-sided assertion concerning what ‘signs existing in space can express’) that ‘painting too can imitate actions, but only by suggestion through bodies’ (p. 78). 22. Cf. E. H. Gombrich (1980) ‘Standards of Truth: The Arrested Image and the Moving Eye’ in W. J. T. Mitchell (ed.) The Language of Images (Chicago and London: University of Chicago Press), pp. 208–17. 23. Langer uses the term ‘virtual time’ to denote the aspect of time in both music and the plastic arts: Langer (1957) ‘Deceptive Analogies’, pp. 81–3. 24. Also Wendy Steiner comes to the conclusion that space and time ‘in fact relate to three very different aspects of the work’ (1982, p. 50). Their aspects are similar but not identical to the three levels that I discern. Cf. the detailed and mostly convincing discussions in J. Levinson and P. Alperson (1991) ‘What Is a Temporal Art?’ Midwest Studies in Philosophy 16, 439–50. Levinson and Alperson, too, conclude that there are three main variants of temporality in arts. 25. For references to relevant research, see Elleström (forthcoming) ‘Iconicity as Meaning Miming Meaning, and Meaning Miming Form’. 26. C. Hartshorne and P. Weiss (eds) (1960) Collected Papers of Charles Sanders Peirce Volume II, Elements of Logic (Cambridge MA and London: The Belknap Press of Harvard University Press), pp. 156–73. 27. Ibid., p. 135. 28. See Elleström (2009) ‘Iconicity as Meaning Miming Meaning, and Meaning Miming Form’. I am well aware of the lack of consensus, not least when it comes to the question of meaning in music, but my point is that no matter how you define the semiotic character of a qualified medium it must include semiotic differences that are at least partly media specific. Even if one does not accept the notion of musical iconicity one must admit that there is a fundamental difference between the way music and, say, literature produce meaning. 29. McLuhan (1994) Understanding Media, pp. 8, 305. 30. ‘all media are mixed media, combining different codes, discursive conventions, channels, sensory and cognitive modes’: W. J. T. Mitchell (1994) ‘Beyond Comparison: Picture, Text, and Method’ in Picture Theory: Essays on Verbal and Visual Representation (Chicago and London: University of Chicago Press), p. 95; ‘All media are, from the standpoint of sensory modality, “mixed media” ’ and ‘the very notion of a medium and of mediation already entails some mixture of sensory, perceptual and semiotic elements’: W. J. T. Mitchell (2005) ‘There Are no Visual Media’, Journal of Visual Culture 4, pp. 257, 260. Cf. the briefer comments in W. J. T. Mitchell (2005) What do Pictures Want? The Lives and Loves of Images (Chicago and London: Chicago University Press), pp. 215, 350.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 43

Media, Modalities and Modes

31. Joseph Garncarz rightly argues that one must see media ‘not only as textual systems, but as cultural and social institutions’: J. Garncarz (1998) ‘Vom Varieté zum Kino: Ein Plädoyer für ein erweitertes Konzept der Intermedialität’ in J. Helbig (ed.) Intermedialität: Theorie und Praxis eines interdisziplinären Forschungsgebiets (Berlin: Erich Schmidt Verlag), p. 253. 32. I refrain from trying to say exactly what ‘aesthetic and communicative characteristics’ are. Actually, any understanding of these characteristics is part of the way the operational qualifying aspect works: all changes in aesthetic conceptions, and even a denial of the point of thinking in terms of aesthetics, take part of the forming of qualified media. 33. Cf. Wolf’s definition of a medium as ‘a conventionally distinct means of communication, specified not only by particular channels (or one channel) of communication but also the use of one or more semiotic systems serving for the transmission of cultural “messages” ’: Wolf (1999) The Musicalization of Fiction, pp. 35–6. 34. Lessing (1984) Laokoön, p. 85. 35. See A. Gaudreault and P. Marion (2002) ‘The Cinema as a Model for the Genealogy of Media’, translated by Timoty Barnard, Convergence 8(4), 12–18. Cf. the case of video which was first launched as a technical medium and then eventually gave birth to a qualified medium with specific aesthetic qualities, as described in Y. Spielmann (2008) Video: The Reflexive Medium, translated by Anja Welle and Stan Jones (Cambridge MA and London: MIT Press). 36. The terminological problem is aggravated by the shifting use of similar words in different languages. Hajnal Király has suggested to me that ‘movie’ would be the closest denomination for the technical aspect and that ‘cinema’ most often rather refers to the sociological, institutional and cultural, that is the contextual qualifying aspect, while ‘film’ is associated with the aesthetically mature medium, determined by the operational qualifying aspect. 37. Discussing television, Müller demonstrates how social, cultural and historical aspects of what I would call the qualified medium of television interact with aesthetic and communicative aspects. All these qualifying aspects are developed well after the step-by-step emergence of the technological prerequisites of the medium. See J. E. Müller, this volume, and idem (2008) ‘Perspectives for an Intermedia History of the Social Functions of Television’ in J. E. Müller (ed.) Media Encounters and Media Theories (Münster: Nodus Publikationen), pp. 201–15. Cf. also J. E. Müller (2008) ‘Intermedialität und Medienhistoriographie’ in J. Paech and J. Schröter (eds) Intermedialität Analog/Digital: Theorien – Methoden – Analysen (Munich: Wilhelm Fink), pp. 31–46. 38. Cf. the genre discussion in Rajewsky, this volume. 39. If we were to assume, in a very simplified way, that the most important modes could be isolated, say (human bodies, other demarcated materiality and not demarcated materiality), (seeing, hearing and feeling), (space and time) and (symbols, icons and indices), and that these modes could be mixed following the principle that there must be at least one mode per modality, and that there can also be all sorts of combinations of modes within the same modality, the possible amount of combinations would be 7×7×3×7=1029. Needless to say, it would be ridiculous to speak of 1029, or more, types of basic media. We have to settle with the fact that some basic modal combinations are commonly distinguishable at a certain time and that the future may hold new conventions and technical solutions that make novel basic media discernible, such as a basic

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

44

40.

41. 42.

43.

44.

45. 46.

medium consisting of, say, not demarcated materiality that can be both seen and felt, that is perceived as both a spatial extension and a temporal flow producing mainly iconic meaning. Assuming that a technical medium capable of mediating such a basic medium were invented, one may expect that a range of qualified media soon would be developed forming aesthetic and communicative conventions and eventually giving rise to more or less demarcated genres and subgenres. ‘ “Intermediality” can therefore be defined as a particular relation (a relation that is “intermedial” in the narrow sense) between conventionally distinct media of expression or communication’: Wolf (1999) The Musicalization of Fiction, p. 37. See also C. Ljungberg, this volume. A similar conclusion is drawn by Irina Rajewsky in her essay in this volume. Cf. Hans Lund’s heuristic distinction between three kinds of word–picture relations: combination, integration and transformation, in H. Lund (1992) Text as Picture: Studies in the Literary Transformation of Pictures, translated by Kacke Götrick (Lewiston NY, Queenston Ontario and Lampeter UK: Edwin Mellen Press), pp. 5–9. Instead of combination and integration, Claus Clüver distinguishes between multimedia texts (separable texts), mixed-media texts (weakly integrated texts) and intermedia texts (fully integrated texts): Clüver (2007) ‘Intermediality and Interarts Studies’, p. 19. Wolf’s distinction between ‘overt’ and ‘covert’ intermediality partly corresponds to my distinction between combination and integration of media and mediation and transformation of media. However, I find it deeply problematic to state that an artefact of ‘overt’ intermediality is distinguished by qualities that are ‘immediately discernible on its surface’ and by way of limiting ‘overt’ intermediality to cases where ‘the signifiers of two media are apparent and distinct’ the category becomes very narrow: Wolf (1999) The Musicalization of Fiction, pp. 40, 50. What is, actually, the surface of a multimodal medium, and what does it mean to say that the signifiers of a medium are apparent? Only when it comes to technical media, actual physical objects or phenomena, is it possible to talk about overt or direct co-presence of media, and since most qualified media are mediated by various and also altering technical media the distinctions overt–covert and direct–indirect have limited value when it comes to qualified media. Theatre is thus definitely extremely multimodal and it integrates many basic and qualified media, but it is an overstatement to say that ‘theatre is a hypermedium that incorporates all arts and media’. See F. Chapple and C. Kattenbelt (2006) ‘Key Issues in Intermediality in Theatre and Performance’ in F. Chapple and C. Kattenbelt (eds) Intermediality in Theatre and Performance (Amsterdam and New York: Rodopi), p. 20. Cf. C. Kattenbelt (2006) ‘Theatre as the Art of the Performer and the Stage of Intermediality’, in the same volume, p. 32. Also, a simple juxtaposition involves some kind of integration. If an image and a visual text are combined, for instance, it matters which one of them is above or to the left of the other. Mitchell mainly deals with ‘verbal and visual representation’, as in Mitchell (1994) ‘Beyond Comparison’. In his later writings, Mitchell’s notion of mixed media becomes more articulated. In Mitchell (2005) ‘There are no Visual Media’, he also straightforwardly acknowledges the differences between media and states that, ‘If all media are mixed media, they are not all mixed in the same way, with the same proportions of elements’ (p. 260). This is a very important step towards the possibility

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 45

47.

48.

49. 50.

Media, Modalities and Modes of creating ‘a more nuanced taxonomy of media based in sensory and semiotic ratios’ (p. 264). However, the relation between the sensorial and the semiotic modalities (in my terminology) is not always apparent in Mitchell’s account. He argues that ‘Subtitles, intertitles, spoken and musical accompaniment made “silent” film anything but silent’ (p. 258), but to me it is not obvious in which way subtitles and intertitles break the silence. Although one may take account of the perceiver’s subvocalization, the basic difference between silent film and sound film remains clear, if not unambiguous. His assertion that photography is so riddled with language ‘that it is hard to imagine what it would mean to call it a purely visual medium’ (p. 260) implies that a particular kind of mainly symbolic semiosis affects the sensorial perception, which is obviously not the case. Symbolic language that can be directly seen on or otherwise deduced from the screen or surface of moving or static images still has its origin in the visual (and has effect only on our photoreceptors), although it is not iconic, in the semiotic sense of the term – but the modes of the semiotic modality are something other than the modes of the sensorial modality. In spite of its conventional signs, neither silent film nor photography can be heard, smelled or tasted – they can only be seen and, in a rudimentary way felt, but that has hardly any bearing on the aspects of texts and language. Mitchell’s important main point is, it seems, to emphasize the blurring of modal borders in the perception of media, but I think one must also emphasize those borders that do exist in spite of the perceptual and cognitive operations of the recipients. Cf. Müller’s distinction between ‘technical conditions’ and ‘medial products’ in J. E. Müller (1996) Intermedialität: Formen moderner kultureller Kommunikation (Münster: Nodus Publikationen), p. 23; see also pp. 81–2. As I define the notion of technical medium, it is narrower than, for instance, the notion of ‘physical media’ as circumscribed in C. Clüver (2007) ‘Intermediality and Interarts Studies’, p. 30. Devices used for the realization of media, but not tools used only for the production or storage of media, are technical media. The brush and the typewriter are tools of production that are separated from the material manifestations of media and cannot be seen as technical media according to my definition, although they count as physical media in Clüver’s sense. Oil on canvas and ink on paper, however, are technical media. The flute and the video camera are partly tools for production and partly devices for the realization of media and can hence also be seen as technical media. Some technical media, such as ink on paper, both store and display basic and qualified media, whereas a computer hard disk, a device for storage only, is not a technical medium in the sense that is emphasized here. See Rajewsky, this volume. In other contexts, of course, it is vital to consider not only the display but also the production and storage of basic and qualified media. When discussing qualified media such as art forms, for instance, many distinctive features that can be seen as operational qualifying aspects are connected to the production and storage of media. Traditional live theatre is produced and displayed by a range of technical media, the bodies of the actors being the most important, but it should not, and actually it cannot, be stored. A filmed theatre performance can be stored, but what is being stored is, as a matter of fact, not the performance, but a transformed version with very different modal and qualified qualities. A painting, on the other hand, is not produced by, for instance, oil paint and canvas, but the paint and canvas both store and display the painting. A motion picture is stored by technical

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

46

51. 52. 53.

54.

55.

56.

57.

equipment that is connected to and yet distinct from the screen that displays a copy of the film. McLuhan (1994) Understanding Media, p. 245. Cf. the much broader notion of remediation in J. D. Bolter and R. Grusin (1999) Remediation: Understanding New Media (Cambridge MA and London: MIT Press). The lack of distinction between various forms of ‘representation’, ‘remediation’ and simply ‘similarity’ is perhaps the major obstacle in Bolter and Grusin (1999) Remediation. It is a very inspiring book, full of interesting observations relevant for intermedial studies, but the authors’ notions of media and remediation are conspicuously vague. In their view, a ‘medium’ seems to be both all kinds of modalities, as understood in the tradition of Kress and van Leeuwen, and all kinds of media as (not) defined by McLuhan. According to Bolter and Grusin, all sorts of media can remediate all sorts of media, whether they are technical, qualified or something else: ‘our identity’ can be remediated by the internet (p. 231), ‘the fatal stillness of Antonioni’s films’ can be remediated by a computer game (p. 268) and ‘the printed book’ is remediated by hypertext (p. 272). Altogether, their account gives a good view of the complexity of media relations, but no theoretical tools to deal with it. Cf. J. Schröter (2008) ‘Das ur-intermediale Netzwerk und die (Neu-)Erfindung des Mediums im (digitalen) Modernismus: Ein Versuch’ in J. Paech and J. Schröter (eds) Intermedialität Analog/Digital: Theorien – Methoden – Analysen (Munich: Wilhelm Fink), pp. 579–601, whose notion of ‘Transformational-ontologische Intermedialität’ is severely limited by its close association to the notion of representation (pp. 589–90). There has been some debate concerning the proper delimitations of the notion of ekphrasis. In S. Bruhn (2000) Musical Ekphrasis: Composers Responding to Poetry and Painting (Hillsdale NY: Pendragon Press), the author demonstrates convincingly that the notion has an unexplored capacity to explain much more than literary transformations of images. Cf. S. Bruhn (ed.) (2008) Sonic Transformations of Literary Texts: From Program Music to Musical Ekphrasis (Hillsdale NY: Pendragon Press). Yvonne Spielmann discusses several ways of understanding intermedial transformation in Y. Spielmann (1998) Intermedialität: Das System Peter Greenaway (Munich: Wilhelm Fink). However, her main arguments are based on notions and distinctions that I find problematic for reasons declared earlier in this essay: ‘textuality’ versus ‘visuality’ and ‘monomediality’ versus ‘multimediality’ versus ‘intermediality’. I. O. Rajewsky (2002) Intermedialität (Tübingen and Basel: A. Francke Verlag), pp. 12–3, 16–7. Rajewsky does not consider ‘Transmedialität’ to be a case of intermediality. Cf. the notion of ‘intermedial translation’ introduced by Regina Schober in this volume. The term ‘media circuit’ is telling. It is used in M. P. Punzi (ed.) (2007) Literary Intermediality: The Transit of Literature through the Media Circuit (Bern: Peter Lang), where ‘intermediality’ mainly refers to the phenomenon here characterized as the transformation of media. Cf. the way the notion of irony is transformed in the discourses on various arts, as described in L. Elleström (2002) Divine Madness, or the way narration can be understood in the context of different media; Marie-Laure Ryan talks about ‘transmedial narratology’ in M.-L. Ryan (2004) ‘Introduction’ in M.-L. Ryan (ed.) Narrative Across Media: The Languages of Storytelling (Lincoln and London: University of Nebraska Press), p. 35. The transmedial notions of complexity, integration and rhythm are discussed as examples

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Lars Elleström 47

Media, Modalities and Modes

of ‘inter-medial factors’ in T. M. Greene (1940) The Arts and the Art of Criticism (Princeton: Princeton University Press), pp. 213–26. Ultimately, also notions such as reception and interpretation are transmedial, of course, and it is vital to recognize that interpretive strategies, contextualizations and ways to communicate the outcome of interpretation cross all media borders. This is a crucial point in Bal (1991) Reading Rembrandt. 58. This problem is emphasized in Müller (1996) Intermedialität, pp. 81–2. 59. Cf. the enlightening overview of intermedial positions and issues in I. O. Rajewsky (2005) ‘Intermediality, Intertextuality, and Remediation: A Literary Perspective on Intermediality’, Intermédialités (6), pp. 43–64. 60. This is very much stressed by Jørgen Bruhn in his contribution in this volume.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

48

Part II

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Media Borders of Qualified Media

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

This page intentionally left blank

Border Talks: The Problematic Status of Media Borders in the Current Debate about Intermediality Irina O. Rajewsky

The debate about intermediality is characterized by a variety of heterogeneous approaches, spanning a wide range of subject matter and research perspectives. A large number of critical approaches make use of the concept, each with their own premises, methodology, terminology and delimitations. Likewise, the specific objectives pursued by different disciplines in conducting intermedial research (for example, media studies, literary, theatre and film studies, art history, musicology, philosophy or sociology) vary considerably. While some approaches focus on general media-historical developments or genealogical relations between media, on medial transformation processes, on the very formation of a given medium or on the process of medialization as such, others aim at questions of media recognition (Medienerkenntnis) or at understanding general functions of media. Further approaches, mostly coming from the realm of literary studies and related fields, such as my own, emphasize various forms and functions of concrete intermedial practices in specific individual texts, films, theatre performances, paintings and so on. Considering this background, it is not surprising that the question – still declared ‘fundamental’ in 2001 – of what ‘the concept of intermediality actually means’1 now seems outmoded and has been reformulated with regard to various intermediality conceptions and their respective heuristic potential.2 While significant differences between the various research traditions become apparent as soon as one looks at them in some detail, there still seems to be a (more or less) general agreement on the definition of intermediality in its broadest sense. Generally speaking, and according to common understanding, ‘intermediality’ refers to relations between media, to medial interactions and interferences. Hence, ‘intermediality’ can be said to serve first and foremost as a flexible generic term ‘that can be applied, in a broad sense, to any phenomenon involving more than one medium’3 and thus to any phenomenon that – as indicated by the prefix inter – in some way takes 51 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

2

Media Borders of Qualified Media

place between media. Accordingly, the crossing of media borders has been defined as a founding category of intermediality.4 Obviously, such a broad definition of intermediality, which seeks to do justice to this concept precisely in its quality as collective term, ultimately tends to be rather bland.5 Nevertheless, it is worthwhile to briefly ponder this broad definition of intermediality because it steers our attention to one of the central basic assumptions of intermediality research, which is also the focus of this essay. A broad conception of this kind – and, as a consequence, any more narrowly defined conception of intermediality – evidently proceeds from the assumption of tangible borders between individual media, of medial specificities and differences. In fact, any reference to intermediality implicitly presumes that it is indeed possible to delimit individual media, since we can hardly talk about intermediality unless we can discern and apprehend distinguishable entities between which there could be some kind of interference, interaction or interplay.6 Quite recently, however, it is precisely this fundamental premise of discernible media borders that has been called into question. As a consequence, the very concept of intermediality itself has come under scrutiny. There are two principal arguments, frequently entangled, that are advanced against the aforementioned premises. Firstly, approaches to intermediality are criticized for widely ignoring the constructed character of any conception of ‘a medium’ and of any reference to ‘individual media’ (Einzelmedien) that, from the other point of view, are conceived of as purely discursive strategies.7 Accordingly, approaches to intermediality are quite frequently associated with dubious essentialist views.8 Secondly, the assumption of medial delimitations and the criterion of a medial border crossing are called into question by referring to various kinds of performances or artistic ‘events’ of the past decades that manifest ‘a still growing tendency towards an annulment, a dissolution of the boundaries between different art forms’.9 As the synopsis of the 2006 conference of the German Society for Theater Studies put it: After decades in which film and video have been seen with such frequency on stage . . . and in which the traditional borders of the arts disciplines have often been completely blurred, we have to ask whether the criterion of medial border crossing can still be so readily applied.10 Similarly, according to Erika Fischer-Lichte, the concept of intermediality quite problematically ‘presupposes that it is possible to distinguish clearly between the different media in play’.11 It is for this reason that Fischer-Lichte is reluctant to adopt approaches of this kind for the investigation of the artistic practices referred to above.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

52

I do not intend to rebut the aforementioned basic considerations with which, taken as such and detached from any judgmental conclusions, I think everybody will agree. The assumption of media borders and of medial delimitations, and (along with it) any reference to ‘individual media’ should indeed be handled with care, and it is true that quite a few approaches to intermediality, especially approaches coming from literary studies, often lack a careful discussion of media theories. At the same time, however, I would like to point out that any kind of theoretical dismantling of the term ‘intermediality’ is confronted with concrete intermedial practices in the arts for which, as I will illustrate later on, media borders and medial specificities are indeed of crucial importance. As I will show in more detail, this is not contradicted by those medial configurations which tend to show the aforementioned tendency towards the blurring or dissolving of borders between different media or art forms. Likewise, with reference to theatre, the fact that films and videos have been seen on stage for decades and that digital techniques are increasingly used hardly gives cause to renounce the basic assumption of current concepts of intermediality regarding the crossing of media borders. On the contrary, the fact that theatre is able to integrate various medial forms of articulation and to present them on stage is made possible precisely by the medial conditions and the fundamentally plurimedial structure of this medium. Despite all medial expansion, theatre is still conventionally perceived – and has been perceived for centuries – as a distinct individual medium. It thus has medially based as well as conventionally drawn borders (which are obviously subject to historic transformation and must in part be seen as fluid). This calls to our attention crucial aspects which should be considered with respect to the fundamental question of the delimitability of individual media and thus with respect to the status of medial specificities, differences and borders in the context of intermedial practices. The first aspect is the specific way in which medial differences, borders and the crossing of borders come into play in a given medial configuration. The second aspect concerns the historical processes of the development and differentiation of so-called individual media. Finally, there is the justifiably designated ‘construct’ character of media conceptions. As I have elaborated in prior publications,12 in this context we should first of all bear in mind that in dealing with medial configurations, we never encounter ‘the medium’ as such, for instance, film as medium or writing as medium, but only specific individual films, individual texts and so on. Whether, following the work of Niklas Luhmann, we distinguish between ‘medium’ and ‘form’, as proposed, for example, by Joachim Paech,13 whether we distinguish between ‘medium’ and ‘medial configuration’, as I will be doing, or whether we use other terms and concepts, we must necessarily take into account that we always only encounter concrete medial forms of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Irina O. Rajewsky 53

Media Borders of Qualified Media

articulation, which moreover are characterized by a multilayered and multimodal complex mediality. Already this very basic and simple observation leads to the conclusion that to speak of ‘a medium’ or of ‘individual media’ ultimately refers to a theoretical construct, to a ‘theoretical abstraction’ (Abstraktionsleistung), as Sybille Krämer calls it.14 Moreover, even a slightly more precise engagement with the variability of media conceptions brings to light the constructed character of each of them. As the debate stands today, it is hardly necessary to discuss the issue further. The question of how a medium should be defined and delimited from other media is of course always dependent on the historical and discursive contexts and the observing subject or system, taking into account technological change and relations between media within the overall media landscape at a given point in time. Analogous conditions apply, for example, to genre conceptions, to introduce a comparative parameter, and yet, as opposed to recent questionings of the concept of intermediality, not only is a heuristic potential still attributed to conceptions of genres as well as of genre mixes or of an undermining of generic boundaries. Rather, literary studies, for example, have sufficiently clarified that – in spite of their constructedness and historical variability – genre conventions, just like discourse traditions, play a decisive role in conveying meaning to literary texts. This holds true both for their production and for their reception. As these considerations already indicate, neither the fact that we are always dealing only with specific individual medial configurations, nor the constructedness and historicity of media conceptions, should lead us to the conclusion that we ought to cease altogether to speak of (historically transformable) medial specificities and differences, of media borders and eventually of intermedial strategies and practices. Rather, we should ask ourselves what exactly we mean when we talk about ‘individual media’, medial specificities or of crossing media borders in this context. Drawing borders of this kind clearly cannot be a matter of ‘fixed’ and ‘stable’ borders between ‘fixed’ and ‘stable’ entities, but if not this, what then? This is a question I shall investigate in more detail, starting from concrete artistic intermedial practices. For this purpose I first introduce some basic differentiations commonly used in current approaches to intermediality, albeit with different terminology and subcategories. Even if we confine ourselves to an understanding of intermediality as a critical category for the concrete analysis of individual medial configurations, as I will be doing, one still has to cope with a vast and quite heterogeneous range of subject matter. In fact, in literary studies as well as in fields such as art history, music, theatre and film studies, there is a repeated focus on an entire range of phenomena qualifying as intermedial. Examples include those phenomena which for a long time have been designated

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

54

by terms such as filmic writing, ekphrasis, musicalization of literature, as well as such phenomena as film adaptations of literary works, novelizations, visual poetry, illuminated manuscripts, Sound Art, opera, comics, multimedia shows, multimedial computer ‘texts’ or installations and so forth. All of these phenomena have to do in some way with a crossing of borders between media and are in so far characterized by a quality of intermediality in the broad sense. However, it is also immediately apparent that the intermedial quality of a film adaptation, for example, is hardly comparable – or is comparable only in the broadest sense – with the intermediality of so-called filmic writing and that both of these are quite distinct from, say, book illustrations or Sound Art installations. If the use of intermediality as a category for the description and analysis of particular phenomena is to be productive, we should therefore distinguish groups of phenomena, each of which exhibits a distinct intermedial quality and – what is even more important in the present context – a particular way of crossing media borders. This allows for drawing distinctions between individual subcategories of intermediality and for developing a uniform theory for each of them. At the same time, it shows that any reference to ‘the criterion of a medial border crossing’ blurs fundamental differences between specific ways in which such a border crossing manifests itself in given intermedial practices. Such differences have implications both for theorizing the phenomena in question as well as for the concrete analysis of given medial configurations and their overall signification of course, in the concrete analysis also differing functions of intermedial strategies have to be taken into account. A closer look at the intermedial practices in question points to three groups of phenomena:15 1. Intermediality in the narrower sense of medial transposition (Medienwechsel), also referred to as medial transformation, as, for example, film adaptations of literary texts, novelizations and so forth. 2. Intermediality in the narrower sense of media combination (Medienkombination), which includes phenomena such as opera, film, theatre, illuminated manuscripts, computer or Sound Art installations, comics, or, to use another terminology, so-called multimedia, mixed-media and intermedia forms.16 3. Intermediality in the narrower sense of intermedial references (intermediale Bezüge), for example, references in a literary text to a specific film, film genre or film qua medium (that is, so-called filmic writing), likewise references in a film to painting, or in a painting to photography and so on. A somewhat more detailed scrutiny of these three groups of phenomena reveals that we are dealing here with qualitatively different conceptions of intermediality. The first category, medial transposition, entails

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Irina O. Rajewsky 55

Media Borders of Qualified Media

a production-oriented, ‘genetic’ conception of intermediality. Here the intermedial quality – the criterion of a medial border crossing – has to do with the way in which a medial configuration comes into being, that is, with the transformation of a given medial configuration (a text, film and so on) or of its substratum into another medium. The ‘original’ text, film and so on, is the ‘source’ of the newly formed medial configuration, whose formation is based on a media-specific and obligatory intermedial transformation process. The category of medial transposition can therefore, to use Werner Wolf’s terminology, also be described as a form of ‘extracompositional intermediality’.17 Unlike medial transposition, both the second and the third category, media combination and intermedial references, aim at an intracompositional intermediality, that is, at a ‘direct or indirect participation of more than one medium’ not only in the formation process, but ‘in the signification and/or structure of a given semiotic entity’.18 Consequently, there is a fundamental difference between medial transposition on the one hand and media combination and intermedial references on the other. As Wolf goes on to say – and what is particularly relevant for analysis – ‘[e]xtracompositional intermediality as such does not necessarily affect the meaning or outer appearance of particular works or performances, while intracompositional intermediality does’.19 If we focus on media combinations and intermedial references, that is, on the different forms of intracompositional intermediality, we note a further significant difference for which once again the moment of crossing media borders becomes most important. This can already be derived from Wolf’s assertion cited above: the issue is whether we are dealing with a direct or an indirect ‘participation of more than one medium in the signification and/or structure of a given semiotic entity’. In the following I will illustrate this central point with examples from dance theatre and photorealistic painting. Film, theatre, opera or, more recently, Sound Art, are evidence that media combinations, from a historical perspective, quite frequently result in the development of new forms which somewhere in the course of this process are themselves conventionally perceived as distinct art or media genres. The plurimedial structure, then, is a characteristic and constitutive feature of these newly emerged genres. Correspondingly, dance theatre is defined by a plurimedial structure which manifests itself, not least, in the combination of theatrical and dance elements and structures. This fundamental plurimediality can, of course, be augmented, as has been shown especially in more recent productions, which increasingly tend to involve digital and other technical media. One example is Wim Vandekeybus’ production Blush (Wim Vandekeybus/Ultima Vez, Brussels 2002). This production involves pre-produced film sequences that are projected on a huge screen during the live performance. In these film sequences the ensemble’s dancers are seen swimming and acting under water. Drawing on the identity of the dancers

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

56

on stage and in the filmic underwater world, the interplay of film and live performance creates highly effective moments: repeatedly dancers seem to jump directly ‘into’ the screen and thus seemingly ‘into’ the film, where, now in their filmic embodiment, they seem to continue their movements without interruption. This effect is made possible by a projection screen that actually consists of several panels put next to one another, leaving some ‘slots’ that allow the dancers to jump behind the screen. Yet, since the live action on stage and the pre-produced action on film are exactly synchronized, the dancers effectively seem to jump into the filmic underwater world and thus into the water; an impression that is enhanced by visual and audible ‘splash’ effects occurring in the film sequences when the dancers seem to hit the water. In this way the identity of the dancers on stage and on film, as well as their seemingly continuous movements, create the illusion of a continuity of what happens on stage and on film. At the same time, two worlds, two medial ‘realities,’ two time levels and two medial forms of embodiment are set against one another. Focusing on the distinction between different categories of intermediality outlined above, it is important that in this case – an instance of media combination – the various medial forms of articulation are all present in their own materiality and contribute to the constitution and signification of the entire performance in their own media-specific way. Here, applying Wolf’s terminology, several media come into play in a direct way. Such a way of employing and staging media in dance performances is in stark contrast to a sequence from the dance theatre production Bodies (Körper) by Sasha Waltz (premiered in Berlin, 2000).20 At the beginning of this sequence, a huge picture-frame-like construction is erected on the stage, equipped with a transparent front and an opaque panel in the back. Positioned between the transparent front pane and the back panel, and supporting themselves in the air by pressing their limbs against the two supports, the dancers move very slowly, heads up and heads down, in every possible direction; as if weightless and freed from the necessity to touch ground. With several other factors contributing to its overall effect, such as the lighting, the dancers’ costumes recalling loincloths and bodies seemingly cut off at the borders of the frame, this sequence as a whole reminds the viewer of a painting, maybe even more specifically of a mannerist one. Here, different medial forms of articulation are not combined with one another, as in media combination; instead, the means and instruments of dance theatre itself – bodies, costumes, movements, lighting, stage props and so on – are employed and fashioned in a way that corresponds to and resembles elements, structures and representational practices of painting, thus creating an illusion of painterly qualities. (Put in terms of cognitive research, the spectator is cued to apply a painterly bound frame.) The evocation of painting is not achieved simply by means of subjective associations that may (or may not) be elicited in the spectator’s mind. Rather, the placement

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Irina O. Rajewsky 57

Media Borders of Qualified Media

onstage of the oversized frame – a device that is iconically related to a picture frame and that effectively frames the action taking place onstage – explicitly designates painting as the medial system being referred to and thus marks the overall mise-en-scène as an intermedial reference to painting. In this way, the sequence as a whole constitutes itself (and is received by the viewer) in relation to painting, simulating but at the same time also expanding the representational modes of the medium referred to. It is as if dance theatre turned into painting, yet explicitly pointing to a medial difference, to its own mediality and to the so-called ‘as if’ character of the whole procedure: it is as if we saw a painting that, supplementing pictorial stasis with the movements of the dancers, is not only populated by bodies in flesh and blood, but also put into motion: a tableau vivant animé. The same kind of intermedial technique underlies photorealistic painting, to provide a further example with which the basic mechanisms of intermedial references can be visualized particularly well.21 Here again, it is not two or more different forms of medial articulation that are present in their own specific materiality. Instead, what we are dealing with is nothing other than painting – but a kind of painting which inevitably evokes in the viewer the impression of a photographic quality.22 Evidently, here too another medium is ‘brought into play’, but this only in an indirect way, as in the case of the sequence of Sasha Waltz’ Bodies. It is not photography which manifests itself materially; rather painting’s own instruments and means are applied and shaped in such a way that experiences, or ‘frames’, are evoked in the observer that are medially bound to photography, leading to an illusion, an ‘as if’, of a photographic quality. Consequently, in the case of intermedial references, only one conventionally distinct medium manifests itself in its specific materiality and mediality. Historically, this medium may of course have emerged through the conflation of different medial forms of articulation and thus, as in the case of dance theatre, can exhibit a (potentially expandable) plurimedial structure. Thus, as already indicated above, the moment of crossing media borders in the case of intermedial references is brought to bear in quite another way than with media combination. A given photorealistic painting as such exhibits less a crossing than a ‘playing around’ its own medial borders in the direction of the system referred to: photorealistic painting constitutes itself in relation to photography and appears to us ‘as a photo’, but it still remains a painting. However, this effect comes about precisely due to the fact that it refers to another medial system: with the medium’s own specific means and instruments, elements and/or structures of another conventionally distinct medium are thematized, evoked or, as in the case of the sequence from Bodies and photorealistic painting, simulated23 – and this is certainly the most interesting variant of this kind of intermedial strategy. The media borders’ cross-over moment in the case of intermedial references therefore does not affect the material manifestation of various media within a given medial

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

58

Irina O. Rajewsky 59

In the sequence of Bodies and in photorealistic painting, the medial difference between the referencing medium and the medium referred to (that is, between live performance and painting, and between painting and photography, respectively), becomes apparent in quite an obvious way, as is typical of intermedial references in general. Yet, especially in the domain of so-called media combinations, we can also find concrete intermedial configurations that do in fact, quite explicitly, show the constructedness of medial delimitations and thus of any notion of an ‘individual medium’. Here, we must consider the wide range of possible realizations of media combinations, reaching from a contiguity or coexistence up to a more or less ‘genuine’ integration or interplay of the medial forms of articulation which in its ‘purest’ form would privilege none of its constitutive elements.24 Depending on the specific form of intermedial relations, such interplay may be experienced as a synthesis or fusion of different modes of medial articulation, but also as an oscillating ‘in-between-ness’, something actually situated between two or more medial forms. The latter becomes especially apparent in certain kinds of Sound Art installations, for instance, installations of the Hamburg artist Andreas Oldörp. In Oldörp’s works sounds are created in a natural, mechanical way, using either air- or steam-driven organ pipes or gas flames in glass cylinders, so-called singing flames.25 Thus, in different constellations, acoustic architectures develop that are distinguished to a great extent through their hybrid quality. First and foremost, this is achieved in that the sculptural, material constructions in the given space are neither combined with independent sounds nor – as in many other sound installations – with sounds (re)produced electronically. Rather, the ‘sculpture’ itself simultaneously functions as a sound-producing instrument. The recipient’s apprehension of sound and his or her perception of the sculptural object are in fact no longer separable from each other. The material constructions are always at the same time ‘sculpture’ and sound instrument. Hence, in such Sound Art works architectural space, material object and sound (and ultimately the peripatetic recipient him or herself), are related in such a way that the resulting effect exists and can only ever be experienced in the ongoing process of their interplay, as transitory and not repeatable. What develops, then, is less the impression of a synthesis, or ‘merging’, of the various medial aspects and qualities than the impression of an oscillating ‘between-the-media’. The movements of the visitors in the architectural space – their approaching some of the sound-producing objects while moving away from others – are continuously accompanied by changes of the sound and of the sound perception. Thus, the medial aspect of sound, its

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

configuration, but rather the specific quality of the reference itself. No matter how corresponding strategies in a given medial configuration may be functionalized in detail, in any case additional layers of meaning will be opened up in this way which must be taken into account in the analysis.

Media Borders of Qualified Media

materiality and mechanical generation, is again and again raised into the visitors’ consciousness. By the same token, the sculpture-like material constructions are held present in their visual-aesthetic dimension and in their relation to the architectural space, but they are at the same time also the source of sound. In this way, it is precisely the smooth transitions and consequently the impossibility of a clear demarcation between the media involved that become evident here. In that the visitors’ seeing and hearing habits are irritated and raised into their consciousness, in that space, sound and material constructions are related to each other in an unusual way and can be apprehended only through their interplay, any kind of ‘clear’ borders between individual media – and equally of distinct qualities in our sensual perception – is presented as being constructed (as opposed to being ‘naturally given’). To recall the arguments that have been advanced against the premise of discernible media borders, what we see here is an artistic practice that definitely manifests the above quoted ‘still growing tendency towards a dissolution of the boundaries between different art forms’ or media. In fact, media combinations expose – or at least can expose – the constructedness of delimitations of individual media. At the same time, however, it is also the oscillating interplay between two or more ‘entities’ as such which media combinations of this kind bring to the recipient’s consciousness. What I would like to emphasize here is that this oscillation per se, and any apprehension of it, necessarily presumes commonly held distinctions between the different media in play. If boundaries between different art forms are said to be dissolved, and if delimitations of media are reflected upon as constructs, what is necessarily presumed in the first place are a priori, conventional delimitations of those media or art forms. With this we arrive at the following point: concrete intermedial configurations show, for one, the constructedness of medial delimitations and (along with it) of any reference to ‘individual media’. For another, however, we may also conclude that by means of intermedial strategies, the possibility per se of delimiting different media re-enters the picture; that is, we may conclude that the ‘idea’ of one or another individual medium can be, and actually frequently is, called up in the recipient. We have seen this in an especially evident way in the case of the reference to painting in Bodies and to photography in photorealistic painting, references which in their functional potential necessarily depend on a perceptible medial difference. I am deliberately emphasizing the notion of an ‘idea’ (Vorstellung) – not least on the basis of conventional attributions – which the recipient associates with a given medium. This is because the ‘idea’ of a particular medium is relevant to the general question of how a medium can be defined and discerned from other media, of how, for instance, the ‘filmic’, ‘painterly’ or ‘musical’ quality can be defined to which a given medial configuration

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

60

could refer. It is obvious that the answer to this question necessarily depends on the historical and discursive contexts and on the observing subject or system; furthermore, we must take into account the historical and contextual variability and conventionality of any defining characteristics attributed to a given medium. Hence, if I have been speaking all along of conventional delimitations and conventionally distinct media, I have been doing this very consciously. In my view, the functioning of intermedial configurations is always based on relations between media or ‘medialities’ that are conventionally perceived as distinct,26 or, to put this in other terms, it is based on the possibility of calling up specific medially bound frames in the recipient. Despite their conventionality and constructedness, these variable conventional ‘ideas’ and concepts associated with specific individual media are nevertheless at one’s disposal, both for the production and inner functioning and the reception of a given medial configuration. Conventional and constructed as they may be, they are still available for partaking in the constitution of a media product’s overall signification. Who (at least in the Western world), to give just one more example, does not think of theatre when watching Lars von Trier’s Dogville (Denmark 2003)? And who does not, at the same time, apprehend the filmic quality of this intermedial experiment that draws, among other things, precisely on the medial difference between film and theatre? In fact, even Sound Art, by emphasizing the constructedness of medial delimitations, at the same time uncovers such delimitations as they are conventionally drawn (at least at a certain, and moreover context-related, point in time). Ultimately then, here too, there is assumed a delimitability, or better, an established, conventional delimitation of media. If the drawing of borders between media is reflected upon as a construct and finally transferred into an oscillating ‘between-the-media’, which applies at least to Oldörp’s works, this nonetheless refers back at the same time to conventionally drawn borders between the media. What is thus reflected upon, and what is in fact made useful for the potential effects of a Sound Art installation on the recipient, are prior, commonplace ideas about ‘individual media’ and medial differences, that is, specific medially-bound frames which are called up in the recipient, but which can also be modified and displaced, not least by the medial performance itself.27 Moreover, the example of Sound Art shows the important role played by processes of habitualization and conventionalization in relation to drawing medial borders: if works of art that are now labelled ‘Sound Art’ were initially seen as an attempt to embark in a new direction and to disrupt established borders (also and particularly by the artists themselves) and could thus be understood as an artistic practice taking place at the margins of current artistic and media conceptions, most recently Sound Art is perceived as an increasingly established art and media genre in its own right. While for this genre the interplay of sound and material constructions

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Irina O. Rajewsky 61

Media Borders of Qualified Media

still remains constitutive, such interplay is meanwhile hardly perceived as a destabilization or disturbance of established borderlines, conventions and norms. Rather, the combination of sound and material constructions has since become a ‘normal’, established element of a genre, which like all other genres has certain prescriptive and restrictive rules. Hence, with the conventionalization and establishment of Sound Art as an art and media genre which in itself is conventionally perceived as distinct, the original destabilizing and disturbing moment of this form of medial border crossing has been lost or, through habitualization, has at least been muffled. Thus, diachronically viewed, practices of border crossings or of dissolutions of established borders – as far as they are accompanied by a sustainable conventionalization and habitualization – may result in other constructions, other borders that again are perceived as conventional, and in turn modified or even entirely new conceptions of individual media and art forms. Up to this point, I have been mainly focusing on the constructed character of any conception of ‘a medium’ and on the fact that when we talk about ‘individual media’ we are actually talking about media that are conventionally perceived as distinct. These considerations can be taken one step further by highlighting an aspect that is usually overlooked, or at least not made explicit, in related arguments. Here again, intermedial references come to the fore, such as the reference to painting in Sasha Waltz’ Bodies or the reference to photography in photorealistic painting. The above remarks on these examples concentrated on differences between instances of media combination and intermedial references. Yet, those remarks actually not only delineate the features of intermedial references as opposed to practices of media combination, but also as opposed to intramedial references, as for instance intertextual relations, or, say, film–film and music–music relations. In this context the criterion of a medial border crossing is again of primary importance. In the case of intramedial references the referencing itself remains within one medium and consequently does not involve any kind of medial difference. Hence, intramedial references, quite significantly, do not come along with a medial border crossing. Instead, in the case of intermedial references a medial difference does come into play; and more precisely, a medial difference that – as a matter of fact – cannot be effaced. What can be achieved by intermedial references is an (more or less pronounced, yet necessarily asymptotical) approximation to the medium referred to; an overall actualization or realization of the other medial system is impossible. Due to its material and medial conditions, dance theatre cannot truly become painting – just as painting itself can never become genuinely photographic, even though this is suggested at times by photorealistic painting. Here medial specificities and borders emerge, which make clear that certain basic medial constraints must be considered. In spite of all the constructedness and conventionality of the derived medial conceptions, these basic medial constraints cannot be neglected.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

62

At the same time such medial constraints point to the fundamentally different status of medial restrictions and possibilities on the one hand and generic restrictions and possibilities on the other. Thus, media specificities can be differentiated from genre specificities. While genre specificities and the prescriptive and restrictive rules of a genre are based solely on conventions that can be played with, undermined and transcended without any problems, medial specificities entail material and operative restrictions that can be played with, but cannot be undermined with the use of the respective media-specific means and instruments. To stress this again, painting cannot become genuinely photographic, just as literary texts cannot really become filmic or musical. What can be achieved in this respect is only an illusion, an ‘as if’ of the other medium. Consequently, if I have been emphasizing the constructed character of any conception of ‘a medium’, this constructedness itself is actually confronted with some limits; limits that are inherent to the materiality and mediality of any form of medial articulation. Accordingly, my above remarks on the potential capacity of intermedial practices to alter established prior ‘ideas’ of a given medium need to be specified: Intermedial strategies, if deployed in the appropriate manner, can indeed be the means through which commonplace ideas of a given medium as well as conventionally drawn borders between, and delimitations of, different media can be apprehended, critically reflected upon, and even displaced and undermined – yet always only in keeping with the basic material conditions of the medial configurations in question. In contrast to genres, conceptions of (individual) media are not based solely on conventions. Rather, they are additionally based on given material and operative conditions, which to a large extent are subject to historical, often technological, change, yet to some extent are also transhistorically valid. In fact, we note a dependency between genre conventions and their respective medial disposition: The emergence of certain genre conventions is at least also bound to technical requirements and thus to medially determined limits and possibilities underlying the respective genre.28 Putting medial boundaries, differences and differentiations at the centre of interest as I have been doing may be conceived as ‘strange’ or somehow démodé, as there seems to be a general tendency in recent research to discard thinking in differences as ontological or essentialist and thus outdated. Currently, efforts are being made to strengthen common and crossover features – not only in intermediality studies, but also, for instance, in the young field of so-called transgeneric and transmedial narratology.29 Contrary to this tendency, I have advanced the thesis that medial differences and the notion of media borders play a crucial and extremely productive role in the context of intermedial practices. In this context it should be emphasized again that the premise of discernible media borders and of the possibility of distinguishing

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Irina O. Rajewsky 63

Media Borders of Qualified Media

between individual media, ultimately inherent to any conception of intermediality, is not challenged even by such artistic or cultural practices that show the tendency to blur, or to entirely dissolve and overcome, established borders and delimitations of different art forms and media. Rather, the respective practices constitute themselves – either intentionally or, so to speak, necessarily – in relation to just these established borders and delimitations. In other words, they necessarily constitute themselves in relation to, and within the scope of, the overall medial and discursive landscape at a given point in time, including the respective delimitations of conventionally distinct art forms and media. Thus, the ‘construct’ character and the historical variability of media conceptions are evoked, aspects which intermediality research must of course take into account. This is exactly why I have been speaking of individual media that (at a given point in time) are conventionally perceived as distinct, and of medially bound frames. These frames – if marked in an appropriate manner – can be called up in the recipient and as such are available for partaking in the constitution of a medial configuration’s overall signification. At the same time, however, we must also take into account that any artistic or, more generally speaking, any cultural practice depends on, and is determined by, its very mediality and materiality, rendering the constructedness of media conceptions and especially the constructed character of medial delimitations at least in part relative. Borders between different medial forms of articulation cannot ‘be drawn differently’ in every respect 30 and it is especially for this reason that media conceptions, as constructed and conventional as they may be, still can be assigned a different status from genre conceptions. Considering the initial question and main thesis of this essay, what is most important here is that, independently of which particular kind of intermedial practices we are dealing with, the effect potential of intermedial practices is always in some way based on medial borders and differences. Hence, in contrast to recent challenges concerning the criterion of a medial border crossing, on the basis of concrete intermedial practices, and thus starting from the objects of investigation as such, it is precisely the concept of the border which can be strengthened. In my view, the concept of a border is the precondition for techniques of crossing or challenging, dissolving or emphasizing medial boundaries, which can consequently be experienced and reflected on as constructs and conventions. It is only due to our constructing borders in the first place that we are able to become aware of ways of transcending or subverting those very boundaries or of ways of highlighting their presence, of probing them, or even of dissolving them entirely. At the same time, it is precisely these acts of transcending, subverting, probing or highlighting which draw attention to the conventionality and (relative) constructedness of these boundaries. My thesis thus encompasses the idea of fostering a process of rethinking the notion of boundaries: it should be

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

64

Irina O. Rajewsky 65

Notes 1. B. Ochsner and C. Grivel (2001) ‘Einleitung’ in B. Ochsner and C. Grivel (eds) Intermediale: Kommunikative Konstellationen zwischen Medien (Tübingen: Stauffenburg), p. 4; my emphasis. 2. Cf. in more detail I. Rajewsky (2008) ‘Intermedialität und remediation: Überlegungen zu einigen Problemfeldern der jüngeren Intermedialitätsforschung’ in J. Paech and J. Schröter (eds.) Intermedialität analog/digital: Theorien, Methoden, Analysen (München: Fink). 3. W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History of Intermediality (Amsterdam and Atlanta GA: Rodopi), pp. 40–1. 4. Cf. I. Rajewsky (2002) Intermedialität (Tübingen and Basel: Francke), pp. 11–15. 5. A broad intermediality concept of this kind allows for making fundamental distinctions, namely between intra, inter and (ultimately) transmedial phenomena, at the same time representing a transmedially useful category. Yet such a broad concept does not permit us to derive a single theory that would uniformly apply to the entire, heterogeneous subject matter covered by all the different conceptions of intermediality, nor does it help us to characterize more precisely any one individual phenomenon on its own distinct formal terms. Accordingly, in order to cover and to uniformly theorize specific intermedial manifestations, more narrowly conceived (and often mutually contradictory) conceptions of intermediality have been introduced, each of them with its own explicit or implicit premises, methods, interests and terminologies (cf. in more detail I. Rajewsky (2005) ‘Intermediality, Intertextuality, and Remediation: A Literary Perspective on Intermediality’, Intermédialités/Intermedialities 6, pp. 43–65). 6. The same is applicable for any reference to ‘interart relations’, as talking about interart relations likewise presupposes discernible borders between different art forms. 7. This was, for instance, one of the theses of the final panel discussion of the 8th Biennial International Conference of the German Society for Theater Studies with a special focus on ‘Theater & the Media’, held at Erlangen, 12–15 October, 2006; cf. H. Schoenmakers et al., (eds) (2008) Theater und Medien/Theater and the Media: Grundlagen – Analysen – Perspektiven. Eine Bestandsaufnahme (Bielefeld: transcript), p. 26 and pp. 545–60. A similar thesis has been advanced by Voßkamp and Weingart: ‘Not only within the traditional text-image genres (emblems, advertising, press photography, comics, etc.), but also when it is presumably “only” a matter of images or texts, monomedial assumptions about the picturality of the picture or about the textuality of the text can ultimately be seen as untenable essentialisms. In the sense of W. J. T. Mitchell’s assertion that “all media are mixed media” it is assumed here that medial purity precepts should themselves be understood as discursive effects and thus, not least, as the result of procedures of power, of inclusion and exclusion’: (2005) Transkriptionen 5, p. 31. See also W. Voßkamp and B. Weingart (2005) ‘Sichtbares und Sagbares: Text-Bild-Verhältnisse’ in W. Voßkamp and B. Weingart

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

shifted from taxonomies to the dynamic and creative potential of the border itself. The borders or – perhaps better – ‘border zones’ between media can thus be understood as enabling structures, as spaces in which we can test and experiment with a plethora of different strategies.

8. 9.

10. 11. 12.

13.

14. 15.

16. 17. 18. 19. 20.

Media Borders of Qualified Media (eds) Sichtbares und Sagbares: Text-Bild-Verhältnisse (Cologne: DuMont), pp. 9–10. Also compare in this context Mitchell’s own remarks on this subject: ‘[T]he interaction of pictures and texts is constitutive of representation as such: all media are mixed media, and all representations are heterogeneous; there are no “purely” visual or verbal arts, though the impulse to purify media is one of the central utopian gestures of modernism’: W. J. T. Mitchell (1994) Picture Theory: Essays on Verbal and Visual Representation (Chicago and London: The University of Chicago Press), p. 5. As in the following I will continue to talk about ‘individual media’, what should be emphasized here is that, in my own understanding, the term ‘individual medium’ (Einzelmedium) does not per se point to any kind of ‘monomediality’ or medial ‘purity’. Instead, in my view, what is at issue here are media that are conventionally perceived as distinct from other media (cf. in more detail below). Hence, so-called individual media can indeed be characterized by a plurimedial structure as, for instance, film or theatre. Moreover, individual media – and this is also true of media often termed ‘monomedial’, such as (literary) texts – are always to be conceived of as multimodal (cf. Lars Elleström’s contribution to the present volume). In fact, Mitchell’s famous dictum might be reformulated into a slightly smoother ‘all media are multimodal (media)’. Significantly, Mitchell himself has recently rephrased his dictum, specifying that ‘[a]ll media are, from the standpoint of sensory modality, mixed media’ (W. J. T. Mitchell (2007) ‘There Are No Visual Media’ in O. Grau (ed.) MediaArtHistories (Cambridge and London: MIT Press), p. 395; my emphasis). Cf. note 7. Quoted from the grant for the International Research Training Group InterArt/ Interart Studies, an international graduate school that has been established at the Freie Universität Berlin in October 2006 (spokesperson: Erika Fischer-Lichte). Quoted from the synopsis of the conference, cf. http://www.theatermedien.de/kongress/sektionen.html, date accessed 4 May 2009. As note 9. Cf. especially I. Rajewsky (2004) ‘Intermedialität ‘light’? Intermediale Bezüge und die ‘bloße Thematisierung’ des Altermedialen’ in R. Lüdeke and E. Greber (eds) Intermedium Literatur: Beiträge zu einer Medientheorie der Literaturwissenschaften (Göttingen: Wallstein). Cf. J. Paech (2002) ‘Intermediale Figuration – am Beispiel von Jean-Luc Godards Histoire(s) du Cinéma’ in A. J. Lehmann and I. Maassen (eds) Mediale Performanzen: Historische Konzepte und Perspektiven (Freiburg i. B.: Rombach). Cf. S. Krämer (2004) ‘Kulturanthropologie der Medien: Thesen zur Einführung’, Paragrana: Internationale Zeitschrift für historische Anthropologie 13, pp. 130–3. Cf. in more detail Rajewsky (2002), (2004) and (2005). With respect to this tripartite division, it is important to note that a single medial configuration may certainly fulfil the criteria of two or even of all three of the intermedial categories outlined below. Cf. below, note 24. W. Wolf (2005) ‘Intermediality’ in D. Herman and M.-L. Ryan (eds) The Routledge Encyclopedia of Narrative Theory (London: Routledge), p. 253, cf. also p. 254. Wolf (2005) ‘Intermediality’, p. 253. Ibid., p. 254. For a more detailed discussion (including illustrations) cf. Rajewsky (2002) and (2005).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

66

21. With respect to ‘photorealism’ and related concepts cf., for example, C. Lindey (1980) Superrealist Painting & Sculpture (New York: Morrow); L. K. Meisel (1980) Photorealism (New York: Abrams). 22. The depictions of streets and buildings by the American painter Richard Estes serve as a paradigmatic example of this (see, for instance, Café Express, 1975, oil on canvas). 23. My use of the term ‘simulation’ is not intended in the sense the word is used in media studies (that is, to designate mathematical simulation processes); rather, it connotes a simulation in the literal sense of the word. 24. Cf. also Wolf (1999) The Musicalization of Fiction, pp. 40–1. At the extreme outer pole of this subcategory of intermediality are phenomena which, taking recourse to another terminology, can also be designated as intermedia configurations. The term ‘intermedia’ was first brought into play by Dick Higgins’ 1966 pioneering essay ‘Intermedia’ (Something Else Newsletter 1, 1 (1966), reprinted in D. Higgins (1984) Horizons: The Poetics and Theory of the Intermedia (Carbondale: Southern Illinois UP)), in which Higgins expresses his conviction that ‘much of the best work being produced today seems to fall between media’ (p. 18). This understanding of the term has become relevant for attempts to delimit so-called intermedia configurations from mixed-media and multimedia configurations. Higgins uses ‘intermedia’ to refer to works ‘in which the materials of various more established art forms are “conceptually fused” rather than merely juxtaposed’ (E. Vos (1997) ‘The Eternal Network: Mail Art, Intermedia Semiotics, Interarts Studies’ in U.-B. Lagerroth, H. Lund and E. Hedling (eds) Interart Poetics: Essays on the Interrelations of the Arts and Media (Amsterdam and Atlanta, GA: Rodopi), p. 325); the quality of medial juxtaposition is ascribed (with certain fine distinctions) to mixed-media and multimedia configurations (see C. Clüver (2001) ‘Inter textus/ inter artes/inter media’, Komparatistik: Jahrbuch der Deutschen Gesellschaft für Allgemeine und Vergleichende Literaturwissenschaft (2000/2001), pp. 14–50). Hence, as can be seen in certain forms of visual poetry or corporate logos, in intermedia configurations the materials of various individual media become inextricably bound to, or even ‘merged with’, one another (cf. ibid., p. 36). 25. For a more detailed discussion (including illustrations) cf. Rajewsky (2002), pp. 20–2, 164–6. 26. Cf. Wolf’s definition of ‘medium’ in Wolf (1999) The Musicalization of Fiction, p. 40; see also W. Wolf (2002) ‘Intermedialität: Ein weites Feld und eine Herausforderung für die Literaturwissenschaft’ in H. Foltinek and C. Leitgeb (eds) Lite-ra-tur-wis-sen-schaft intermedial – interdisziplinär (Vienna: Verlag der Österreichischen Akademie der Wissenschaften), p. 165. 27. Here a performative potential of intermedial strategies becomes apparent (cf. also below). 28. In narrative film, for example, the formation of certain narrative conventions, such as specific ways of introducing flashbacks, can at least also be traced back to the fact that film, as far as the image track is concerned, is medially restricted to present-tense narration. Hence, to visually convey analepses, film narration must necessarily rely on certain filmic codes and conventions. 29. Cf. in more detail I. Rajewsky (2007) ‘Von Erzählern, die (nichts) vermitteln: Überlegungen zu grundlegenden Annahmen der Dramentheorie im Kontext einer transmedialen Narratologie’, Zeitschrift für Französische Sprache und Literatur 117, pp. 25–68.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Irina O. Rajewsky 67

Media Borders of Qualified Media

30. Compare in this connection general considerations on the constructed character of border drawings, discussed particularly in the 1990s with special attention to the observing subject or system. Referring to the concept of hybridity, Irmela Schneider has pointed out that to the extent that differences are conceived as observer-dependent differentiations, these are at the same time recognized as modifiable (cf. I. Schneider (1997) ‘Von der Vielsprachigkeit zur “Kunst der Hybridation”: Diskurse des Hybriden’ in I. Schneider and C. W. Thomsen (eds) Hybridkultur: Medien, Netze, Künste (Cologne: Wienand)). Accordingly, differences and borders are not seen and discussed as ‘natural’ but as set or drawn borders, which could also be set and drawn differently.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

68

3 Axel Englund

As the title of this volume illustrates, we often tend to think of arts and media in terms of geographic areas delineated by borders, and consequently of interartial and intermedial studies as a kind of topographical description, a charting of territories and their positions in relation to each other. This conception is often a helpful way of imagining the arts and media, and one which is deeply connected to the way they have been functioning for the last couple of hundred years or so. My point in this essay is not to argue that the topographical model ought to be discarded, but to contrast it with a different model, namely the conceptualization of intermedial relations in terms of metaphoricity.1 My examples will concern the interplay between music and literature, specifically works of Western art music (Ravel and Mozart) and poetry (Bertrand and Celan) that bring the ‘other’ medium into play. Most modern theories conceive of metaphoricity not in terms of substitution, but as a tension between separate elements that cannot be paraphrased without losing its essential meaning.2 In other words, metaphor has a cognitive import in its own right, and what it communicates cannot be exhaustively recast in any literal wording. In this context I will be making use of the conception of metaphor advocated by Max Black, which he calls the ‘interaction view’ of metaphor.3 In short, a metaphor consists of two separate subjects – Black labels them principal subject and subsidiary subject – interacting with each other. Each subject has a system of associated implications (which might be roughly described as a list of ideas commonly associated with the subject) and when placed together in a metaphor, the implications of the primary subject are affected so that analogies with subsidiary subject are foregrounded. As a result, our perception of the primary subject is altered by the metaphorical interaction. As the word ‘interaction’ implies, however, this is not the whole truth: our perception of the secondary subject is altered as well, since those implications that are analogous to implications of the primary subject are foregrounded by the metaphor. In short, the metaphor organizes our conception of its two constituent subjects. My discussion of particular examples departs from the idea that the 69 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Intermedial Topography and Metaphorical Interaction

Media Borders of Qualified Media

‘music’ part and the ‘literature’ part of many musico-literary artefacts can be interpreted as the constituent subjects of a metaphor. The emphasis on the interpretative activity is important: what we are dealing with is less an inherent quality exhibited by intermedial works than a fertile strategy for approaching these works. At the same time, as has been suggested by theoretical concepts as different as Gadamer’s Wirkungsgeschichte and Derrida’s dissemination, a definitive distinction between text and interpretation, between internal and external, is neither possible nor desirable. What implications does Black’s theory have when applied to intermedial relations? For one thing, his conception of metaphor suggests that the subjects involved in the interaction are best regarded not as ‘things’ but as ‘systems of things’,4 or, even better, each subject could be conceived of as a ‘system of ideas’.5 Such a system, he stresses, is ‘not sharply delineated, and yet sufficiently definite to admit of detailed enumeration’.6 Hence, if the interacting subjects are music and literature, these are to be thought of as systems of ideas rather than one sharply delineated ‘thing’ with a definable essence. On a general level, these ideas could be the physical form of presentation (literature is stereotypically presented as a book, music as performed on musical instruments), the semiotic mode (literature stereotypically employs a symbolic mode of reference, music stereotypically consists of abstract sound or, in some cases, sound with iconic reference) or the sensory channels through which they reach the percipient (a text is stereotypically read, a musical piece heard). These notions, and many others, are all part of the system of ideas connected to the words ‘music’ and ‘literature’. This way of considering music and literature is fully analogous to definitions of media not according to a single, palpable essence, but rather as the configuration of a number of distinguishable modalities.7 It is not, however, entirely compatible with the topographical model, since it tells us that there are indeed no distinct borders. Any alteration of the configuration of modalities would entail a slight movement on the map, and it would be impossible to pinpoint the place in which one medial territory ends and another starts. Most epochs, particularly modernity and post-modernity, have seen a number of works that do not allow themselves readily to be placed on the map in its contemporary state – liminal cases that, in terms of the topographical model, would have to be regarded as a no man’s land or disputed zones. Much like nations, then, the systems of ideas that constitute arts and media are historically contingent and if one wishes to speak about their borders, it needs to be kept in mind that they are subject to gradual but constant change. In Black’s view, we understand a metaphor by searching for analogies between the ideas associated with the two subjects. The ideas, according to Black, may be either commonplaces about the subject or ideas ‘established ad hoc by the writer’.8 This double-edgedness is an important part of applying the metaphorical perspective to musico-literary artefacts. Any musico-literary metaphor would activate the system of ideas that make up

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

70

our conception of ‘music’ and ‘literature’, provoking us to find analogies between them. These are the kind of general ideas mentioned in the preceding section. However, such general ideas are not the only ones put into play by the interaction, unless the metaphor in question is simply the sentence ‘literature is music’. In addition, each work incorporating music and literature also contains its own unique configuration of material ideas, by which I mean its particular words, sentences, notes, chords, rhythms, large-scale structures – in short, all that constitutes the physiognomic identity of the given work. The ‘system of ideas’ associated with principal and subsidiary subject respectively thus has a very wide spectrum, incorporating everything from concrete material shapes to semantic connotations. Since all such aspects are potentially essential parts of the artwork – a phenomenon akin to what has been termed ‘repleteness’ by Goodman9 – they may consequently be activated and reorganized by the metaphorical interplay. Furthermore, much as there are no watertight seals between langue and parole, there are none between these medial-general and work-specific ideas. When a work is understood in terms of a metaphorical interaction between music and literature, both categories, as they are configured in the particular artefact, partake of the reorganization of the systems of ideas, altering our conception of categories as such. As Irina Rajewsky stresses, we never encounter the abstract notion ‘the medium’ as such, but always as articulated in an individual work.10 If metaphorical interaction is based on a tension between the principal and subsidiary subject, it also implies a tension between our preconceptions of the subjects involved and the way in which they actually appear in the present context, the former demanding continual revision when faced with the latter. The far-reaching consequence of this perspective is that each musico-literary work can be read as a metamedial utterance with the inherent potential of altering the system of ideas known as ‘music’ and ‘literature’, thus redrawing the borders of medial territories. The way in which the ‘music-part’ and the ‘literature-part’ relate in a given work has the power to change our conception of the way ‘music’ and ‘literature’ relate in general. Just as a metaphor changes our conception of the world, the metaphorical dynamic of musico-literary artefacts changes our conception of media. From this perspective, the topographical model becomes untenable: a work of verbal or musical art is not an object located in one territory or another, but is a shaping force in our conception of these ‘territories’. Two further comments should be made here regarding the concept ‘metaphor’, namely regarding the criteria of two ‘systems of ideas’ such as ‘music’ and ‘literature’ being read as interacting according to the rules of metaphor. First, metaphor is commonly thought of as a trope based on similarity. It is also, however, fundamentally dependent on difference. If love is said to be ‘a red, red rose’, we are dealing with metaphor. Where there is no difference and no term is alien to the other, there is no metaphor,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Axel Englund 71

Media Borders of Qualified Media

but the pointless statement that ‘a rose is a flower’, or even a figural shortcircuit such as Gertrude Stein’s ‘a rose is a rose is a rose is a rose’. Now, if metaphoricity presupposes difference, the relation between music and literature can never be described as metaphorical as long as the arts are not conceived as different and distinguishable from each other. For instance, the use of the Greek term mousike techne, subsuming dance and drama as well as music and poetry, renders impossible any notion of metaphorical interaction between the arts – they were not thought of as separate at all and cannot constitute the ‘distinct subjects’ described by Black. One step in this historical development was taken when poetry began to be written to be read silently rather than performed in song. A definite separation did not come about until the mid eighteenth century, when aesthetics became a philosophical field and instrumental music gradually began to be conceived of as a genre of its own, distinct from language.11 The difference needed to create a metaphorical tension between the concept of ‘music’ and the concept of ‘literature’ was thus guaranteed: the arts became each other’s Other, which opened up the possibility of cross-illumination. As is well known, this possibility was not left unexploited: especially in Germany, the advent of Romanticism saw a proliferation of the use of the other art as a model for that which music or literature should aspire to. If the notion of literature and music as independent and fundamentally different arts was necessary for a metaphorical interaction to take place between them, it stands to reason that it also promoted the topographical model, which to a certain degree counteracts the metaphorical perspective: differences easily came to be thought of as borders delineating territories. If difference is one prerequisite of the fecundity of the metaphorical perspective, the pretension to identity is the other, in the textbook example of metaphor usually brought about by the use of the copula ‘be’. Max Black exemplifies his interaction view with the sentence ‘man is wolf’, which in essence consists of nothing but a principal and a subsidiary subject and the verb suggesting an identity between them. This ‘is’ need not be explicit; it can equally well be implied by a genitive (‘the chess game of the cold war’) or predicate (‘the chairman ploughed through the discussion’). Intermedial works that somehow suggest an identity between music and literature seem to encourage a metaphorical interpretation. The pretension to identity, one might argue, is the necessary catalyst for a metaphorical understanding of the intermedial relations. In a work bringing music and literature together, then, this understanding does seem to require a meta-level: an identity between the arts has to be suggested by the artefacts themselves. How, then, does this suggestion come about? The most obvious place to look for such a pretension to identity is the title of a work, whose most important function, as Genette has pointed out in his meticulous discussion on paratexts, is to designate the work, to give it an identity.12 If the identity suggested by the title clearly belongs to the other art, its metaphorical

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

72

quality is indisputable; it is, in fact, a clear illustration of the Aristotelian definition of metaphor as ‘giving the thing a name that belongs to something else’.13 It is tantamount to saying that ‘this piece of music is a poem’ or ‘this poem is a piece of music’. A quick look at the works commonly studied by musico-literary research today confirms a dominance of works signalling their intermedial status in this fashion.14 A similar intermedial signal can be found in the titles of instrumental pieces commonly studied in musico-literary contexts, where many titles are borrowed from a specific literary work rather than a genre, which nevertheless has the effect of ascribing the music to a medium other than its own. Such is the case with my first example: Ravel’s piano piece Gaspard de la nuit, whose title is borrowed from Aloysius Bertrand’s collection of poems. Siglind Bruhn has carried out a beautiful analysis of the second piece from Gaspard de la nuit, ‘Le gibet’, which, like the whole suite, has borrowed its name from Bertrand.15 Rather brutally oversimplified, this poem has the form of an inquiry as to the origin of a sound. This inquiry is repeated through five strophes, all containing the interrogative ‘serait-ce’ and the suggestion of a possible answer. These are the chirping of a cicada, the buzzing of a fly and other insect activities. In the sixth strophe, the answer is given: it is a bell on the city walls sounding the death-knell for the hanged man. Now, as can be seen in Figure 2, Ravel’s piece begins with a B flat in two octaves,

Figure 2

The first measures of Maurice Ravel’s piano piece Gaspard de la nuit

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Axel Englund 73

Media Borders of Qualified Media

which persists throughout the piece, although in different metric positions. It is subsequently surrounded by different kinds of music, a technique that is suggested already in the first measures of the piece. The bell tolling throughout the poem (which we recognize only in retrospect) and the insistent repeated pitch of the piano piece is an obvious analogy, and one that Bruhn’s analysis elaborates on. She interprets the gradually developing relation between this note and its surroundings as analogous to the relation between the hanged man and the insects that become ever more encroaching as the poem progresses.16 The metaphorical perspective would not oppose this interpretation – it would acknowledge the specific analogies discussed by Bruhn, although in slightly different terms. It would, for instance, speak of the interaction between music and poem as endowing the repeated note (an idea in the system of the principal subject, Ravel’s music) with the sinister timbre of a death knell (which is an idea in the system of the subsidiary subject, Bertrand’s poem). These are, in the terms suggested above, the specific or material ideas, but the metaphorical perspective would also add a further notion to support the interpretation. The general concepts of ‘music’ and ‘literature’ form the background against which the material analogies are understood, and the individual work of art is constantly measured against these pre-established notions of the media it involves. For instance, the notion of referentiality, undeniably an important component in the system of ideas thought of as ‘literature’, is projected upon the system of ideas known as ‘music’. This analogous implication organizes our conception of Ravel’s music, so that the idea of referentiality – which is sometimes thought to be a part of music, but which is not central to it – is foregrounded. By dint of the metaphorical interaction between ‘literature’ and ‘music’, the latter is endowed with the former’s tendency to signify – although in a semiotic modality focused on the iconic rather than the symbolic mode – which makes the interpretation of the B flat as a death-knell, as well as its further implications for the interpretation of the piece, all the more pertinent. Heard in this way, this composition becomes a meta-utterance on the character of music and its relation to verbal art, taking part in the historical process of medial evolution by emphasizing their common ground in the semiotic modality. Furthermore, due to its paratexts, Ravel’s piece is particularly susceptible to a metaphorical reading: ‘Le gibet’ is an excellent illustration of the pretension to identity between music and poetry encouraging metaphorical interaction. First, the title uses the name of a piece of literature to designate a piece of music. Second, Ravel has furnished his score with the subtitle, ‘Trois Poèmes pour Piano d’après Aloysius Bertrand’. These paratexts make the metaphorical suggestion ‘this piano piece is a poem’, indeed a specific poem, which necessitates an interpretation of Ravel’s piece in the light of the poem ‘Le gibet’. The paratext, then, constitutes a link to the important intertext of Bertrand’s poem, which points to a crucial issue: the paratexts

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

74

play an essential part as the catalysts of metaphorical interaction. In fact, music without verbal elements has very limited means – if any – to make a meta-reflective suggestion of its own being literature. Therefore, if there is no title or other authorial paratext (such as epigrams, program notes and so on) suggesting the relation to a literary artefact, a piece of instrumental music cannot be recognized as being based on such an artefact. This notion is related to the question of how to categorize a piece like Ravel’s. As Bruhn points out, several labels exist already: she discusses transposition, transformation, transcription, translation and transmutation, before settling with transmedialization.17 In other words, what is continually altered is the ending of the word, whereas the prefix is retained throughout. Now, ‘trans’ etymologically implies something being moved from one place to another, as is exemplified by Rajewsky’s definition of the term ‘transmediality’ as the appearance of the same matter in different media, where ‘the assumption of a contact-giving medium of origin is not important or possible, and would not be relevant for the constitution of meaning in the current medial product’.18 If this were the case, then, we would not need the source medium to access this something once it had been carried across into the target medium, just as the point of a translation is to make a text accessible to those who do not have access to the original language. Quite obviously, this is not the case in our example. Without the title designating at once Ravel’s music and Bertrand’s text, and thus bringing the latter into the interpretation, we would have no particular reason to hear this piano piece in the light of this poem rather than any other. The repeated B flat would hardly be endowed with the ominous character of a death-knell, or at least it might equally well be heard as a number of other things. This fact might be illustrated, for instance, by the similarly repeated A flat of Chopin’s op. 28:15, which, due to the paratext of the piece, is often interpreted as a drop of rain. Any term using the prefix ‘trans’, then, denies the necessity of keeping ‘target’ and ‘source’ simultaneously in mind, by implying that the notes themselves are able to express the content of a poem without being illuminated by a verbal paratext. Even if one adopts a postmodernist perspective from which the notion of ‘music itself’ as absolutely distinguishable from ‘language’ appears highly suspect, the paramount importance of paratextual elements in generating meaning has to be acknowledged.19 The notion of a musico-literary artefact as transmedialized is potentially hypostasized by the topographical model, since it suppresses the notion of simultaneity and interaction by emphasizing the notion of moving from one place, art or medium into another, leaving the original behind. Rajewsky notes this aspect implicitly in referring to transmedial works as Wanderphänomene – phenomena of wandering. In this intermedial vagabondage, the border is crossed and the property of another area brought into foreign territory. The notion of media as adjacent, geographical areas, then, suggests that an artefact of the verbal field can be transported into a musical field, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Axel Englund 75

Media Borders of Qualified Media

core of its identity remaining unaltered even when the original place is out of sight. By contrast, the notion of musico-literary relations as metaphorical interaction between two subjects or system of ideas, stresses the fact that the original, verbal artefact and the musical piece need to be held in mind simultaneously. For instance, Paul Ricœur, quoting Douglas Berggren, holds that the ‘possibility of comprehension of metaphorical construing requires . . . the ability to entertain two different points of view at the same time’.20 Black, too, underscores the simultaneous appearance of principal and subsidiary subject, speaking of the latter as a filter through which the former is perceived.21 Ravel’s ‘Le gibet’, from this perspective, is not a content transported from literature into music, but a piece of music demanding to be perceived through the filter of a piece of poetry. Unlike music, literature does not need to rely on its paratexts in order to create the pretension to identity needed to set off a metaphorical interaction between itself and another art. A piece of verbal writing, in other words, has the ability to meta-reflectively thematize its own medial status and make the metaphorical claim of belonging to another medium. For my next example, I turn to Paul Celan, whose poem ‘Anabasis’ from the 1963 collection Die Niemandsrose contains the following lines: sekundenschön hüpfenden Atemreflexen -: Leuchtglockentöne (dum-, dun-, un-, unde suspirat cor), ausgelöst, eingelöst, unser.22

secondlovely skipping breathreflexes -: lightchimenotes (dum-, dun-, un-, unde suspirat cor), released, redeemed, ours.

One could spend a great deal of time with these lines, not to mention with the poem from which they have been severed here, but let us focus on what they say about the relation between language and music.23 First, the Latin phrase – meaning approximately ‘whence our hearts sigh’ – comes from Mozart’s motet for soprano and orchestra, Exsultate, Jubilate.24 Immediately preceding this quotation, three syllables that seem to be emptied of referential content present themselves. Importantly, these two modalities – abstract, organized sound and referential language – are joined by a gradual, almost seamless modulation: ‘dum-, / dun-, un-, / unde’. The Mozart quotation as well as the sounds empty of referential meaning – tönend bewegte Formen, to quote Hanslick’s famous definition of music – are contained within a parenthesis, which seems to suggest that they exemplify the preceding word, ‘light- / chimenotes’. Thus, the poem makes a claim to verbal imitation of musical sounds. Moreover, the gradual transition from an abstract-auditive

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

76

to a semantic modality in combination with the evocation of Mozart contributes to the suggestion of an identity between music and language: the border, as it were, is being blurred. Alternatively, these verses mimic the blurring of the border – there is never any doubt as to the fact that ‘Anabasis’ is a verbal text rather than a piece of music.25 Thus the interplay between identity and difference needed for a metaphorical interaction to be set off is present in these lines. There are, in fact, further reasons for interpreting this gesture as a pretension to identity between music and literature. One of them lies in a general tendency in Celan’s poetry to be meta-reflective: one of the main issues of his oeuvre, only partly stemming from his engagement with Adorno’s famous caveat, is the possibility of German poetry in the wake of Auschwitz. In view of this, every single element of his poems can potentially be understood as turned towards poetic language itself, questioning, problematizing and analysing it, and musical references such as the ones displayed in ‘Anabasis’ are no exception. Although a particularly salient feature of Celan’s work, the inclination of poetry towards poetology is of course a very widespread practice in modernist as well as postmodernist literature. A further historical note could be added: in the chaotic age of modernity, one of the most conspicuous techniques in music as well as literature has been the juxtaposition of seemingly disparate and unrelated elements, disregarding those laws of causality to which art had hitherto adhered. In poetry, paratactic collocations as well as a number of other ways of deconstructing syntactic structures could be mentioned, and in music the dissolving of tonality and highly fragmented formal language exemplifies a similar tendency. In short, the relations between the elements of a work of art became less and less obvious, and had to be inferred by the percipient. Although he preferred the term imago, the following brief and well-known poem by Ezra Pound is doubtless an example of metaphorical interaction in Black’s sense: ‘the apparition of these faces in the crowd; / Petals on a wet, black bough’.26 Even though no explicit statement of identity is made, the juxtaposition clearly prompts us to look for analogous implications and to regard the depiction of the subway scenery through the filter of the second image. The proliferation of this kind of writing and composing, one might argue, has made the percipients of modernist art attuned to the search for relations other than causal ones. There is no reason why this change of attitude should not affect our understanding of the juxtaposition of music and literature within the same work. The question of the relation between the medial components of a piece of text or music has become increasingly important in modernism, which is patently illustrated by verses such as those of Celan’s quoted above. Having thus argued that these lines from ‘Anabasis’ are another example of metaphorical interaction between literature and music, as well as an illustration of a verbal text setting off such interaction without the aid of paratexts, I would briefly describe the resulting interaction. Celan’s verses

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Axel Englund 77

Media Borders of Qualified Media

themselves would have to be the principal subject of this metaphor, whereas the secondary subject would be the andante movement of Mozart’s motet. One very general notion included in the system of ideas known as ‘music’ is that of a strong connection to emotions, a commonplace that remains widespread to this day. This idea finds its analogy in the ‘cor’ mentioned in the text, as well as in the sounds ‘dum-, /dun-’, which might be interpreted as an onomatopoetic reference to the beats of the sighing heart (or, for that matter, to the sound of chimes). Also, the religious character of the Mozart motet activates the traditional notion of music functioning as a link between humanity and divinity, fundamentally important to Friedrich Hölderlin among others. In Celan’s text, this notion is mirrored by the elated mood and almost epiphanic connotations of the lines ‘re- / leased, re- / deemed, ours’. A more specific, material idea lies in the adverb ‘secondlovely’, which could be taken as referring not only to the time unit of a second, thus stressing the fundamentally temporal quality of musical appreciation, but also the musical interval of a major or minor second. If one examines Mozart’s Exsultate, Jubilate at the entrance of the quoted phrase – that is, the music evoked by the textual quotation – 21 intervals out of 25 turn out to be seconds (the remaining being a fifth and three thirds).27 The predominance of seconds is of course a very common phenomenon in the melodic idiom of Mozart. Thus it seems reasonable to understand this musical interval as one of the connotations of ‘secondlovely’. Furthermore, the term ‘breathreflexes’ actualizes the notion of breath, which is not only a central notion in Celan’s poetics,28 but also inextricably linked to the human voice, be it speaking or, as in Mozart’s piece, singing. In this example, it is rather obvious that we are not dealing with a ‘trans’ anything: music is not carried into the poem. Without the aid of a paratextual marker, Mozart’s music is evoked as an intertext through which we may perceive the non-referential syllables as well as the Latin phrase – Mozart’s motet serves as a metaphorical filter through which certain aspects of Celan’s verses might be read. By using examples from the categories labelled ‘literature in music’ and ‘music in literature’ in Steven Paul Scher’s classic essay,29 I have aimed to suggest a further important advantage of the metaphorical perspective: it is able to serve as an umbrella concept for both of these types, since they can both be regarded as metaphorical constellations. This approach, I believe, might serve as a possible starting point for the collocation and comparative study of works from all three of Scher’s categories, a kind of intermedial examination that is arguably hard to access and of which we have not yet seen many convincing examples. I have also tried to shed light on some implications of the topographical model which describes media as adjacent areas separated by borders and gaps, and contrast this model with an understanding of musico-literary

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

78

phenomena in terms of metaphorical interaction. For an artefact to open itself up to such an interpretation, it needs a meta-reflective element, giving rise to a pretension to identity between music and literature. In verbal artefacts, this pretension can be expressed either by the text proper or a paratext. In musical artefacts, by contrast, a paratextual marker such as a title, a program note or an epigraph, is perhaps the only way. Hence, a literary content or form cannot be transported into music and recognized as such without the simultaneous presence of the original work, a fact blurred by the topographical model as well as by the use of the ‘trans’ prefix. Although efficient in many ways, I have argued that it suppresses the notion of simultaneity. The metaphorical perspective, then, requires a suspension of the topographical model in order for the arts to exist simultaneously in the mind of the percipient. Moreover, I have argued that, like Black’s subjects, these arts are better understood as systems of ideas than as neatly mapped out medial areas delineated by clear borders. Through the metaphorical interaction, in accordance with Black’s perspective, the ideas that constitute analogies between the subjects come to the fore in our understanding of the musico-literary artefact. These are general, notional ideas as well as specific, material ones and the reorganization of the systems of ideas to which the analogies between them give rise can be read as meta-medial utterances, potentially altering our view of ‘music’ and ‘literature’ as such. Each instance of a musico-literary metaphor actively partakes in the historically developing notions of arts and media, reflecting on and reshaping their imaginary territories, and thus continuously rendering obsolete each new map of the intermedial topography before it is finished.

Notes 1. The understanding of musico-literary relations as metaphorical has cropped up in recent intermedial studies by Lawrence Kramer, Eric Prieto and Michael Spitzer, whose work has given important incentives to this essay. Cf. L. Kramer (1995) Classical Music and Postmodern Knowledge (Berkeley CA: University of California Press), pp. 69–71; E. Prieto (2002) Listening In: Music, Mind, and the Modernist Narrative (Lincoln NE: University of Nebraska Press); M. Spitzer (2004) Metaphor and Musical Thought (Chicago: Chicago University Press). 2. Cf. for instance M. Black (1962) ‘Metaphor’ in Models and Metaphors (Ithaca NY: Cornell University Press), pp. 25–47; P. Ricœur (1979) ‘The Metaphorical Process as Cognition, Imagination, and Feeling’ in S. Sacks (ed.) On Metaphor (Chicago: The University of Chicago Press), pp. 141–57. 3. Black (1962) ‘Metaphor’, pp. 38–46. 4. Ibid., p. 44. 5. Ibid., p. 40. 6. Ibid., p. 41. 7. Cf. Elleström, this volume. 8. Black (1962) ‘Metaphor’, p. 44. 9. N. Goodman (1976) Languages of Art (Indianapolis: Hackett), pp. 229–30.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Axel Englund 79

Media Borders of Qualified Media

10. Cf. Rajewsky, this volume. 11. Cf., for instance, C. Dalhaus (2002) ‘Die Idee der absoluten Musik’ in Gesammelte Schriften 4 (Laaber: Laaber), pp. 11–128. 12. G. Genette (1997) Paratexts: Thresholds of Interpretation, translated by Jane E. Lewin (Cambridge: Cambridge University Press), p. 76. 13. Aristotle (1995) The Complete Works of Aristotle, Jonathan Barnes (ed.) (Princeton: Princeton University Press), vol. 2, p. 2332. 14. Cf. for instance the works studied in W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History of Intermediality (Amsterdam and New York: Rodopi), pp. ix–x. 15. S. Bruhn (1999) ‘Piano Poems and Orchestral Recitations: Instrumental Music Interprets a Literary Text’ in W. Bernhart, S. P. Scher, and W. Wolf (eds) Word and Music Studies: Defining the Field (Amsterdam and New York: Rodopi) pp. 277–99. 16. Ibid., pp. 288–91. 17. Ibid., pp. 296–7. 18. I. Rajewsky (2002) Intermedialität (Tübingen: A. Francke), p. 13. 19. Cf. Lawrence Kramer’s use of the term ‘designator’ in Kramer (1995) Classical Music, p. 69. 20. Ricœur (1979) ‘The Metaphorical Process’, p. 152. 21. Black (1962) ‘Metaphor’, pp. 39–41, 22. P. Celan (1992) Gesammelte Werke in Sieben Bänden, B. Allemann and S. Reichert c 1963 (eds) (Frankfurt am Main: Suhrkamp), vol. 1, p. 256. Die Niemandsrose  S. Fischer Verlag GmbH, Frankfurt am Main. 23. For further discussion of the musico-literary aspects of this poem cf. A. Englund (2009) ‘Modes of Musicality in Paul Celan’s Die Niemandsrose’, Seminar: A Journal of Germanic Studies 45(2), pp. 138–58. 24. O. Pöggeler (1993) ‘Die göttliche Tragödie: Mozart in Celans Spätwerk’ in C. Jamme and O. Pöggeler (eds) Der glühende Leertext: Annäherungen an Paul Celans Dichtung (München: Weilhelm Fink), p. 68. 25. For equivalent examples involving the visual arts, cf. Rajewsky’s discussion of intermedial reference in this volume. 26. E. Pound (2003) Poems and Translations (New York: Library of America), p. 287. 27. Cf. W. A. Mozart (1963) Geistliche Gesangswerk: Werkgruppe 3 (Kassel: Bärenreiter), p. 167. 28. Cf., for instance, the famous formulation from ‘Der Meridian’: ‘Poetry: that can signify a turn of breath’ (Celan (1992) Gesammelte Werke, vol. 3, p.195). 29. S. P. Scher (1982) ‘Literature and Music’ in J.-P. Barricelli and J. Gibaldi (eds) Interrelations of Literature (New York: Modern Language Association), pp. 225–45.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

80

4 Christina Ljungberg

Laurie Anderson’s multimedia performance White Lily opens with a computer-animated projection of a figure running in slow motion. Anderson then enters into view backwards against the animated runner who disappears to the left, while Anderson moves across to centre stage to electronic music punctuated by clock chimes. Dressed in a white suit and accompanied by her silhouette shadow generated by a strong circular projection, Anderson has her movements doubled by the shadow, as she presents a short text about her memory of a brief conversation in Rainer Werner Fassbinder’s1 film Berlin Alexanderplatz.2 This short sequence is reshaped into a poem recited by Anderson, at the same time as she makes punctuating gestures with her right arm, ending by making a backwards sign. The performance concludes with the projection of the central symbol, the white lily, held by the hand of a white silhouette in a still that is left standing against the dark, as Anderson moves out of the centre and disappears to the right. The animated figure then reappears and fades out, with only the music playing in the dark. With its intricate intermingling of image, sound and gesture, the performance of White Lily3 is a complex multimedial restaging of the short scene in Fassbinder’s film, in which Anderson directly addresses a number of questions concerning mediality, modality and art form. Anderson’s reconfiguration of the scene in Fassbinder’s movie alludes to Alfred Döblin’s famous 1929 novel Berlin Alexanderplatz4 which was the source of the film Fassbinder made for television in 1980. Its impetus as an intermedial work is partly drawn from Döblin’s text, which is in itself both intermedial and intertextual. With its strikingly cinematic and journalistic character, Döblin’s work is a modern epic, which, not unlike James Joyce’s Ulysses, rewrites antique myth into a modernist urban setting and which, influenced by both Futurism and Dadaism, tries to capture not only the visual mosaic of the city’s frenetic and oscillating surfaces but also its polyvocal discourses. This is the cultural memory that Fassbinder recreates in his epic 15-hour film, which transforms the (mainly verbal) montage technique Döblin employs in his large 400-page polyphonic novel into an audiovisual masterpiece, and which 81 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Intermedial Strategies in Multimedia Art

Media Borders of Qualified Media

Anderson reconfigures in her 1 minute and 16 seconds performance of a key event in Fassbinder’s film and Döblin’s novel. These intersemiotic transformations, from a dialogic and intermedial novel to, initially, a brilliant filmic visualization of Döblin’s imagination and, eventually, to Anderson’s masterpiece, that is, from one medium to another to yet another, raise the question of the notion of ‘medium’ in an especially pointed and forceful manner. What is a medium? A key concept in semiotics, one way of defining ‘medium’ is to say that it is the necessary channel or conduit of communication which allows the transmission of a message to a receiver. As Winfried Nöth5 suggests, even the air functions as a medium, since it carries the sound waves from speaker to listener. This basic meaning resonates with the definitions given by the Oxford English Dictionary (OED), which describes it as a means by which something is expressed, communicated or achieved; a substance through which a force or other influence is transmitted; a form of storage for computer software, such as magnetic tape or disks; a liquid with which pigments are mixed to make paint; with the plural ‘mediums’, a person claiming to be able to communicate between the dead and the living, that is, between our lived world and an imaginary one; or the middle state between two extremes. Although the first sense given by the OED may be the most commonly used in contemporary life, all these aspects of ‘medium’ suggest a dynamic and fluid meaning, characterizing a transient function more than a fixed or stable property. In Peircean semiotics, a sign is itself a medium since it dialogically interacts in its various modes, the iconic, the indexical and the symbolic, in an ongoing flow of signs mediating between the life-world we live in and our interpretation of it. Although all sign aspects are necessary for a sign to function, they are differently foregrounded in the various sign systems. If music is predominantly iconic, photography and film are, due to their mode of production, mainly indexical media, whereas painting and verbal communication, insofar as they depend on cultural conventions, are symbolic. In communication and media studies, the concept of ‘media’ is used to refer to the classical mass media newspaper, book, radio, popular music, film and television. More recently, the concept has been extended to cover writing or even speech in general, music, painting, photography, video, the Internet or computer games, which no longer qualify as media interacting with the ‘masses’. Intermediality, then, concerns the transgression of the borders between such media, for instance, between different sign systems and/or the iconic enactment of one medium within another. It also involves the sensorial modality of a specific medium, mainly the visual, oral or tactile (the use of olfactory signs, as in Divine’s Odorama, was a short, even if rather successful, attempt) and the semiotic register of sign functions. What happens to these modalities and to these functions when various media interact? What precisely constitutes the result, that is, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

82

Christina Ljungberg 83

• radically performative, as we are confronted with hybrid forms that generate something new and unique • strongly self-reflexive, since they focus attention both on their own mode of production and on their own semiotic specificity, which is heightened by the increasing digitalization of interacting media • a highly effective communication strategy, as they give readers, viewers and listeners access to different levels of meaning. These are the issues at stake in my contribution, which will discuss two very different examples of intermedial art mapping time and space by the performer, artist and writer Laurie Anderson and digital artist Lucia Leão. What I want to explore are the strategies they use to achieve their unique and innovative intermedial effects.

The medium as sign As mentioned at the outset, in a technical or material sense, ‘medium’ can be described as the channel enabling communication between a sender and a receiver. In its broader semiotic sense, however, the sign itself, as defined by C. S. Peirce, functions as a medium. In the triadic semiotics of Peirce, a sign is anything that stands for (represents) something, called its object, to generate another sign as its interpretant. According to this definition, the sign is itself a mediator or medium, acting, so to speak, as a translator between its object and its so-called interpretant, which is the result of its interpretation. The sign is therefore defined in terms of a triadic process called semiosis, or sign generation. According to one of the most quoted definitions, a sign is ‘something that stands for something to somebody in some respect or capacity’.6 The sign initiates a process which makes it interact relationally or functionally with its object. Signs are not necessarily material objects, nor even a class of objects7 : they exist in the mind of their interpreters, in other words, they have a cognitive effect on their interpreters. In signification, a sign dialogically interacts with its various sign aspects, the iconic, the indexical and the symbolic, in an ongoing flow of signs mediating between the life-world we live in and our interpretation of it. Although all sign aspects are necessary for a sign to function, these aspects – iconic, indexical or symbolic – are, as mentioned above, differently foregrounded in various sign systems. That is why painting which, as a

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

phenomenon we call intermediality, which concerns the negotiations of the borders between various media. What do these ‘border talks’, as Irina Rajewsky (in this volume) calls them, effect in such intermedial transgressions? What are the possibilities and limitations that such intersemiotic translations from one art to the other bring into being and how is one medium reflected in another? I will argue that these instances of intermediality are

Media Borders of Qualified Media

visual sign, would seem to be primarily indexical as it is always embodied in some singular materiality8 displays mostly symbolic aspects: paintings adhere to the styles and cultural (and ideological) conventions dominant in the period in which they were executed. Not even so-called ‘pure painting’, which was advanced at the end of the nineteenth century as a form separate from so-called ‘narrative painting’ and supposedly demonstrating ‘pure opticality’, is ‘pure’ but is, as W. J. T. Mitchell suggests, the ‘discourses of theory, of idealist and critical philosophy’.9 Recognizing these underlying discourses is as vital for understanding modernist painting as the knowledge of the Western narrative canon and familiarity with its myths required for comprehending classical narrative painting. Music, another example, and the most iconic of all medial forms as it comes to us in the form of mere quality, can also be indexical and, though marginally, symbolic. It is indexical not only because it always indicates a certain style or genre (insofar as the recognition of its characteristics depends on similarity, it is again iconic), but also because its tonal flow calls us to ‘things in this [particular] world’. 10 Symbolic aspects appear, for instance, in the case of a national anthem.11 That is why, just as there are no ‘pure’ sign forms, there are no ‘purely’ visual, verbal or aural media: in Mitchell’s words, ‘all media are mixed media’.12 This does not take away a medium’s particular characteristics, its ‘specificity’ as it were, but instead, enables media mixtures and innovations and the transformations of old media by new techniques. What happens when various sign systems interact, which is the phenomenon that we call intermediality? What does it do to the specific character of each particular system? It would have to be assumed that, as in all border transgression, transformations and substitutions take place. One of the oldest intermedial relationships is the evolution from image over pictorial writing to ideographical writing systems, which, as has recently been shown, developed according to underlying principles governing the shapes of human signs.13 And is not also narrative language in itself intermedial, as we structure our sentences in verbal diagrams and as spatial diagrams on the page? It would thus seem that intermediality is intrinsic to narrative texts, whether in the form of rhetorical structure14 or in the interplay with other sign systems such as photographs or maps.15 This explains, I would argue, why forms of self-referentiality and iconicity are intrinsic to literary texts – and vice versa. This intermediality demonstrates how such a seemingly simple transgression is inherently performative, since it creates something new and unprecedented, at the same time as one medium is reflected in the other. In intermedial art forms, this relationship becomes radicalized, as the difference in interaction in multimedia art and performance determines the different degrees of performativity and self-referentiality of the work of art and its communicative effect.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

84

Christina Ljungberg 85

What precisely is meant by performative here, and what is its relationship to intermediality? If ‘performance’ is an execution of an action, the fulfilment of a claim, promise and so on or a presentation to an audience, ‘performative’ is the very expression that effectuates it. Both concepts entail memory. As Mieke Bal points out, in a performance, playing a part or role requires memorizing that part or that score and practicing the gestures, expressions and diction suiting the role, just as a performative act needs a cultural – and appropriate – context to function. This subordinates individual intention to social convention and makes performative acts instances of an ‘endless process of repetition involving similarity and difference, and therefore relativizing and enabling social change and subjects’ interventions, in other words, agency’.16 The concept of performativity and its link with performance has also been theorized by the theatre historian Erika Fischer-Lichte17 who argues that it is the transformative potential arising from the shared ritual practices surrounding a theatre performance that makes them both self-referential and capable of constituting reality. She differentiates three different kinds of performativity: in the weak sense, that something is done by someone saying it; in the strong sense, whereby language creates a new reality against a backdrop of stable conventions; and in the radical sense, by which all these processes create a new social reality.18 Intermediality always entails performativity in the radical sense owing to its hybridity. Within our cultural life, performative utterances and acts bring something into being. In the simplest cases, in uttering words (for instance, ‘I promise you to be there at seven’) an act is performed, one entailing various consequences and expectations. The performance of a traditional ritual (for instance, a marriage ceremony or christening) is one thing, that of a yet to be instituted form of ritual and performance quite another. Of course, the latter are enacted against the vast, vague background of our complex, myriad inheritances. Even so, they bring something into being, not in accord with traditional forms of ritual, but as an attempt to establish what is not now recognized or authorized. What is attempted in the quiet dramas of quotidian life is also undertaken in more manifest ways in numerous contemporary artworks, which therefore also involve analysis and interpretation. But is not the very act of analysis in itself a performative act in which a new, further developed object of study is created? As C. S. Peirce points out, an object exercises influence over the sign in so far that it can ‘guide or constrain’ the process of semiosis or sign generation.19 This is what Bal refers to when she points to the fact that the analysis mutates from just applying theory to ‘a performative interaction between object (including those of its aspects that remained invisible before the encounter), theory and analyst’ making the very process

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Intermediality and performativity

Media Borders of Qualified Media

of interpretation part of the object for the analyst to investigate. In this way, ‘objects enable reflection and speculation, and can contradict projection and wrong-headed interpretation . . . and thus constitute a theoretical object with philosophical relevance’.20 Such awareness is particularly called for in the analysis of complex multimedia performances such as White Lily, in which the relationship between performativity and performance indeed becomes radicalized since the switching between different media automatically generates new objects and new realities for their ‘analysts’, that is, the audience trying to grasp the intricate layers of meaning. Anderson’s performance – and also the work of Lucia Leão – therefore poses interesting questions as to what happens when various media interact and what such intermedial relationships effect. Anderson’s profound and multilayered transformation of Fassbinder’s film based on Döblin’s novel into her own performance demonstrates exemplarily the extent to which intermediality involves the transformations of myths, poetry or prose, modes of narration and styles of writing into other forms such as paintings, films, video arts, performances or other works of art which in some way quote, adapt, rework or just allude to literary texts. At the same time intermediality also refers to the very act or process of transformation that literature undergoes under the influence or in contact with other art forms as it absorbs and adapts motives, plots and even modes of writing of the visual arts.21 It also takes up intermedial dialogues with the other art forms by reflecting, in writing, on what these interactions bring, as well as producing new hybrid and multimedial forms of art, of which Döblin’s intermedial novel Berlin Alexanderplatz is a particularly pertinent example. In Anderson’s performance, the interplay among the visual, the vocal and the gestural is highlighted by the complex interaction between the technical media, the qualified media and their various modes,22 performatively calling something entirely new and different into being. Anderson’s brilliant syncopation of a key event in Fassbinder’s film and Döblin’s novel abstracts the core issues of time and temporality in her ‘pretexts’, transforming them into a multi-layered mediation about time, memory and mortality. The split focus on Anderson and her shadow is iconically matched by the sound tension simultaneously created between the electronic background music with its effects of a bell that chimes and Anderson’s voice dramatically reciting her memory: What Fassbinder film is it? The one-armed man comes into a flower shop and says: What flower expresses days go by and they just keep going by endlessly pulling you into the future?

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

86

Christina Ljungberg 87

Anderson’s condensation of the question of time and memory into her one minute and 16 seconds refiguration of this key event in Fassbinder’s film and Döblin’s novel brings up the question of the relationship between performance, performativity and cultural memory in a very intriguing manner. The event itself, the quotidian act in which a man comes into a flower shop to buy flowers, may seem banal enough. It constitutes, however, a crucial trait in the portrayal of the hulking, child-like ex-convict Franz Biberkopf. When this scene takes place, Franz has recently come out of prison, fully convinced that from now on he will become an ‘honest soul’ and lead ‘a decent life’.23 But every time his life seems to be taking a turn for the better, he is betrayed by men he trusts and considers his friends. Such a betrayal is implied in Anderson’s performance of the flower shop scene, when Franz wants to make a decent gesture towards a woman who has been nice to him. In Döblin’s story and Fassbinder’s film, Franz meets a girl, Polish Lina, whose uncle, Otto Lüders, lets him into his business as a door-to-door shoelace salesman. On one of his peddling rounds, Franz meets a lonely widow, whose husband he physically resembles, and has sex with her. She is grateful and generous to him, but Franz’s childish delight at his good luck has him tell Lüders about it. Lüders immediately pays a visit to the widow, first to insult her, saying that word about her ‘generosity’ is afoot, and then to rob her. Franz, unsuspectingly looking forward to his next dalliance with her, goes to buy her flowers. This is where Fassbinder’s poetic imagination stages the short scene that Anderson is alluding to, a scene that does not occur in the novel which only tells us that Biberkopf ‘slowly walks up the stairs with a bouquet wrapped in oiled paper,’24 only to have the door slammed shut in his face, and, disappointed and angry, throws the flowers into the gutter. Only later, receiving a letter from the widow, does he learn of Lüder’s betrayal, which demolishes him: when Lina comes home, she finds him sitting in his room, apathetically picking his alarm clock to pieces, as if time could be stopped and turned back to be repackaged, reordered and replayed. In his film, Fassbinder superimposes Biberkopf’s visit to the flower shop onto the underlying issue of time and temporality. Franz enters the shop and asks the florist what flowers express the meaning ‘that the past always runs after you and drives you further and further to where there is no future’. To his surprise, the florist suggests white carnations, which to Franz symbolize ‘flowers of death’. Fassbinder then has Biberkopf buy the widow a bouquet of red roses instead – which then end up the gutter when he is refused entry

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Days go by endlessly endlessly pulling you into the future. And the florist says: White lily.

Media Borders of Qualified Media

to the widow’s apartment. At this time, Franz is not yet the ‘one-armed man’ in Anderson’s memory – he still has both his arms. Moreover, not only does Anderson change the white carnation into a white lily but she also weaves the flower shop scene in Fassbinder’s film together with a later betrayal, which costs Franz his arm and nearly his life when he is thrown out of a car by another envious ‘friend’ – who later proceeds to kill his girlfriend, making Franz temporarily insane. By mapping the two betrayals, temporally distinct but causally related incidents (as both concern betrayal by ‘friends’ whom Franz naïvely trusted) onto one another, Anderson skilfully and poetically syncopates Franz’s existence as an agent caught up in fields of intersecting forces beyond his control, a victim of his time and circumstances. By only ‘learning things that he had rather not wanted to learn’25 too late, and not being able to adapt to the ruthless life of the modern metropolis, he shows himself as too naïve, too innocent and above all, too trusting. The network thus spun over Alexanderplatz could therefore be looked at as an ‘agential space’, a space in which agents are at once caught up transcending their immediate control and implicated in the effective exercise of their somatic, social agency. In other words, these agents are such situated and embodied forces that the exercise of agency is best understood in terms of introducing disturbances into this field or as tracing these intersecting force patterns. This notion would seem to be appropriate to describe both Fassbinder’s film and Döblin’s multiperspectival story about the interaction of people caught in the corrupt urban landscape of Weimar-era Germany and which is brought to such a fulcrum in Anderson’s performance.

Intermediality and self-referentiality Intermediality also displays degrees of self-referentiality. Iconic self-reference is typical of the aesthetic sign. One of the characteristics of the aesthetic sign is that it calls attention to various aspects of itself, above all to its sensuous qualities and formal structures, its actual materiality and its rhetorical strategies. At least, this becomes evident and explicable when we approach self-reference in general and the self-referentiality of aesthetic signs according to C. S. Peirce’s doctrine of signs. In particular, his second trichotomy of signs (icon, index and symbol), which is based upon the character of the relationship between a sign and its dynamical object, is especially illuminating here, since it provides 1) a way of understanding reference in terms of indexicality (the indexical sign being defined as that in which there is a spatiotemporal or causal relationship between sign and object) and 2) a way of understanding self-referentiality (at least in part) in terms of iconicity. All iconic signs are self-referential, which could appear paradoxical since signs should really stand in for something else. The reason

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

88

why signs can represent other signs is because, in Peircean semiotics, the object ‘does not need to be a piece of the so-called real world at all, since signs or ideas can be the object of a sign. The object of the sign is something which precedes and thus determines the sign in the process of semiosis as a previous experience or cognition of the world.’26 The sign’s referent (Peirce’s object of the sign) can be another sign, and self-reference can be a chain of signs referring to other signs. Oral interpretation, as in the case of Anderson’s performance, utilizes the self-referencing qualities of language and literature which involve the interpreter as both actor and reader. In the case of digital media, self-reference is coupled to the mathematical generation of numerical images. These images are in perpetual metamorphosis, ‘oscillating’ between the actual image on the screen and the virtual image or potential set of images.27 Because there is no analogy between the algorithms that generate it and the image on the computer screen, this image is highly iconic. It generates experiences that are not ‘real’ but formalized and repeatable calculations, which makes the synthetic image synonymous with virtuality and simulation. That is why the digitalization of pictures and films has contributed strongly to the increase of self-reference in the media – at the same time as it has liberated the media from ‘the bonds of factual reference to a world which they used to depict’.28 Quotations, allusions, adaptations, influences and borrowing from texts, films or any other medium also generate intertextual self-reference.29 When various media are involved, for instance a film referring to a painting (Girl with a Pearl Earring) or a novel in a film (for example, Conrad’s Heart of Darkness in Francis Ford Coppola’s Apocalypse Now, which has the novella lying on the protagonist’s bedside table), there is intermedial self-reference: reference from one media to the other. Such borrowings from other texts or media are alloreferential, which means that ‘the object of the quoting sign is a quoted sign from which it differs’,30 since one medium refers to another. The difference here is that when one film makes an intertextual reference to its own cinematic medium by quoting another film, and not to the world which they ultimately represent, they are examples of intertextual self-reference, in particular since quotations always entail repetition and sameness, which is ‘the source of iconic self-reference’.31 As discussed above, White Lily is a highly complex intertextual act since its ‘pretexts’ resonate throughout the performance, albeit mainly for an audience familiar with Fassbinder’s film and even more so for those who have read Döblin’s novel. Quotations and allusions are generally considered referential, since they refer to something else, an object, in a different context. That would make Anderson’s literary quotations referential and in this sense indexical since they have a particular context in mind. However, whenever her music refers to other pieces of music and her visuals to other visuals (and not to any world beyond the world of music or visual representation), her

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Christina Ljungberg 89

Media Borders of Qualified Media

‘quotations’ of other musical and visual works are self-referential since they are references from the media to the media. Furthermore, Anderson’s work is pervaded with repetitions and recursions of words, phrases or ideas, which are typical and striking forms of self-reference – whether in music, texts, images or films – since they always refer back to the preceding instances. So are the reuses and quotes of her own work, which make up much of Anderson’s œuvre. Self-referentiality is something that digital art shares with other postmodern art forms. Hybrid forms of art and media heighten the degree of self-referentiality: switching between or among various media not only forces its viewing or, rather, participating, audience32 to make comparisons among them but also exposes the particularities of the various semiotic systems that each medium embodies. Virtual reality requires both specific aids and a technological environment, which calls attention to the necessary procedures involved. Focusing attention on the artist and her bodily self, as both generating and participating in the work of art, also increases self-reference. However, these works have a marked indexical ingredient, too, in the sense of referring to other ‘real’ works, contexts or bodies. Even in virtual reality, an awareness of the physical body is necessary for orienting ourselves in and understanding the particular digital work of art. Hence, there are varying degrees and forms of self-reference characteristic of various types of digital art and media. Virtual reality is not separate or even separable from embodied reality, but is rather an attenuated and reconfigured form – or array of forms – of our embodied being in interaction, intermedially, with anything and everything. Another factor contributing to the self-referentiality of digital art is the emphasis artists put on the very process involved in producing art by more or less smoothly integrating the various media into an intermedial whole. As Mitchell points out (referring to McLuhan), what is important is the ‘ratio’ in which the various ‘ingredients’ are mixed and timed, questions of dominance/subordination, the phenomenon of synesthesia, or what Mitchell calls ‘nesting’, in which one medium appears within another as its content, or ‘braiding’, the seamless weaving together of disjunctive elements into a ‘seemingly continuous narrative’.33 His own emphasis on process is in effect an interrogation of time, of the textures and forms of temporality, but since the flux of time leaves sedimented layers of meaning and identity, these sedimentations deserve our critical attention – not only the result of the temporal flux but also the power of this flux to unsettle, to disturb. This does not emanate from the artist only but is created in the space between bodily co-presence between actor and spectator, and in the filmed recording of a performance. Anderson’s multimedial performance can be seen as ‘braiding’ since one medium appears within the other as its content though in a highly complex form: the scene from Döblin’s novel transformed into Fassbinder’s film, then the filmed recording of Anderson’s performance of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

90

her memory of Fassbinder’s film. There is however also a ‘braiding’ taking place in the viewer’s mind in the act of interpretation as this utterly condensed performance must be mapped onto our own personal and cultural maps of knowledge and experience. Viewed thus, there is an expansive network of performative relationships, which contributes to the artwork’s particular power. Interactive works of art such as Lucia Leão’s Hermenetka34 would seem to offer a different kind of self-referentiality and intermediality, as it offers the user the possibility to produce his or her own map dynamically in an everchanging cartographic configuration. The meshwork Leão spans across her Mediterranean map is a project of Net Art that generates fortuitous cartographies from search engines in various databases. Hermenetka is a telescope word coined from associations to Hermes, god of communication and commerce, protean shape-changer and inventive creator as well as the interpreter and intermediary between the divinities and the mortals; Net, to internet, and Ka to the divine concept in ancient Egyptian mythology representing the life-force and consciousness. Leão chose the geographical area and the cultural concept of the ‘Mediterranean’ because of its etymological meaning as the ‘sea between territories’ or its cultural meaning of being a ‘sea in the middle of the Earth’ – both literally and metaphorically – and therefore being a ‘space in-between’ – which is thus embodied in unbounded cybernetic flows and data exchanges. The project’s basic intent is to generate plural cartographies of the seas of data populating cyber culture. Hermenetka is constructed as two types of mappings. The first enables the generation of a map in real-time centring on the concept of the Mediterranean: countries, cultures, histories and so on. The map is entered by clicking on one of the various topics circulating in hypertext on the map surface; frontiers, myths, limits, plurality, territories, flows, memory, flavours, aroma or rhythm. You can then explore each site’s layered images by doubleclicking them – ‘rhythm/pulse’, for instance, begins with Luciano Berio, Italian composer, with a link to the Wikipedia entry on Berio, the Albanian singer Edi Zara and the Spanish rap artist La Mala Rodriguez. ‘[L]imits’ introduces us to the singer Fortuina, then links us up with Wilson Sukorski’s poetic Noosphere, focusing on boundary-crossing artists presented in continuously shifting images such as Okay Temiz, the Turkish jazz percussionist whose unique ethno-jazz is the result of his crossing both physical and cultural borders and confronting different music styles, or the Franco-Algerian musician Rachid Taha. All these entries are accompanied by artists’ websites or biographies, geographical maps, blogs, bibliographies and photologs. At the time of downloading, the music databank provided 33 different kinds of music from all parts of the Mediterranean zone which, apart from the front map with its meshwork opening itself to the manifold images, is the only ‘constant’ element in this dynamically altering interactive mapping enterprise.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Christina Ljungberg 91

Media Borders of Qualified Media

Figure 3 Lucia Leão, Hermenetka (2005–2007): ‘What is the Mediterranean for you?’ Courtesy of the artist

The second kind of mapping consists in answering the question ‘What is the Mediterranean for you?’ in an html-window (see Figure 3). This encourages map users to contribute their own images, texts and links, triggering further cyberspace research to be entered on the site, thus constantly composing a unique and instantaneous map. In both mappings, images are generated at random and composed of different sizes and levels of transparent overlaying of images and texts. It is hard to escape the impression that the context here is that of advertising propelling us into the illusion that the world and everything in it is simply what we – with enough resources and ingenuity – are able to make it into. At the same time, it is indicative of the extent to which the ethos of consumerism permeates our sensibility as the interactive character of the site flatters the informed and enlightened consumer as setting the terms of interaction. But these are in fact constantly negotiated and renegotiated, and here we, too, are agents in a field of crisscrossing forces in which we can never achieve full control. Populating the various territories with a seemingly unending supply of sources of historical entanglements, myths and sensorial inputs as streams in which we are caught up and carried along in the form of visual and aural media both deconstructs known space and reconstructs it all over again. The interaction between the various media – images, film, television,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

92

music, books, the internet – uses the visual, the aural and its evocation of smell, taste and tactility, thus playing on our sensory apparatus. The sensorial inputs in their attempts to evoke flavours and aromas play no small part here, making the site both performative and self-referential. Moreover, it is continuous, as the various levels flow into one another; it is also dynamic, constantly developing new, augmented alternative cartographies of images and sounds from visitors’ suggestions and new combinations with constantly changing perspectives and positions. To conclude, what becomes obvious here is that an intermedial analysis demands a multi-level specification of the particular elements of matter, energy, skill and technology that are involved in medium-specific practices. However, considering the extent to which we are caught up in media and the ways that our actions and reactions are functions of the energies and trajectories of the media themselves, an investigation into what spaces of interaction, what degrees of performativity and of self-referentiality are generated in such reconfigurations is even more called for. This becomes yet more urgent since our involvement is such that both our identity and agency can be defined in and through immersion in media, which makes necessary a penetrating understanding of the relation not only of practitioner to practice but also of the interaction among media effects. This interplay results in both artworks in something new and unique and is therefore radically performative. Anderson’s condensation of Fassbinder’s transformation of Döblin’s novel produces a performance which is both precise and poetic and will also differ whenever it is experienced. This makes each viewing instantaneous since it projects new space emanating from the cultural, political and social memory provided by the ‘pretexts’ it evokes. This space is liminal and unique; apparently, it is also immediate, though upon analysis it turns out to be a complex mediated affair. Such uniqueness also applies to interactive digital art works like Leão’s dynamic mapping: its fortuitous cartographies consist of an immense amount of incalculable combinations and variations as well as being potentially unlimited in the sense that, whenever a new element is added to it, the constellations change. The ‘actual’ map is in continuous metamorphosis, responding to the input of the user or interactor. They may seem to involve different degrees of performativity – one theatrical, related to the space generated in the performative interaction between performer and audience, the other in the dynamic space produced in the interactive response between the mapper and the computer. Nevertheless, they are not only both radically performative but also produced by not entirely dissimilar processes, as they both address time and cultural (as well as personal) memory. Both take place within a certain ritual: Laurie Anderson’s polyphonic amalgamation of various cultures, media and time is played out to an audience as a live performance with all its inherent rituals – even as a video recording. In a sense, this also pertains to Lucia Leão’s Hermenetka, which involves a ‘cyberart ritual’ (turning on

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Christina Ljungberg 93

Media Borders of Qualified Media

the computer, opening the website, clicking on the various links). Its performative relationship is rather similar to the one created by Anderson’s performance during which the audience as much as the artist becomes responsible for what is brought into being by such dynamic works of art. After all, the cybercartographer equally perceives and experiences the mappings from the point of his or her subjectivity and maps them into his or her mental geography. Moreover, both in Anderson’s White Lily and Leão’s Hermenetka self-reference is used as an aesthetic sign whose formal structures and rhetorical strategies evoke the materiality of the performance, but in different ways. That is also why they involve different degrees of intermediality: as we have seen in White Lily, Anderson’s multimodal Gesamtkunstwerk which consists of invisibly sutured ingredients, there is a self-referential use not only of syntax and sound in her lyrics (prosody, assonance, alliteration and the sound editor) but also in the ways she deploys music, gesture and visual animation. Whereas Leão’s cybermapping is closer to the concept of ‘nesting’, in which one medium appears within another as its content, Anderson’s intermediality could be defined as ‘braiding’, the multimodal but seamless weaving together of disjunctive elements into a seemingly continuous narrative. Intermedial works of art therefore not only involve intermedial border negotiations and media transgressions, they also concern performativity and high degrees of self-reflexivity. Furthermore, it is precisely the efficacy of these dialogic negotiations that makes them meta-negotiations, or are they re-negotiations? Are not the terms on which we undertake such negotiations always already set by existing structures of power, thereby forcing us to renegotiate the very framework of negotiation itself at the same time as we are renegotiating some specific point of negotiation or contestation? All this may explain why intermediality is such an efficient, complex, elusive and exhilarating communicative strategy.

Notes 1. Anderson’s first performance appeared one year after Fassbinder’s death, which hardly seems coincidental. 2. R. W. Fassbinder (1980/2006) Berlin Alexanderplatz (Munich: Süddeutsche Rundfunk Cinematek). 3. L. Anderson (1986) Home of the Brave (Warner Bros). 4. A. Döblin (1929) Berlin Alexanderplatz, translated by E. Jolas (London: Continuum). 5. W. Nöth (2000) Handbuch der Semiotik, 2nd edn (Stuttgart: Metzler), p. 467. 6. C. S. Peirce (1931–1958) Collected Papers vols. 1–6, C. Harthorne and P. Weiss (eds), vols. 7–8, A. W. Burks (ed.) (Cambridge MA: Harvard University Press), 2.228. 7. W. Nöth (1990) Handbook of Semiotics (Bloomington: Indiana University Press), p. 42. 8. L. Santaella (2001) Matrizes da linguagem e do pensamento (São Paulo: Iluminuras).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

94

9. W. J. T. Mitchell (2005) ‘There are no Visual Media’, Journal of Visual Culture 4(2), pp. 257–66. 10. V. Colapietro (forthcoming) ‘Pointing Things out: Exploring the Indexical Dimensions of Literary Texts’ in H. Veivo, C. Ljungberg and J. D. Johansen (eds) Redefining Literary Semiotics (Newcastle-on-Tyne: Cambridge Scholars Press), pp. 109–33. 11. Nöth (2000) Handbuch der Semiotik, p. 436. 12. Mitchell (2005) ‘There are no Visual Media’, p. 258. 13. M. Changizi, Z. Qiong, Y. Hao and S. Shinsuke (2006) ‘The Structures of Letters and Symbols throughout Human History are Selected to Match those Found in Objects in Natural Scenes’, American Naturalist 167(5) E, 117–39. 14. See M. Nänny (2002) ‘Ikonicitet’ in H. Lund (ed.) Intermedialitet (Lund: Studentlitteratur), pp. 131–38. 15. C. Ljungberg (2004) ‘Between Reality and Representation’, Peirce and the Notion of Representation [Special Issue] VISIO 9, pp. 67–78. 16. M. Bal (2005) Travelling Concepts in the Humanities: A Rough Guide (New York: Routledge), pp. 175–6. 17. E. Fischer-Lichte (2005) Theatre, Sacrifice, Ritual: Exploring Forms of Political Theatre (London: Routledge). 18. E. Fischer-Lichte (2004) ‘Culture as Performance’, keynote lecture at the Symposium Performativity: A Paradigm for the Studies of Art and Culture in Copenhagen, 30 November. 19. V. Colapietro (1997) ‘Peircean Reflections on Gendered Subjects’ in C. W. Spinks and J. Deely (eds) Semiotics 1996 (New York: Peter Lang), pp. 178–88. 20. M. Bal (2003) ‘Visual Essentialism and the Object of Visual Culture’, Journal of Visual Culture 2(5), p. 24. 21. W. Nöth and L.Santaella (forthcoming) ‘Literature and the Other Arts: The Point of View of Semiotics’ in L. Block de Behar (ed.) The Role of Comparative Literature in the Sharing of Knowledge and in the Preservation of Cultural Diversity (Oxford: Eolss). 22. Cf. S. Moser’s (2007) analysis in E. Tabakowska, C. Ljungberg and O. Fischer, Insistent Images (Amsterdam: John Benjamins). 23. Döblin (1929) Berlin Alexanderplatz, p. 2. 24. Ibid., p. 83. 25. Ibid., p. 2. 26. W. Nöth (2007) ‘Self-Reference in the Media: The Semiotic Framework’ in W. Nöth and N. Bishara (eds) Self-Reference in the Media (Berlin: Mouton de Gruyter), p. 19. 27. L. Santaella (1997) ‘The Prephotographic, the Photographic, and the Postphotographic Image’ in W. Nöth (ed.) Semiotics of the Media: State of the Art, Projects and Perspectives (Berlin: Mouton de Gruyter), pp. 121–32. 28. Nöth (2007) ‘Self-Reference in the Media’, p. 3. 29. Ibid., p. 19. 30. Ibid. 31. Ibid. 32. Not only can digital art forms be accessed anywhere and at all times but they also demand interactivity on the part of the addressee. 33. Mitchell (2005) ‘There Are No Visual Media’, p. 262. 34. L. Leão (2005) Hermenetka, http://www.lucialeao.pro.br/hermenetka.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Christina Ljungberg 95

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

This page intentionally left blank

Part III

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Combinations and Integrations of Media

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

This page intentionally left blank

‘Media’ before ‘Media’ were Invented: The Medieval Ballad and the Romanesque Church Sigurd Kværndrup

My principal aim is to demonstrate a special variant of the spatiotemporal modality, which I propose to call the ‘numinous mode’, a result of the interface between a sacred and a secular space, a meeting that took place in the very Romanesque church building (cf. Elleström, this volume). Understanding this mode is necessary in order to grasp how a seemingly secular art form such as the medieval ballad could be created and performed in the church, as we can see happen in a Danish church painting from around 1300: the ballad dance in the church of Ørslev (see Figure 4). It would hardly be controversial to describe the ballad as a pan-European art form with roots in the Middle Ages, but I understand it as an intermedial art form in close connection with the church of the early High Middle Ages. This is controversial for at least two reasons. Firstly, one consequence is an early dating of the art form, whereas the tendency in much international ballad research over the last 30 years has been to deny the ballad’s ancient origins. Secondly, the ballad is normally understood as an aspect of a non-Christian sub-culture in this Catholic age.1 In the Middle Ages, most art forms were born as intermedial and were performed in direct contact with a public which often functioned as active co-artists in the performance. The church interior, which in cathedrals unites many different media and modalities into a grandiose unity, could probably be looked upon as an early mass medium, as it communicated its messages to many people at the same time, and not only through the spoken word. This way of thinking about the notion of medium is originally due to the works of Marshall McLuhan, which shall be considered below.

Understanding medieval media Before the publication in 1964 of Marshall McLuhan’s Understanding Media, the normal state of affairs in medieval studies was ignorance about media. Certain Nordic philologists began to use the notion in the late sixties in 99 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

5

Figure 4 The ballad dance sublime; the church of Ørslev, Denmark, ca. 1300

relation to, for example, Icelandic family sagas. This orientation towards the notion of medium, no doubt, was inspired by American media theory – regarded by some with considerable scepticism – while others found that McLuhan’s essay contained very interesting perspectives. What made this highly regarded work a new beginning in fields like media and mass communication? First of all, Understanding Media offered a universal notion for medium in defining it as ‘the extensions of man’, tools to prolong the physical and spiritual capacity of mankind.2 In this way, McLuhan built up his notion of media from the bottom, for not only the book, the painting, the radio and other electronic media are extensions of man, but so too are the wheel, the house and the bridge. In addition to this, he offered a provocative and inspiring motto, ‘the medium is the message’, which threatened to drive classical humanists mad. For humanists the content was the message. McLuhan defended his motto this way: ‘This is merely to say that the personal and social consequences of any medium – that is, of any extension of ourselves – result from the new scale that is introduced into our affairs by each extension of ourselves, or by any new technology’.3 Looking back upon the development in media technology over the last 30 years, one can hardly disagree with McLuhan that all of these new ‘extensions of man’ have carried with them new messages, new publics and new possibilities, as well as a radical change in culture, which became part of ‘the global village’.

The writing and the law No doubt McLuhan also has something to offer the study of medieval media reality, primarily in the ninth chapter of his book, titled ‘An Eye for an

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

100 Combinations and Integrations of Media

Ear’, which describes writing as a medium. Its main point is that the phonetic alphabet, which was re-diffused during the Middle Ages, is essential for the transition from local tribal societies to European feudalism based on commonly accepted law. Ever since the ninth century, manuscripts of law were produced in greater amounts, not for public enlightenment but as holy tokens, supporting the authority of thing-men who had knowledge of the powerful script. Authorities used the medium of writing to create common standards of law, administration of justice and, thereby, social order, based on feudal principles in the broadest sense. Apart from this historical aspect of Understanding Media, McLuhan’s greatest importance for the study of premodern culture and communication lies in the way he opens up a new field of studies by breaking down the division between what is normally called media and other forms of ‘extensions of man’, first and foremost in the field of architecture.4 It seems that McLuhan anticipates a main line, one also found in intermedial theory, which is characterized by this very broad sociological notion of media that are treated in rather mechanistic terms. Of course, this causes problems for any intermedial research that is interested in showing the boundaries between media and the interaction between them. Therefore, his approach requires some demarcation, for if we go back to McLuhan’s notion of media, it is generalized to a level where medium is more or less the same as the mediaeval notion of Ars (art and handicraft). Ars also covers the wheel, the house and the bridge – everything that man through his or her Godgiven, creative power could add to Natura, God’s creation. Even though the Middle Ages did not normally distinguish between craft (everyday creativity) and art (exceptional and highly qualified forms of expression), this is a necessary distinction from the point of view of media theory. More specifically, it was also with the rediscovery of ideas from Antiquity that efforts were made to develop a narrower concept of art in the High Middle Ages, based on the Greek doctrine of the nine muses, including those for the epic, dance, comedy, tragedy, pantomime and last but not least, the song-dance (Erato).5 Aristotelian aesthetics also enjoyed some diffusion towards the fourteenth century, and in Dante Alighieri we already see traces of this. In his De vulgari eloquentia, Dante supplies us with a very important key to understanding why ballad texts are not normally recorded during the Middle Ages: he compares balata with canzona, and says that the latter is a finer art than the former and it is therefore written down by ‘friends of the book’. The canzona has a text which may stand alone, whereas the balata needs plausores (dancers and clappers) to be performed.6 Thus, the ‘ballata’ was born as an intermedial construction, and in Dante’s eyes, it is, therefore, of less value than a real poetic text. In the twelfth century the handwritten, vernacular liber was spreading all over Europe and was later taken over by Scandinavian writers. The book to the modern mind is a mass medium. It is an interesting fact that ‘a book’ in old Scandinavian sources is not something that one reads from; in the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sigurd Kværndrup 101

almost totally illiterate North, the book was understood as a holy object on which one could swear an oath in front of the altar – in other words, it was also looked upon as a ‘medium’, but this was understood as a channel to the godhead.7 Thus, some notion of medium did exist and was used as part of the scholastic speculations about the spiritual world. It was obvious to everybody who lived in the ‘modern’ world, that is, in early feudal society, that there was a need for mediation to the faraway and sometimes invisible lords of this world – be they divine or human, mediation was the central element in communication. In early feudal society one met the vertical distribution of power through vassals and servants, mediating between power and people, and parallel to this in the spiritual world, one could find a special sort of ‘media’, the saints and not least the Virgin Mary, called the Mediatrix of all graces, an advocate contacted in prayers. Here we see the original meaning of medium, which, in Latin, means to go-between two extremes; medium is the bridge of communication between two parties who cannot communicate directly, such as servant and king, man and God. By using the Latin understanding of medium, it becomes possible to recognize a fundamental feature of McLuhan’s method, in which he stresses the fact that the notion of metaphor in Greek means ‘to carry over’ or ‘to transport’. From this perspective, the meaning of ‘metaphor’ becomes the same as ‘medium’ in classical culture, a bridge. For McLuhan, the inventor of the wheel was able to think metaphorically about running man, thus creating a new ‘extension of man’, and therefore he can write: ‘All media are active metaphors in their power to translate experience into new forms’.8 Medium and metaphor – the combination of Latin and McLuhan’s ideas shows how these two notions are closely related: they both have to do with the connection between a message, a channel and an interpreter. But whereas ‘medium’ in the Middle Ages primarily referred to a spiritual, even a transcendent, communicative function, the concept of medium in our time is material, perhaps to the extent that we are inclined to forget this original dimension of the concept.

The early medieval church as mass medium After the huge Viking halls had disappeared, the church buildings became the greatest ‘extensions of man’, though the notion of kyrka (church) comes from Greek kyriakon meaning ‘an extension of the Lord’ (kyrios). Its holiness was originally differentiated, however, into two very distinct parts: towards the east the choir with apse ascends, while the longhouse or nave (from the Latin word for ‘ship’) was directed towards the west and the sunset. In the early High Middle Ages, this division had particularly significant consequences for the church as a sort of mass medium: a rood screen or ‘triumphal wall’ divided the Romanesque church into two separate parts, as does the icon wall in contemporary Orthodox churches; the choir was accessible from

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

102 Combinations and Integrations of Media

the nave through a door which was opened at Eucharist and at Easter. Only the ecclesiastical personnel had direct access to the choir, which was, therefore, also called the presbyterium. Furthermore, the choir was conceived as a parallel to the Holy of Holies in the Jewish temple, but whereas the Lord was believed to be present in the Jewish temple, the choir was thought of as a bridge to the Lord, metaphorically the ‘mountain’ on which the Lord of Hosts revealed himself to Moses. Therefore the choir was, ideally, to be lifted a few yards above the ground and entered by steps, a metaphor for the celestial staircase which the patriarch Jacob beheld in his dream. In this way there was room for a crypt (from Greek kryptos, hidden), an intimate underground room with an altar, a saint’s sepulchre and often tombs for noble people; here, the laity might have access to a holy room – being a metaphor for the access Maria Magdalena had to the empty tomb of Christ. In the Jewish temple, access to the Holy of Holies was absolutely closed by a heavy curtain and barred by two cherubim, similar to those who guarded the gate to Paradise. Only the high priest entered once a year at the feast of atonement. To the Christians, admission was symbolically obtained when the curtain to the Holy of Holies was split from top to bottom at the death of Christ, thus reopening the gate to Paradise (Matthew 27:51–5). The aristocratic paintings in the Romanesque choir would normally depict the creation, the history of salvation and the Kingdom of God coming to Earth. The only choir painting that was sometimes visible to the common congregation represented the Majestas Domini, the victorious Christos Pantokrator, sitting in a mandorla on the rainbow above the globe as the true ruler of heaven and earth, with his personal weapon, the book, in his left hand, and reading ‘Sum Via, Veritas et Vita’ (John 14:6), his right hand lifted with a typical rhetorical gesture, teaching the world a new road to truth and life – by use of the alphabet. Thus, while the idea behind the Romanesque choir was deeply rooted in Mosaic religion, the nave was a typical Roman edifice. It was normally built on the model of the secular congregation hall of antiquity, the basilica, since the prince or emperor (Greek basileus) had raised seats at the western end of the nave, while there were no chairs for ‘the people on the floor’. The nave functioned as a congregational house, and there was access for both men and women. The sexes were separated, however, and each had an entrance: the north door for women, the south door, which was through the porch, for men. In the western end of the nave stood the font, whose natural waters gave the congregation access to life and to the common ‘ship’, where the priest delivered his sermon in the vernacular, while the mass was held in the holy language, Latin. The nave was not permeated with the same sacred secrets as the choir; we know that the nave was also used for meetings, feasts, defence and for chorea (antiphon combined with circle dance). In this division of the church into two parts (largely forgotten today) lies the explanation of why carols

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sigurd Kværndrup 103

104 Combinations and Integrations of Media

and ballads might be performed in the church itself. This is only revealed, however, in the later pictorial development of the nave during the Gothic period.

In the Romanesque period (before 1200), the nave was mainly decorated with stylized sacred paintings, and they mostly told stories from the Old Testament which were ante-types for the ‘new covenant’. Additionally, the nave often showed pictures of the 12 apostles and the four evangelists. The pictures were clearly inspired by the philosophy of the period, conceptual realism.9 With the more powerfully ostentatious Gothic church, drafting the church interior as a more or less holy unity, the building of bipartite churches ended – and, with that, the construction of crypts. In many church naves, the old wooden ceilings were replaced by vaults, giving the nave a choir-like character, and the whitewashed walls offered room for new paintings. The triumphal wall was partly broken down, leaving a triumphal arch of sorts, upon which Christ was seen judging mankind at Doomsday, often with a dramatic chasm of hell flaming to the left and to the right the peaceful city of Jerusalem reserved for the redeemed. In the Gothic period, not only the building but also the function and ideology of the church decoration changed fundamentally. With a popular didactic purpose, the paintings become intermedial in a new way. They now describe human life as an eternal struggle between good and evil, between virtues and vices, and the intermediality is completed with bands of text, which interpret the pictures with pedagogical messages. Thus, it might be said that the Gothic style of decoration took its departure from everyday life, as did, in fact, nominalism, the typical philosophy of the later Middle Ages. At the same time, the Gothic decorations lent weight to secular authorities, concerning the formulation of common laws. The pictorial programmes behind the new decorations were, to some extent, international, such as Biblia Pauperum. Certain local, vernacular stories might also be used as sources, especially at the western end of the nave and in the porch where the road back to ‘the World’ might be anticipated with frescoes of the conditions ‘out there’, where Satan was lord. Here, elements from the grotesque world of the jocular ballads may be seen. During a period around 1300, however, one could also see paintings of a virtuous lifestyle, which might have been borrowed from knightly ballads and novels.

The numinous mode and the sublime The Romanesque church interior could thus be described as a compound medium, built of architectural metaphors and sacred icons. By combining

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

The essential features of the iconic development

the four modalities – the material, the sensorial, the spatiotemporal and the semiotic – within the same church, a special mode of numinous experience is created: a sensorial perception beyond words and, therefore, perhaps a fifth modality arising out of the meeting between sacred and secular space. In looking at the material and sensorial modalities in the church, one experiences an enormous and static space, unlike in any other building. The windows are small, so the sensorial impression of the interior is dark and more intimate than the impression given from the outside – in accordance with the wish to keep a solemn and normally silent atmosphere that could sometimes be broken with the aural impression of plainsong. The church was never heated, so in winter the congregation would not only have to get used to the darkness but also to the cold. The paintings became gradually visible, opening up a unique spatiotemporal reality as the supplicant’s eyes adjusted to the darkness, and this widened the spatial perspective with a fourth and temporal dimension: most of the paintings portrayed holy people from the past and told stories from the Bible; a few gave contemporary impressions, but the most alarming were those glimpses of Doomsday images from the future. While the sacred space in the choir was mastered by the educated elite, the nave was for lay people. The elite lived in a meditative and somber reality, which was almost as static as the building, while the laity was part of a movable and transitory physical world. Through the congregational choros and antiphon in the nave, dynamic life was sometimes given to the church, but in a stylized manner that combined the static spatial order with a temporal perspective, simply by telling stories that were not ancient, but that mirrored contemporary life, as did many church paintings. The ballad is one of these art forms that could be performed in this setting, sub limes, meaning under the border to the holy space and to God.10 The church also gave room to a rich semiotic modality, a symbolic dimension expressed in a holy order defined through conventional signs, assembled in an all-encompassing symbolical dimension, which was substantiated in the philosophy of conceptual realism. This brings Platonism to mind, and historically this way of thinking was indeed partially rooted in Hellenistic Neoplatonism. Its basic idea – expressed in Anselm of Canterbury’s famous proof of God’s existence – is that mankind, made in God’s image, is able to create and live by certain concepts and ideas that are even more real than earthly reality (hence conceptual realism).11 As a fallen creature under the influence of Satan, man may also dedicate himself to certain false ideas, for instance, one which states that God does not exist. This realism meant that earthly conditions were understood in analogy with those of heaven – with certain reservations, stemming from original sin and the unspeakable greatness of God. This duality of truth and illusions, or misconceptions, is depicted visually in many churches – and in ballads.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sigurd Kværndrup 105

The intermedial relation between choir (‘heaven’) and nave (‘Earth’) may be interpreted from this observation: the nave is a multimedial and multimodal space that is not holy itself, but gains its special atmosphere and meaning from its meeting with a sacred space, thus creating the numinous mode. This ‘medium border’ inspires man to prayer, incantation and thanksgiving, and such spaces may function as performance arena both for the enacting of everyday stories (ballads), for misconceptions and ideas concerning the Kingdom of God that the congregation had or was taught by the priests and auctores. The road from idea to action (song, drama, art) passed through realist philosophy, being an invisible, spiritual medium that forged a bridge from heaven to Earth. An example may shed light on this notion: music up to around 1300 was also built on the philosophy of conceptual realism. The rich ecclesiastical singing imitated two sorts of heavenly song: firstly the angelic choirs, imagined to be performed with polyphony, secondly the spherical music of the seven planets.12 Both musical ideas also inspired European composers, throughout time, to compose sublime, polyphonic music.

The ballad and the problem of cultural interpretation As an art form, the ballad in the Faroe Islands and the Balkans today combines antiphonal singing, epic poetry and ring dance; it is, in short, inherently an intermedial art form where song and text are always integrated, while there is some uncertainty as to whether the dance was originally integrated or just combined with the songs. My position is that the ballad is originally and primarily a choral dance (Greek: choros), closely related to the carol, a French term that indeed derives from choros. I understand the ballad as a total work of art, a Gesamtkunstwerk, which integrates four arts that we would probably call media today: chain dance, antiphonal song, poetry and image.13 The majority of the East Nordic ballads belong to the genre called ‘ballads of chivalry’, their narrative universe seemingly links them to courtly culture, a culture that has been viewed as being relatively independent of the Catholic Church. If the Middle Ages was a non-secularized period, then, the ballad was interpreted as one side of a residual and relatively secular and vernacular aspect of medieval culture. This view seemed to be fully corroborated by another ballad genre, the jocular ballad, which does not appear to have any affinity at all with Catholic piety; on the contrary, since M. Bakhtin this genre has been portrayed as an impious, carnivalesque culture of laughter.14 How, then, should we understand the ballads of the supernatural, which often have a Christian theme, to say nothing of the legendary ballads, which can even have the distinct elements of the late medieval sermon? This dichotomous way of interpreting medieval artefacts is outmoded – as medievalists will know; but it is far from being eradicated, least of all in

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

106 Combinations and Integrations of Media

ballad studies. For my own part, it was a lecture about the carnival, delivered by Bishop Joseph Ratzinger at the Saga Conference at the University of Munich in 1976, that was the most important eye-opener. Here, the future Pope Benedict XVI interpreted the carnival in biblical terms and based on the Catholic tradition, which created, and still energetically maintains, this festive practice. Since ancient times, it has comprised customs such as noisy processions whose participants can sometimes perform naked or dressed as absurdly as they please, while vigorously beating empty barrels and shaking rattles. They thereby illustrate the anti-numinous point of the carnival, which is to show off the naked, biological human being, without Christian charity and without knowledge of God. Bishop Ratzinger demonstrated that this pre-Lenten festival of the forsaking of meat (carne vallo) has its biblical foundation in the words of Paul in 1 Corinthians 13: ‘Though I speak with the tongues of men and angels, and have not charity, I am become as sounding brass, or a tinkling cymbal’. Obviously, it was not the Church that invented the pagan art forms through which the carnival expresses itself, but the relationship illustrates that the Catholic Church in the Middle Ages was surprisingly spacious.

Choros as an ecclesiastical medium The choir was a kind of mirror of heaven and when joyful events happened in heaven, such as the coronation of Mary as the heavenly bride of Christ, it was written that the angels danced circle dances (chorea) in joy. As an analogy to this, the congregation could dance chorea of joy in the nave, which was seen as the antechamber of paradise.15 Does that mean that people could dance and sing carols in the church itself? Certainly; but such activities were forbidden by numerous councils and bishops, revealing that they did occur anyway.16 Carol (from Greek choros) integrates circle dance with a text that is sung in interaction between one or two solo voices, and burdens that are performed by all as a choric response to the solo. The strophic Nordic ballad genres are normally of exactly the same form. The response burdens may emphasize a particular point of view on the outcome of the story and contain more or less hidden references to the context of the performance.17 At Yuletide, this dance activity was especially widespread. One well-known Christmas carol, ‘A Child is Born in Bethlehem’, has been translated from a medieval antiphony and is performed in the East Nordic rhymed ballad form of the short-lined type, a couplet with two burdens and with four rhythmic beats in each line, alternating with three beats. As a form, this strophe is also known from popular legendary ballads like ‘Jungfru Maria och Jesus’ (‘The Virgin Mary and Jesus’), which was widespread over Scandinavia. The idea that governs this ballad is that the child Jesus only grows up and recognizes his mission on Earth the moment he runs away from his mother. She is

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sigurd Kværndrup 107

108 Combinations and Integrations of Media

anxiously chasing around, searching everywhere, to find her boy (here the E version from Västmanland in 1830):

–Låfvadt vare Guds heliga namn! (burden with four beats) (Praise to God’s holy name) Der födde hon Jesum kär sonen sin (four beats) (There she gave birth to Jesus, her dear son) –Och så den helige and! (burden with three beats) (And likewise The Holy Ghost) E. L. Backman demonstrates in his Religious Dances in the Christian Church and in Popular Medicine (1945, English version 1952) that the ecclesiastical choir-dances may be traced back to the Christian church of Antiquity.18 They had their roots in Greek and Jewish festive traditions but there are also examples from the New Testament that might serve as models, one of these being in St Luke’s Gospel (7:32) where Jesus says to the Jews: ‘We have piped unto you, and ye have not danced’. One of the strengths of Backman’s thesis is that he has read the scriptures of the Fathers and translated their statements about dancing. It is quite clear that most of the Fathers are ambivalent, for though dancing is a pious way of expressing the joy of God, according to the Bible, how it is practiced by former European pagans is another matter – particularly its content and effect. Dancing for joy in heaven is found in a church painting in the church of Ørslev in southern Zealand from around 1300. Here, an open chain of beautiful virgins and noble knights is dancing the choros before Christ and the Virgin Mary, who are symbolically united in the Holy Marriage. The painting very clearly connotes the universe of the ballad, for on the next wall the same painter has shown vernacular wooing and wedding images from the motif world of the knightly ballad. The great Swedish painter Albertus Pictor (ca. 1440–1507) painted such ring dances in three Upplandic churches: 19 in Husby-Sjutolft one sees the same sort of dance of joy, but here the occasion is the birth of Christ. The ecclesiastical choros must have been one of the origins of the medieval ballad, but certainly not the only one, and time and again the Church turned against the art form and banned it. However, this should not lead ballad researchers to forget that this old intermedial art form also had its place in the church.20 Indeed, the folk ballad in stanza-form was originally an ecclesiastical medieval carol. Like the many people originally baptized in the church and who later left it, however, the ballad also parted from its ecclesiastical origin. The carol integrates chain dance with a text that is sung in interaction between one or two solo voices, and burdens that are performed

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jungfru Maria till Bethlehem gick (four beats) (The Virgin Mary went to Bethlehem)

by everybody. The folk ballad makes exactly the same performance. When dealing with the early media of the Middle Ages, one has to take into account primarily that they were born as intermedial forms and, secondarily, that integration and even transformation of media from one cultural sphere to another was of much greater importance than one would imagine today – the Church being the most important melting pot for media exchange before the Late Middle Ages, when the market place took over.

Notes 1. I have at least one forerunner in this undertaking, namely the American ballad scholar L. Pound (1921) Poetic Origins and the Ballad (New York: Macmillan). Louise Pound took the standpoint that the earliest Anglo-Saxon ballads were of ecclesiastical origin. 2. The full title of the work is: M. McLuhan (1964) Understanding Media: The Extensions of Man (New York: McGraw-Hill). 3. Ibid., p. 7. 4. Cf. C. Dreyer (1997) ‘Architecture as a Mass Medium’ in W. Nöth (ed.) Semiotics of the Media: State of the Art, Projects, and Perspectives (Berlin and New York: Mouton de Gruyter). 5. Cf. U. Eco (1986) Art and Beauty in the Middle Ages (New Haven CT: Yale University Press). 6. Dante Alighieri (1966) De vulgari eloquentia: Über das Dichten in der Muttersprache (Darmstadt: Wissenschaftliche Buchgesellschaft), p. 54. 7. See further description by T. Damsgaard Olsen (1984) in S. Kaspersen, S. Kværndrup, L. Lönnroth and T. Damsgaard Olsen, Dansk litteraturhistorie 1. 800–1480 (Copenhagen: Gyldendal), p. 13. 8. McLuhan (1964) Understanding Media, p. 57. 9. See M. H. Carré (1946) Realists and Nominalists: An Introduction to Medieval Views of Knowledge (London: Oxford University Press). 10. It is quite likely that through that numinous mode, art (in the modern sense of the word) was separated from ars, thus creating that gulf between handicraft of lower and higher or even sublime value, which is so characteristic of European culture since the Renaissance. 11. The proof is very short: ‘And, indeed, we believe that thou art a being than which nothing greater can be conceived’: Saint Anselm (1962) Basic Writings, translated by S. W. Deane, 2nd edn (La Salle IL: Open Court Publishing Company), p. 7. 12. One of the first examples of imagined polyphony is depicted in Le chevalier au lion by Chrétien de Troyes around 1180, in the scene where a knight is experiencing a miracle by a magic fountain. 13. These may be based on ekphrasis of everyday images, produced by women in different visual media. I argue to this effect in S. Kværndrup (2006) Den østnordiske ballade – oral teori og tekstanalyse: Studier i Danmarks gamle Folkeviser (Copenhagen: Museum Tusculanum) (The East-Nordic Ballad: Oral Theory and Textual Analysis). Danmarks gamle Folkeviser means ‘The Ancient Popular Ballads of Denmark’, vol. 1–12, 1853–1972 (Copenhagen: Universitets-Jubiläets danske Samfund). 14. Cf. O. Solberg (1993), Den omsnudde verda: Ein studie i dei norske skjemteballadane (Oslo: Solum) (The World Upside Down: A Study of the Norwegian Jocular Ballads) and T. Olofsson (2008) ‘Den magstarka käringen: Om roliga

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sigurd Kværndrup 109

15.

16.

17.

18.

19.

20.

medeltidsballader’ (The Unbelievable Woman: About Funny Medieval Ballads) in G. Byrman (ed.) En värld för sig själv: Nya studier i medeltida ballader (Växjö University Press) (A Special World: New Studies in Medieval Ballads). Each ballad type is a unique organism, and each ballad version is a unique poem. In B. R. Jonsson, S. Solheim and E. Danielson (1978) The Types of the Scandinavian Medieval Ballad (TSB) (Oslo: Universitetsforlaget), p. 14, the editors write: ‘A ballad is an “idea” in the Platonic sense; there is no archetype; from the point of view of the folklorist, every text is as good as any other.’ Cf. C. Sachs (1937) World History of the Dance (New York: W. W. Norton & Co). P. Bourcier (1978) Histoire de la dance en Occident (Paris: Le Seuil), gives many examples of ecclesiastical prohibitions against carolling in church, beginning as early as 774, when Pope Zacharias decreed a ban against ‘les chants et caroles des femmes à l’église’ (p. 52). Bourcier adds that the persistence of the bans proves the persistence of the dance (p. 53). In the ballad content the dual reality is very often pictured, and in quite a few burdens the duality between ‘world’ and ‘heaven’ is expressed, for instance in SMB 48, where the first burden sings, ‘Om sommaren’ (In summertime), and the second ‘I himmelen är en stor glädje’ (In heaven there is great joy). SMB is short for B. R. Jonsson, M. Jersild (tunes) and S.-B. Jansson (1983–2001), Sveriges Medeltida Ballader 1–5, (Stockholm: Almquist & Wiksell International). E. L. Backman (1945) Den religiösa dansen inom kristen kyrka och folkmedicin (Stockholm: Norstedt). English version: (1952) Religious Dances in the Christian Church and in Popular Medicine (London: George Allen and Unwin). In Härkeberga and Täby the paintings are rustic ring dances around the golden calf; the hand position of the dancers reveals that they are dancing a ballad. Albertus Pictor has painted four clear motifs from ballads in Floda church. Some researchers of ballad tunes saw a near relationship between ecclesiastic tonalities and some ballad tunes. The relationship may be interpreted to the effect that the origin in church music can be glimpsed behind several generations of singers who have transformed the tunes beyond recognition.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

110 Combinations and Integrations of Media

6 Håkan Sandgren

In field guides the mediating practices used are called media combinations (see Lars Elleström and Irina Rajewsky, this volume), that is, a multifaceted medium which contains two or more kinds of media modes organized and combined in such a way as to create a certain perceptive and cognitive effect. It is therefore not of any major relevance in this context to speak of border crossings, due to the fact that the genre makes use of the differences and delimitations of the media contained in it, rather than transgressing or dissolving them. The media transgressions made while using a field guide nevertheless seem mainly to be a side effect of our way of perceiving the media modes it contains. Our attention wanders from the object we would like to identify to image, text, maps and out to the environment surrounding us, and eventually back to the physical object we are trying to identify, in a process that at least superficially seems to be related to the cognitive activity triggered when decoding a page containing both text and image.1 That the field guide should be considered an art form is a more dubious proposition, because the purpose of a field guide is never restricted to evoking aesthetical effects or giving the user an artistic experience by means of different media. If a field guide, nevertheless, in some way or other, should be considered a work of art by a recipient, this must be seen as a side effect, not as its primary aim. It is, though, in constant change; primarily the changes affect those aspects of the medium which Elleström in this volume calls ‘contextual qualifying aspects’ and ‘operational qualifying aspects’ due to the fact that the field guide has become a necessary part of a social context, for example, the birdwatching community. Birdwatchers have created a discourse including a qualitative stratification of the different items of the genre due to their interest in how well the field guides relate to current research and to contemporary reports of bird observation. Apart from this, the genre is also defined by an instrumental, or rather operational, multimodal form, which is a necessary condition of the genre to function as intended: as a medium used for identification of natural objects in the field. 111 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

The Intermediality of Field Guides: Notes Towards a Theory

That said, it is obvious that studies of field guides could be carried out in the same way as we study other forms of intermedial and multimodal forms of communication. Owing to the historical and popular dominance of bird guides, this essay is based mainly on examples taken from this quite abundant sub-genre. Hopefully, this will not affect the general conclusions being made, for, in any field guide, we will find combinations and interactions of images and prose, describing the object in a simple way, sometimes in narrative form, with genre-specific types of verbal figures. Description is by way of a visual, textual or auditory medium invoking different kinds of cognitive processes and experiences of a physical reality, as convincingly and unambiguously as possible.2 The field guide typically contains a short introduction, a user’s guide to the best way to identify the objects in question. If the field guide is a guide to flora or fauna, the evolutionary relation between orders, families, genera, species, sub-species and races is explained. In the introduction we will probably find images, illustrations, maps and sometimes sonograms, the latter serving as visual explications of bird song.3 Field guides of animals and plants are often organized in line with a taxonomical order based on the interrelation between the different species. Although based on scientific research, intradisciplinary discussions are rare in field guides, rather, recent research results tend to ‘trickle down’ to the books, however slowly, and sometimes randomly, and will sometimes be included in new, revised editions.4 The guides are often equipped with so-called keys; this is especially true of plant guides, where keys present stylized images to illustrate similarities and differences in the shape and form of the different kinds of leaves and flowers. For the ordinary user, identification is equivalent to naming the object. Scientific naming of birds follows a strict but ever-changing nomenclature,5 but for the common user the popular name is often the most often searched for, a fact that has encouraged the authors of guides to ‘invent’ popular names for organisms. The argument of this essay is that the intermediality of a field guide is based on the cooperation, dialogue and mediating process between image, text, map and transcription or interpretation of bird sound; mediated description is connected to the natural ‘field’. Not all of the mediated aspects need to be simultaneously activated by the reader and observer, but the user has to take part in an intermedial process and make use of all different kinds of media in a field guide, while he or she compares the medium with the physical reality. For a field guide to work properly we must assume that it combines in a pragmatic way different media modes in an act of gaining the highest possible informative effect. As Elleström proposes in this volume, media could be understood in terms of four kinds of modalities, and the field guide is very much defined by the semiotic modality, but as is stated in this essay, it is a genre of combined media, taken into active use in the field,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

112 Combinations and Integrations of Media

which means that some specific kind of material mode also must be associated with the genre. This is due to the fact that in the case of the field guide the function, not the aesthetic value or effect of it, is of primary interest. If such a standpoint is taken, the reader of the field guide will first and foremost become a user who by cognitive processes treats the different kinds of information which text, image, transcription and maps are communicating, in such a way that he or she succeeds in his or her goal of identifying the object.

The image The image serves a certain purpose: to assist the user in identifying a specific species. Because of this, distinctiveness is of the utmost importance. Images in bird guides often take the form of black-on-white silhouettes showing typical traits such as the shape of a wing or tail.6 Drawings in black and white may be used as illustrations to a piece of narrative, as part of ‘how to use this book’ passages7 or to give an approximate image of a very rare species.8 The main bulk of images in a field guide, however, are paintings – in watercolour, oil, gouache or acrylic. In older guides the artist often used stuffed birds or preserved skins as models; today, live birds, caught for ring marking or observed in the field, are preferred. Birds in different variants of plumage are illustrated due to the fact that they could be confused with other, similar species, or differ considerably within the species. Variants of plumage used in a field guide include adult, sub-adult, male, female or wearing breeding plumages, winter plumage, moult and so on; some common birds, such as the mallard, show a definite difference between male and female plumage during breeding season. Other birds are similar, regardless of sex, but carry a winter and summer plumage, and different kinds of plumage at different ages. The object could be placed in a stylized natural habitat to give the observer an idea of where and how the bird is most likely to be discovered; more commonly the object is seen against a neutral background. The most frequent way to present a species visually is by a so-called ‘mug shot’.9 In a ‘mug shot’ the bird is seen in profile against a neutral background. Certain species may be painted in motion – for example, ducks, which on the plates seem to swim upon a water surface, a position that obscures their legs from view.10 Some images take into account a certain wing or tail pattern which can only be observed when the bird is in flight.11 The way the artist uses light is very important; objects must be shown in an even light to facilitate identification and the artist’s choice of perspective thus becomes an interpretative act which differs from the way a nature painter would choose to construe and depict a natural object as, for instance, partly dissolved, out of focus or in shade. Some artistic possibilities are definitely excluded for the artist illustrating a field guide: an artist may create an image of a moving bird in its natural habitat, but he or she cannot take such liberties as to cover parts

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Håkan Sandgren 113

of the object or to deviate from what we may call the standard observation position. Images in field guides are referential and they guide identification – that is their main purpose. Identification is, in turn, based upon recognition, an act in which the observer must be able to connect the visual artefact to the natural object. This is why distinct and separating traits are highlighted in the image and sometimes noted by a discreet marker, such as an arrow or a thin line.12

The text The style in field guides is matter-of-fact and the text has to act together with the images to make identification possible. This does not stop the author from using rhetorical tricks, sometimes quite artfully, or using narratives.13 A standard text accompanying an image gives us facts about the length of the bird from beak to the end of the tail, wingspan and whether there is a considerable difference in size between the sexes. Short notes indicating how common or uncommon the species are, following a defined categorization, such as ‘common’, ‘scarce’ and ‘very rare’, are necessary. Description of the bird’s habitat is essential, as is information about its most common mating localities. The text is often supported by a map which shows the ranges of the species, during breeding season or during migration. To serve its goal, the text is primarily neutral and descriptive. Sometimes the writer, when verbally mediating the bird’s behaviour, gives it poignant, human-like traits. The bird could also be associated with concepts of conscience or states of mind. In this particular context, anthropocentrism and anthropomorphism signify the way the text describes the bird as a part of an environment dominated by humans. Anthropocentrism can take many forms, but its main purpose is to describe bird behaviour in such a way that it is possible for us, as Homo sapiens, to understand it. Identification of a species is facilitated by knowing how the bird will react when a human is in the vicinity; the difference in human-induced behaviour could help us determine which species we encounter.14 Now and again we will find examples of animalcentrism in a bird guide, a concept that in this context would try to help describe the behaviour of the bird in the presence of other birds (for example, raptors). The text often contains a verbal interpretation of the bird’s behaviour, appearance, qualities and particularities. It has to be as transparent as possible by creating a verbal representation of the natural object, sometimes with the help of tropes and figures typical of the genre. This leads the writer to focus on differentiating details and traits, thereby helping the user to identify the species by separating it from other similar ones. ‘Field marks’ – prominent and distinctive details in the plumage, the shape of the bill, song or calls – make it possible for the observer to separate one species from another.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

114 Combinations and Integrations of Media

Håkan Sandgren 115

A common feature in bird guides is the writer’s attempt to translate or transcribe the sounds of birds into verbal equivalents or onomatopoetic combinations of vowels and/or consonants, put together in such a way as to make it possible for the user to compare his or her sound impressions with the phonetic effects invoked by the arrangement of letters. Transcription of bird song is a subject under constant discussion,15 and verbal representation of bird calls has a long history, for instance, in poetry.16 When the sound of the call or song is described rather than transcribed, the use of metaphors and similes is predominant in the verbal rendering. This should not come as a surprise considering that the ordinary usage of these figures tends to establish a resemblance between two objects – in this case the authentic sounds of the bird and the textual ‘picture’ created by the writer to conjure up these sounds in the mind of the reader. The metaphors and similes most often used are taken from the sphere of nature; surprisingly often a comparison is done by using other birds’ calls, a feature that seems to presuppose a thoroughly ‘informed reader’. Occasionally the vehicle of the metaphor is taken from culture, such as ‘unnatural’ sounds (‘rubbing rubber tubing’), musical instruments (‘jarred trumpet fanfares’) or sounds made by humans (‘a laughing tremolo’).17 In cases when transcription is made, the interpreter makes use of the onomatopoetic potential of letters. Transcription is based on more or less successful representation of natural sound (for example, ‘ksch i’i’i’t’,’küll-küll-küll’, ‘ptrr’).

Examples discussed The two examples chosen as a ground for a discussion are generally considered to be the most important of Swedish field guides. A classic Swedish field guide is Erik Rosenberg’s Fåglar i Sverige [The Birds of Sweden]. The popularity of this book has led to a number of revisions since it was first published in 1953 and new species have been added continuously. The illustrations were executed by the famous Swedish nature painter Harald Wiberg, an artist in his own right.18 Apart from the fact that Rosenberg’s book was both a result of and a cause for the rising popularity of birdwatching in Sweden, it became part of the environmental movement. In this book, Rosenberg, as he did in his other work, pointed out changes in population range that he knew were due to the impact of mankind on the environment.19 It is of some importance to point out that this field guide, like almost every other field guide, limits its scope to a well-defined geographical-political unit. Rosenberg’s book contains a preface, a piece on the habitats of birds and a survey of taxonomy. The main part of the book presents the birds on colour plates with texts accompanying the images. A curious feature is the particular chapter in which Rosenberg (1972) addresses the reader and urges him or

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Transcription of bird sounds

her to inform the writer of observations concerning the red-flanked bluetail (Tarsiger cyanurus), a rare bird that had recently been reported, though with some uncertainty. Indeed, I have seen no similar passage in any of the field guides with which I am familiar, but it informs us of the kind of communication necessary to make a guide work. The reader (that is, user) is addressed in a way that presupposes a committed user, who goes out in the field to observe and identify natural objects and thereby, in one way or another, put them into some kind of mental archive.20 Lars Jonsson’s Fåglar i Europa med Nordafrika och Mellanöstern [Birds of Europe, with North Africa, and the Middle East], the first edition of which was published in 1992, differs in many ways from the work of Rosenberg, as will be discussed shortly. It is important to note that Jonsson is both illustrator and writer, thus creating a closer affinity between image and text. In the Preface, Jonsson stresses that he has taken on the subject of writing a field guide mainly from an artist’s perspective.21 As the title indicates, Jonsson’s field guide covers a larger geographical area than Rosenberg’s and as a result more species are included. Even though the geographical boundaries are somewhat arbitrary, the question of political borders is of no importance here. In addition, Jonsson’s field guide is more in line in its scope and ambition with important international ones such as Birds of Eastern and Central North America and Collins Bird Guide. Jonsson’s field guide goes into greater detail than Rosenberg’s for two reasons: it covers a larger area and it strives to satisfy the demands of current birdwatching, a popular pursuit that is in constant development.

Images compared To study different kinds of image usage in field guides, I will take as examples the pages in Rosenberg and Jonsson showing the pintail (Anas platyrhynchos) and the (Northern) shoveler (Anas clypeata). Wiberg’s illustrations are plain ‘mug shots’ and are apparently modelled on stuffed specimens or photographs.22 The eight birds (on the single plate there are seven different species) are seen in profile against a white background, floating tranquilly on an imagined surface of water and arranged so as to appear at the same distance from the viewer, which makes differences in size apparent on the page. A kind of symmetry is created by the fact that four birds seem to swim to the left of the page, while the remaining four swim to the right. All birds, apart from a female mallard, are male specimens clad in breeding plumage. Names in the vernacular in capital letters appear on the page, as do the signs for male ♂ and female ♀. Jonsson’s images are of a totally different kind.23 Seven birds are seen on the page, but they are of only two species – pintail and shoveler – seen in different kinds of plumage. Symmetry is achieved by letting the four centrally positioned birds – male and female pintail and male and female shoveler in eclipse plumage – swim in opposite directions. The

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

116 Combinations and Integrations of Media

image signals depth by presenting two pintails in the upper part of the picture, as if they are swimming at some distance away from the other birds. The most significant part of the page is the image of the male shoveler in breeding plumage. Distinct from the other birds, it stands on the shore showing its red duck’s feet. Its head is turned towards the onlooker, which makes its already prominent beak even more significant. As a way to underline movement and realism, Jonsson has placed the objects in a sketchy, but distinctive habitat – a lake and its shore – and the conceived movements of water are underlined by a stylized surface, rings in the water and reflections. The two examples chosen show how field guides can vary in the way they present their visual material. This material can be of a very neutral kind, without any artistic ambition, or it can come close to a work of art. The function of the image, though, is always to make identification possible.

Text comparison An intermedial relation exists between text and image when different categories of media are not only combined with each other, but also work as complementary factors; the text focuses on differential visual patterns, and the image draws attention to traits commented on in the text. A plate in a field guide is always in need of this kind of intermedial interaction. Let us see how a common bird – the European robin (Erithacus rubecula rubecula) – is verbally presented in the books studied. In Rosenberg the bird is described in the following way: ‘The bird is curious and will happily jump out of the shrubbery to peacefully observe the human . . . Sometimes he curtsies . . . At dusk he is at his most spirited humor’.24 We can notice how the writer in the cited passage is using anthropomorphic expressions which mediate the bird into a human context. This device could be explained as being a way to enhance the possibilities of identification, enabling the observer to name the bird with the help of verbal description. Possibly this is facilitated if the bird’s behaviour is vivified by comparing it with human behaviour. Identification of the European robin seldom presents a problem, but occasionally the bird is hidden and detection must be done by way of recognizing its calls. Here a comparison between Rosenberg’s and Jonsson’s ways of song transcription and interpretation will be carried out, starting with Rosenberg: The song of the robin is instantly recognizable; it resembles a musical fantasy, a few high-pitched, extorted chirps, followed by a long, extended, trembling, quivering coloratura, which after a while almost dies away, and then, a sudden shake-up and after that a pearl-string of crystal clear diminutive noises.25 Note how the passage starts by pointing out the song as ‘instantly recognizable’, a conventional concept in this particular context. The way the bird’s

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Håkan Sandgren 117

specificity is pointed out makes it easier for the identifier to reduce the range of possibilities. A description of the song follows, finding its rich metaphors from what could be a haut-bourgeois musical performance where the bird song is compared to classical music (‘musical fantasy’, ‘coloratura’) and to the accessories worn by the conventionally dressed audience, and found in the environment of the event (‘pearl-string’, ‘crystal’). The sound of the bird has little to do with music, but the metaphorical transformation of the sound makes it possible for the listener to recognize the bird, provided the fanciful description is interpreted by an imaginative and creative mind, that is. Jonsson’s description is more low-key: Its song is unmistakable; a murmuring rivulet of crystal clear tones with capricious changes of meter. Of a defining character is a discreet ticking noise: ‘tick’, often repeated ‘tick-ick-ick . . . ’ as when a mechanical plaything is wound up, often heard from a shrub . . . Also has a thin, sucking ‘siih’ and from night time travellers a thin ‘sie’ or ‘sisie’.26 In this passage the simile makes use of natural elements that act as vehicles in the comparison (‘rivulet’). This places the bird in a different context from the Rosenberg example. Jonsson, too, is using musical metaphors by pointing towards changes in meter, and to mechanics. ‘Crystal clear’ is a metaphor which finds its comparative point in a natural object, but this phrase being a cliché, connoting certain kinds of singing (bel canto, for instance), it is associated with a cultural artefact. Of further interest in this example is the way this very problematic transformational mediation develops some kind of intertextuality, based on a given set of metaphors and similes. Both Rosenberg and Jonsson use the word ‘crystal’ to describe the characteristics of the European robin’s song, showing us a kind of verbal tradition or conventionality even in this genre. In the cited passage by Jonsson we can also notice the anthropomorphic and anthropocentric traits. It is informative to compare Rosenberg’s and Jonsson’s transcriptions of a quite different and more exclusive species than the robin. The chosen bird is a warbler, a shy and quite plain kind of bird that is seldom seen, which forces the listener to identify the species by its characteristic song. This is how Rosenberg describes the song of the icterine warbler (Hippolais icterina): As a singer the Hippolais is in the same advanced class as the bluethroat and the marsh warbler; he’s a virtuoso, and a composer of masterful strophes. The song consists of short, swift melodies, e.g.: t r a l l a l í t r a l l a l á mixed with mimicry of other species’ sounds, such as the barn swallow’s s i f l i t , the common tern’s k i t t , the chaffinch’s f i n k , the oystercatcher’s k u b i k , all of them repeated several times. From time to time a chorus is inserted, a scraping g í e g í e g í e (with the i-note similar to the sound

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

118 Combinations and Integrations of Media

Håkan Sandgren 119

The text contains many verbal figures and tropes which characterize this kind of mediation from one medium to another, which invites the author to use a kind of verbal equivalent to translate the song into words. As a start we may observe Rosenberg once again using words and phrases taken from the discourse associated with classical music: ‘singer’, ‘advanced class’, ‘composer’, ‘melodies’ and so on. In addition, his description presupposes knowledge of other birds’ singing to be fully decipherable. This is a peculiar use of intertextuality limited to the bird guide. To complicate things even further, the authentic song of the icterine warbler includes auditory mimicry, a fact that makes the bird an intertextualist in its own right. The sound of the wasp noted in the passage is not a mock sound, but a simile invented by the author, but it is nevertheless interesting because the simile’s vehicle is the sound of another organism. We can also recognize examples of classical forms of anthropocentrism in the passage (‘the unwanted visitor’). Jonsson describes the song of the icterine warbler in this way: The song is masterful, varied, and rich in mimicry. Every strophe is repeated 2–5 times, the beginning of the strophes often starts with a starling-like sound ‘shrr, shrr, shrr . . . ’ Through it all a typical, whining, violin-like tone is heard, like a soft, dwindling ‘sjuh liu’. The gathering call, a soft yet explosive ‘tett e’eytt’ or ‘hippol’yit’ is also woven into the song . . . a nasal ‘tjepp’ or ‘the’, agitated often combined with a ‘the the the te-lü’, the short staccato sounds house sparrow-esque.28 There is no need to go into detail when commenting on Jonsson’s description of the warbler’s sounds, but, as we can see, the species pointed to as interartial comparison are different. The transcription is quite close to Rosenberg’s, but the system of notation is somewhat different. The combined vowel and consonant sound ‘hippol’ yit’ is also the onomatopoetic scientific name of the genus to which this warbler belongs (Hippolais).

Ecocritical and intermedial conclusions Whether or not field guides should be considered an art form may be a matter of debate, but I will not venture up that road, limiting myself to making a humble statement pointing out the necessary existence of creativity, rhetorical and metaphorical fancifulness, and an intertextual and intermedial skill in the producer of a field guide. That the genre is a true intermedial medium

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

of a wasp) and h i p p o l ý i t - g í e g í e g í e h i p p o l ý i t . The song is exceptionally sprightly . . . When the species is nesting the adult ones nervously fly around in the trees surrounding the unwanted visitor, swinging their heads from side to side in a funny way while they are heard to utter a melodic h y i t or a quick thrust t a c k t a c k t a c k h i p p o l ý t .27

is beyond doubt, though. The user must make use of image, text, maps and recurrent observation of the field surrounding him or her for the field guide to have any apparent meaning. It is therefore important to stress the necessity of media combination and the cognitive operations performed by the user to make this combinatory mediation of visual, semantic and audible modes work in a satisfactory way. If we take this discussion further, we may come to the conclusion that nature, in itself, is not only the basis of information being processed by the different kinds of modes represented in the genre, but actually a necessary element (though probably not a mode) which in combination with the aforementioned ones makes up the specific characteristics of the particular medium ‘the field guide’ (cf. Lars Elleström, in this volume). Consequently questions concerning the processes of transferring physical experiences to verbal and visual media should always be the focal point of the producer and the user of a field guide. I have come to this intermedial subject by way of ecocriticism. Ecocriticism is a school of cultural criticism that takes an interest in how humans’ relation to and interactions with nature are mediated in cultural artefacts. Field guides are both explicitly and implicitly engaged in this mediation. From an ecocritical standpoint, Lyon (1995) tries to construe a taxonomy of nature writing by creating a subject-based spectrum. This spectrum ranges from descriptive texts to texts dealing mainly with philosophical treatises on humanity’s relation to physical nature. At the end of the axis of describability sits the field guide, a place it shares with the scientific paper. The main goal of both genres is to submit, and mediate, information, and the philosophical, personal or literary traits we find in them are mainly accidental. Lyon calls this dimension of nature writing ‘natural history information’.29 Primarily physical nature is what the artefact is trying to mediate and guide us towards. Nature becomes the most important, albeit absent, feature in the different kinds of intermedial relations created by a field guide. The relation to this mediated medium, nature, is far from uncomplicated, as Dana Phillips has pointed out in an essay, where ecocriticism and field guides are discussed. Phillips wants to tell us that a field guide is an open work, meaning that the user has to divert his or her attention from the guide, and its combination of media, to the field, the reality in which primary observation takes place, and then back to the guide in an ongoing process of signification. In this way, writes Phillips, the user must become a ‘reader’ in that he or she has to master the techniques of identifying such features as the birds’ behaviour, their habitat and their different ways of flying. By this thought experiment, the physical nature where the observer is situated will be defined as a medium in which birds and other organisms are signs, or can be combined into signs, and as an effect of this be ‘read’ by the observer. If that description does not settle the issue of a bird’s identity, then the birdwatcher must resort to the habitat maps in the back of the book

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

120 Combinations and Integrations of Media

Håkan Sandgren 121

The image, the text, the transcriptions and the maps all work together, creating a unified perspective, but to reach this unifying point, verbal and visual representation must at the same time be complementary media, for instance, by focusing on specific and differentiating details in the object. This urges the artist or illustrator to be utterly aware of ‘the moment of choice’31 because the importance of positive identification, recognition and referentiality are crucial aspects of every field guide. The artist’s or the writer’s ‘interpretation’ of the natural object has one main objective, namely to make naming and identification possible. The kind of artistry that the producer must strive for is exactitude and faithfulness in the way the natural object is mediated into the media used, and to reach this objective is truly an utterly refined form of artistry.32 In this lies a paradox: the mediated image or description of, for instance, a bird that we would find in a field guide – no matter how faithful to reality it is – will never replicate the visual data we conceive as a bird when observing it in the field. First of all the image and the description found in the guide are merely rough ideas of how a natural object probably will appear when we find it in front of our eyes. Second, the field as medium does not allow the kind of scrutiny made possible by the guide, and its lifeless objects. In the field, the object is ever-moving, sitting in the shade, partly covered or fleeing. Mediated in the medium of the field guide the object is once and for all given to us. Apart from this, nature persistently eludes us with its constant flux; what yesterday could be said to denote ‘the bird species of Europe’ is today pointing to a different part of the total quantity of bird species, a totality which in itself has to be an approximation. Nevertheless, nature is always there when we use a field guide; it is the element we have to come to terms with as users of the genre, and as producers of it. Nature is always present in a field guide, a necessary part of the way in which the creators and users of the genre combine the media modes included therein, and as a goal to which our perception is directed while making use of the guide.

Notes 1. J. Holšánová (2008) Discourse, Vision, and Cognition (Amsterdam: J. Benjamins). 2. W. Wolf and W. Bernhart (eds) (2007) Description: In Literature and Other Media (Amsterdam: Rodopi). 3. For example, R. T. Peterson, G. Mountfort and P. A. D. Hollom (1985) Europas fåglar: En fälthandbok (Stockholm: Norstedt), B. King, E. C. Dickinson and M. W. Woodcock (1991) A Field Guide to the Birds of South-East Asia (London: Collins),

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

(included in its recent editions). ‘Field-marks,’ it should be clear, are not limited to visual features, but also comprise things like geographic range, habitat preferences, typical behaviours such as interspecies flocking, and flight patterns. Such being the case, the birdwatcher must become a ‘reader’.30

4.

5.

6.

7.

8. 9.

10.

11.

12. 13. 14. 15.

16. 17. 18. 19.

20.

L. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern (Stockholm: Wahlström & Widstrand), E. Rosenberg (1995) Fåglar i Sverige (Stockholm: Norstedts) and L. Svensson, P. J. Grant, K. Mullarney and D. Zetterström (1999) Fågelguiden: Europas och Medelhavsområdets fåglar i fält (Stockholm: Albert Bonniers förlag). J. Karnicky (2007) ‘Included in this Classification: Encoding American Birds’ (unpublished paper delivered at the conference of the Society for Literature, Science, and the Arts, Portland, Maine, November 2007). For the birds of Northern Europe, see K. H. Voous (1977) List of Recent Holarctic Bird Species (London: BOU) and M. Beaman (1994) Palearctic Birds (Stonyhurst: Harrier). For example, Peterson et al. (1985) Europas fåglar, inside of cover; King et al. (1991) A Field Guide to the Birds of South-East Asia, p. 219 and R. T. Peterson and V. M. Peterson (2002) A Field Guide to the Birds of Eastern and Central North America (Boston: Houghton Mifflin), inside of cover. For example, Peterson and Peterson (2002) A Field Guide to the Birds of Eastern and Central North America, pp. 8ff.; Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, pp. 7, 11, 21. E. Rosenberg (1972) Fåglar i Sverige (Stockholm: Almqvist & Wiksell). A term coined by D. Phillips (1999) ‘Ecocriticism, Literary Theory, and the Truth of Ecology’, New Literary History 30(3), p. 593. Roger Tory Peterson’s famous guides set the standard for this way of illustrating birds. Peterson and Peterson (2002) A Field Guide to the Birds of Eastern and Central North America, p. 73, and Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 113. Peterson and Peterson (2002) A Field Guide to the Birds of Eastern and Central North America, pp. 106, 108, and Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, pp. 166f. Peterson et al. (1985) Europas fåglar, and Peterson and Peterson (2002) A Field Guide to the Birds of Eastern and Central North America. Rosenberg (1972) and (1995) Fåglar i Sverige. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 202. A. Saunders (1951) A Guide to Bird Song: Descriptions and Diagrams of the Songs and Singing Habits of Land Birds and Selected Species of Shore Birds (New York: Garden City); L. Lutwack (1994) Birds in Literature (Gainsville FL: University Press of Florida); D. Rothenberg (2006) Why Birds Sing: A Journey into the Mystery of Bird Song (New York: Basic Books). Lutwack (1994) Birds in Literature, pp. 1–17 and Rothenberg (2006) Why Birds Sing, pp. 15 ff. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern. I. Storm (1985) Harald Wiberg: Ett konstnärsliv i naturen (Stockholm: LT) and B. Svanberg (1991) Harald Wiberg: En konstnär (Umeå: SIH läromedel). C. Curry-Lindahl (1952) Festskrift tillägnad Erik Rosenberg på 50-årsdagen 19 17/8 52 (Stockholm: Svenska naturskyddsföreningen) and C. Thor (2002) Fågelliv betyder orden: Vandringar i Erik Rosenbergs riken (Örebro: Gullars). The relation between user and writer, and questions concerning the social function of birdwatching, are studied by M. Cocker (2001) Birders: Tales of a Tribe (New York: Grove Press) and J. Karnicky (2004) ‘What Is the Red Knot Worth? Valuing Human/Avian Interaction’, Society & Animals 12, pp. 253–66.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

122 Combinations and Integrations of Media

21. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 5, see also L. Jonsson (2002) Birds and Light: The Art of Lars Jonsson (London: Christopher Helm). 22. Rosenberg (1995) Fåglar i Sverige, p. 48. 23. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 93. 24. Rosenberg (1995) Fåglar i Sverige, p. 315. 25. Ibid., p. 316. 26. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 384. 27. Rosenberg (1995) Fåglar i Sverige, p. 349. 28. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 434. 29. T. J. Lyon (1995) ‘A Taxonomy of Nature Writing’ in C. Glotfelty and H. Fromm (eds) The Ecocriticism Reader: Landmarks in Literary Ecology (Athens: University of Georgia Press), pp. 276–81. 30. Phillips (1999) ‘Ecocriticism, Literary Theory, and the Truth of Ecology’, p. 594. 31. E. Hodnett (1982) Image and Text: Studies in the Illustration of English Literature (London: Scholar Press). 32. S. Jay Gould gives special attention to this skill in (2000) Wonderful Life: The Burgess Shale and the Nature of History (London: Vintage), p. 100.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Håkan Sandgren 123

7 Sami Sjöberg

It is commonly recognized that intermedial works of art may mediate information in ways that individual media are not capable. The intermedial work plays with different modes of signification and produces effects impossible for language alone. Peter Wagner defines intermediality as the practice of describing one medium by means of another.1 Wagner refers solely to ekphrasis, although intermediality is a complex phenomenon and requires that ‘medium’ is defined. Lars Elleström defines ‘medium’ based on the shared, yet dissimilar, qualities of media as communication.2 A ‘medium’ is a construct born and delimited in the process of definition. Thus intermediality – the engagement of more than one medium – highlights the constructedness of medial borders and establishes a kind of ‘border zone’.3 Accordingly, media can be mixed in ways that struggle to overcome the construct nature of ‘intermedia’. Because the practice of applying many media differs from the scholarly term describing this practice, this essay4 seeks to ‘renegotiate’ intermediality in terms of artworks radically disrupting the coherence of the work itself. Coherence is always relative and the success of such disruption debatable, but by realizing these limitations we can study works that play with intermediality; while by doing so, artworks may also demonstrate the term’s inadequacy due to its constructedness. This essay argues that intermedial works play with nothingness and question the validity of signification in general by showing its limits with obscure techniques, including invented signs. The themes of incoherence and nothingness form the base of the œuvres of the French Lettrist movement. The Lettrists’ interest in nothingness derives from preceding developments in philosophy and art. In France, the interwar era had made the theme of nothingness prominent in Sartrean existentialism as also in the Dadaist works of Tristan Tzara.5 Influenced by the trend in philosophy and aesthetics, Isidore Isou (Ion-Isidor Goldstein 1925–2007), the Romanian-born founder of Lettrism, published the manifesto of Lettrist poetry (Le Manifeste de la poésie lettriste, 1947), claiming that words could express none of the individual’s feelings, because words 124 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Media on the Edge of Nothingness: Visual Apostrophes in Lettrism

were stereotypes.6 Isou criticized the inadequate means to mediate radically differing experiences. There can be no certainty, however, regarding the fundamental similarity or dissimilarity of emotions due to the structure of communication. Language is based on concepts that are already generalizations, hence no comparison of the experiential is possible without objectification into language. Nevertheless, Isou’s definition of language as stereotypical communication manifests his profound distrust of this medium.7 He appears to consider language as inevitably exterior to the individual. Isou’s Manifesto implies that language sets the frames for the artist’s creative efforts although it should be the other way around.8 Therefore he focused on what language could not mediate: nothingness and the incomprehensible. Furthermore, Isou tests how these motifs could be expressed in writing,9 seeking to subvert the notional language of logos to individual use of anticonceptual language. Although Isou’s point of departure is utopian and highly problematic, his quest for individual expression results in a unique response to the modern crisis of language: ‘That what for preceding authors represented the dimension of silence and the unknown will become, thanks to me, the dimension of the known and of the new kind of speech’.10 Isou considers this ‘new speech’ the means to overcome the limits of language, which do not correlate with those of experience. However, Isou’s ‘new speech’ is inevitably in relation to language: what is defined as incomprehensible is incomprehensible only when one seeks to communicate it through notional language. Hence writing requires autonomy, distance from notional language, and Isou seeks to emphasize the act of writing so that Lettrist writing approaches writing on canvas.11 However, although he tries to recontextualize writing, Isou does not abandon the book, and hence Lettrist works create an intermedial tension between the written and the visual.12 By prioritizing the act of writing Isou simultaneously effaces the mediating role of language. For him writing apparently deals with the arousal of connotations and pre-linguistic affects, toying with the reader’s ‘semiotic imagination’, engaging possible interpretations such that the markings acquire a plenitude of meaning – they can mean anything. This study concentrates rather on those qualities in Lettrist writing that seem to precede and/or resist objectification into language (see Figure 5). The concrete effects of such resistance are markings that appear meaningless and thus emphasize the medium itself. Lettrist writing must be treated as a special case of writing because it functions differently from notional language: its visual elements emphasize the presence of the text. Visuality was highlighted from 1953 onwards by hypergraphics (hypergraphie), enabling artists to combine all known writing systems without a particular ‘grammar’ and invent ‘signs’ of their own.13 The invented markings14 are a method for distancing the act of writing from language. They are applied to express what defies expression in notional language, in the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sami Sjöberg 125

Figure 5 An excerpt of Alain Satié’s ‘Sur le pont d’Avignon’. Courtesy of the artist

sense of being supposedly outside the production and application of concepts. In this light, invented markings are not based on ciphers and are at best quasi-iconic with no recognizable resemblance or convention to connect them with pre-linguistic experiences. The invented markings thus highlight the individual writer’s ‘presence’ in Lettrist writing, producing a tension when text is combined with invented markings. In such cases, Lettrist writing fragments conventional text by means of another medium. I call these interventions visual apostrophes. Besides indicating a missing letter or a word in the text, the original literal meaning of the word apostrophe (Greek: apostrophos ‘turning away’) is also preserved. In conventional text, the visual apostrophe represents a turn away from notional language that makes room for another medium. However, the visual apostrophe maintains its relation to language, because often the apostrophe is linguistically motivated: the apostrophe can replace a word or a letter and can hence be regarded as intermedial. Furthermore, there is a third, subtler meaning of apostrophe. The Greeks used it to refer to an orator’s ‘turning aside’ to address some individual.15 In this context, the hypergraphics address the individual reader by showing him or her the limits of notional language, thereby leading him or her to reconstruct the meaning of invented markings. I suggest that this meaning of the apostrophe requires a departure from the intermedial approach, although the aforementioned definitions are necessary steps towards the other approach, meontologization or the dynamics of signification and nothingness, which is studied in the third part of this essay.

The restrictions of the intermedial Critical inquiry may not always benefit the complexity of an artwork, especially in the case of intentional ambiguity. This is because intermediality

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

126 Combinations and Integrations of Media

makes certain fundamental ontological assumptions affecting the depiction of a work. One such assumption is that in the work there are at least two media differing in their ways of signification. For instance, verbal and visual signs differ in their mode of signification. In Lettrism, intermediality refers to the interaction between the two distinct media, namely the written and the pictorial. Hence, in order to approach such difficult cases of intermediality as that of hypergraphics , the work must be subjected to a certain deontologization. Deontologization refers to the lowering or eventual removal of media borders, yet, as W. J. T. Mitchell argues, ‘all media are mixed media, and all representations are heterogeneous’.16 Mitchell means that writing too is visual and material. In addition, no modern pictorial work can avoid the text.17 Consequently, every intermedial work is subject to some degree of deontologization. In the context of this essay, however, deontologization is insufficient because Lettrist writing is not based on shared unproblematic representations. Hypergraphics suggests a manifold relation to language. On the one hand, hypergraphics is a form of visual expression, but visual only in the limits imposed by written language. Invented markings have the semblance of signs, because they are black and white and appear in contexts where written signification is found (see Figure 5). In short, they emulate the conventions of writing. On the other hand, invented markings fragment the text with their apparent lack of meaning. Once grasped in the context of language, the lack of linguistic signification is supplemented by invented markings apparently concealing their meaning. This supplementation engages the reader’s semiotic imagination. Yet the materiality of hypergraphics arouses a feeling of the presence of the text and of a presence ‘behind’ the text: someone in particular wrote these. Therefore, the more notional language fails, the more the presence of the unobjectified is pronounced. Such presence manifests in the following visual apostrophe by Isou: The individual wanted to see the surfaces it spoke of calculate them visually.18

and

Although the particular apostrophe is not as advanced as those applying hypergraphics, it nevertheless emphasizes the co-presence of the visual and the written in the script.19 No linguistic meaning is lost by the introduction of the black rectangle. The fragment is intermedial – in the Wagnerian sense – in that the text emphasizes seeing and materiality through ekphrasis. The notional language discusses the visual element before the rectangle is introduced. In addition, the fragment also works as an iconotext. Wagner writes: ‘Iconotext refers to an artefact in which the verbal and the visual signs mingle to produce rhetoric that depends on the co-presence of words and images’.20 Thus the rectangle is included in the sentence as a visual sign, which acquires that quality through the text. Yet its mode of signification is 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sami Sjöberg 127

ambivalent. The rectangle may be either a mere self-referential presentation as in both cases above or it may be depicted as representation. In the latter case, an interpretation based on iconotextuality recognizes the black rectangle as comparable to note clusters in early twentieth-century modernist music.21 As such, the rectangle is a ‘condensation’ of every possible sign superimposed on each and obliterating individual markings. These different modes of signification are not contradictory. The nature of the visual apostrophe is such that it presents a pause or a cut in the text, it shuns one medium in order to make way for another, but it may leave a trace of what is removed from the first. For example, consider replacing the rectangle with an adverb, such as ‘automatically’ (automatiquement). This supplementation is not linguistically necessary but possible and exemplifies how easily the rectangle is interpreted as something other than itself. Therefore, the lack of language or the traces of it are ambiguous in visual apostrophes and the ambiguity arouses the interpreter’s semiotic imagination. The possibility of replacing the absent signified is insufficiently described by any deontologization of intermediality, because deontologization ignores the potential signification. The co-presence of different ways of signification occurs because coherent sign categories are based on certain distinct ways of signification. Deontologization explores both the reciprocal differences between sign categories (that is, verbal, visual) and their co-presence, but excludes the lack of a clearly defined signified. Hypergraphic writing entails that the lack of signified not only prevents the definition of an unambiguous semiotic category, but this may also be deemed an intrusion of ‘pre-linguistic elements’ into notional language. This is to say that hypergraphics explores the possibility of ‘notionless’ writing. However, such a formulation is clearly paradoxical, because writing is by definition the mediation of ideas through concepts. Hypergraphics plays with this key idea in communication and deliberately engages the interpreter to seek meaning through connotations and associations, which the work neither affirms nor negates. This strategy of ambiguous signification suggests that the loosely definable connection of hypergraphics to existing sign categories should be analysed as such instead of cataloguing the various possible interpretations. Hence I coin a new term, meontologization.22 It refers to the study of what remains outside notional language; to what has not yet been objectified or is beyond objectification. The perception of such a lack of concepts as nothingness is practical, because nothingness cannot be made into an object of thought. This is to say that nothingness does not lend itself to semiotic imagination, and as a counterweight to interpretation, it shapes the coherence of the artwork. As deontologization refers to diminishing the limits of what is present (signified), meontologization focuses on the disappearance of distinct limits between what in representation is present and absent. In other words, instead of the iconotextual co-presence

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

128 Combinations and Integrations of Media

Sami Sjöberg 129

How to do nothing with words Although hypergraphics supposedly refers to what falls outside notional language, Isou’s theory is in many ways problematic. Due to his utopian attitude he sought to make the incomprehensible palpable by ‘writing nothingness’ (écrire le rien).23 Isou was undoubtedly referring to experiences beyond the limits of communication, yet he did not discuss the obvious problem of mediating something unobjectified. The existence of unmediated experiences is debatable, because if we experience something we do not simultaneously conceptually reflect it.24 Although the possible existence of such experiences is not denied, their mediation is certainly impossible, because it would require the objectification of the very experiences one seeks not to objectify. Mediation necessitates objectification, because the experiences cannot be present as such. In addition, the claim to ‘write nothingness’ is obviously paradoxical; how can material production, such as writing, produce nothingness? Undoubtedly these are problematical issues, which Isou ignores. However, intermediality or iconotextuality is a characteristic result of Isou’s search to express nothingness. When the signification of invented markings or any kind of signs is effaced, their visuality is emphasized. The reader then seeks to make hypergraphics meaningful as presentations – as abstract forms – or to interpret them as representations. Due to the ambiguity of signification in Lettrist writing, Heidegger’s view of nothingness corresponds to visual apostrophes. He stated that nothingness can be surmised but not logically proven.25 That is, nothingness responds poorly to analytical thinking based on notional language. Hypergraphics is writing that defies interpretation and this resistance is both intentional and significant to the coherence of the work. In this case, hypergraphics is interpreted as an effort to mediate experiences that cannot be made into objects of thought, the challenge being not to interpret nothingness as something else. Arguably the visual apostrophe can be approached as nothingness. Its structure proves that what is taken away from representation is also a representation – as a trace of representation. Representation is the only way to approach nothingness, because nothingness cannot be presented. Meontologization further allows depiction of the trace of representation, the absence of a given signified, as an intentional effect in the work. In visual apostrophes, meontologization may indicate an absence or predict the anxiety about nothingness.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

of the verbal and the visual, meontologization is the ‘cooperation’ of presence and absence manifest in verbal and visual signs. By meontologization hypergraphics is understood as invented markings that are notionless, but have the potential to change. Meontologization prioritizes the actual coherence of the work and its possible disruptions instead of the hermeneutical assumption of its unity, which may lead to hasty interpretations.

Alain Satié’s work, ‘Sur le pont d’Avignon’ (On the bridge of Avignon) (1970, see Figure 5), exemplifies these functions.26 There are three discernible levels of writing and meontologization elaborates their interaction. Firstly, the typewritten text forms the common level open to all. Secondly, the handwritten headline forms the individual level, because although written in French, another person cannot reproduce Satié’s handwriting. Thirdly, the private level is formed by invented markings that disrupt sentences and even individual words. In this work hypergraphics is not intended to widen signification but to underline the shortcomings of notional language. However, the work sporadically applies methods pointing to the absence of a particular element. For example, the fragment between the fourth and fifth lines (compa[ ]ables) suggests that the invented markings can be replaced only by the letter ‘r’, thus forming a readable word in French, comparables (comparable). The length of the ‘imaginary cartouche’ suggests a longer replacement than ‘r’ due to the sign-like nature of the fragment, which the reader identifies with language.27 As the ‘r’ seems to be the only logical replacement, in this case the hypergraphics leads us to search for an absent particularity. Such absence depends on our expectations of finding something that is not there.28 Therefore the nothingness in comparables is inherently particular, meaning that there is a trace of something particular that is absent. In some visual apostrophes the lack of language refers to absence, as in the example above. Moreover, invented markings resemble representations because the reader replaces the fragment with what it seems to stand for. The text around the visual fragment is motivated by the hypergraphics charged with semantic content. In addition to the meontological cooperation of presence and absence, the copresence of words and images is essential. Hence, these examples may be called iconotextual hypergraphics. The relation to other objects is established and the hypergraphic fragment becomes a quasi-linguistic sign. Thus visual apostrophes utilize the particularity of absence: the interaction between the common and the private levels is necessary, because individually the private level structures only an unfinished signification. Here something present mediates the absence of some particular, but what is missing can still be depicted. The reconstruction of other parts of the composition is not equally uncomplicated, however. In places a whole word or multiple words are missing. For example, the invented markings on the second and third line in Satié’s work suggest that the visual apostrophe either replaces multiple words or none at all. However, in the case of multiple missing words, the interpretation is more ambiguous than the previous one. Hence no univocal supplementation but only indeterminate possibilities can be found. What if there is no appropriate unambiguous particular replacement? In the case of such profound indeterminacy, invented markings would not represent the absence of something in particular, but a more general nothingness. Here, Heidegger’s notion of nothingness has a structural

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

130 Combinations and Integrations of Media

homology with the visual apostrophe. According to him, beyond the limits of a whole there ‘is’ nothingness.29 Heidegger understood beings and things as finite, and therefore nothingness draws their very limits. Nothingness constitutes being and this constitution is known in anxiety. The anxiety derives from the limitedness of our existence and our awareness thereof. Therefore nothingness emerges as ‘a slipping away of the whole’.30 The slippage undermines the coherence of the finite (an individual, an artwork). The hypergraphics evokes nothingness by showing the reader the limits of his or her being. The implied existence of personal experiences is revealed by invented markings, but the experiences remain radically other. According to this approach, invented markings are a vestige of the presence of the unknown other (writer, meaning) within the text. Moreover, visual apostrophes are not mere changes in media or modes of signification, but breaks in the coherence of the text revealing the absence of the writer and exposing the reader to the lack of language and meaning. The slippage of the whole results once the limits of being are foregrounded and we grasp that there is something beyond our field of knowledge. By effacing meaning, though, Lettrist writing undermines the capability of language to correspond with the world it describes. Satié’s work does not radically question the limits of being but those of notional language. Hence, the work’s ‘meaningless’ visual apostrophes in fact criticize the assumption that the individual and experiential could be fully communicated through language. Such examples of visual apostrophes may be called notionless hypergraphics. In this case, the hypergraphics ‘deconstructs’ the co-presence essential to the iconotext because it establishes no coherent sign category.31 Therefore, in notionless hypergraphics the verbal and visual signs do not mingle, but proclaim their autonomy. Iconotextual hypergraphics is subordinate to notional language because it replaces a grammatical sign (a letter, a word) in the text. However, if hypergraphics does not acquire equal grammatical motivation, the hierarchical relation between notional language and hypergraphics is the opposite. Notionless hypergraphics cuts the text, opening it to the reader’s semiotic imagination. The reader is called to fill in the blank, which means that through interpretations alone the blank can potentially become something other than nothingness. Furthermore, the lack of meaning in the artwork is in itself meaningful, thanks to its intentionality. Although hypergraphics challenges the capabilities of writing to serve as a medium and communicate by mediating information, meontologization highlights hypergraphics as a medium that mediates the ‘presence’ of the unobjectified in the artwork. This unobjectified, however, can only be detected through the dynamics of meontologization, which is present in the visual apostrophe. Fundamentally, Satié’s work is fragmentary if the aspects of absence and nothingness are not taken into account and if iconotextual and notionless hypergraphics are not regarded as separate. The title refers to an impassable bridge. Arguably in Satié’s piece language also forms a bridge that enables

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sami Sjöberg 131

132 Combinations and Integrations of Media

Conclusion Lettrist visual apostrophes demonstrate the absence and potential of nothingness by showing us our limits. By shunning language with invented markings, there is a feeling of concealed expression. This concealment causes the unfinished signification to emphasize the materiality of hypergraphics. The pure materiality of writing results in what Isou called ‘hidden signification’ (signification cachée).32 This hiddenness not only highlights absence, but also reveals the limits of being and, henceforth, arouses the anxiety of things eluding us. The meontologization of Lettrist writing causes an effect of deficiency, as hypergraphics points out the limits of notional language. Such is the ultimate contribution of hypergraphics, although it disregards Isou’s utopian claims. We must realize that language exists because the things it denotes are not present. This practical view assumes that language enables us to discuss concrete absent things or abstractions that were never there. According to this idea, hypergraphics reveals the fundamental emptiness of representations in language by invented markings and unfinished significations, which highlight the unattainability of the intended meaning. In other words, although the intended meaning is always inaccessible, hypergraphics foregrounds the separate realities of individuals instead of producing a fictive one. Thus the empty representations evoke an anxiety of different, yet simultaneous, realities where the slipping away of the whole becomes evident. However, hypergraphics implies another result of meontologization: invented markings merely propose their own meaninglessness without affirming it. Accordingly, it may be concluded that unfinished signification is the very possibility to represent. From the perspective of notional language hypergraphics signals that something is taken away – from representation – in which case the possibility to represent is simultaneously a vestige. As such, the pronounced presence of hypergraphics brings about the possibility of experiences that cannot be made into objects of thought. Therefore, to neglect the meontologization in artworks is to neglect the philosophical power of works of art.

Notes 1. P. Wagner (1996) Icons, Texts, Iconotexts: Essays on Ekphrasis and Intermediality (Berlin and New York: Walter de Gruyter), pp. 10–11, 17. 2. Cf. Elleström and Jørgen Bruhn, in this volume.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

one to cross to the other bank at the end of the text, yet invented markings subject the bridge of language to constant disruption. To ensure a safe crossing, the actual bridge requires its missing pieces and likewise, meontologization supplements the bridge of language by showing what is not there.

3. Cf. Rajewsky and Sándor, in this volume. 4. This essay is a part of the project Literature, Transcendence, Avant-Garde, funded by the Academy of Finland (1121211). 5. Although Isou never produced a coherent theory of nothingness, he drew on the prevailing philosophical accounts of nothingness. See I. Isou (2000) Amos ou introduction à la métagraphologie (first published 1953) (Marseille: la Termitière), p. 26. 6. I. Isou (1947) Introduction à une nouvelle poésie et à une nouvelle musique (Paris: Gallimard), pp. 11–12. 7. Isou criticized media as means of communication, but the concept ‘medium’ was understood rather conventionally in Lettrism. 8. See Isou (1947) Introduction à une nouvelle poésie et à une nouvelle musique, pp. 11–12. 9. I. Isou (2000) La dictature lettriste: Cahiers d’un nouveau régime artistique (first published 1946) (Paris: Cahiers de l’externite), p. 16. 10. ‘Ce qui pour des auteurs antérieurs représentait la dimension du silence et de l’inconnu deviendra, grâce à moi, la dimension du connu [et] de la parole novatrice’. I. Isou (2003) La créatique ou la novatique, 1941–1976 (Romainville: Éditions Al Dante), p. 23. 11. See Isou (2000) Amos ou introduction à la métagraphologie, p. 26. 12. The genre of the work is also bound to the technical requirements of a given medium (cf. Rajewsky, in this volume). 13. R. Sabatier (1989) Le lettrisme, les creations et les createurs (Nice: Z’éditions), pp. 27–8. 14. I call ‘signs’ invented by the writer ‘invented markings’ to avoid confusing their sign-like nature with actual signs. ‘Hypergraphics’ refers to the combination of every kind of sign system and therefore I rather apply ‘invented markings’, which also emphasizes the act of mark-making. 15. J. D. Kneale (1991) ‘Romantic Aversions: Apostrophe Reconsidered’, English Literary History 58, pp. 141–65. 16. W. J. T. Mitchell (1995) Picture Theory: Essays on Verbal and Visual Representation (Chicago and London: The University of Chicago Press), p. 5. 17. Cf. Jørgen Bruhn, in this volume. 18. ‘L’individu voulait voir les surfaces dont il parlait . . . et les calculer visuellement’: Isou (2000) Amos ou introduction à la métagraphologie, p. 20. 19. Isou’s apostrophe had numerous antecedents, such as Man Ray’s anonymous poem (1924), consisting of black lines that form the poem, and a book by Robert Carlton Brown (1886–1959), Gems: A Censored Anthology (1931). See J. Rasula and S. McCaffery (2001) Imagining Language: An Anthology (Cambridge MA: MIT Press), pp. 201, 288. 20. Wagner (1996) Icons, Texts, Iconotexts, p. 16. Wagner’s definition of iconotext is congruent with some approaches to intermediality introduced in this volume (cf. Jørgen Bruhn). However, I retain iconotext in order to distinguish between the numerous intermedial ways in which hypergraphics criticizes language. 21. Henry Cowell pioneered the technique in The Tides of Manaunaun (1917). In France, especially Olivier Messiaen applied note clusters in the early 1940s. 22. Meontology, from Greek to m¯e on ‘nonbeing’. 23. Isou (2000) La dictature lettriste, p. 16. Isou may have absorbed this rhetoric from negative theology, which also struggled with the mediation of inexpressible experiences. In addition, linguistic negation and absurd texts diffuse ambiguous

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Sami Sjöberg 133

24. 25. 26. 27. 28. 29. 30. 31. 32.

uses of language. However, in nonsense literature and negative theology, for instance, distrust of language does not manifest as radically as in invented markings. Here I treat as nonsense and negative theology such texts that rely on conventional language although invented markings can be regarded as nonsensical and were also utilized in different branches of mysticism. However, it is difficult, if not impossible, to depict the genre of texts that apply invented markings. Cf. R. K. C. Forman (1997) The Problem of Pure Consciousness: Mysticism and Philosophy (Oxford: Oxford University Press). See M. Heidegger (1993) Basic Writings (London: Routledge). The title has a double reference to the real world. The bridge of Avignon is a historical site and ‘Sur le pont d’Avignon’ is a traditional French song. This is a case of confusion between ‘visual’ and ‘iconic’ (cf. Elleström, in this volume). See J.-P. Sartre (1943) L’être et le néant: Essai d’ontologie phenomenologique (Paris: Gallimard), pp. 42–4. The ‘whole’ refers to all entities whose meaning to us constitutes our world. Cf. Heidegger (1993) Basic Writings. Ibid., p. 102. For a similar non-totalizing strategy cf. Sándor, in this volume. Isou (1947) Introduction à une nouvelle poésie et à une nouvelle musique, p. 18.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

134 Combinations and Integrations of Media

Part IV

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Mediations and Transformations of Media

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

This page intentionally left blank

Penrose, ‘Seeing is Believing’: Intentionality, Mediation and Comprehension in the Arts Siglind Bruhn

In 1934, the Swedish artist Oscar Reutersvärd created, with an uncommon arrangement of cubes, what has become known as an ‘impossible triangle’. In 1980, the Swedish government honoured the inventive artist by commissioning photographs of three of his works for Swedish stamps. The stamps were produced in 1982 and issued for about two years. Twenty years after his Swedish counterpart, in 1954, the British mathematical physicist Roger Penrose (born 1931), after attending a lecture by the Dutch graphic artist M. C. Escher, rediscovered the impossible triangle, drew it in what is today its most familiar form, and described it in a 1958 article published in the British Journal of Psychology. Penrose’s impossible triangle, unlike Reutersvärd’s earlier version, was drawn in perspective, which added a further paradox to the figure. Penrose described the ‘tribar’ as ‘impossibility in its purest form’. It does indeed contain various dimensions of impossibility: its three straight beams meet at right angles at the vertices of the triangle they form, and the three surfaces of each beam are also at 90 degree angles to one another. In the context of intermediality the object seems thought-provoking for its strictly non-hierarchical nature. No face of any of the three arms can be said to be privileged in that it is ‘always front’ or ‘tracing the inner circle’. Instead, each of them constantly changes its spatial designation and its apparent visual reality, as so many surfaces in M. C. Escher’s drawings typically do. This essay, then, is a meditation on various art forms, media and modalities – thoughts that are inspired by this and other triangles. In particular, I hope to incite discussion of a number of questions concerning the angles linking the three beams in another strange triangle – what is often referred to as the ‘artistic triangle’. I will end by analysing and interpreting the three-step intermedial transposition of a work created by an uncle of the designer of the tribar, the British Surrealist painter Roland Penrose.1 137

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

8

138 Mediations and Transformations of Media

Some time ago I undertook an in-depth study of an opera by the American composer, Philip Glass. The opera is entitled Satyagraha. In terms of the dramatic action developed on stage, it deals with the early life of Mahatma Gandhi and his development of the movement of non-violent resistance in South Africa. The lyrics, however, are entirely taken from the ancient Indian poem, Bhagavad Gita. Thus my task, before even commencing the analysis of Glass’s music, was to understand how the ancient poetic dialogue between a divine avatar and the warrior relates to an early twentieth-century story of racial discrimination, and how particular verses from the Gita may inform and reflect this particular instance from Gandhi’s struggle for human dignity under a white supremacist government. In the middle of my several weeks of reading up on the Bhagavad Gita, and entirely unrelated to my work on Glass, I was invited to participate in a panel discussion on the topic of the relation between the Trinity and the ‘artistic triangle’, which is understood to consist of (a) artistic creator, (b) performer or mediator and (c) audience or beholders. While I knew, of course, which Trinity was being referred to, I found myself inspired to think of the question in terms of the Hindu trinity. This in turn led me to a different kind of artistic triangle, which I have since come to consider a foil in an impossible dimension of the well-rehearsed one, linking (a) art as an offering with (b) art that is passionately involved in the world and (c) art that ‘constructively destroys’ or dethrones. The first deity in the Hindu trinity is Brahma, the Creator. Brahma creates the world as a kind of offering. Having created it, Brahma is no longer involved in the world, neither imposing rules nor fixing problems, neither punishing nor rewarding. The second deity in the Hindu trinity is Vishnu, the Preserver. Vishnu is supremely concerned with the world, incarnating again and again to help remedy what human beings invariably mess up. (Krishna, one of the protagonists in the Bhagavad Gita, and Rama are perhaps the best known incarnations or avatars of Vishnu.) Finally there is Shiva, the Constructive Destroyer. Shiva destroys so that new things may arise; it is said that Shiva ‘dances the world to pieces’. Shiva is also very much interested in abstractions and in philosophical conceptualizations. The artistic triangle that emerged in the context of my reflections on the Hindu trinity may be seen as intersecting with the more familiar triangle of artistic creator, performer or mediator and audience. One corner of my alternative artistic triangle would be taken up by art and music that serve as an offering; Byzantine icons and religious mosaics come to mind just as much as hymns whose composers have remained anonymous. This kind of artistic attitude relates to Brahma. A second corner of my triangle would be occupied by art and music that is passionately and compassionately involved with the human world. This includes art exploring human

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Various artistic triangles: options at the creator’s angle

bodies (such as Michelangelo’s Vitruvian Man) and art representing the world in which humans live – from the interiors of Vermeer and other Flemish artists to Romantic representations of small man in overwhelming nature, music that incorporates human dances, folksongs and the like, all the way to the nineteenth-century dramas and novels. This attitude relates to Vishnu. Finally, there is the artistic attitude that ‘constructively destroys’: art that dethrones the idea of representation, music that dethrones overtone-based harmonies, literature that dethrones the idea of a logically developed plot or a fully impersonated protagonist with whom readers can identify. This art corresponds to Shiva in that it may become increasingly concerned with conceptualizations, either by turning to abstract modes of communication or by engaging irony, as Lars Elleström has shown so persuasively in his book Divine Madness.2 The beauty of this triangle is that just as the three corners are not unrelated points in space but linked as the three sides of a strictly contained geometric figure, the three attitudes are intricately interrelated. A work of abstract music may well be created in the spirit of an offering, as may be a representational still-life and so on, but then, Brahma, Vishnu and Shiva are really three aspects of a godhead that is essentially One: Trimurti.

The limits of perception: obstacles at the beholder’s angle With regard to what is more commonly called the artistic triangle, one question has haunted me for a long time: how and to what extent does it matter whether or not the audience can see, hear or in any other way grasp details, structures and intentions, or whether these only become apparent with the aid of notation or analysis? In a masterpiece of literature, music or plastic art, a first exposure will typically only reveal a general impression and a fragment of the extant details. Renewed exposure and atypical approaches can and usually will take the beholder’s appreciation to ever deeper levels. This includes reading a work of literature not consecutively but selectively or by comparing disjunct passages, looking at the score of a musical composition rather than only listening to the sounding realization, voicing the thoughts that possibly inform the visual images in a painting and perhaps extrapolating from them into realms beyond the visually represented. There is thus a development on the beholder’s side from perception to an understanding of the deeper signification and possibly on to a sense of a work’s transcendent meaning. From a first glimpse at this third corner, considerations will return to the level of perception and retrace the triangle a number of times, in the manner that Gadamer described for the hermeneutic circle. If one were to arrange artworks along a spectrum that spans from those to which access is immediate all the way to those that appear exceedingly complex, veiled and layered, one would find oneself confronted with extreme questions. At the one end there are creations, presumably in any medium

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Siglind Bruhn 139

and modality, which may reveal no more at prolonged or repeated exposure than at the very first glance. At the other end, there are works whose allusions and significations can only be ‘understood’ by specialists. Works belonging to the one extreme may collapse the artistic triangle to a twodimensional relationship, as there may be no need or justification for any ‘mediation’, and where a musical composition still requires a performer in order to be heard, this unfortunate go-between may seem flattened to a presence that is, psychologically and spiritually, paper-thin. At the latter end, one may wonder where creative artists on the one hand and audiences on the other draw the line between what can be understood and what needs to be understood?

Case study 1: sensually or intellectually inaccessible spiritual signifiers I would like to give two examples from my own field, music. In both cases, neither the signifier nor the signified are utterly mysterious for anybody, even those who know almost nothing about music. One of the phenomena is impossible to grasp without conscious knowledge, the other is only ever felt subconsciously. In 1991, the British composer John Tavener (born 1944) composed an opera about a fifth-century desert hermit revered as St Mary of Egypt.3 The work bears the subtitle ‘an ikon (sic) in music and dance’, thus alerting us to the fact that it combines various art forms. It involves not only music, drama and stage design as all operas typically do, but also includes dance, as some works did since the beginnings of the genre in the early seventeenth century. Moreover, this opera makes reference to a kind of visual representation that has no connection to music-drama: the icon. Tavener, a member of the Orthodox Church since 1977, and his librettist, Mother Thekla, the abbess of an Orthodox convent, had many levels in mind when they decided to use an icon as a frame for their representation of an ancient penitent. On the literal level, they envisage a three-panelled icon on stage. At the beginning of the performance, the outer panels are closed in on the central panel (see Figure 6).

Mary

Mary and Zosíma

Zosíma

the penitent whore

meeting in the desert

the righteous monk

(acts I + II)

(acts III–V)

(acts I + II)

Figure 6 The ‘musical icon’ of John Tavener’s Mary of Egypt

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

140 Mediations and Transformations of Media

When the two wings open a few minutes into Act I, the space inside them is occupied by the two main characters: on the left, Mary who gives herself up to the pleasures of a sensuous life in Alexandria, and on the right, a devout monk in his monastery in Palestine who will eventually learn from her that there is a devotion greater and more God-pleasing than his selfrighteous piety. The separate, parallel arrangement in the triptych’s two opened side panels is maintained throughout the first half of the opera. The work’s second half then takes place in the central panel, which represents the desert where the penitent’s and the monk’s worlds intersect. Shortly before the end of the final act, the icon panels close again, presumably allowing us to comprehend that while the particular story is completed, the larger one (and the music, for a little while) continues. When Tavener refers to his work as a ‘living icon’, he has more in mind than simply the visual aspect. The music is as stylized as the stage setting and acting, described as using ‘primary colors as the icon painters do’.4 Tavener’s ‘musical icon’ presents a fascinating compendium of musical symbols. The entire composition is crafted from a limited catalogue of ten components, distinguished by tonal arrangement, metric organization, melodic features and timbral realization, with very little overlap between one group and the other. Of the ten musical elements, five are introduced ‘before the curtain rises’ as it were, that is, before the icon’s side panels on stage swing open and reveal the protagonists. These five musical elements must thus be comprehended as signifiers of an eternal, never-changing, divinely inspired universe that exists before, during and after the particular human story. Once the stage scenery has become visible, they are complemented by five thematic components that embody the players in the dramatic action. The first musical symbol that listeners consciously perceive while the stage still suggests – to those who have eyes to see – the extra-dramatic world of the closed icon, is a sequence of chiming sounds created by hand bells struck with soft sticks. The performance indication requests that they ring ‘with unearthly stillness’ and allows freely chosen distances of 1–2 seconds between the individual strokes. The sequence, encompassing 25 notes, is built as a palindrome, that is, in perfect horizontal symmetry (see Figure 7). Musical palindromes are interesting above all for their spiritual significance. In the realm of human experience, the irreversibility that defines all acts, be they physical or linguistic, the irreversibility that defines the course of a day or life and the expected execution of a plan, are of a quality

Figure 7 John Tavener, Mary of Egypt: schematic representation of the hand bells of ‘unearthly stillness’

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Siglind Bruhn 141

intrinsically different from reminiscences, regrets, nostalgia and other acts or feelings turned toward the past. Imagining a point where this distinction no longer applies means leaving the realm of time as humans know it – that time which, together with space, provides the coordinates for life in our universe. On this ground, musical palindromes offer themselves as signifiers of timelessness, of the annihilation of time or, in a religious context, of eternity. The device is also associated with connotations of perfection, an essential quality of the Divine. If the effect of symmetrical designs in works of visual art is one of balance and aesthetic pleasure, the result of similar mirroring processes in art forms that evolve in time is quite different. Whenever the segment that will turn into its own retrograde comprises more than five or seven – at most, nine – notes, the resulting symmetry is beyond the grasp of human listeners. In other words: the distinguishing attribute of this melodic line has a profound spiritual meaning, which however lies outside the perceptive faculties of all human beholders. Although each note, and the succession, is easy to hear, the line’s overall shape, and with it its transcendent message, are not. (Spiritually speaking, it is of course utterly appropriate that we should not be able to perceive what is by definition outside of human experience.) The opera’s very first signifier escapes listeners for a different reason. It is in fact the most pervasive sound phenomenon in the composition: a protracted drone on the pitch F. This sound seems to emerge imperceptibly from the silence preceding the composition and to vanish equally gently back into this silence. In the music of the Byzantine rite, such a drone is used to accompany liturgical singing.5 With regard to the spiritual message conveyed in this opera, it stands for the eternal presence of God, imperturbable no matter what happens. With regard to Tavener’s ‘musical icon’, it is analogous to the golden background that surrounds painted images in the icons of the Orthodox rite. Just as the faithful praying in front of a visual icon will focus on the stylized features of the saint and largely neglect the backdrop, so also in this music: the drone is so unimposing that it seems to fade out of the hearing range of listeners. It is only interrupted when, on three successive occasions, Mary threatens to become entirely disconnected from God. In these instances, even lay listeners who claim to have had no conscious awareness that there ever was such a backdrop sound comment that ‘something intangible has changed’.

Interpretation or digression: risks and freedoms at the mediator’s angle In the two examples discussed above and in many others, performers and conductors can do nothing to facilitate the audience’s access to the signification that is being offered. In other cases, interpretation makes all the difference, and it therefore matters what and how much the interpreter

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

142 Mediations and Transformations of Media

understands. In particular, genres involving a multiplicity of media raise the question of the role of choreographers and opera directors. Many an operatic staging captures the audience so powerfully through visual images – especially where video projections are added to real-time actions on stage – that the music is relegated to the role of a cinematographic sound track: a sonic tapestry that is meant to highlight the emotional qualities of a story presumably told primarily through verbal and visual means. While that may be fine in the case of works that are composed with just this effect in mind, one wonders whether it is equally appropriate in the case of operas whose music speaks a language rich in hermeneutic baggage? Does not an approach privileging visual effects thwart the audience’s potential access to the musically conveyed messages? A related question concerns choreographers wedded to the late-twentiethcentury concept of ‘non-redundancy’ and therefore striving for, as one of them explained to me, ‘a dance that dialectically engages rather than duplicates the music’.6 After the first thrill of this ‘dialectic engagement’ wears off, such choreographers often receive feedback telling them that the disjuncture they create is such that spectators can only take in the dance if they tune out the music as one would the muzak in a restaurant during an interesting conversation with a friend. Reflections on the immediate versus the successively revealed disclosure of a work’s message inevitably lead to the role of the mediator. If the performance of a work, while no doubt a creative act, is meant as an ‘interpretation’ that facilitates rather than veils the beholders’ access to the conveyed signification, then musicians, actors and dancers, would ideally invite their audiences to respond to the primary artwork’s content, rather than to their skills. This raises a question regarding a medium that seldom speaks through interpreters: plastic art. Is the fact that visual artists do not (normally) rely on, much less depend on, interpreting mediators, only an advantage or can it also sometimes foreclose avenues of access? How do artists feel about the (still fairly unusual, always secondary, but often most interesting) interpretation of a work of theirs in poetry, music or dance? And this is only one of the many ways in which a primary representation is re-presented in another medium? In the final component of my essay, I will briefly discuss a musical piece based on a poem that responds to a painting, in other words: a song whose lyrics are an ekphrasis and whose music adds a novel perspective to what James Heffernan has succinctly called ‘a verbal representation of visual representation’.7

Case study 2: musical signifiers enriching an ekphrastic poem In 1937, the English painter Roland Penrose, who had worked in the Parisian Surrealist circles for many years and introduced Surrealism to the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Siglind Bruhn 143

British Isles, completed a canvas alternatively entitled Voir c’est croire (Seeing is Believing) or L’Île invisible (The Invisible Isle).8 The painting shows simultaneity of night and day. At the top, a dark sky with a new moon, a few stars and white clouds seem to bleed out into the shoulders of a young woman whose head hangs upside-down, covering much of the central section, which is bathed in sunlight. Her wavy blond hair spills into the ocean below her and forms the backdrop and visual enclosure for a small, rocky island occupied by more buildings than the limited space seems able to hold. One distinguishes a port with the masts of sailing boats. The reflections on the water suggest bright weather; at the same time, the sky at the right-hand side releases a sheet of rain. From the barely suggested shore in the foreground, two hands in a size roughly corresponding to the scale of the woman’s head reach up – one real, the other a mere shadow. Their gesture is vertically directed toward the lovely face but horizontally separated from it by the expanse of water and the depth of the island.9 The French composer Olivier Messiaen, a contemporary of Penrose, saw a reproduction of the painting in the Swiss art magazine Forme et Couleur and included his response to it as number ten in a cycle of 12 songs for dramatic soprano and piano, Harawi, composed in the summer of 1945. Messiaen loved the poetry of Paul Éluard, Pierre Reverdy and André Breton. For the five song cycles he composed during the years 1936–48, he wrote his own poetry, in a style that became ever more Surrealist. His interest in Surrealist art was of a very particular kind, however: a devout Catholic fascinated with miracles and supernatural occurrences of any kind (from fairytales to biblical stories), he searched for the religious dimension in non-real representations. Here is the poem Messiaen wrote in response to Penrose’s painting, together with my translation. X – ‘Amour oiseau d’étoile’ ‘Love Star-Bird’ Oiseau d’étoile, Star-bird, Ton œil qui chante, Your eye that sings, Vers les étoiles, Toward the stars, Ta tête à l’envers sous le ciel. Your head reversed under the sky. Ton œil d’étoile, Your star-eye, Chaînes tombantes, Falling chains, Vers les étoiles, Toward the stars, Plus court chemin de l’ombre au ciel. Shortest path from the shadow to the sky. Tous les oiseaux des étoiles, All star-birds, Loin du tableau mes mains chantent. Far from the scene my hands sing. Étoile, silence augmenté du ciel. Star, increased silence of the sky. Mes mains, ton œil, ton cou, le ciel. My hands, your eye, your neck, the sky. (Olivier Messiaen, ‘Amour oiseau d’étoile’, from Harawi: chant d’amour et de mort)

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

144 Mediations and Transformations of Media

Messiaen’s poem suggests that he interpreted Penrose’s painting as symbolizing the relationship between earth/water and sky/heaven, between body and spirit. He describes the young woman’s head as ‘reversed under the heaven’, believes her eyes to be ‘singing toward the stars’ and her freely falling hair (‘falling chains’) to be mediating between the painting’s two worlds. He identifies with the hands in the foreground, which he knows to be ‘far from the scene’ defined by the woman linking night sky and sunlit island. The poem’s final line traces the central images once again in their upward-leading path: ‘My hands, your eye, your neck, the sky’. The poetic text also includes several of Messiaen’s favourite terms, even though what they describe is not evident or at least not explicit in Penrose’s painting: bird, song, star and silence. In the beginning, the poetic voice addresses the young woman as ‘oiseau d’étoile’ (star-bird); later, as the speaker turns to the hands that represent him, he takes up the image by alluding to ‘Tous les oiseaux des étoiles’ (all star-birds). The inner connection between eye, singing and star, established in the symbolic imagery of the woman’s eye singing ‘toward the stars’, finds its equivalent on the interpreter’s side when his observation that his hands sing leads immediately back to the stars. The attitude expressed in the longingly raised hands and their distance to the intuited closeness ‘of shadow and heaven’ accounts for the increased silence of the heaven. The music Messiaen composed for this song imports its own nuances of interpretation. Some of them are universal, others could be described as a kind of idiosyncratic semiosis typical of this composer at the period when he wrote this song cycle, and a third category relies on our recognition of a specific musical quotation (or rather, a paraphrase) of material for which Messiaen is well known. I would like to begin with the third category, so that I can lead readers from the specific to the universal. One year before he composed the song cycle Harawi, Messiaen had written a cycle of twenty pieces for piano solo entitled Vingt Regards sur l’Enfant-Jésus. That cycle thus celebrates various contemplations of the child in the manger or gazes cast on the infant Jesus. The principal cyclical theme in this composition is labelled in the score: thème de Dieu, ‘theme of God’. Its basic phrase consists of five chords. The melody harmonized with these chords is very simple, consisting of a threenote repetition, a climax a little higher and a return to the earlier pitch. The rhythm of this little phrase is also very simple: attacks of the three-note repetition are all equal in length, the climax is twice as long and the final relaxation, three times as long. The tempo of this piece – as it were: the temporal dimension of God’s eyes falling on the infant Jesus – is extremely slow: Messiaen imagines each of the repeated, ‘faster’ note values to last for three normal heart beats. In the song ‘Amour oiseau d’étoile’, the main line – ‘Oiseau d’étoile’ – is composed in the same key as ‘Regard du Père’ and has almost exactly

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Siglind Bruhn 145

the same melodic line as the ‘theme of God’, except that the note repetition occurs on the level of the climax, not on that of the release. Also, the rhythm is slightly changed: instead of beginning with three beats of equal length followed by ever longer note values, the duration now increases incrementally all along (that is, instead of counting 1-1-1-2- -3- - one now counts 1-2- -3- - -4- - - -5- - - - -). This is the thematic link: a kind of varied quotation. There is also a direct structural link: the initial movement of the piano cycle, devoted to ‘The Father’s gaze on the infant Jesus’ and focusing entirely on the ‘theme of God’, is formed in accordance with the triad of the Pindaric ode, with two stanzas (strophe and antistrophe) rounded off by a substantial coda (epode). The same layout is found in the song ‘Amour oiseau d’étoile’. The second realm of musical signification in Messiaen’s song on the poem he wrote in response to Penrose’s painting is a case of what I like to call idiosyncratic semiosis. This aspect begins on the level of the poem and continues in the music. Olivier Messiaen was a bona fide ornithologist who is today recognized worldwide as the person who has most accurately transcribed the song of hundreds of birds. From the 1950s onward, Messiaen wrote a number of compositions exclusively devoted to the musical representation of bird calls, with some sonic descriptions of the birds’ characteristic habitat: Le Merle noir (The Blackbird), 1951, for flute and piano; Réveil des oiseaux (Awakening of the Birds), 1953, for orchestra, piano and gamelan; Oiseaux exotiques (Exotic Birds), 1956, for orchestra, piano and gamelan; Catalogue d’oiseaux (Catalog of Birds), 1958, for piano; La Fauvette des jardins (The Garden Warbler), 1970, for piano; Des Canyons aux étoiles (From the Canyons to the Stars), 1974, for orchestra and soloists; Petites Esquisses d’oiseaux (Little Bird Sketches), 1985, for piano; Un Vitrail et des oiseaux (A Stained-glass Window and Birds), 1986, for orchestra, piano and gamelan. All through his life, Messiaen interpreted the music of songbirds as a sign of God’s presence in nature. During the 1940s in particular, however, when the song under discussion originated and his birdsong transcriptions were not yet fully developed, he often used birds to signify human silence, especially the silence filled with awe in view of the transcendent.10 Indeed, ‘birds’ appear in the title of Messiaen’s poem, ‘Amour oiseau d’étoile’ (‘Love Starbird’) and recur in several lines, although I have not been able to detect a single bird in Penrose’s painting. For birds, one may thus read ‘silence’. In the music of the Penrose-inspired song, the birds rejoice in the piano after every single phrase. The most significant representative of the first, universal category of interpretation in this song is an element of musical symmetry. This time, however, the symmetry is vertical and not horizontal, spatial and not temporal. Thus there is no constitutional hurdle for human understanding, as in the case of a sequence read backward. While most of us will still not necessarily grasp details about the relative distance between simultaneously

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

146 Mediations and Transformations of Media

major triad

with in added first sixth inversion

Figure 8 Messiaen’s favourite vertically-symmetric chord: the generation of its spatial pattern and its distribution on a keyboard

sounded pitches when hearing a chord, it is a strange fact that almost all people can, somehow, hear and recognize the effect created, especially the one achieved in the most frequently used vertically-symmetric chord. The technical description of this chord is not essential here; for musicians, its basic form is a major triad with added sixth, in first inversion. For an easy way of making this vertical symmetry of sounds visual, it is helpful that Messiaen uses only, and always when he uses this chord, the one in F# major, and in this key, the chord even looks symmetric on any keyboard of a piano, organ, xylophone and so on (see Figure 8). In the piano piece ‘Regard du Père’, the very same chord is heard as a concluding chord repeated 24 times. A little later in the piano cycle, the chord also participates in another prominent theme, one that Messiaen’s score identifies as thème d’amour, ‘theme of love’. As a result of this dual appearance in the gaze of the loving heavenly Father and in the ‘theme of love’, I like to speak of this chord itself – the chord that is vertically symmetrical both in its interval structural and in its visual appearance on the keyboard – as the ‘chord of love’. This brings me to another word in Messiaen’s poem that is not entirely explained by Penrose’s painting: the word ‘love’, which appears only, but very prominently, at the outset of the title ‘Amour oiseau d’étoile’. If the remainder of Messiaen’s poem does not explicitly stress the quality of love, Messiaen’s music does so all the more. For listeners, the predominant symbolic message the composer conveys in this piece resides in the conclusion of each phrase. The first ten segments of the song (eight in the stanzas and two more at the beginning of the coda) each conclude with two F-major six-five chords; the coda’s two final segments each add one more instance of the characteristic chord. More than half of these harmonically equivalent chords are cast in a vertically symmetrical format, thus appearing as variants of Messiaen’s ‘chord of love’. One could expand this symmetrical chord by repeating one or more pitches on either side. Messiaen does so with considerable regularity, mostly adding the same number of pitches above the piano chord’s highest and

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Siglind Bruhn 147

below its lowest note. The composer uses the vertically symmetrical chord of love at the end of every vocal phrase, in different expansions. In the coda’s long final phrase, the chord on the downbeat of m. 23, which provides the backdrop for the most extensive bird song in this piece while separating the summary remark (‘My hands, your eye, your neck, the sky’) from the body of the poem, presents the most expansive of the vertical symmetries: a chord of ten vertically symmetrical pitches. Thus the word ‘amour’, prominent in the song’s title but not verbally present in the poetic text that is sung, is embodied (and even ubiquitous) in the music. The musical language confirms the composer’s trust that, while the union of the two lovers suggested in Penrose’s canvas may seem unattainable, they themselves are reliably supported by God’s love.

Conclusion As I have attempted to show, many of the triangles that are habitually constructed in the field of the arts have at least one angle that may make the necessity of closure and geometric logic seem ‘impossible’. This is true not only for the notorious ‘artistic triangle’ supposedly linking the three participants involved in the life of an artwork – creator, mediator and beholder, but it is also true in the case of triangles defining each of these participants individually. The example I fashioned with the Hindu Trinity as a metaphoric trio shows this with regard to three essentially different creative aims; my case study from Tavener’s opera documents three conditions in which the limits of human sense perception displace the beholders’ appreciation from the conscious to the subconscious realm; my example from Messiaen’s art song demonstrates at once the extraordinary richness of an artistic mediator’s input and the highly individual reading of the first work of art he or she may incite in the final appreciators by presenting it through the lens of his or her own, secondary medium. Were one to attempt to come full circle, from the piece of music interpreting a poem that in turn responds to a painting and now back to Penrose’s (and Reutersvärd’s) ‘impossible triangle’, one could observe that the repeated mediation does indeed lead to an interestingly twisted appearance of the three-dimensional object: returning to the point of departure after having travelled through the three hinged legs, one notices with wonder that none of the surfaces is where it was before. Mediation – in the various meanings of this term – is a creative act that changes its object’s appearance and message.

Notes 1. Sir Roger Penrose, OM, FRS (born 8 August 1931) is an English mathematical physicist and Emeritus Rouse Ball Professor of Mathematics at the Mathematical Institute, University of Oxford. Sir Roland Penrose (1900–1984) was an English artist, historian and poet. He was a major promoter and collector of modern art, an associate of the Surrealists in the United Kingdom and a friend of Picasso.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

148 Mediations and Transformations of Media

2. L. Elleström (2002) Divine Madness: On Interpreting Literature, Music, and the Visual Arts Ironically (Lewisburg PA and London: Bucknell University Press). 3. John Tavener, Mary of Egypt: An ikon in Music and Dance, was composed in 1991. The score was published by Chester Music, London, in 1992; a compact disc recording was issued by Collins Classics in 1993 (no. 70232). 4. ‘John Tavener in conversation with Michael Stewart on Mary of Egypt’, recorded interview added as track 20 to the Collins Classics recording of Mary of Egypt. 5. For more details on this liturgical music, see K. Levy (2001) ‘Music of the Byzantine rite’ in S. Sadie (ed.) The New Grove Dictionary of Music and Musicians, 2nd edn (London: Macmillan) pp. 553–66. 6. Peter Sparling, professor of dance at the University of Michigan School of Music and choreographer of the Peter Sparling Dance Company, in a private conversation with the author, fall 2003. 7. See J. A. W. Heffernan (1993) Museum of Words: The Poetics of Ekphrasis from Homer to Ashbery (Chicago: University of Chicago Press), p. 3. In a more encompassing wording, Claus Clüver defines ekphrasis as ‘the verbal representation of a real or fictitious text composed in a non-verbal sign system’; cf. C. Clüver (1997) ‘Ekphrasis Reconsidered: On Verbal Representations of Non-Verbal Texts’ in U.-B. Lagerroth, H. Lund and E. Hedling (eds) Interart Poetics: Essays on the Interrelations of the Arts and Media (Amsterdam: Rodopi), p. 26. 8. The original was long believed to have been lost during World War II. However, in a recent book by Penrose’s son, it is listed as ‘held in a private collection’: A. Penrose (2001) Roland Penrose: The Friendly Surrealist. A Memoir (Munich: Prestel), p. 48. 9. A full-colour reproduction of Roland Penrose’s oil-on-canvas composition of 1937 Voir c’est croire (L’Île invisible) can be found at http://home.vrweb.de/∼ editiongorz/bruhn5-06.pdf (p. 215), date accessed 13 August 2009. 10. Cf. both ‘les oiseaux du silence’, mentioned in the commentary for the Regard du Fils sur le Fils (no. V), and Regard du silence (no. XVII).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Siglind Bruhn 149

9 Valerie Robillard

Theories [are] instruments, not answers to enigmas, in which we can rest. We don’t lie back upon them, we move forward, and, on occasion, make nature over again by their aid. Pragmatism unstiffens all our theories, limbers them up and sets each one at work. —William James1 As media continue to merge in new and interesting ways, the various theories and definitions formulated to describe and articulate the nature and complexity of their interactions have generated compelling, and often conflicting, critical discourses. Indeed, the current plethora of perspectives on ‘intermediality’ not only demonstrates the slipperiness of the term but also suggests that there may be more than one theoretical inroad by which to fully understand the multiplicity of intermedial operations. Current research into intermediality, with some notable exceptions,2 has primarily focused on defining the terms of the field; however, it is becoming increasingly clear that definitions, although essential in laying out common terms of discourse, do not fully contribute to our understanding, or articulation, of the various types and degrees of medial interaction. The purpose of this essay is to demonstrate the need to employ ontological systems to delineate medial types, systems that have proven indispensable to other disciplines (such as the natural sciences and linguistics) in determining the relative positions of concepts and categories with respect to one another. Although the notion of ‘exact disciplines’ can be debated, the sciences have proven to be more easily ordered into categories than have the arts, as the latter are closely tied to fluctuating cultural and aesthetic codes. This essay will explore the advantages of applying a system of categories to intermedial inquiry by focussing on the intertextual exchange between verbal and visual texts within the hybrid artwork. The advantage of exploring intermediality (partly) in terms of intertextuality lies in the fact that the interaction between the visual and verbal arts can be considered in terms of their interreferentiality and semiotic encoding. As Peter Wagner has 150 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Beyond Definition: A Pragmatic Approach to Intermediality

Valerie Robillard 151

intermedial exploration of the working of both linguistic and pictorial signs in one medium. Intertextuality in art thus becomes a possibility in research . . . in a more extensive variety of ekphrasis, broadly understood, which considers visual art as a space serving as both meeting ground and battle ground for encoded, rhetorical, sign systems that refer to texts and images.3 Wagner’s argument for subsuming verbal and visual operations under the heading of ‘intertextuality’ offers intriguing possibilities for the categorization of distinct intermedial types. Heinrich Plett, for example, has suggested that intermediality be considered a sub-category of intertextuality, as ‘it is not usually single signifiers which are exchanged for others but themes, motifs, scenes or even moods of a pretext which take shape in a different medium.’4 Both Wagner and Plett make a good case for considering intertextuality within the intermedial paradigm (although the hierarchy implied by Plett will be questioned later in this essay). To address the operation of intertexuality within an intermedial framework, I will explore one small theoretical corner of the intermedial debate, that of ‘ekphrasis’ and ‘illustration’, both of which, as I have argued elsewhere,5 can be analysed in terms of their intertextuality. I hope to demonstrate that such complex combinations require a broad epistemological basis and ontological system by which to explore and articulate the variety and types of their interaction.

Categorizing intermedial operations Why employ categories in aesthetic discourse since, by their very nature, categories are imperfect representations of phenomena? Can the demand that categories be ‘mutually exclusive’ (the ‘bottom line’ for scientific models) be useful (or even possible) for studies in the humanities whose objects are often far more ‘subjective’ than the empirical sciences? In other words, categories, because of their perceived rigidity, may seem inappropriate for organizing concepts that are generated by cultural or aesthetic codes. On the other hand, categories, unless they are mutually exclusive, will not accurately represent difference.6 In spite of the obvious pitfalls associated with classifying intermedial types, the formulation of a system of categories which takes into account the ‘slippage’ associated with cultural or aesthetic encoding might successfully contribute to understanding the operations and interrelations of media through isolating and categorizing their various domains and characteristics. The typology presented here, which I have called ‘The Differential Model’, is

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

suggested, the discourse on intermediality needs a reunification of writing and painting under ‘the common banner of representation’ and an:

152 Mediations and Transformations of Media

Differential model This section will present a conceptual framework devised to differentiate types of medial interactions, most specifically between the verbal and visual arts. As, similar to verbal ekphrasis, the majority of ‘illustrated’ texts enlarge, reduce or in some way alter their pretexts – and for similar reasons – these operations can also be traced within a categorical framework. The typology in Figure 9 is designed to be (a) non-hierarchical and (b) one whose categories serve as vehicles for making generalizations about an artwork’s representative, referential and interactive capacity. This model differentiates between explicitly-marked visual or verbal texts and those which signal a more subtle, associative relationship with their pre-texts. Each heading is divided into subheadings. The first category in the typology, labelled as ‘Referentiality’, reflects the assumption that ekphrastic texts, because they are specific to particular works of art, will, in the rhetorical spirit of ekphrasis, mark their pre-texts to some degree. In a poem, for example, the name of the artwork or artist might be explicitly mentioned in the title or elsewhere in the text. Similarly, in a visual work of art the verbal pre-text might be mentioned in a label next to the painting or attached to the frame, or, as in some modernist texts, appear within the painting itself.

naming Referentiality

allusion indeterminate marking selectivity

Re-presentation structurality

mythos/topos Association dialogicity Figure 9 The differential model

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

divided into categories and sub-categories and focuses on types and degrees of intertextual relationships.7 This model, originally designed for the purpose of delineating and differentiating logo-centric ekphrastic operations, has been altered to accommodate intertextual operations on both sides of the relationship (verbal-visual, visual-verbal).

The second sub-category, ‘allusion’, subsumes more subtle references in which the pre-text is called to mind through the presence of some (generally) familiar aspect present in the target text. Finally, the sub-category ‘indeterminate marking’ accounts for the presence of some reference that would not be generally understood as connected to a pre-text, but would be accessible to a viewer (or ‘ideal reader’) who has subject-specific knowledge or a particular ‘cultural memory’. The second category, labelled ‘Re-presentation’, differentiates between the ways in which the textual details of the verbal or visual pre-text are present in the target text. The sub-category, ‘selectivity’, refers to the detail from the pre-text that is transferred to the target text;8 the second sub-category, ‘structurality’, refers to the manner in which a text follows the physical organization of its pre-text. (In the visual arts, for example, this is often encountered in the triptych, where the narrative structure of the original story is followed in a linear fashion; in the verbal arts, a poet might organize the objects of the poem around the structural conventions of, say, a cubist painting.) Finally, the ‘Associative’ category accounts for those texts concerned with conventions or ideas associated with the visual or verbal pre-text (mythos/topos) or are ‘dialogical’ in that they deliberately establish a tension (both semantic and ideological) between the original text and the new, wherein the former is cast in a new, opposing framework.9 (A filmic example of ‘reframing’ would be Peter Greenaway’s Prospero’s Books, based on Shakespeare’s play The Tempest.) This final category is linked to the referential properties of the text (Category I), as reception of these intertextual elements will depend on level of ‘readership’.10 The following section will explore the possibilities of such a typology by considering the complexities of the Scottish artist Calum Colvin’s Fragments of Ancient Poetry (2002), focussing on the manner in which both verbal and visual texts interact to convey meaning.11

The intermedial artwork Colvin’s Fragments of Ancient Poetry, a series of photographs based on James MacPherson’s 1760 text of the same title, is unique in its movement toward a complete intermedial performance, as it is situated, according to Colvin, ‘on the boundaries of Painting, Photography, Sculpture and Electronic Imaging’.12 Through his employment of a wide variety of media, Colvin establishes a multi-levelled relationship with the original text. Before analysing Colvin’s work as an intermedial ‘event’, however, I would first like to address the aesthetic conventions that it challenges – namely those connected with ‘illustration’. Most definitions of illustration suggest that the translation of the verbal to the visual necessitates some level of exemplification of the pre-text in which the visual becomes an emblem for the verbal. Yet nearly all (art) historical periods have produced a wide variety of visual approaches to verbal texts

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Valerie Robillard 153

that far exceed simple transcriptions. Meyer Schapiro has suggested a variety of ways in which illustrations engage their visual pre-texts, ranging from those that ‘enlarge’ the verbal text, adding details, figures, settings and so on, that are not present in the original, to those which are extreme reductions of the text in which only a few details are represented. These artistic choices are often the function of cultural, social and/or aesthetic concerns of a particular time and place, or reflect individual artistic style or ideology.13 I would like to postulate that, as these operations apply equally to verbal and visual ‘texts’, both can be subsumed under the heading of ‘ekphrasis’ (in a departure from the logo-centric origin of the term). Proceeding from this understanding of the term, the textually-based aspects of Colvin’s Ossian series, therefore, can be classified as a type of ‘visual ekphrasis’ which belongs to the overarching notion of ‘intertextuality’. As will become clear, however, this intertextual aspect accounts for only a small part of the intermedial and multimedial nature of Colvin’s Fragments, which are, as Tom Normand points out, ‘created as constructed sets, painted with iconic subjects, decorated with symbolic references, and, finally, photographed. The photographic images are subsequently digitized and presented on canvas. Sculpture, environment, collage, painting, photography, and computer-art combine in a paradoxical and fantastic vision.’14 What are the implications of attempting to categorize both textual and extra-textual media in works such as Colvin’s? Colvin’s Fragments depicts the discrepancies between a culture’s perception of its past and modern realities through exploring the epic nature of MacPherson’s Fragments of Ancient Poetry, supposedly the poet Ossian’s account of the heroic deeds of Fingal, a third-century Scottish king. In spite of ongoing scepticism concerning authorship, this literary work not only had an important impact on Romantic sensibility, but has become an integral part of Scottish identity. These epics, which ‘conjured a world of heroic northern warriors whose savagery was tempered by distinguished codes of honour, sentiment, and a recognizable morality’,15 form the basis of Colvin’s intermedial artwork. In a series of eight photographs, Colvin addresses the relevance of these epic conventions to contemporary Scottish identity by juxtaposing these with objects from Scottish popular culture and more recent literary past. The importance of Colvin’s artwork to intermedial studies in general (and ‘visual ekphrasis’ in particular) is significant, as it far exceeds simple exemplification of its pre-text and the variants of illustration suggested by Schapiro. According to Iain Gale, Colvin ‘wants to peal away the layers of historical image-making which obscure Scottish culture’ and with works such as Fragments, ‘he is reflecting on how Scotland has become a nation of stereotypes’.16 Colvin achieves this dialogical effect not only by juxtaposing fragments of popular culture with subtle references to the Ossianic tradition, but also by making use of wider references to cultural

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

154 Mediations and Transformations of Media

archetypes and literary texts associated thematically with the issues raised by the original: Macpherson’s Fragments of Ancient Poetry. In Fragments, Colvin arranges several objects against a stone backdrop, objects that serve as literary references (primarily to MacPherson, Walter Scott and Robert Burns) or perform a metaphorical function (see Figure 10). Colvin begins the series with a Maori head projected onto a movie screen, an image suggesting the sense of a Celtic past.17 Over a sequence of eight photographs, Colvin transforms this central image into a cultural cliché: a tipsy Scotsman sporting a ‘Jimmy hat’. Eventually, this image too disappears, ‘crumbling into dust, becoming the stuff of legend’.18 Further, these photographs contain oblique references to the epic tradition embodied by the Ossian texts through the incorporation of fragments of Colvin’s portraits of Robert Burns and Walter Scott: an eye borrowed from Burns and an ear from Scott, ironically placing the heroic epic against the romanticized

Figure 10

Calum Colvin, Fragments of Ancient Poetry IV. Courtesy of the artist

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Valerie Robillard 155

Highland adventures of Scott and the lyrics of Burns. Also included in this stone landscape are modern icons that establish a similar ironic connection between the past and present, between ancient and popular culture: a slide projector showing images from the film Brigadoon, a record player (with the drunken Scot on the turntable – later to develop into the central image) – and a souvenir Celtic lollipop. These are just three objects among the many that Colvin adds to this ruined landscape, but they are sufficient for our discussion here: the intermedial tension achieved by placing literary and cultural icons (both past and present) within a shared visual space.

Intertextual components of Fragments To what extent is Colvin’s Fragments ‘intertextual’? More importantly, what aspects of this work cannot be contained in such a text-based framework? I will explore these questions by turning to the ‘Differential Model’. The first question that we will want to ask is this: in what way is Colvin’s artwork ‘referential’? In other words, does it clearly indicate that it is based on Macpherson’s Fragments of Ancient Poetry? Because the pre-text is identified in the title of Colvin’s work, referentiality is high. We also find several literary allusions and some intriguing indeterminate marking that are indexical of the original text. Furthermore, the inclusion of the (partial) images of Burns and Scott place the work within more than one literary context, and these visual allusions fall under the sub-category of ‘dialogicity’, which will be discussed below. Moving to the ‘Re-presentative’ category, it is clear that few details of the original text have been selected. However, what at first glance appears to be a series of still-life images turns out to contain a narrative progression achieved by a rearrangement of objects and the transformation of one image into another. Although the artwork does not follow Ossian’s narrative line, it nevertheless literally performs the larger issue of cultural change: the ‘measured disintegration’19 of a people that governs Macpherson’s text and both artworks in Colvin’s series (Blind Ossian and Fragments). This brings us to the final, ‘Associative’, category. If Colvin’s Fragments does not explicitly engage its pre-text, either through reference or re-presentation, it does, nevertheless, have a highly symbolic and metaphorical relationship to its pre-text and this relationship is ‘associative’ through its play on cultural memory and associated myths. Furthermore, we can claim that dialogicity in this work is high since the issue of Scottish identity lies at the heart of the artwork’s conception and, most importantly, its construction. Recalling that dialogicity refers to the way in which a verbal or visual text ‘reframes’ its pre-text or places it in another context, the use of Macpherson’s text is dialogical in that it raises the issue of the demise of mythic stature in the modern world. Aside from its dialogical stance, Colvin’s Fragments is concerned with the ‘mythos/topos’

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

156 Mediations and Transformations of Media

that generated Macpherson’s text and its cultural and literary aftermath primarily by stressing the absence of ancient warriors and a noble race. The placement of the head of Walter Scott in this construction is associative in the sense that, influenced by the romanticism of the Ossian fragments, Scott popularized Highland history through his novels, yet was also regarded as responsible for its trivialization and for generating much Scottish ‘kitsch’ (tartan kilts, Jimmy hats and so on), criticized as ‘the fatal legacy of Scott’s achievement’.20 The inclusion of the film Brigadoon (more romantic kitsch) in all eight plates serves to underline the overall theme of a lost or misunderstood national identity. Finally, the work is replete with semiotic codes for the modern world: for example, the ‘Celtic’ lollipop wrapper, the drunken Scot and the Jimmy hat, become the ‘potent symbol of Scotland’s self-denigrating identity crisis’.21 In view of the fact that Colvin uses Macpherson’s text as a template upon which to explore questions concerning cultural heritage and its deconstruction, we can arguably conclude that the intertextual relationship of Colvin’s visual text to its verbal pretext is primarily ‘associative’ (more specifically, dialogical) and not in any way depictive – ultimately rendering an artwork that is a far cry from the conventional notion of ‘illustration’.

Intertextuality? Intermediality? Multimediality? The complexity of Colvin’s Fragments raises interesting questions concerning the ways in which we define intermediality and categorize its operations. If, following Plett, we understand intermediality to function as a variant of intertextuality, then we will be concerned with textual semiotics and with the manner in which the diverse elements of a pre-text are incorporated into the new text (in this case, the artwork). In applying the ‘Differential Model’ to Colvin’s Fragments, however, we find that there is much present in this complex work that cannot be accounted for within an intertextual framework. Although the iconic (and ‘found’) objects that have been added to Colvin’s artwork are not related to Macpherson’s Fragments of Ancient Poetry but to contemporary popular culture, these additions can be categorized as ‘dialogical’, as the placement of ‘life media’ in juxtaposition with other (oblique) references to Macpherson’s text render a kind of spatiotemporal contextual ekphrasis, as defined by Hans Lund. According to Lund, timecontextual ekphrasis is generated by references to other artworks or texts which arouse a reader’s associations or ‘memories’ and it is the reader ‘who links the verbalized image to the memory of other images’.22 Spatiocontextual images are linked to each other through ‘the physical space in which they are contemplated’.23 In the case of Colvin’s Fragments, the placement of various objects within the shared visual space evoke several levels of reading on the part of the viewer and the resulting dialogical effect is crucial to Colvin’s overall theme of cultural loss.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Valerie Robillard 157

However, what the above analysis has demonstrated is that there is no vehicle within an intertextual system of categories to account for the materiality of Colvin’s artwork nor is there a way to categorize the process of composition, which involves the interaction of a number of media. As we know, Colvin’s work is based on a complex process: an image first constructed in three-dimensional space is painted, photographed and finally destroyed. All that remains are the photographs: ‘layers and traces that must be uncovered by the reader’.24 Colvin also produces meaning in his entire Ossian series by the use of computer imaging. It has been argued that Colvin’s employment of the computer for the series addresses an important and controversial aspect of the original text: that Colvin was ‘inspired to resurrect Macpherson’s texts by the dubious authenticity of the Ossian myths and their similarity to the deceptions possible with modern digital photography, which also distort and subtly change images of reality’.25 Colvin’s work, therefore, not only takes a dialogical stance towards Macpherson’s original text but also takes issue with the very process of its making. Colvin’s Fragments become even more interesting, for this study, when placed against major theories of intertextuality, namely those of Michael Riffaterre, whose reader-oriented theories focus on the hypogram. As Colvin points out, his use of the fixed viewpoint of the camera is designed: to collect the manipulated and constructed image in order to create elaborate narratives. These narratives have the quality of being both open and closed. They are closed in that they clearly refer to given icons and archetypes of Western culture, but open in that they accommodate any number of potential readings. These readings, in turn, reflect the contemporary cultural climate and the unique authorial role of the viewer.26 This notion of the ‘closed’ and ‘open’ narrative recalls Riffaterre’s differentiation between obligatory intertextuality, which demands that the reader take account of some ‘hypogrammatic origin’, and aleatory intertextuality, which allows the reader to read a text through the prism of all and any familiar texts.27 Furthermore, Riffaterre has posited three kinds of ‘semantic indirection’ by which the reader/viewer is led to consider the hypograms that lie behind the target text. These are: (1) displacement, where the sign shifts meaning, as in metaphor and metonymy; (2) distortion, which functions through ambiguity, contradiction or nonsense; and (3) creation, in which textual space serves as a principle of organization for making signs out of linguistic (and semiotic) items that may not be meaningful otherwise.28 Colvin’s series clearly makes use of all forms of ‘semantic indirection’, as set forth by Riffaterre, and in so doing has achieved the creative manipulation of visual space necessary to form a symbolic link between past and present, text and ‘life media’, and visual and verbal texts. As these links depend on

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

158 Mediations and Transformations of Media

the translation of one text to another, we can conclude that there is much in the series that is intertextual. However, there is more than one operation at work that cannot be contained (or explained) within an intertextual framework. The materials and media employed are not ‘texts’ and therefore are not referential; furthermore, in a work such as Colvin’s where media are used and then erased to produce the final artwork, the work bears traces of process which need to be accounted for. Here is where the slipperiness of terminology and categories comes into play. As it is clear that overall meaning in Colvin’s Fragments depends on both textual and medial interaction, the question arises as to how one system of categories or set of definitions can contain the interactive potential of such hybrid works.

Categories and medial difference We must be systematic, but we must keep our systems open. —Alfred North Whitehead29 Categories delineating medial types are, almost by definition, bound to be somewhat inaccurate in their attempt to contain various genres as well as to keep apace with the ever-developing and innovative ‘cross-over’ experiments of contemporary artists and writers. As Dick Higgins suggests, ‘much of the best work being produced today seems to fall between media’.30 This essay has argued in favour of the categorization of one aspect of intermediality: the (inter)textual aspects of hybrid artworks. Yet even the notion of ‘text’ is problematic, as our agreement as to what constitutes a ‘text’ remains elusive. According to Winfred Nöth (quoting Petöfi), ‘since textuality “is not an inherent property of certain objects, but is rather a property assigned to objects by those producing or analysing them,” it is not surprising that semioticians of the text have been unable to agree on a definition and on criteria of their object of research’. However, as Nöth goes onto explain, ‘In the pragmatic view, the text is defined by criteria of communication . . . the pragmatic criteria determine the text within its situational context. This context comprises textual and extra-textual phenomenon’.31 Taken strictly, the notion of ‘text’ may apply only to written texts or, taken more widely, to the visual arts, or it may be extended to all types of communicative processes such as film, performance (and gesture), iconic objects and so on. It is this wider sense of the word that gives us the freedom to explore the intertextual (and therefore encoded) nature of the composite artwork and to differentiate this type of medium from those which are ‘extratextual phenomena’. In other words, we are in a position to distinguish the ‘message’ from the ‘material’. Of course, this differentiation has raised serious questions of its own. If we can find ways to categorize intertextual operations within the intermedial framework, how do we account for the ‘material’ (extratextual) aspects of these composite artworks? For example, the depiction of a camera

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Valerie Robillard 159

and film screen within Colvin’s Fragments bears an encoded message relevant to its context and therefore is intertextual; the use, however, of a camera to construct this work is purely material and calls for different categories altogether, those that take process into account. For example, I have referred throughout this essay to Colvin’s Fragments as ‘photographs’, yet photography is only one medium used for the total artwork. How do we account for other medial traces within a categorical framework? This brings us back to the problem of categories in general. Returning to Plett’s suggestion that intermediality be considered a subcategory of intertextuality, the whole issue of ‘genus, species, class’ is raised in which we are again confronted with definition and nomenclature. Plett here addresses an important question of hierarchy: which is the largest, most inclusive category and which the sub-category? In other words, if intermediality is the overarching term, where does intertextuality, clearly a crucial element in intermedial discourse, come into play? Perhaps a way forward might be to reverse Plett’s hierarchy so that ‘Intermediality’ becomes the overarching concept, with one of its major categories labelled ‘Intertextuality’, which will account for those aspects of the composite artwork that are communicative and encoded; a sub-category of Intertextuality, then, might be labelled as ‘ekphrasis’, to be shared with sub-categories that account for other types of intertextual exchange (verbal–verbal or visual–visual transference for example). In this vein, a second category under Intermediality might then be classified as ‘multimedial’ (a term explored by Jürgen Müller in this volume and elsewhere). This category would include the materials employed (film, paint, computer technology and so on). The manner in which these could be placed into sub-categories might be drawn from cognitive categories such as those formulated by Lars Elleström (in this volume), which differentiate between types of sensory impact generated by the medial ‘event’. It is perhaps this type of category which could account for the layers (and traces) of media used by Colvin to create the final artwork, as these elements are not encoded messages but pure materials and therefore sensory. Once we have been able to place the various medial types within a categorical framework, we might then be in a position to address more general questions, such as those raised by Werner Wolf on determining ‘medial dominance’.32 It is clear that what categories offer us is the mechanism by which to schematically represent intertextual varieties and medial oppositions. Simply stated, the sciences (including linguistics) have taught us the usefulness of models in organizing experience and knowledge. Whether or not categories can be as accurately employed in literary discourse as they are in other disciplines, they do, nevertheless, bring us one step closer to a shared language and ontological system by which to advance our discourse on intermedial topics. According to Nelson Goodman, understanding the world of art is very much like understanding the world of science, as it always requires

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

160 Mediations and Transformations of Media

interpretation of symbols and their relations among themselves.33 Developing a shared system of categories, which would have the advantage of delineating subtype/super-types and articulating part–whole relationships, is as relevant in determining the positions of the aesthetic concepts with respect to one another as it is in categorizing genus and species within the more ‘exact’ disciplines. Recalling William James’s statement on pragmatism which began this essay, devising pragmatic categories need not set artificial parameters around our topic nor limit its extension to other potential developments within intermedial discourse. The real challenge, as I see it, is to find useful frameworks by which to articulate fundamental differences among the media before we can begin to explore the nature and extent of their interaction.

Notes 1. B. Kuklick (ed.) (1981) W. James, Pragmatism (Indianapolis and Cambridge: Hackett Publishing Co.) (first published in 1907), p. 28. 2. See, for example, essays by Lars Elleström, Jørgen Bruhn and Jürgen Müller in this volume. 3. P. Wagner (1996) Icons, Texts, Iconotexts: Essays on Ekphrasis and Intermediality (Berlin: Walter de Gruyter), p. 36. 4. H. Plett (1991) ‘Intertextualities’ in Intertextuality: Research in Text Theory (Berlin: De Gruyter), p. 20. In his essay, Plett suggests an intertextual framework based on ‘verbal and non-verbal signifiers, including the pictorial as a form of intertextual discourse’. 5. V. Robillard (1998) ‘In Pursuit of Ekphrasis: An Intertexutual Approach’ in V. Robillard and E. Jongeneel (eds) Pictures Into Words: Theoretical and Descriptive Approaches to Ekphrasis (Amsterdam: VU University Press), pp. 53–72. 6. This problem has been addressed by Elinor Rosch, who has suggested in her prototype theory that some disciplines need to employ ‘fuzzy categories’ to describe relations between concepts. See G. Lakoff (1987) Women, Fire, and Dangerous Things: What Categories Reveal About the Mind (Chicago and London: University of Chicago Press), p. 15. 7. The typology employed here was developed to delineate ekphrastic types inspired by Manfred Pfister’s notions of intertextuality. See Robillard (1998) ‘In Pursuit of Ekphrasis’, pp. 60–2. 8. The terms ‘selectivity’ and ‘dialogicity’ have been adapted from Manfred Pfister’s Intertextual Scales set out in his article, ‘Konzepte der Intertextualitität’ in U. Broich and M. Pfister (eds) (1985) Intertextualität: Formen, Funktionen, anglistische Fallstudien (Tübingen: Max Niemeyer). 9. Ibid. 10. This is where the idea of ‘mutual-exclusivity’ of a textually-based model breaks down. A future restructuring of such a model would need to provide several branches that account for the indeterminacy of reception. 11. One of Scotland’s leading contemporary artists, Calum Colvin is Professor of Fine Arts at the University of Dundee. His works have been exhibited in venues such as the Metropolitan Museum of Modern Art in New York, the Museum

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Valerie Robillard 161

12. 13. 14. 15. 16. 17.

18. 19. 20. 21. 22.

23. 24. 25. 26. 27. 28.

29. 30.

31. 32. 33.

of Fine Art in Houston, Texas, the Victoria and Albert Museum in London, the Scottish National Portrait Gallery in Edinburgh, which exhibited Colvin’s Ossian series, as well as the Gallery of Modern Art in Glasgow (The Mad Hatters’ Review, http://www.madhattersreview.com/issue7/contributors7.shtml, date accessed 13 August 2009). The Ossian series includes Blind Ossian and Fragments of Ancient Poetry. I would like to express my appreciation to Calum Colvin for generously granting permission to use his images as the basis for this essay. Quoted after J. Lawson, executive curator, National Portrait Gallery, Edinburgh, http://www.calumcolvin.com/thumbs8.htm, date accessed 13 August 2009. M. Schapiro (1973) Words and Pictures: On the Literal and the Symbolic in the Illustration of a Text (The Hague and Paris: Mouton), pp. 9–16. T. Normand (2002) Calum Colvin: Fragments of Ancient Poetry (Edinburgh: Scottish National Portrait Gallery), p. 11. Ibid. I. Gale, Scotland on Sunday, 22 September, 2002. It has been suggested that the head might be seen as an ‘ironic leitmotif representing the fallibility of history and the problem, even the absurdity, of those narratives that search for essences, particularly racial essences’: Normand (2002) Calum Colvin, p. 22. P. Miller, Glasgow Herald, 14 September, 2002. Normand (2002) Calum Colvin, p. 22. Ibid., p. 44. Gale, Scotland on Sunday, 22 September, 2002. H. Lund (1998) ‘Ekphrastic Linkage and Contextual Ekphrasis’ in V. Robillard and E. Jongeneel (eds) Pictures Into Words: Theoretical and Descriptive Approaches to Ekphrasis (Amsterdam: VU University Press), pp. 173–88 at pp. 176–7. Ibid., p. 177. Gale, Scotland on Sunday, 22 September, 2002. Miller, Glasgow Herald, 14 September, 2002. Quoted in Calum Colvin, http://www.calumcolvin.com, date accessed 13 August 2009. M. Worton and J. Still (eds) (1990) Intertextuality: Theory and Practices (Manchester and New York: Manchester University Press), p. 26. Riffaterre has set out these theories most notably in (1980) Semiotics of Poetry (London: Methuen) and Text Production (1983) (New York: Columbia University Press). Quoted in J. F. Sowa (2003) Ontology, http://www.jfsowa.com/ontology/. D. Higgins, Synesthesia and Intersenses: Intermedia, http://www.ubu.com/papers/ higgins_intermedia.html, date accessed 13 August 2009. Originally published 1966 in Something Else Newsletter I. W. Nöth (1990) Handbook of Semiotics (Bloomington and Indianapolis: Indiana University Press), pp. 329–32. W. Wolf (2009) The Musicalization of Fiction: A Study in the Theory and History of Intermediality (Amsterdam and Atlanta GA: Rodopi), p. 37. N. Goodman (1998) Languages of Art: An Approach to the Theory of Symbols (Indianapolis: Hackett Publishing).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

162 Mediations and Transformations of Media

Translating Sounds: Intermedial Exchanges in Amy Lowell’s ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’ Regina Schober

‘Imagine Media!’ This motto gives reason to reflect upon the concept of mediality, the problem of medial boundaries and possible spaces for intermedial exchange and cross-fertilization.1 The word ‘imagine’, however, also points to the permeability and constructedness of allegedly established media boundaries, calling into question the validity of such commonly accepted notions as, for instance, ‘literature’, ‘music’ or ‘art’. Apart from drawing attention to the artificiality and arbitrary nature of media definition and classification, this slogan also contains an inherent notion of intermediality, of a reciprocal relationship between different media, as we conventionally understand them. The question of what a certain medium is always depends on the conception, the ‘imagination’ of the beholder – be it the recipient or the medium itself. For not only do we as scholars, performing artists or audience have a specific understanding of what we recognize to be a certain medium, but also the medium itself intrinsically incorporates and conveys its ‘image’, its version of mediality, either by self-referentially and self-reflexively pointing to its own medial status, by referring to other media and thus determining its own medial boundaries, or simply by displaying performative features of its own mediality in dissociation from other media. In the case of intermedial ‘imagination’, a medium defines itself by its own potentials, but also with regard to its limitations. Especially when turning to another medium, it automatically embarks on a dialogical contemplation about possible borrowings, impulses and insights which could contribute to enhancing its own (limited) expressive power. With the Modernist movement, self-reflexive and experimental tendencies increasingly occurred in all media. Ezra Pound’s appeal to ‘MAKE IT NEW’2 is paradigmatic for the general Modernist quest for revolutionizing the arts by leaving trodden paths, dispensing with outdated aesthetic models and venturing into new expressive fields. One of the crucial elements in the 163 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

10

process of the revolution of the arts at the turn of the twentieth century was the questioning and expansion of established media boundaries. More than ever before media turned to other media, assimilating media-external techniques and qualities into their own medial realm. In fact, Modernist artists were particularly drawn to the musical medium. Walter Pater’s famous assertion that ‘all art constantly aspires towards the condition of music’3 reflects a common tendency among Modernist writers, but also painters, to turn to music as an aesthetic model.4 In this essay I will discuss a very specific case of intermedial ‘imagination’ on the basis of a musico-literary experiment of American Modernist poet Amy Lowell. I will show how Lowell’s poem ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’ takes up a particular piece of music and verbalizes it within the poetic medium.5 By ‘imagining’ its own mediated version of another medium, the poem engages in what I call ‘intermedial translation’ and thus creates not only a concept of the medium music, but at the same time also self-reflexively redefines, or ‘imagines’ its own medial condition. As Jane Ambrose reveals in ‘Amy Lowell and the Music of her Poetry’,6 Amy Lowell held a lifelong interest in music as poetry’s ‘sister art’. Several of Lowell’s poems either allude to music or display musical analogies on a structural level. In 1915, Lowell heard Igor Stravinsky’s ‘Three Pieces “Grotesques”, for String Quartet’7 performed in concert in Boston, which inspired her to convert an entire work of art from one medium into another, thus performing an act of intermedial translation. This transformatory process falls under the category of what Lars Elleström calls ‘mediation and transformation of media’,8 but has also been described by terms such as ‘intermedial transposition’,9 ‘Medienwechsel’,10 adaptation, transposition d’art and so on.11 This example of intermedial translation corresponds to the notion of ekphrasis, as it is a ‘verbal representation of a real . . . text composed in a non-verbal sign system’.12 However, in this context I would like to use the term intermedial ‘translation’, for it emphasizes the tension between the notion of accuracy on the one hand and creative originality on the other hand. Furthermore, the idea of ‘translation’ implies a perspective that is not restricted to the semiotic level, in terms of a simple shift between different sign systems. As with any translation process, intermedial translation also has considerable cultural and aesthetic implications resulting from the interaction of two culturally and historically embedded artefacts.13 As I will show, Amy Lowell not only transgresses medial boundaries by referring to and engaging with another medium. She also raises our awareness with regard to the very translatability and exchangeability between different media by highlighting at once transmedial relations as well as media-specific distinctions and divergences. The concept of intermedial translation will thus also

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

164 Mediations and Transformations of Media

Regina Schober 165

Processes of translation undeniably played an important role in Modernist literary production. Modernist writers not only produced a great number of translations, they also frequently drew on foreign, often classical, writings which were integrated into their own texts, either as translations or in their original form.14 The practice of translation did not solely serve hermeneutic purposes, but instead participated in the Modernists’ experimental urge to employ alternative materials in a general rewriting process. In Modernism, translation was no longer regarded as a purely functional and inferior form of writing, but rather attained the ‘status [of] a primary mode of Modernist literary production in its own right’.15 Along the lines of T. S. Eliot’s notion of tradition and originality, translation functioned ‘as a kind of dynamic procedural lens through which the Modernists could at once view both the past as well as other cultures and . . . focus their images of these traditions in their own times’.16 Obviously, translation became an integral part of Modernist literary practice, in that it transformed already existing texts, which Eliot calls ‘tradition’, into ‘the new’.17 Translation thus grew to be an integral part of the original and creative writing process, while the boundaries between original and translated text became increasingly blurred. Modernist writers held the strong view that only through an active engagement with the ‘other’ texts and media could the ‘self’ be comprehended and developed. In his famous essay ‘The Task of the Translator’, Walter Benjamin describes the aim of a good translation to ‘express the central reciprocal relationship between languages’.18 In highlighting the reciprocity of the translation process and the repercussions which the translation has on the original text by giving it an ‘afterlife’,19 Benjamin’s considerations provide a valuable basis for the question of intermedial translation processes. Although translation studies have conventionally been concerned with written texts only, the concept of translation could as well prove suitable for the context of intermedial relationships. If translation is regarded as a transformation of a pretext of a particular system into another text or system, certain intermedial processes could be considered as just another form of ‘translating’ from one medium to another. For a long time, ‘translation’ in the literary or linguistic context has commonly only been associated with the transfer of a text from one language to another.20 According to the definition in the Oxford English Dictionary, there is, however, already an indication of the notion of ‘intermedial translation’ in the nineteenth century, when ‘translation’ is referred to as ‘the expression or rendering of something in another medium or form, e.g. of a painting by an engraving or etching’.21 This early use of the concept of intermedial translation has been taken up by Roman Jakobson in his essay ‘On

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

serve to reflect upon general processes of transfer and appropriation between the two media: music and poetry.

Linguistic Aspects of Translation’, which differentiates between three types of translation; ‘intralingual’, ‘interlingual’ and ‘intersemiotic’ translation.22 While only the second type corresponds to the commonly accepted notion of translation between different languages, the third concept, ‘intersemiotic translation’, implies the translation between different semiotic systems, for example, between written texts and film or music. Even though Jakobson’s ‘intersemiotic translation’ is somehow restricted in that the source medium is always a written text, we owe it to him that the concept of translation is extended from its strictly linguistic to a broader semiotic context, including transformations and intersections not only between different languages but, more generally, between any different sign systems. Recent translation studies have increasingly followed this expanded and often metaphorical understanding of translation, disassociating it from its strictly linguistic meaning and instead concentrating more on the universally and cross-disciplinarily applicable transformational processes involved. As maintained by Mieke Bal and Joanne Morra, ‘today translation is gaining ground as a crucial trope, idea, concept, metaphor and mode of interpretation’.23 In accord with Shirley Chew and Alistair Stead, who understand translation ‘in its transferred or metaphorical senses, most fundamentally as a process of change or a passage from one state to another’,24 I would like to place the idea of translation, ‘translate’ it, into the context of intermediality. Intermedial translation25 , as I understand it, can thus be regarded as a generic term for various different expressions which have been applied to describe the processes of change from one (medial) state to another, to use Chew and Stead’s turn of phrase, such as ‘transposition’, ‘transcription’ and ‘adaptation’. Whether the aim of intermedial translation is to make the original medium ‘readable’ or to try experimentally to make a text fitting for another medial context is a matter that is still open to debate, as is the question of the degree to which the transformation process also implies modification and alteration, and to what extent intermedial translation is either total or partial. The fact that a translation, as an original artwork in itself, is always a modified ‘version’ of the original, containing features quite distinct and divergent from the reference medium, applies, of course, all the more to the case of intermedial translation. Even if the target medium employs trans or intermedial components which let it appear like the original medium, there is always a certain degree of modification and change. Not the medium itself, but an ‘imagined’ version of the medium, realized by means of another medium, is the outcome of the intermedial translation process. In order to elucidate further the concept of intermedial translation, I will illustrate my theoretical considerations by means of a musico-poetic transcription by Amy Lowell. The musical pieces she refers to, Igor Stravinsky’s Three Pieces for String Quartet, were written in 1914 and were originally titled

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

166 Mediations and Transformations of Media

‘Dance’, ‘Excentrique’ and ‘Cantique’. The pieces were supplemented by a short introduction written by the composer himself, which was read out before the performance and which most likely formed the basis of Lowell’s poem. The introduction described the setting of the pieces as a peasant dance, a grotesque Pierrot scene and a tragic funeral service.26 Although an approximate narrative thread is discernible throughout the three pieces, they can well be regarded individually.27 In this context, I will mainly restrict my analysis to the first of the three pieces. If we take a closer look at the different translation processes involved in this musico-poetic experiment, it is especially striking how accurately the poem takes up the music and represents it by both ‘explicit’ and ‘implicit’ intermedial references.28 Lowell both imitates the music with regard to its compositional material and describes her sound impressions by adding a semantic sphere to the music. The vivid sound imitations, descriptions and thematic associations are all the more remarkable, since Lowell must have been able to listen to the music just once.29 In her poetic transcription of Stravinsky’s music, Lowell retains the overall title of the pieces,30 as well as its three-part structure, although she interprets the three autonomous pieces as movements of a larger whole by grouping them together. The first of Stravinsky’s Three Pieces, originally entitled ‘Dance’, calls to mind a folkloristic peasant dance, first and foremost evoked by the seemingly ‘simple’ melody with its relatively small pitch range and its repetitive and cyclical structure. The first part of Lowell’s poem takes up this thematic setting of a peasant dance in a ‘market-place’ (line 11) with ‘sabots slapping the worn, old stones’ (line 12), articulating a potential program of the music on the basis of very accurate aural impressions. The kinetic nature of the piece is supported by its rhythmic layout which is characterized by the recurrence of extremely regular ostinati which are, however, set against each other and thus create rhythmical displacement and instability. Lowell’s poem captures this sense of rhythmic disruption, which is clearly perceivable in the music, by bringing up the image of bulky sabots dancing on cobblestones: ‘Clumsy and hard they are, / And uneven, / Losing half a beat’ (lines 15–17). Written in free verse, the poem is liberated from regular meter, so that particular rhythmic elements of the music can be imitated.31 The basic rhythmical unit of the music, the alternation of a single quarter and double eighth note, is reflected in the repeated words ‘sharp (and) cutting, sharp (and) cutting’ or, more vividly, in the onomatopoetic ‘Bump! Bump! Tong-ti-bump!’ (line 7), and ‘shuffle, rap’ (line 23). While Lowell’s poem clearly suggests a peasant dance scene, Stravinsky’s piece, with its lack of clear structure and melodic circularity, seems almost constructed. Rainer Sievers is right in observing a certain mechanical quality about the piece.32 Like a perpetual motion machine, the melody could continue endlessly, repeating the patterns over and over, until it suddenly comes to a halt in a rather abrupt manner.33 Obviously, the artistic

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Regina Schober 167

alienation of a traditional folk dance is as evident as its imitation. Stravinsky does not attempt to reproduce authentic folk music, but by combining it with new compositional techniques, he treats it in an experimental way in order to create a new, idiosyncratic means of expression in line with his Modernist aesthetics. By employing folk elements, but at the same time dissociating himself from these influences, Stravinsky performs a typical Modernist move. Furthermore, through his alienation of the traditional material towards a mechanical and machine-like gestus, he establishes a link between artistic modernism and the notion of ‘modernization’ in terms of technological and industrial progress. Stravinsky’s experimentalism also shows in his unconventional treatment of tonal colour. The traditional chamber music formation of a string quartet is transformed into a folk band, in which the typically delicate string instruments produce harsh and vulgar sounds. The almost excruciating crudeness of the sounds are described by Lowell as ‘sharp and cutting’ (line 4), probably particularly referring to the dissonant minor ninth played in the opening by the viola, portrayed as a ‘screeching thread’ (line 3) which is so discordant that ‘it hurts’ (line 5). Following Stravinsky’s detailed performing instructions such as ‘sur le sol’ (on the G-string), ‘glissez avec toute la longueur de l’archet jusqu’ à la fin’ (use the entire bow) or ‘au talon’ (at the frog), the instruments generate a breathy and at times raucous timbre. The crudest sound is presented by the second violin. The four notes, which display the loudest dynamics (ff ) of the piece, are again played ‘sur le sol du talon’ and, additionally, extremely secco (‘excessivement sec’), a sound effect created by the sequence of four up/downbows and staccatissimo accents (wedge-shaped accents as opposed to the dotted accents, such as in bars 5/6). Taken as a whole, the sound quality of the piece is dominated by rough noises, as Rudolf Stephan observes: Already the complexion is singular: Noises such as glissandi, harsh grace notes, scratching at the bridge, playing at the frog, striking the string with the wood, pizzicato etc. are predominant. This prominence of noises has its function: the compositional detail is not to be lost in the comforting sound of the quartet.34 The prevalence of the compositional material, indicating a high degree of self-referentiality, is certainly characteristic of Stravinsky’s avant-garde aesthetics. The music’s harsh and unrefined sounds, however, also serve to represent the allegedly primitive and unrefined sound quality of peasant music which is portrayed in the piece. While the sound descriptions in Lowell’s poem start off as rather accurate attributions to musical instruments, they increasingly develop into rather free associations regarding the market place setting. There are ‘thin Spring

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

168 Mediations and Transformations of Media

Regina Schober 169

Given the precision with which the poem represents the music, would it be right, however, to speak of Lowell’s poem as a mere replica of the music in another medium? Does the poetic adaptation simply involve a one-toone reproduction without any degree of originality? Certainly not, even if Lowell, with regard to this translational experiment poses the question: ‘Could I reproduce the effect of the music in another medium?’35 In fact, this question provides a first clue. The reproduction of the music itself is not implied, but only the reproduction of its effect. Verbally imitating music by means of employing the transmedial features of rhythm and sound, the poem virtually mediates between the two art forms by taking up elements, or, according to Elleström,36 medial modes which are innate to both media and thus bridge the narrow gap between the already closely related art forms. Elleström’s assertion, that ‘media are both different and similar’, is perfectly illustrated by this example. The poem abounds with musical effects – in this regard it blends in with many other Modernist poems. On the other hand, however, the poem does not become music, nor can it be merely reduced to its experimental quality of transforming sounds into words, since it is important to maintain that intermedial translation, as I have claimed above, includes more than the structural and formal, but also touches upon on the cultural and aesthetic realm of a medium. Interestingly, Lowell’s poem refers to a contemporary piece of music by a composer who was associated with the European avant-garde. Stravinsky’s pieces must have seemed extremely modern, unconventional and eccentric at the time, especially to the rather conservative audience of the Boston cultural elite. Taking up this particular piece of music, the poem thus selfreflexively positions itself in an avant-garde context. Besides, the poem takes up certain cultural themes which are inherent in the music or its titles and which are then developed and elaborated upon by means of semanticization and cultural association. By turning to what is considered European Modernism, the poem consequently not only engages in intermedial translation, but at the same time also in a form of cultural translation. As has already become apparent in the analysis, one of the culturally relevant topics involved in both the music and this part of the poem is the discourse on folklore and primitivism in European and American Modernism. Stravinsky overtly employs compositional devices associated with folk music. The notion of a peasant dance, which is evoked by both the original title ‘Dance’ and Stravinsky’s introduction, is thus also inherent in the compositional structure of the music, however alienated by Modernist compositional techniques. Stravinsky uses folkloristic elements not mainly as

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

leaves’ that ‘shake to the banging of shoes’ (lines 20–21) as well as ‘little pigs’ voices’ (line 25), obviously an association evoked by the croaking and grunting sound of the second violin.

a source of compositional material, but possibly to try to revive facets of a departing culture of the ‘Old Russia’, which he expected to find in the allegedly primitive Russian folk music, while at the same time embedding it within his experimental compositional practices.37 Elements of primitivism also enter Lowell’s poem. The depiction of the folk dance suggests not only an atmosphere full of different noises which appear simultaneously and thus evoke a chaotic scenery, but also give the impression of sensual and physical ecstasy. Allusions to ‘delirium flapping its thigh-bones’ (line 33), for instance, or ‘coarse stuffs and hot flesh weaving together’ (line 39) create an unrefined, sexual and almost vulgar atmosphere. The scene depicted expresses a high amount of energy, caused by the rhythmic use of words and the dynamic imagery employed. Not only the sexual imagery, but also the prominence of colours, such as ‘red, blue, yellow’ (line 34), blurred in a delirium-like state of ‘steaming’ ‘drunkenness’ (line 35) until they appear again in a different order ‘red, yellow, blue’ (line 36), add to an atmosphere frequently associated with a primitivist setting in literary modernism. That Lowell was familiar with the primitivist discourse of her time becomes obvious in her essay, ‘Some Musical Analogies in Modern Poetry’: It is obviously impossible to go back to the beginning of any art. That rhythm was the starting place of all, seems, however, practically indisputable. It would appear as though the more simple rhythms should have developed first, but that is lost in the mists of time. And it is a strange fact that now the savage or semi-civilized races employ exceedingly subtle rhythmic effects. . . . the complicated syncopation of the American negro has captivated the world under the vulgar and misleading name of ragtime. That he never learnt this from us is plain . . . At what period did civilized man lose this power of retaining psychological beats in his head without necessarily expressing them?38 This passage shows that Lowell, not being free from the prevalent racial ideologies of early Modernism, links her concept of the ‘primitive’ with the idea of a primal and an instinctive sense of rhythm, which has been lost in the civilization process and which has to be retained and rediscovered. The inherent irony of primitivism, the paradox between idealizing the ‘primitive’ as a means of criticizing one’s own society while on the other hand establishing and maintaining the border between ‘us’ and ‘them’ can well be observed in Lowell’s rhetoric, which draws a clear line between the ‘American negro’ and ‘us’. Lowell’s fascination for the ‘primitive’ is not particularly differentiated, for she subsumes various ethnic groups under the concept of the ‘savage’; the Egyptians she encountered during a boat trip on the Nile, the American Indian or the ‘American negro’. Because Lowell, as a Boston Brahmin, had no direct affiliation with the supposed primitive element of African-American culture, it was probably as ‘exotic’ and ‘foreign’ to her as African or Chinese cultures. 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

170 Mediations and Transformations of Media

This is exactly where Lowell’s primitivism differs from Stravinsky’s interest in folk music. Although both utilize ‘primitive’ elements as compositional material, above all that of rhythm, their concepts of ‘primitivism’ are grounded on totally different premises. Whereas Stravinsky uses materials from the Russian folk tradition, belonging to his ‘own’ culture, Lowell has in mind foreign, remote cultures, the ‘other’. Her poem ‘Stravinsky’s Three Pieces “Grotesques”’ does not make explicit references to the Russian cultural context, but rather considers and portrays Russian folklore as one of many ‘primitive’ traditions. Russian folklore elements, which play a crucial part in Stravinsky’s Three Pieces, turn into ‘primitive’ elements, which are almost exchangeable, in Lowell’s text. While Stravinsky is interested in traces of a particularly Russian culture, Lowell engages in a more universal search for a primal state of being as a means of reinforcing modern civilization. Unlike Stravinsky, she does not turn to the ‘primitive’ in order to return to her own cultural identity, but rather to depart from it in favour of a turn to an imaginary pre-civilized state of mankind. To conclude, Amy Lowell’s poem ‘Stravinsky’s Three Pieces for String Quartet’ exhibits processes of intermedial translation which engage both on a formal and cultural or aesthetic level. The poem verbally represents a piece of music by imitating and semanticizing it. Lowell’s translation not only attempts an accurate representation of the original, but also foregrounds the experimental borrowing of the material of the musical medium and thus self-reflexively comments on its own aesthetics, while emphasizing and appreciating its expressive potential. The poem also translates and thus transforms the music’s cultural ‘meaning’. In the imaginative description of the folk dance scene, Lowell creates her own response to the notion of the primal inherent in Stravinsky’s music. The Russian folk element captured in the music is translated into Lowell’s own cultural context and her particular version of ‘primitivism’. However, the processes of intermedial translation are not unidirectional. Not only can Lowell’s poem be understood as a potential program to the music, but the translation process itself elucidates and reflects upon the very quality of mediality. By ‘translating’ a piece of music, the verbal medium creates its own imagined version of the ‘other’ medium and at the same time also of its own medial status. Thus, in translating from music to literature, Lowell’s poem both overcomes and transcends medial, cultural and aesthetic boundaries, but at the same time illuminates medial differences, pointing to the very essence of their materiality.

Notes 1. ‘Imagine Media!’ is the title of the eighth conference of the Nordic Society for Intermedial Studies, held at Växjö University (from 2010: Linnaeus University), Sweden, 25–8 October 2007, where a preliminary version of this essay was first presented. 2. E. Pound (1977) ‘LIII’ in The Cantos of Ezra Pound (New York: New Directions Books), p. 53.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Regina Schober 171

3. W. Pater (1910) The Renaissance: Studies in Art and Poetry (London: MacMillan), p. 135. 4. For the influence of music on Modernist writers, see B. Bucknell (2001) Literary Modernism and Musical Aesthetics: Pater, Pound, Joyce, and Stein (Cambridge: Cambridge University Press). With regard to music’s impact on Modernist painting, see K. von Maur (1999) Vom Klang der Bilder (Munich: Prestel). 5. A. Lowell (1955) ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’, The Complete Poetical Works of Amy Lowell (Boston MA: Houghton Mifflin Company), pp. 148–9. 6. J. P. Ambrose (1989) ‘Amy Lowell and the Music of Her Poetry’, The New England Quarterly 62(1), 45–62. 7. I. Stravinsky (1922), ‘Trois Pièces pour Quatuor à Cordes’, F. H. Schneider (ed.) (Edition Russe le Musique, printed by arr. Boosey & Hawkes). 8. Cf. Lars Elleström, in this volume. 9. W. Wolf (2002) ‘Intermediality Revisited: Reflections on Word and Music Relations in the Context of a General Typology of Intermediality’ in S. M. Lodato, S. Aspden and W. Bernhart (eds) Word and Music Studies: Essays in Honor of Steven Paul Scher on Cultural Identity and the Musical Stage (Amsterdam: Rodopi), pp. 13–34. 10. I. O. Rajewsky (2002) Intermedialität (Tübingen and Basel: Francke). 11. Cf. Irina Rajewsky, in this volume, for an overview of the terms describing processes of media transformation. 12. C. Clüver (1997) ‘Ekphrasis Reconsidered: On Verbal Representations of NonVerbal Texts’ in U.-B. Lagerroth, H. Lund and E. Hedling (eds) Interart Poetics: Essays on the Interrelations of the Arts and Media (Amsterdam: Rodopi), p. 26. S. Bruhn (2001) discusses the term ekphrasis in the context of intermedial studies in ‘A Concert of Paintings: Musical Ekphrasis in the Twentieth Century’, Poetics Today 22(3), 551–605. 13. Recent studies of intermediality have increasingly focused their attention on cultural implications and possible functions of intermediality. Significant contributions to this field include E. Hedling and U.-B. Lagerroth (eds) (2002) Cultural Functions of Intermedial Exploration (Amsterdam: Rodopi) and L. Eckstein and C. Reinfandt (2006) ‘On Dancing about Architecture: Words and Music between Cultural Practice and Transcendence’, Zeitschrift für Anglistik und Amerikanistik 54(1), pp. 1–8. Also see Jørgen Bruhn’s contribution to this volume for ideological and cultural implications of intermedial studies. 14. S. G. Yao (2002) Translation and the Languages of Modernism: Gender, Politics, Language (New York: Palgrave Macmillan), p. 5. 15. Ibid., p. 157. 16. Ibid., p. 7. 17. T. S. Eliot (1975) ‘Tradition and the Individual Talent’ in F. Kermode (ed.), Selected Prose of T. S. Eliot (London: Faber and Faber), pp. 37–58. 18. W. Benjamin (2004) ‘The Task of the Translator’, translated by Harry Zohn, in L. Venuti (ed.) The Translation Studies Reader (New York: Routledge), p. 17. 19. Ibid., p. 16. 20. The term ‘translation’ is also used in natural sciences, the science of religion and astrology; however, in all cases the basic notion of the term is the transformation of one state into another. 21. ‘Translation’ in Oxford English Dictionary, 2nd edn (Oxford University Press 1989, reprinted 2007).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

172 Mediations and Transformations of Media

22. R. Jakobson (2004) ‘On Linguistic Aspects of Translation’ in L. Venuti (ed.), The Translation Studies Reader (New York: Routledge), p. 139. 23. M. Bal and J. Morra (2007) ‘Editorial: Acts of Translation’, Journal of Visual Culture 6 (5), p. 5. 24. S. Chew and A. Stead (1999) ‘Introduction’ in S. Chew and A. Stead (eds) Translating Life: Studies in Transpositional Aesthetics (Liverpool: Liverpool University Press), p. 2. 25. The term ‘intermedial translation’ has been used sporadically, for example, in the context of visual culture by Bal and Morra and by Dick Higgins with regard to sound poetry (‘A Taxonomy of Sound Poetry’). However, it has so far not been applied to the musico-literary context. 26. According to a review of the concert by Philip Hale in The Boston Herald (3 December 1915), a text was read in connection with the pieces, which unfortunately does not seem to exist anymore. 27. Although calling the pieces ‘Grotesques’, Stravinsky probably considered them as individual works grouped together arbitrarily, for the orchestrated version of 1928 (Four Studies for Orchestra) contains an additional fourth piece titled ‘Madrid’. 28. Concerning the terms ‘explicit’ and ‘implicit’ intermedial references, see W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History of Intermediality (Amsterdam and Atlanta GA: Rodopi). 29. Since the contemporary pieces were played from manuscript by the players, Lowell did not have access to a score – the first printed edition was not published until 1922. 30. Considering some of the ideas put forward by Axel Englund in this volume, the explicit and exact reference to the music’s title could be regarded as indicative of a metaphorical relationship between poem and music, since the poem represents and thereby purports to be music, its medial ‘other’. 31. The direct imitation of musical rhythms corresponds with Lowell’s understanding of poetic rhythm as set out in her essay ‘Some Musical Analogies’ (at p. 139, see below note 34). Experimenting with the possibilities of free verse, Lowell takes music, especially that of Debussy, as a model. Due to its neglect of syllables in favour of equally long time-units, free verse acquires a musical flow which is not dominated by regular meters, but instead by larger time divisions. As a result, single word lines, which are often onomatopoeic sound imitations, such as ‘Whee-e-e!’ (line 12) are of the same length as lines with more words and syllables and thus create a musical resonance and reverberation. Due to their exposed position and the combination of the voiced glide /w/ and the long, shrill /i/, they have a similar effect as the accented quarter notes of the first violin, slurred into an eighth note g (for example, bars 7/8). The rhythmical effect of free verse is supported by a frequent use of sharp consonants, especially fricatives, affricates and stops as in ‘screeching thread’ (line 3), ‘sabots slapping’ (line 13) or ‘shaking and cracking’ (line 14). 32. R. Sievers (1996) Igor Strawinsky: Trois Pièces Pour Quatuor à Cordes: Analyse und Deutung (Wiesbaden, Leipzig and Paris: Breitkopf & Härtel), p. 71. 33. At the end of the piece Stravinsky demands sans ralentir in order to make it explicit that a ritardando is not intended. 34. R. Stephan (1972) ‘Aus Igor Stravinskys Spielzeugschachtel’ in L. U. Abraham (ed.) Erich Doflein: Festschrift zum 70. Geburtstag (7. August 1970) (Mainz: Schott), p. 27. 35. A. Lowell (1920) ‘Some Musical Analogies in Modern Poetry’, The Musical Quarterly 6(1), 148.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Regina Schober 173

36. Cf. Lars Elleström, in this volume. 37. As demonstrated in R. Taruskin (1996) Stravinsky and the Russian Traditions: A Biography of the Works through Mavra, 2 vols (Berkeley CA: University of California Press), Stravinsky’s works are undeniably strongly influenced by Russian folk music, although the composer himself denied any influence of that kind. His explicit detachment from those influences might be explained with his antipathy towards Red Russia and the socialist call for a nationalist art. Yet, according to Stephen Walsh, Stravinsky developed a ’sudden and intensive interest in folk poetry’ (Walsh (1999) Stravinsky: A Creative Spring: Russia and France, 1882–1934 (New York: Knopf), p. 56) during his Swiss period, which not only affected his Three Pieces, but also larger works such as Les Noces (1914 and 1923) and Renard (1916). As a result of his involuntary exile, Stravinsky developed an interest for his home country, which Walsh calls a ‘wartime quest for an idealized folk modernism’ (1999, p. 537). 38. Lowell (1920), ‘Some Musical Analogies’, pp. 130–1.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

174 Mediations and Transformations of Media

11 Claus Clüver

The various forms of ‘New Media Poetry’ documented and theorized in 1996 in an issue of Visible Language edited by Eduardo Kac1 included such forms as digital poetry in a number of forms and genres (some of it interactive), other forms of cyberliterature, videopoetry and holographic poetry (or holopoetry). The concept of ‘poetry’ involved in this practice and its theorization expanded the convention of considering as ‘poetry’ all forms of manipulation of and experimentation with the verbal medium and its written and aural representations which dated back to the beginning of the twentieth century and had come to be labelled as visual, concrete or sound poetry, respectively. The Brazilian poet and artist Eduardo Kac (b. 1962) first gained international recognition in the early 1980s with his computer-generated holopoetry. Over the past 20 years, he has more radically explored the possibilities of contemporary media technology for artmaking, a development documented in Telepresence & Bio Art: Networking Humans, Rabbits, & Robots (2005), a collection of Kac’s essays published between 1992 and 2002,2 with the essays grouped under the headings ‘Telecommunications, Dialogism and Internet Art’, ‘Telepresence Art and Robotics’ and ‘Bio Art’. The most notorious of his ‘bio art’ creations is the GFP Bunny (2000), the green fluorescent rabbit Alba, a product of what Kac calls ‘transgenic art’ – ‘a new art form based on the use of genetic engineering to create unique living beings’.3 The first of Kac’s ‘transgenic’ artworks, Genesis (1999), was also his first ‘biopoem’. Kac has written an essay4 detailing the conception and execution of the work and reflecting on its implications. In describing it I shall have to rely heavily on Kac’s own description. The work was executed in three phases. The first led to the creation of an ‘artist’s gene’ – ‘a synthetic gene that I invented and that does not exist in nature’.5 It involved the transformation of a verbal text into DNA. The text chosen was a statement by the God of the biblical Genesis as rendered in English in the King James Version of the Bible: ‘Let man have dominion over the fish of the sea and over the fowl of the air and over every living thing that moves upon the earth’ (Genesis 1.26). 175 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

‘Transgenic Art’: The Biopoetry of Eduardo Kac

The ironic appropriateness of choosing these lines to be transformed into a gene is obvious. Kac has pointed out that the choice of a translation, instead of the original Hebrew, and of the King James Version with its claims, its complicated history and its ideological contexts, was deliberate. Next, the letters of the alphabet were replaced by the symbols of the Morse code. Kac explains that ‘Morse code was chosen partly because, as first employed in radiotelegraphy, it represents the dawn of the information age – the genesis of global communications’.6 I do not know what other code could have provided Kac with such a suitable base for conversion into DNA. Kac used the Morse dashes and dots for two of the four letters in his own conversion code: dashes represented the letter T (thymine), dots the letter C (cytosin); word spaces represented the letter A (adenine), letter spaces the letter G (guanine). The entire DNA resulting from this conversion represents the new synthetic gene. Kac reports that he had to ‘e-mail the gene to a company specializing in DNA synthesis’ and two weeks later received a ‘package with a vial containing millions of copies of the gene’. He realized that by itself the gene cannot do anything because . . . to be meaningful it needs a context. The context of the gene is the body of an organism, and the context of the organism is its environment. In the case of my Genesis, the organisms are bacteria . . ., and their environment is at once their dish, the gallery, and the Internet.7 The Petri dish with the bacteria into which the Genesis gene was introduced was displayed in a gallery. In fact, it contained two kinds of bacteria, both genetically engineered to glow: one kind, which contained the synthetic gene, emitted blue light, the other kind, which did not, emitted yellow light. Spectators could not only observe these bacteria in their dish, they could also turn on an ultraviolet light that would cause real biological mutations in the bacteria. The entire work exists as a multimedia installation in a blackened room (see Figure 11). According to Kac, it employs a petri dish with the bacteria, a flexible microvideo camera, a UV light box, and a microscope illuminator. This set is connected to a video projector and two networked computers. One computer works as a Web server (streaming live video and audio) and handles remote requests for UV activation. The other computer is responsible for DNA music synthesis. The original music, which employs the Genesis gene, was composed by Peter Gena. The local video projection shows a larger-than-life image of the bacterial division and interaction seen through the microvideo camera.8 In the image of the installation reproduced below, the wall to the right of the projection shows a large-scale painted inscription of the biblical text,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

176 Mediations and Transformations of Media

Figure 11 Eduardo Kac, Genesis, 1999. Transgenic work with artist-created bacteria, ultraviolet light, Internet, video (detail). Collection Instituto Valenciano de Arte Moderno (IVAM), Valencia, Spain.9 Courtesy of the artist

opposite the projection is the text in Morse code, and on the left wall the DNA. Even thus made visible and tangible through the employment of many media, the project still seems to require a verbal narrative in the form of annotations or a descriptive text to enlighten the viewer about the project and the processes involved. Kac reports that, at the end of the first exhibition of Genesis in 1999, the gene ‘was decoded and read back in plain English’, but the mutations had caused slight changes: ‘The mutated sentence read: LET AAN HAVE DOMINION OVER THE FISH OF THE SEA AND OVER THE FOWL OF THE AIR AND OVER EVERY LIVING THING THAT IOVES UA EON THE EARTH’.10 There were two other phases to the project that I will not describe in any detail here. One dealt with the protein produced by the synthetic gene. It involved ‘the visualization of the three-dimensional structure of proteins produced by sequenced genes’ and investigated ‘the logic, the methods, and the symbolism’ of the process ‘as well as its potential as a domain of art making’. The last phase ‘focused on giving tangible expression to important aspects of the genomic and proteomic developments of Genesis’.11 A pair of large black granite tablets contains the laser-etched texts that were painted on the walls in the installation, as well as the inversion, with the mutated DNA sequence on top, their Morse equivalent below and the changed verbal text at the bottom. According to the artist, the ‘triadic configuration of [these] Encryption Stones critically exposes the intersemiotic operations that lie at the heart of the contemporary understanding of life processes’.12 There were four additional sets of objects that made aspects of the project tangible in more material form.13

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Claus Clüver 177

178 Mediations and Transformations of Media

[a] critical stance is manifested throughout the Genesis project by following scientifically accurate methods in the real production and visualization of a gene and a protein that I have invented and that have absolutely no function or value in biology. Rather than explicating or illustrating scientific principles, the Genesis project complicates and obfuscates the extreme simplification of standard molecular biology descriptions of life processes, reinstating social and historical contextualization at the core of the debate.14 Instead of entering into the ongoing discussion of the scientific and ethical implications of this transgenic artwork and of the others Kac has created, I shall briefly explore, in the context of the present project, the intermedial processes in its constitution, among which transposition and transformation are dominant. I shall focus on the first phase, and more specifically on the processes leading to the Petri dish with the glowing bacteria. I shall consider the ways in which the entire work as a set of intermedial transformations and ongoing processes is communicated to the reader/observer, and its implications for the discourse on media and intermediality and for the notion of ‘poetry’. The initial medium is verbal language as represented alphabetically in writing, using modern spelling and no punctuation. The fact that the text is a specific and familiar translation leads to reflections about the changes of meaning that will have occurred in its transformations from its earliest oral formulation and about other versions of this passage in the same language and in others. These considerations will also involve the realization that any other version would have led to a different DNA – and the question whether this would have changed the final result in any significant way, given the self-contained nature of the project. The slight changes of the original text that would have occurred by the time of the re-‘translation’ of the mutated genes would be different but of no imaginable consequence, especially since any decoding, including that of the original text, would always have different and uncontrollable results, which would also depend on the time allowed for the process of mutation. The re-presentation of the alphabetic English version of the biblical text in the signs of Morse code does not necessarily depend on a written text, although seeing the letters will undoubtedly help; nor does the text in Morse code need to be written out. In fact, the code was invented to send telegraphic signals, electronic impulses received primarily as sound (and replicable through knocking or other rhythmically produced sounds). However, the subsequent transformation into a genetic code, represented by a string of capital letters of the Latin alphabet, is difficult to imagine without the model of a visual version of the Morse text.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

In his essay, the artist claims that

It is only when the three texts are seen on the walls of the gallery space that we are physically confronted with the fact that writing is a medium of its own, though usually intimately connected with verbal language. The letterforms used for the verbal text suggest to the modern reader the era when the translation was produced, and the font used for the DNA string will be associated with contemporary technology. The visual version of the text in Morse code of the very same text spelled out in letters, signals a change in the production and means of communication: the verbal message is the same, and there has been no change in alphabet, but the lines and dots and spaces serve only to visualize the groups of electronic impulses representing the letters conveyed over a distance. Moreover, as painted on the wall (not shown in the illustration) and etched into the Encryption Stones, the visual symbols do not indicate the spaces between the letters in the encoded text. In my view the representation of letters in Morse code, even as visualized for this project, does not constitute a form of writing. That may sound like a precarious statement, because it may seem to question the status of so much writing that occurs in electronic media and is perceived in virtual space. Media poetry ‘takes language beyond the confines of the printed page’, as Kac wrote in his ‘Introduction’ to New Media Poetry, ‘simply because the textual aspirations of the authors cannot be physically realized in print’.15 Nevertheless, in its concrete manifestations the physical appearance and organization of letters and letter combinations and their placement in (three-dimensional, dynamic) space are manipulated with even greater abandon and inventiveness than previously happened in print and manuscript and calligraphy, with transformations only now made possible by the new media, and approaching illegibility even more freely than some two-dimensional experimental poetry. All these possibilities for manipulating written signs that are inherent in any writing system and have been explored for centuries in many cultures are independent of verbal language, even when the writing serves to communicate a verbal text, and quite frequently in order to enhance such a text. The choices of typeface, letter size and visual arrangement of the biblical text above the Morse code version, and the DNA string below it, are a modest indication of the effectiveness of writing as a medium. The dots and dashes and spaces of the Morse code version do not invite or even permit such manipulation; and while the physical (or, to use Lars Elleström’s term, ‘technical’) media may be changed, the sequence of the signs cannot. Moreover, removed from that particular context, they fail to function altogether, while there have been many stimulating visual compositions made of elements of writing without any ostensible connection to verbal language, especially in the past hundred years.16 In this reading, a written verbal text constitutes a combination of two media, although this fact only becomes significant when the shape, size and placement of letters attract attention to themselves as constituents of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Claus Clüver 179

the text’s meaning. Since the visual codes on which such a text draws are not part of the verbal sign system, we would be reading an intermedia text. Contrary to multimedia and mixedmedia texts, the media involved in intermedia texts (for example, many logotypes and graffiti, concrete and holographic poetry) are fused and not separated by any boundaries.17 All written languages are based on particular codes that have developed over long periods of time. In alphabetic languages, the letters are written signs that must be distinguishable one from the other but are independent of the specific fonts or typefaces used in the production and manipulation of ‘writing’. It is the alphabetic version of the biblical text, which can also be spelled out orally, that serves as the basis of the text in Morse code. The text’s transformation into a version in Morse code, which is entirely reversible, does not affect its status as a verbal text at all, but it makes it difficult if not impossible to read, especially as painted on the wall, without separation of the coded letter signs. However, if properly transmitted by telegraph, the signals will be received, and can be decoded and pronounced, as letters. It is best to consider the Morse code as exactly that, and the transformation as a switch of codes of the representation of alphabetical verbal signs, and not as a change of media, even though the physical media involved are different: the voice or visual signs in manuscript or print on one side, telegraphic media for the text in Morse code on the other. The next step in the sequence of transpositions, however, is entirely different, even though its visual representation may at first sight show similarities to the original written text. The uninterrupted chain of capital letters is the result of an identification of the four elements of the visual version of the biblical text in Morse code (the dots, the dashes, the pauses between letters and the pauses between words) with the four elements that constitute the genetic code. These elements have names, and the initial letters of these names function, as capitals, to identify them. The capitals have been assigned, apparently in an arbitrary fashion, to the four elements of the Morse code, and the exact sequence in which these elements appear in the transformed biblical text thus determines the DNA of the gene that results from this process. While the DNA is represented in alphabetical form, the gene itself is an entirely different medium. Is it, in fact, a medium altogether? Even though it was formulated over two decades ago, and in spite of the current general agreement that any single-phrase definition of ‘medium’ is bound to be inadequate, it may still be useful in this context to cite the definition proposed in 1988 by Rainer Bohn, Eggo Müller and Rainer Ruppert, which I have translated as: ‘That which mediates for and between humans a (meaningful) sign (or a combination of signs) with the aid of suitable transmitters across temporal and/or spatial distances’,18 for it provides a few terms to be considered in our investigation. If we apply it to the artificial genes contained in the bacteria in the Petri dish, our question about the medial

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

180 Mediations and Transformations of Media

nature of the gene does not find a straightforward answer. The blue bacteria convey to us, as they were instructed to, that they contain the gene. They do that both in the gallery and electronically whenever the work is installed in a gallery and the installation is accessible via the internet, including the possibility of clicking a button to turn on the ultraviolet light. The bacteria might thus be considered suitable transmitters of a meaningful sign invisible to the naked eye – genes constituted by the transformation of the original text, in a process that can be reversed. Are the bacteria transmitters of signs, or are they themselves the sign, the media configuration considered as a ‘biopoem’? If the latter is the case, then what is the medium in which they can be configured as a sign? The core of the actual work, the product of several transformative intersemiotic processes, consists in the synthetic genes inhabiting the bacteria in the dish. However, it is crucial that the disk also holds bacteria that do not contain the artificial gene, because the project requires the possibility of mutations under the influence of ultraviolet light. The medial core configuration, thoroughly manipulated and controlled, is thus the Petri dish with two different-coloured sets of bacteria, one of which contains the artificial gene, accessible by the light. The medium enabling this configuration is any suitable controllable environment where bacteria can interact under certain conditions. Such an environment ordinarily does not serve to enable the transmission of meaningful signs between humans – part of the definition cited above. Arguably it becomes a ‘medium’ only by being made to function as an integral part of the Genesis project. The complex sign communicated by this particular laboratory setting is primarily self-referential: it concerns the use of ordinary biological procedures to make us aware of the possibility of genetic engineering through the creation of a gene resulting from particular transformative processes, and through the consequent, less controllable life processes resulting in mutations that can be demonstrated by decoding. That is indeed all the re-translation can tell us, besides the fact that the original text has been substantially preserved. The actual changes that have occurred will inevitably make the text less intelligible, and their extent, while otherwise uncontrollable, will depend on the time allowed for the process of mutation to take place.19 The gene itself is entirely inaccessible to the gallery visitor. Its ‘meaning’ in this specific case is completely artificial; in ordinary biological circumstances the ‘meaning’ of a genetic code is understood and used in totally different contexts. The alphabetical representation of the DNA, by itself unintelligible to the ordinary viewer, nevertheless makes it accessible in a different medium – writing – which also mediates the verbal text from which it originated via its translation into Morse code (also contained in the display and known to most viewers, at least in principle). However, its crude representation in a substantially different medium, alphabetic writing, indicates only the first step in the process of the gene’s production. On the other hand, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Claus Clüver 181

fact that the reverse process of decoding will end up in a very similar chain of capital letters that can then be re-translated, via the Morse code, into an intelligible and identical (though slightly distorted) verbal phrase indicates that the letter chain, though not a part of the genetic medium, is an integral part of the constitutive transformative process. Any understanding of the projected image of the bacterial interaction will depend on an entirely different readerly competence. Again, most viewers will bring some knowledge of these processes to the experience, in different degrees; and these differences will be reflected in the readings of the entire project that viewers will produce. A few may even come close to the artist’s own assessment, who states emphatically that he is interested in creating artworks that reflect on the multiple social implications of genetics, from unacceptable abuse to its hopeful promises, from the notion of ‘code’ to the question of translation, from the synthesis of genes to the process of mutation, from the metaphors employed by biotechnology to the fetishization of genes and proteins, from simple reductive narratives to complex views that account for environmental influences. The urgent task is to unpack the implicit meanings of the biotech revolution and, through art making, contribute to the creation of alternative views.20 For viewers interested in questions of intermediality the large-scale projection of the bacteria in the Petri dish will raise a set of additional questions and trigger reflections concerning the media-specificity of the enlarged living organisms moving around and emitting the two kinds of light. As we observe them, they are entirely the creatures of human imaginative activity exploiting recent scientific discoveries. It is this knowledge, derived from an external source, which makes us see them not only as biological specimens but as elements of a media discourse. We know that they owe their existence to a profound shift of codes in which the original meaningful sign, the verbal text, was obliterated to be reconstituted as a ‘biopoem’, a living organism that yet contains in its genetic structure that very text, subject only to the mutations provoked by bacterial interactions. While these interactions can be influenced by human intervention (activating the ultraviolet light), the poem-carrying bacteria in the dish constitute a medial configuration whose behaviour follows its own rules or, as Kac put it somewhat anthropomorphically, its own ‘interests’: ‘the biopoem’s internal interests as a living creature (which are independent of you and me, or even of the verbal components of the poem), i.e., the poem, being alive, has interests that go beyond its status as a poem’.21 The poet offers another observation about a consequence of transforming a verbal text into a living organism, this one concerned with what Elleström (in this volume) calls the ‘spatiotemporal modality’:

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

182 Mediations and Transformations of Media

There is also the fundamental question of what I call ‘biological time’, i.e., contrary to the temporal structures already known in poetry (oral performance, silent reading, simultaneities, recording and manipulation of voice, use of video editing, etc.), the biopoem evolves according to its own time, from a dynamic relationship between its internal metabolism and its response to the environment, to environmental conditions, which include the care we give to it (or not). Biological time gives the poem its own irreducible pace, following the rhythm of life as we live it.22 If the bacteria in the dish constitute the actual biopoem, it is a text made up by the fusion of at least two usually very distinct media and is as such an intermedia text. One can argue, however, that what is really communicated is the creative concept, especially since that which makes it a poem, the presence of a radically transformed verbal text in the DNA (which can nevertheless be recovered by a process of decoding), is invisible to the eye (and thus, like other concept art, it raises questions about the ‘sensorial modality’, to use another of Elleström’s terms). On the other hand, the entire project is not only a multimedia installation appealing primarily to the sense of vision and exchanging the materiality of the printed page for the painted shapes on the walls, which thus appear materially similar to the projections of the enlarged image of the Petri dish, while the technical media retain the most solid material presence. The materiality of the work’s other aspects in the various media involved is also made tangible by the additional objects created as part of the work but not of the installation. There is another medium involved in the work as installation: electronic music, on which Eduardo Kac offers no further comments. It is apparently generated by the DNA and does not exist independently.23 It most likely does not directly and immediately affect our understanding of the transformation project; but being surrounded by Peter Gena’s ‘gene sound’ will certainly affect our experience, and on reflection we realize that the genesis of a sonorous dimension in yet another medium is a further demonstration of the interplay of natural language, genetics and binary logic that forms ‘the triple system of Genesis’, according to the artist, who sees it as ‘the key to understanding the future’.24 At the core of the work is the act of what Irina Rajewsky, following others, has called ‘medial transposition’: ‘the “original” text, film, etc., is the “source” of the newly formed media product, whose formation is based on a media-specific and obligatory intermedial transformation process.’25 Her classification of this as a ‘production-oriented, “genetic” conception of intermediality’ seems particularly appropriate in this case, but the case is also unusual, and not only because the target medium is not normally involved in intermedial operations or an intermedial discourse. While in

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Claus Clüver 183

such media transpositions as film adaptations of literary texts, elements of the source text will always be incorporated, more or less transformed, into the new media constellation, in the case of Genesis the entire source text, radically transformed, has been incorporated into the biopoem – in fact, its presence makes it a poem, but also an intermedia text. As such, it falls into Rajewsky’s second category of intermediality, that of ‘media combination’. Intermediality serves in this project as a bridge between usually unrelated discourses. What enables the inclusion of the genetic medium in the game of intermedial transformations, at least in this case, is the fact that the gene is the carrier, and indeed a version, of a verbal text. Mimicking all genetic engineering, the conversion of this arrogant ancestral self-definition attributed to their god in an imaginative act that questions it by carrying the phrase to an unexpected extreme can indeed be considered as another extension of poetry. Kac told me again in 2007 that in the end he considers himself a poet. In ‘Biopoetry’, a contribution to a book currently in press,26 Kac proposes ‘the use of biotechnology and living organisms in poetry as a new realm of verbal, paraverbal and nonverbal creation’ and outlines 20 projects, including a few, like Genesis, already completed. Project number three reads thus: Marine mammal dialogical interaction: compose sound text by manipulating recorded parameters of pitch and frequency of dolphin communication, for a dolphin audience. Observe how a whale audience responds and vice-versa. Here any kind of human interaction, a fundamental component in the definition of ‘medium’ cited above, is at best secondary to the communication among other mammals. If we again consider the project a specimen of concept art where the idea is more interesting than its execution, we remain entirely in the human sphere, of course. Whether executed or not, however, the substitution of any element of verbal language by human-induced sounds of dolphin language takes the concept of ‘poetry’ well beyond the presence and role of verbal language in Genesis and also beyond most if not all boundaries of ‘poetry’ accepted in the work of experimental poets of the last hundred years, from the inarticulate vocal productions of Dada performers and concrete ‘sound poets’ to the illegibility of certain printed texts or calligraphic productions. What connects this particular project with such radical versions of ‘poetry’ as well as some types of media poetry is the nonmetaphorical use of ‘language’, albeit that of non-human mammals. Poetry can exist in many media, and its boundaries are constantly being redefined. On the other hand, ‘biopoetry’ occupies only one area in the apparently thriving realm of ‘bio-art’.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

184 Mediations and Transformations of Media

Claus Clüver 185

1. E. Kac (ed.) (1996) New Media Poetry: Poetic Innovation and New Technologies. Special issue of Visible Language, 30(2). For the revised and expanded edition published in 2007, Kac decided to drop ‘New’ from the title, ‘because now, ten years later, digital and electronic media are no longer new in society in general or in poetry in particular’ (E. Kac (ed.) (2007) Media Poetry: An International Anthology (Bristol, UK and Chicago: Intellect), p. 7). 2. A collection of essays published between 1982 and 1988 in Portuguese appeared in 2004 as E. Kac (2004) Luz & Letra: ensaios de arte, literatura e comunicação (Rio de Janeiro: Contracapa). 3. E. Kac (2005) ‘Transgenic Art’ in E. Kac, Telepresence & Bio Art: Networking Humans, Rabbits, & Robots (Ann Arbor MI: The University of Michigan Press), p. 236. 4. E. Kac (2005) ‘Genesis’ in Kac, Telepresence & Bio Art, pp. 249–63. 5. Ibid., p. 249. 6. Ibid., p. 251. In a long note, Kac added that he ‘employed Morse code not out of a technical need but as a symbolic gesture meant both to expose the continuity of ideology and technology and to reveal important aspects of the rhetorical strategies of molecular biology’ (p. 261). Morse was an ardent nativist: ‘the nativist platform was racist, anti-immigrant, anti-Catholic, and anti-Semitic.’ To Kac, ‘the translation of the KJV Genesis passage into Morse code represents the continuity from fierce British colonialism to the bigotry of nativist ideology’ (p. 261). 7. Ibid., p. 251. 8. The image was taken at the installation of Genesis in the Julia Friedman Gallery in Chicago. An eight-page fold-out catalogue, Eduardo Kac: Genesis (May 4–June 2, 2001), contains an essay by David Hunt, ‘Eduardo Kac: Metaphor into Motif’, and colour reproductions of all the elements of the work, including the text faces of the two Encryption Stones. 9. Ibid., p. 251. 10. Ibid., p. 254. 11. Ibid., pp. 254–5. 12. Ibid., p. 256. 13. There is also Transcription Jewels, ‘a sculpture encased in a custom-made round wooden box’ and consisting of a small glass bottle containing purified Genesis DNA and ‘an equally small gold cast of the three-dimensional structure of the Genesis protein’ (Kac (2005) Telepresence & Bio Art, p. 257). There is further a set of Fossil Folds, a series of sculpted tablets based on Kac’s ‘artist’s protein’ (ibid.), a five-page portfolio called The Book of Mutations and In Our Own Image, ‘a pair of digital video-sculptures that present, respectively, moving images of Genesis bacteria and the Genesis three-dimensional protein’ (ibid., p. 260). In 2001, all these elements were brought together in the solo exhibition in the Julia Friedman Gallery. 14. Ibid., p. 249. 15. Kac (1996) Visible Language 30(2), 98; reprinted in Kac (2007) Media Poetry, p. 11. 16. Examples of such compositions can be found, among many other sources, in R. Kostelanetz (ed.) (1970) Imaged Words & Worded Images (New York: Outerbridge & Dienstfrey); K. P. Dencker (ed.) (1972) Text-Bilder: Visuelle Poesie international: Von der Antike bis zur Gegenwart (Köln: DuMont Schauberg); F. W. Block et al. (eds) (1990) Transfutur: Visuelle Poesie aus der Sowjetunion, Brasilien und deutschsprachigen

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Notes

17.

18.

19.

20. 21. 22. 23.

24.

25.

26.

Ländern (Kassel: Jenior und Pressler); J. Peignot (ed.) (1993) Typoésie (Paris: Imprimerie nationale). My definitions of these categories were first published in Swedish in 1993. They were restated in C. Clüver (2001), ‘Inter textus/inter artes/inter media’ in M. Schmitz-Emans and U. Lindemann (eds) Komparatistik 2000/2001: Jahrbuch der Deutschen Gesellschaft für Allgemeine und Vergleichende Literaturwissenschaft (Heidelberg: Synchron Publishers) (in German) and again in C. Clüver (2008) ‘Intermediality and Interarts Studies’ in J. Arvidson, M. Askander, J. Bruhn and H. Führer (eds) Changing Borders: Contemporary Positions in Intermediality (Lund: Intermedia Studies Press). R. Bohn, E. Müller and R. Ruppert (1988) ‘Die Wirklichkeit im Zeitalter ihrer technischen Fingierbarkeit’ in R. Bohn, E. Müller and R. Ruppert (eds) Ansichten einer künftigen Medienwissenschaft (Berlin: Ed. Sigma Bohn), p. 10. Gallery visitors and internet viewers can interfere in the process by switching on the ultra-violet light; but it would be misleading to think of this as an interactive situation, because these viewers do not respond to any sign coming from the dish. Kac apparently attributes a greater significance to these viewer-induced mutated texts: ‘The ability to change the sentence is a symbolic gesture: it means that we do not accept its meaning in the form we inherited it, and that new meanings emerge as we seek to change it’ (‘Artist’s Statement’ quoted by Jennifer Eberbach (2007) ‘Eduardo Kac’ in B. Stirrat and L. Stephenson (eds) Human Nature. Exhibition catalogue (Bloomington: Indiana University School of Fine Arts Gallery), p. 66). Kac’s Genesis was shown as an installation in the SoFA Gallery’s ‘Human Nature II’ exhibition, 9 February–9 March 2007. Kac (2005) Telepresence & Bio Art, p. 255. E-mail to the author. E. Kac, e-mail to the author, 20 April 2009. The 2001 catalogue of the installation in the Julia Friedman gallery notes on p. 8 that ‘The Genesis Net installation has original DNA music by composer Peter Gena’. Kac likens it to the three writing systems recorded on the Rosetta stone, by which Jean François Champollion, in the nineteenth century, produced ‘the key to understanding the past’, because it made the Egyptian hieroglyphs readable (Kac (2005) Telepresence & Bio Art, p. 254). I. O. Rajewsky (2005), ‘Intermediality, Intertextuality, and Remediation: A Literary Perspective on Intermediality’, Intermédialités: histoire et théorie des arts, des lettres et des techniques / Intermedialities: History and Theory of the Arts, Literature and Techniques VI, p. 51; cf. Rajewsky, this volume. E. Kac (forthcoming) ‘Biopoetry’ in S. A. Glaser (ed.) Media inter Media: Essays in Honor of Claus Clüver (Amsterdam and New York: Rodopi), pp. 283–8.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

186 Mediations and Transformations of Media

Photo/graphic Traces in Dubravka Ugreši´c’s The Museum of Unconditional Surrender Katalin Sándor

In Dubravka Ugreši´c’s novel, The Museum of Unconditional Surrender,1 the reader ‘enters’ the text through a photograph, the single photograph of the text: in different editions this can be found either on the cover or on the first page of the book. What follows is a disjointed archive, the first person narrative of a female Croatian intellectual in exile after the dissolution of the former Yugoslavia, a memory-work about a dismembered cultural, historical and geographical ‘body’, about personal and collective pasts.2 Ugreši´c herself retells in an essay3 the story of Simonides, the ‘inventor of memory’ from Cicero’s De oratore: during a banquet the hall collapsed on the guests, disfiguring their bodies. Only Simonides was able to identify, to re-member the unrecognizable fragments by using space as a mnemo-technical strategy, remembering where the guests had sat at the table. Ugreši´c’s novel itself seems to work as a somewhat Simonidesian experiment, often appropriating photography, albums, collage, film, cards as ‘mnemonic’, (re)mediating strategies for remembering the dismembered body of the past, which appears not as something to be accessed but as something to be (re)constructed, (re)assembled, interrogated. The wrecks of/from the past are recollected in a text that remains unsettlingly dissected, traversed by white lines of interruption. In this process, photography is not primarily relevant as a document, as an indexical evidence of/from the past (being often connected to oblivion4 and to questionable referentiality), but as an intermedial modality to frame, thematize and/or critically reflect on identity, on the representability of personal and historical past, on remembrance and forgetting, on what Ricoeur would call ‘the enigma of memory as presence of the absent encountered previously’.5 The text of this novel is connected to photography in many respects: (1) the photographic medium and the scopic experience is thematized throughout the narrative, the text itself recurrently foregrounding 187 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

12

photography and albums as its own self-interpreting, metadiscursive patterns; (2) the single photograph of the book is rewritten and displaced in two similar ekphrastic texts; (3) the narrator repeatedly adopts the position and the lens of the camera as representational modalities, making textual ‘clicks’ and inserting ‘snapshots’ into the text. This is at times connected to a narratorial attitude of limiting her role to that of a collector, withholding commentary and judgement, sketching a picture, inserting a quotation, a piece from a story and leaving it to the reader (‘let him be patient: the connections will establish themselves of their own accord.’6 ); (4) the text interrupted by textual ‘snapshots’, by numbered fragments and quotations, is not only a discourse on albums, but also an unfastened album-like arrangement; (5) photography and albums are also intermedial practices in relation to which historiographic and biographic representation are problematized by foregrounding their mediated and mediating-constitutive aspect precisely through the ‘objective’ of the camera. Thematizing photography is also a way to expose the medial heterogeneity of discourse. In this sense, the text might be called ‘photo/graphic’ – alluding to Mitchell’s notion of the ‘image/text’ which designates ‘a problematic gap, cleavage, or rupture in representation’7 , and ‘an unstable dialectic that constantly shifts its location in representational practices, breaking both pictorial and discursive frames’.8 In Ugreši´c’s text (which might also be an ‘imagetext’ – as Mitchell9 understands it – combining text and photography), the ‘photo/graphic’ seems to work as a gap or break in the text, an interruption or an unsettling enigma retaining silence within discourse, or a mediating modality that shapes remembrance and representation. The photo/graphic could be an instance of intermediality as Paech defines it: intermedial processes manifest themselves as events of difference, ‘as configurations or as transformative inscriptions of mediality in a work, text or intertext’.10 The in-between rhetoric of intermediality produces medial displacements, figurations of medial difference and the ‘re-transcription of the medium as form’ might insert ‘breaks, gaps and intervals’11 into representation. Thus the term ‘photo/graphic’ (written with a slash) does not refer only to the single photograph in the book, but also to the way in which the text (desires or) aims at approaching (and thus simultaneously displacing or erasing) a medial ‘other’, as well as to the way photographic techniques of seeing, framing, focusing, clicking become ‘readable’ as figurative, textual transcriptions of the absent medium of photography or the absent pages of albums. Paech distinguishes this kind of intermediality as ‘symbolic’ (as opposed to ‘material’), since the medium is not present in another in its materiality, but ‘on the level of depiction’12 , of textual (re)constitution. However, the novel which ‘cites’ photography verbally but also incorporates a ‘visible’ photograph, could be located at times in-between symbolic and material intermediality.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

188 Mediations and Transformations of Media

Alternating between temporal unfolding and the open, spatial co-ordination of collage, this ‘mosaic’ or ‘album-novel’ can be read as a discursive attempt to recollect the dismembered past, and (re)read the present within historical and discursive conditions that do not offer any reference point around which a coherent narrative or identity could be constructed. The text on exile and displacement seems to be a text in displacement as well: several fragments are dislocated as if in transit, circulating and returning in the process of textual repetition. In this nonidentical recurrence, the narrator experiences different modes of identification and self-effacement in different geographical and socio-cultural settings: Belgrade, Berlin, New York, Lisbon and so on. The process of textual, narrative displacements is often figured by ‘the map of the unreal’13 , by the interim dream-rhetoric of emigration, through the figures of heterotopic places like museums, mirrors, swaying houses, albums or the mutant, divided city of Berlin. These also outline a non-linear, self-quoting and selfinspecting discourse (at times more essayistic than narrative) in which some short textual fragments seem to return as the traces, specters or echoes of others. A further metadiscursive pattern exposes the text as non-totalizing (and ‘visibly’ fabricated) discourse through ‘exhibiting’ the stomach contents of Roland, a walrus formerly living at the Berlin Zoo. This unusual display, which enlists the most common and yet the most unimaginable objects from the pink cigarette lighter to a baby’s shoe, becomes a site of de and recontextualized ready-mades, reflecting on the divergent, collage-like process in the (personal and collective) practices of remembering, forgetting, contextualizing and meaning-making. As Rothberg notes about collective memory (and memory in general), ‘Memories emerge in the interplay between different pasts and a heterogeneous present. I call this the multidirectionality of memory: the interference, overlap, and mutual constitution of seemingly distinct collective memories . . .’.14 The text abounds in sites and tropes of heterogeneity, multidirectionality and collage: Roland’s exhibited stomach, the Berlin flea-market, an artist’s studio, the German capital as a divided, mutant museum city, the tarot cards through which the stories of the eight friends and the eight different ‘histories’ of Yugoslavia are told, all scatter any singular, reliable narrative about the past whose coherent representability is contested throughout the text. Sontag claims that photographs expose history, present and past, as a set of anecdotes and ‘faits divers’15 . In this sense, the decontextualizing, fragmenting aspect and the contingency of photography (and collage) might also be relevant from a historiographic point of view: it questions the possibility and the politics of a master discourse, of an exclusive, hegemonic, uncritical and/or ethnocentric narrative about the recent past of former Yugoslavia by leaving room for multiple histories contesting and constituting each other.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Katalin Sándor 189

190 Mediations and Transformations of Media

In the processes of political, cultural and biographical dispersal, the narrator seems to be fixated on an old photograph of three unknown women bathers from the beginning of the twentieth century, an enigmatic image torn away from any specific context. The only clue is that the photograph was taken near the Pakra river, close to the narrator’s birthplace. The narrator takes the photograph everywhere and uses it in her intimate rituals of remembrance. This fetishization of the photo, which may already mark a displacement (being an exile’s surrogate for a loss), could possibly place it at the very core of the text. An especially interesting approach to this novel, that of Ilinca Iura¸scu, defines the position of the photograph as more or less central, as if it were the generator of the narrative: a ‘pivotal element of the Museum’ around which the text ‘will gravitate’16 ; the ‘ “silence” of the image will be interrupted within the narrative discourse and insistently counteracted by the language of gendered self-representations’.17 Nevertheless from another point of view the photo can be just an image, a recurrent one in the album-like arrangement of the text, a trace through which the past might address the viewer and might itself be addressed. At one point, the photograph is called ‘an old yellowing photograph’18 in a list of other random bits of memory. The novel allegorized through the exhibited stomach of the walrus – which frames and rearranges the contingent – resists being read as a centred, hierarchical structure. The text evades any centralizing reading, the photograph of the unknown swimmers ‘disappears’ and reappears among other memory-fragments, or in the ekphrastic transcriptions which multiply its readability and (in)accessibility: it is a fetish, but also a laterna magica of textual anamnesis, a context for (textually) exposing other images, just an old yellowing photograph, or the only recurrent (and ‘transportable’) reality in the continuous dislocations of exile. When the narrator’s mother and her missing photographs are thematized, we might recall Barthes’s Camera Lucida19 and the way Mitchell comments on it. According to Mitchell, in Barthes’s text the missing and only textuallyrecuperated photograph of the mother seems to be placed at the core of the imaginary labyrinth made of all the photographs of the world; the text is a thread that leads to the centre of the labyrinth, which is an absence, a lack, a loss.20 With Ugreši´c we also have absent, only textually-recuperated photographs of the mother, who is very much preoccupied with rearranging the family albums, the ‘archives’ of her own past. The constitutive aspect of remembrance is also relevant in the way in which the process of looking at the past and present and anticipating a future is connected to different modalities of identity-construction. The mother’s face and her photographs are figured as sites for practices of identification (and sometimes anxieties), as mirrors returning the narrator’s glimpse from the future, the glimpse of a

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

A photograph at the centre?

future (‘older’) self, a subject yet-to-be, splitting the sense of presence, oneness and sameness. The absence of the mother’s photograph is also linked to the narrator’s reluctance to take pictures, to the perception of the photograph as an act which prepares the body as a mummified trace – as Bazin21 would suggest – freezing the temporal existence of the subject into a still, lifeless picture, into absence and ‘Death’.22 When the sick mother asks the narrator to take a picture of her, the latter is terrified by the thought that it would be her last photograph.23 Through the lens of the camera she watches her mother, a helpless prey, and is afraid of pressing the shutter to take a picture, which could bring the loss of the mother by preparing for or multiplying her absence. The camera is conceptualized by the latent metaphor of the weapon, as the lens of death, producing a picture and/but erasing a subject. The desire to preserve the body as a photographic trace is overcome by the anxiety that the photo will become a memorial image irrespective of any intentionality that produced it. Recurrent as they may be, in Ugreši´c’s text none of the visual or textual fragments can acquire the position that encapsulates all the others in the rhetoric of reading: the exhibition of the stomach of the walrus as a metadiscursive frame encourages not the trope of the labyrinth with a centre (even if that is an absence) but rather the pattern of collage in which the text does not lead the reader on a trajectory like Ariadne’s thread; a collage with multiple nodes in which the old yellowing photograph and the absent photos of the mother intersect with other texts or textually (re)constructed images.

Rewriting the ‘fetish’ The old yellowing photograph on the cover of the book is thematized several times, in various allusions or in two almost identical ekphrastic transcriptions. In both texts, the photograph is referred to as a ‘little fetish object’,24 and this is related to the way in which it is used by the narrator. In the theoretical discourses on photography there seems to be a kind of overlapping in the conceptualization of this medium: one might notice the recurrence of certain features, such as fetishizing fragmentation, metonymy, cut, rupture, displacement, lack or absence. In Bazin’s realistic reading, the photograph is conceptualized as an act of mummifying time,25 for Sontag photos are memento mori or a ‘thin slice’26 of time and space, Barthes also relates it to lack, absence, death and a missing mother,27 and Mitchell, commenting on Said and Mohr (especially on the latter’s photographs about anonymous Palestinian exiles), observes that ‘photography re-doubles the exile of image from referent’.28 However, naming the photograph through the concept of the fetish is also likely to invoke psychoanalytic, Marxist and feminist discourses on fetishization. Relying on such discourses, Iura¸scu rightfully observes that the photograph is the site for the fragmentation and fetishization of the female

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Katalin Sándor 191

body, for its reduction to an ‘item of visual inspection and exchange’29 by a phallocentric scopic regime, as well as a site for reifying gendered, racial, national otherness within hegemonic power relations. Still – according to Iura¸scu – photography can be read in a subversive way by incorporating the image into a linguistic framework through commentary and caption. This is how ‘the photographic image may be allowed to “speak”, ( . . . ) and therefore to do away with the confines of “silence”, “death” and “absence” which have been shown to define the feminine subject within the phallocentric scopic regime’.30 We can completely agree with Iura¸scu that we should understand identity, memory, image and ideology in terms of technological constructedness, which opens them up to contestation. Nevertheless, she assigns the capacity of contesting, subverting the ideologically inscribed photo only to language and discourse, and seems to preserve to some extent the ideological pattern of dualistic oppositions in which the image is essentialized as static, silent, ideologically-fixed, feminine and lacking agency, whereas language is connected to voice, temporality, agency, intervention and the capability of subversion. Such an approach might play down the potentiality of photographs and albums to ‘intervene’ in the text as themes, representational frames and gaps, textual ‘clicks’ or mutable snapshot-like descriptions within a constant textual (!) displacement towards a medial ‘other’.

Photography and the photo/graphic: reading, looking at and diving into the picture Ugreši´c’s text on the old photograph (and photography in general) outlines certain ways in which the narrator uses the image, as well as the way her gaze adopts the perspective of the camera-eye. In the narrator’s scopic experience the photo is not reduced to a single (fetishizing) use: sometimes it is a ‘little fetish object’ whose real meaning is not known, a piece of material which keeps returning in the displacements of exile; sometimes it is an enigmatic, seductive image, or a laterna magica for unleashing memory-work: ‘like a lamp lit in a murky window, a heartening secret gesture with which I draw pictures out of indifferent whiteness’.31 In the ekphrastic description,32 when the photo is ‘read’ as a picture, the narrator tries to ‘decipher’ it by applying historical and iconographic pretexts and codes: she recognizes (through discursive knowledge) the four gourds as old-fashioned swimming rings and sees the shape of the women’s arms as wings. The photo seems to be read through the iconography of the angel (the angel of history or that of oblivion), a recurrent trope in the text which also echoes the reminiscences of Wim Wenders, Walter Benjamin, Rilke and Klee, as well as allusions to Christian tradition. However, the conceptual indeterminacy of the photograph as a kind of unsettling silence cannot entirely be broken, it seems to resist utterance. At one point, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

192 Mediations and Transformations of Media

narrator enunciates (only) what is not there, what does not appear as an attempt to preserve a gap, a void – namely, the absent, invisible something the women seem to be waiting for: ‘Around them hovers an oniric haze full of restrained light. They seem to be expecting something. For some reason I am certain that what they are expecting is not the click of the camera.’33 The women’s look (‘expecting something’) undoes the frame of the photograph and exposes the act of taking pictures as decontextualization, as ripping out from a temporal and spatial continuum. The ekphrastic remediations of the photo might work in this way as never fully completed attempts to reread, recontextualize the image and invest it with meaning. But just as the women hide their hands in the photograph, something remains ripped away, unsaid and unsettling, in excess to meaning or to the readable. This may be somewhat similar to the way in which Barthes’s punctum34 or wound in photography touches the viewer by eluding or disturbing its studium, its culturally coded and intelligible meaning. In the use of the photograph as a means for exposing different images from or of the past, the photograph cracks and is looked through (and not looked at) as a medium of anamnesis: the narrator’s gaze searches a temporal and spatial fissure, ‘a hidden passage’35 to step out of linear or historical time and homogeneous space into the multidirectionality of memory. The narrator’s scopic experience is described not only as a self-interpreting, self-inspecting, critical practice, but also as a self-effacing, immersive process, a surrender to the image. This is articulated in expressions that make the direction and the activity of the gaze indeterminate: ‘attracts my attention, hypnotically’, ‘stare at it’, ‘not thinking about anything’, ‘dive into them’, ‘plunge attentively into’ and ‘slip’.36 Ambivalence also persists in the way the photograph is dealt with throughout the text: aside from the more theoretical-essayistic, sometimes even too-persistently self-reflective, demystifying practice of thematizing photography, there is also a more amateur, naïve, non-professional surrender to the seductive power of the oneiric, enigmatic old photograph. The narrator admits that she has no rational explanation for carrying this photo everywhere. She also makes a distinction between photographic ‘amateurism’ and ‘professionalism’: the advantage of the former over the latter ‘is contained in the point of indistinct pain, pain which an amateur work (like extrasensory perception) can touch and thus provoke the same reaction in the observer/reader’.37 This process of losing one’s sense of self, of giving up to the (power of the) image, is somewhat similar to Barthes’s approach in which he disclaims a more professional, scientific, critical discourse in order to recuperate something from the magic of photography38 through being touched, attracted, wounded by the picture. However, the desire to dissolve the medium and immerse in some memory behind it, is continually counterbalanced by multiple framing, by disclosing the act of mediation: the photo placed in a window frame (‘in the left-hand corner of the window, where the end of the lake can be seen’39 ) is exposed

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Katalin Sándor 193

as a result of mediation and framing – a static, black-and-white square interrupting the continuity of the moving image of the lake which is itself an already framed, pre-mediated sight. The photograph as a requisite of exile transforms shifting contexts and is transformed by them. The oscillation between the (cultural) readability of the photo and its unreadable wound-like, absorbing effect between its studium and punctum might also be observed (in a different sense) in the way the narrator’s voice continually alternates between a more critical, self-reflexive, distancing discourse exposing its own fabricated, ideological aspect (Was is Kunst? is a recurrent question) and a more intimate, confessional, immersive way of recounting the past.

Remediation, discursive remedy or the (critical) intervention of the image? Conceptualizing the photograph as fetish does not automatically position the working of language as subversive and contesting, but rather as a dialectical relation, in which the image can also elude or (actively) intervene in verbal discourse. The narrator calls the photograph ‘a little fetish object’, which might position it as a substitute for the loss in what she considers to be the ‘neurosis’40 of exile. Elizabeth Grosz asks in Lesbian Fetishism what the difference is ‘between the psychotic and the fetishist if both share disavowal and a rejection of a piece of reality’.41 She concludes that the difference ‘seems to lie in the opposition between hallucination and substitution’42 as far as the absent reality is concerned. In this instance, the narrator who uses the photograph as a fetish does not (cannot) hallucinate the presence of a lost reality, but continually substitutes and displaces it in the process of the desire to return. Or, as Edward Said notes in his comments on the photographs of Palestinian exiles: ‘You learn to transform the mechanics of loss into a constantly postponed metaphysics of return’.43 The dialectical relation between photograph and text cannot be exhausted by the discursive, ‘voiced’ deconstruction of the (fetishizing) photographic representation. Iura¸scu’s most challenging approach seems to assign agency, voice and deconstructive potential only to language and discourse: ‘the death of the image allows for the emergence of narrative re/collection’44 and ‘the photograph gains a voice which speaks “in memories” and therefore disrupts the silence imposed by the camera eye’.45 In this interpretation the critical, active, temporal aspect of speech, discourse and narration might seem to be overemphasized as opposed to the so-called static, freezing, fixating aspect of the (photographic) image which is unable to speak for itself. Language is conceptualized not only in terms of remediation but rather as a kind of remedy to the ideological constraints imposed by the photograph: it is words and discourse that contest the reifying representations of the past, the female body and the subject.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

194 Mediations and Transformations of Media

However, from a different point of view, photography also seems to ‘intervene’ in discourse, sometimes even in a quite ‘visible’ way: the text transcribes photographic techniques of image-making, often giving up narrative linearity and inserting textual clicks and interruptions; it performs a multidirectional memory-work which invokes the representational framework of albums, museums, artistic installations, flee-markets and collage. The narrator’s gaze often appropriates the perspective of the camera in ‘taking’ textual pictures, exposing the medial awareness and the constitutive aspect of the point of view and of framing, as well as the contextualizing, mediating role of other pictures and texts (present and past) in the act of perception and image-making. Thus the description of three women bathing in the Adriatic in the narrator’s present is related to the photograph of the three women from the beginning of the twentieth century: the depiction of the scene is already ‘photo/graphic’, produced by an ‘inner click’46 through the ‘lens’ of the yellowing photograph. This textual snapshot produced through the memory of a picture could be read in-between discursive and pictorial frames: it produces a short, cut-out, mutable fragment in the text, the action is freeze-framed by a click, and/but the words ‘click’ and ‘record’ are the medial metaphors of the act of writing. Thus we cannot reduce the image– text relations to the one-way appropriation, subversion and voicing of the image by the text. The slash in ‘photo/graphic’ would signal a split within discourse, a textual displacement towards a medial ‘other’, as well as the modality in which the text transcribes photographic techniques figuratively, producing and erasing the signs of their ‘presence’. By citing the poetics of the album and photography, the text is cut up into fragments, whose arrangement allows for exchangeability and fluid structures. Whether it is the self or history to be narrated, the text works as an open, rearrangable archive. Being traversed by the white, unfilled lines of interruption, it remains critical towards the concept of any homogeneous, exclusive, self-sufficient historical truth. One of Iura¸scu’s most relevant conclusions is that ‘the “productive look” can act as a trigger for the resurgence of “othered”, marginal hi/stories which have been “muted” by the discourses of dominant ideologies’.47 The narration of the past displays – often through the absent lens of the camera or through the poetics of photography and albums – an awareness of its own partial, mediated and mediating aspect and also of the ‘plural’ realities it tries to recount. In Part III the eight friends’ stories named after tarot cards suggest an understanding of history, of personal and collective past in which different truths can be asserted simultaneously. The non-hierarchical working of collage or the continually rearranged albums and cards as strategies of relativization do not remove the problem of truth (and, consequently, falsehood) from the scene, but necessarily multiply it and expose it as relational, constructed and open to negotiation. Such problems are also addressed by contemporary theories of historiography, which,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Katalin Sándor 195

according to Gábor Gyáni, do not abandon the problem of distinguishing truth and falsehood (this being related to the specific status of historiography within other collective practices of remembrance), but accept the fact that different truths can be affirmed simultaneously.48 The textual anamnesis, alternating between self-reflective, critical distance and immersive proximity, makes collages and albums rather than unambiguous judgements; it scatters the fragments of life-stories through collages, albums and snapshots whose ‘silence’ may nevertheless ‘speak’ for itself. The text does not complete a synthesis of heterogeneous memories; rather, it struggles with something which might (still) be outside the representable. Even if the narrator calls this something a ‘still verifiable’49 reality, she does not transform the verifiable into an explanatory narrative, into ‘the story of the local apocalypse’.50 If, in Mohr’s photographs ‘photography re-doubles the exile of image from referent’51 , then in The Museum of Unconditional Surrender (photo/graphic) anamnesis – which produces a sense of otherness about the past – redoubles the exile of the remembering subject from her memories, making it difficult, if not impossible, to feel ‘at ease, at home (heimlich) in the enjoyment of the past revived’.52

Coda The image–text relations in the text call for an understanding of intermediality in which the question of medial difference and aporetical tension (between and within media) is not necessarily settled. Therefore, the present essay does not attempt to straighten out the dialectical, equivocal rhetoric of the photo/graphic image/text. Neither does it try to reinforce the ideological opposition between a silent, static inert image and a temporal, voicing, empowering discourse. Rather, it places the problem back into the allegorical stomach of the walrus. Just like the ‘ready-mades’ in that unusual collage, image and text may relate to each other in multiple and unpredictable ways, and at the same time may intersect with other questions: the different modalities of the subject’s (self)constitution and (self)effacement; the discursive and medial conditions of reminiscence, of historiographic and biographic representation; the contingency of photographic and linguistic archives; the multidirectional and medially heterogeneous work of remembrance. A non-totalizing (intermedial) approach should not necessarily reconcile but rather enter the ‘image/text’ aporia and trace the continuous dislocations of discursive and pictorial frames.

Notes 1. D. Ugreši´c (1999) The Museum of Unconditional Surrender, translated by Celia Hawkesworth (New York: New Directions).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

196 Mediations and Transformations of Media

2. Whether this text can be read as autobiographical – a possibility apparently ‘discouraged’ in the text but, consequently, foregrounded as a problem – would be a relevant question to ask, but it is beyond the limits of this essay. 3. D. Ugreši´c (2001) ‘Elkobzott emlékezet’ [Forfeit Memory], Magyar Lettre Internationale, 40, http://www.c3.hu/scripta/lettre/lettre40/ugresic.htm, date accessed 5 May 2007. 4. Photographs not only ‘archive’ the past, but may also erase it: the framing act of photography also performs an exclusion, a rubbing out of what has been excluded: ‘I tried to remember something else, but my memories stayed tenaciously fixed on the contents of the photographs’: Ugreši´c (1999) The Museum of Unconditional Surrender, p. 24. 5. P. Ricoeur (2006) Memory, History, Forgetting (Chicago and London: University of Chicago Press), p. 39. 6. Ugreši´c (1999) The Museum of Unconditional Surrender, unpaginated. 7. W. J. T. Mitchell (1994) Picture Theory: Essays on Verbal and Visual Representation (Chicago and London: The University of Chicago Press), p. 89. 8. Ibid., p. 83. 9. Ibid., p. 89. 10. J. Paech (2000) Artwork – Text – Medium: Steps en Route to Intermediality, http://www.uni-konstanz.de/FuF/Philo/LitWiss/MedienWiss/Texte/interm.html. 11. Ibid. 12. Ibid. 13. Ugreši´c (1999) The Museum of Unconditional Surrender, p. 113. 14. M. Rothberg (2006) ‘Between Auschwitz and Algeria: Multidirectional Memory and Counterpublic Witness’, Critical Inquiry 33 (Autumn), 162. 15. S. Sontag (1999) A fényképezésr˝ ol [On Photography] (Budapest: Európa Kiadó), p. 33. 16. I. Iura¸scu (2001) Re/collecting Gendered Memory: Photo/graphic Constructions of Remembrance in Dubravka Ugresic’s The Museum of Unconditional Surrender and Leslie Marmon Silko’s Storyteller. Master of Arts thesis (Budapest, Central European University) p. 14. 17. Ibid., p. 3. 18. Ugreši´c (1999) The Museum of Unconditional Surrender, p. 9. 19. R. Barthes (2000) Világoskamra. [Camera Lucida] (Budapest: Európa Kiadó). 20. Mitchell (1994) Picture Theory, p. 305. 21. A. Bazin (1967) What is Cinema? Vol. 1. (Berkeley CA: University of California Press). 22. Barthes (2000) Világoskamra, p. 35. 23. Ugreši´c (1999) The Museum of Unconditional Surrender, p. 20. 24. Ibid., pp. 4, 169. 25. Bazin (1967) What is Cinema? 26. Sontag (1999) A fényképezésr˝ ol, p. 33. 27. Barthes (2000) Világoskamra. 28. Mitchell (1994) Picture Theory, p. 316. 29. Iura¸scu (2001) Re/collecting Gendered Memory, p. 5. 30. Ibid., p. 10. 31. Ugreši´c (1999) The Museum of Unconditional Surrender, p. 173. 32. Ibid., pp. 169–70. 33. Ibid., p. 170. 34. Barthes (2000) Világoskamra, p. 31.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Katalin Sándor 197

35. 36. 37. 38. 39. 40. 41.

42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52.

Ugreši´c (1999) The Museum of Unconditional Surrender, p. 4. Ibid., pp. 4, 169. Ibid., p. 28. Barthes (2000) Világoskamra, p. 23. Ugreši´c (1999) The Museum of Unconditional Surrender, p. 4. Ibid., p. 113. E. Grosz (1993) ‘Lesbian Fetishism?’ in E. Apter and W. Pietz William (eds) Fetishism as Cultural Discourse (Ithaca NY and London: Cornell University Press), p. 108. Ibid., p. 109. E. Said, quoted in Mitchell (1994) Picture Theory, p. 314. Iura¸scu (2001) Re/collecting Gendered Memory, p. 25. Ibid., p. 26. Ugreši´c (1999) The Museum of Unconditional Surrender, p. 6. Iura¸scu (2001) Re/collecting Gendered Memory, p. 26. G. Gyáni (2007) ‘Hamis és igaz a történelemben’ in idem, Relatív történelem [Relative history] (Budapest: Typotex Kiadó), p. 219. Ugreši´c (1999) The Museum of Unconditional Surrender, p. 194. Ibid. Mitchell (1994) Picture Theory, p. 316. Ricoeur (2006) Memory, History, Forgetting, p. 39.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

198 Mediations and Transformations of Media

The Dance of Intermediality: Attempt at a Semiotic Approach of Medium Specificity and Intermediality in Film Hajnal Király

During the centuries following the Italian Renaissance, numerous philosophers, theologians, literary men and artists found it necessary to delimitate poetic and visual arts, and, accordingly, to establish an accurate hierarchy of them. Intriguingly enough, this comparative tradition has persisted after the advent of the film, considered from the beginning – although pejoratively – a ‘mixed art’. The long-lived textual era, though it managed to level the differences between different arts by imposing a universal terminology and interpretation methodology (considering all works of art as simply texts, that is, as readable sign systems), mostly provided close readings of isolated texts, without attempting to place them in a wider, cultural and specific sign system, characteristic for different arts or media. As Mitchell puts it in his Picture Theory, the ‘pictorial turn’ has engendered ‘a postlinguistic, postsemiotic rediscovery of the picture as a complex interplay between visuality, apparatus, institutions, discourse, bodies and figurality. It is the realization that spectatorship (the look, the gaze, the glance, the practices of observation, surveillance, and visual pleasure) may be as deep a problem as various forms of reading (decipherment, decoding, interpretation, etc.), and that visual experience or “visual literacy” might not be fully explicable on the model of textuality’.1 There is instead – we could add with Murray Krieger – an increased preoccupation with the socio-cultural-political subtext of works of art.2 Moreover, the late twentieth century witnessed the overturning of the classical narrative of art history (with, among others’, A. C. Danto’s subversive essays on ‘the end of art’3 ). Different arts are no longer simply responding external theories, recipes, discourses, manifestos, illustrations of what they then become, but they tend to be the discourses themselves, and, most importantly on themselves, their own mediality or/and intermediality. Marshall McLuhan’s famous utterance ‘the medium is the message’ has never been this actual: the 199 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

13

medium is not only mediating, holding the message, but it is the message (of) itself. Beginning with the 1990s, two powerful cinematographic trends have addressed – from various institutional backgrounds – the problem of medium specificity and intermediality in films: the so-called ‘writer’s movies’4 and the contemplative, extremely slow-paced movies, mostly coming from the Far East, defying all complex narratological accomplishments of the film-medium. The writer’s movies, many of them from the popular and ‘midcult’ register – for example, Shakespeare in Love (John Madden 1996), Quills (Philip Kaufman 2000), Adaptation (Spike Jonze 2003), TupsyTurvy (Mike Leigh 1999), A Cock and a Bull Story (Michael Winterbottom 2005) – represent writing and observing (spectatorship) as complementary and reciprocally conditioning activities. We are often seeing the writer struggling to write the story we are watching; or, conversely, we are witnessing the effort of a director to mediate between the writer and spectators. This is no longer simply a ‘self-reflective’, stylistically identifiable feature (formerly an exclusive characteristic of arthouse movies), but a mediatic gesture of selfawareness: both the writer and director are stepping over the limits of their own medium to satisfy the expectation of their spectators. Film does not need to define itself as ‘art’ anymore – the urge to delimitate itself from other arts, to prove its representational competence has expired. Instead it regards itself as a medium, inseparable – as W. J. T. Mitchell puts it in his Picture Theory – from its socio-cultural, institutional background and foreground, its spectators and the various discourses of spectatorship.5 This accentuated self-awareness, which deals with its own intermediality or mixed mediality (so ferociously attacked by the long lasting comparative tradition of ut pictura poesis . . .) , is systematically overturning the strict delimitation between literature and film along with the idea of conceptuality of the first and visuality of the latter. In fact, the conceptuality of film has found defenders from the very beginnings of its history (the avant-gardes, Russian film theories, the Nouvelle Vague, the semiologic approach and, lately, the cognitive theory of David Bordwell6 ) – but this time the ‘meaning making’ is often happening (is being modelled) in the film diegesis itself. The huge amount of visually extremely rich movies coming from the Far East, with their intensely chronotopic imagery and slow-paced, minimalistic narrative, on the other hand, continuously turn the ‘running time’ of the narrative into ‘space’, a static, ‘plastic’ visual work of art: a picture (for example, Zhang Ke Jia: Still Life, 2006). This is already an intermedial relationship involving film and painting: we are watching the film as we contemplate a painting, ‘scanning’ and ‘making meaning’ of all the details included in the frame. These versions of intermediality in films – the self-conscious inclusion of literary and painterly modalities – are not loudly proclaimed as they once were in the form of manifestos, as with isolated works of Jean-Luc Godard or Peter Greenaway, but appear as naturally integrated in films with the most varied institutional backgrounds, belonging already to the everyday

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

200 Mediations and Transformations of Media

experience of the film medium. Films have become our medium, our ‘natural environment’. This is partly due to the ‘nature’ of the medium: as Joachim Paech puts it in his medium–form comparison, the medium is not observable in itself, only the form is, and the medium appears in whatever form it makes possible. ‘The medium can only be observed in its Other’. Its disappearance aids the other’s emergence, in which it participates in a ‘parasitic way’. He continues: The only possibility to reach the medium behind the form consists in selfobservation of the observation and the re-entry of the medium as form or as a back link, in which mediality as the constitutive difference in the oscillation between medium and form becomes observable as the ‘parasitic third’, whose background noise renders the event of the difference, thus, the message, perceptible and comprehensible.7 However, as already mentioned above, the ‘noise’ made by this parasitic third, the medium or the ‘intrusion’ of another medium, is becoming less and less perceptible: the ‘surprising’ forms of montage, superimpositions, framing techniques or sound-effects are now common characteristics for any film register – popular, midcult or arthouse movie. Instead, the film medium tends to appear in the complete lack of these learned or technological features – or, according to Paech’s terminology, in the ‘breaks, gaps and intervals’ of the form: extreme (narratological) minimalism, uncomfortably long shots and almost complete lack of dialogue. Together with this obvious return to its origins – considered a sign of maturity by Rudolf Arnheim and Erwin Panofsky, among others – along with a growing interest in cultural and post-colonial studies, film is also acquiring an increased socio-cultural responsibility. It is showing, unmasking, symbolically representing cultural, social, political reality. These minimalist movies rely on the aesthetics of the frame, instead of that of the cut, and the changed role of the spectator consists of ‘scanning’ and interpreting the signs and symbols it contains. It is a more active form of spectatorship: a continuous effort of meaning-making instead of losing ourselves in a perfect diegesis. We become increasingly aware of the presence of the medium and our role as spectators. The – almost forgotten – semiotic analysis appears to gain relevance in the interpretation of these symbols, but, this time, it is strongly related to cultural discourses of the film as medium. As Marc Laverette argues in his contribution to Image and Narrative,8 the necessity of a discipline embracing both semiotic and medium theory principles – a so-called ‘semiotic mediatics’ – has never been so actual. The delay of this merger is possibly due to the fact that, with the emergence of medium theory, semiotics is already considered exhausted and old-fashioned: Medium theorists need to incorporate semiotics into their paradigm to gain a respect for content and the overarching importance of meaning. 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Hajnal Király 201

202 Mediations and Transformations of Media

Below I attempt a semiotic analysis, going beyond close textual reading, of Hungarian director Béla Tarr’s adaptation of Satan’s Tango (1994).9 This seven-and-a-half hour long film merges both trends mentioned above; the writer’s movie, raising the issue of film–literature relationship and of the adaptation, and the slow-paced, contemplative trend incorporating the painterly tradition. Interestingly enough, in this ménage à trois or complex intermedial relationship, the painting mediates between the other two: the aesthetics of the frame, chronotopes such as the perspective, the circle, the interior–exterior (house and road) opposition and the threshold, are all transmediatic symbols which reconcile mediatic differences between literature and film. This approach, using Greimas’s semiotic square model, aims to contribute to the methodology of analysing medium specificity and mixed medium or intermediality, in the spirit of an old or new discipline: semiotic mediatics.

Defining (inter)mediality: the tango metaphor As often happens at the very beginning of a new theory or discipline, metaphors fill the gap of a missing terminology: early film theory is crowded with musical, painterly and architectural metaphors meant to grasp the ‘essence’ of the new medium. Similarly, intermediality has been described as a ‘dialogue’, ‘in-between-ness’, ‘translation’, ‘oscillation’ or ‘flickering’ between the involved media. As we know from W. J. T. Mitchell, there are always at least two media involved, for every medium is mixed by nature. Béla Tarr’s adaptation raises a series of issues related to mediality, medium specificity, limits and limitations of media, medium experience and intermediality, the boundaries of media involved in adaptations, challenging classical oppositions such as spatiality–temporality, or perceptual–conceptual. Accordingly, the ‘tango’ or ‘dance’ metaphor from the title can be regarded not only as a thematic, but also, at the same time, as an extended theoretic figure. As a thematic figure, it interprets the hesitation of the characters to step out from their lethargic environment (an abandoned farm) which keeps them prisoner, and from the moral devaluation presented in the brughelian dance party just before the devilish pact with the false prophet (another possible meaning of ‘Satan tango’). As a theoretical figure it presents, first of all, the experience of the medium as a dance of the spectator. Satan’s Tango crosses the conventional, institutional boundaries of the medium by exposing the viewer to a contradictory medium experience: at the beginning, the low-key images and an extremely slow camera movement make it difficult to watch, increasing the awareness

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

And while medium theory needs semiotics to better understand the signs of life, semioticians need medium theory in order to better understand the ‘allness’ of our signified environment.

of our physical reality and body; then there is a gradual absorption due to the hypnotic pace – or dance – of camera movement. We forget about our body, being here as spectators of the movie and then in-there, as one of the film’s observing characters. We continuously step in and step out, engaged in a tango between the film diegesis and our reality. The camera itself engages in a continuous oscillation between stasis and slow movement, a tango which emphasizes the spatio-temporality of the film medium. Moreover, due to its unusual length, the film is screened with three intervals, and the screening room along with the entire institutional apparatus (a small art-movie theatre in Budapest, which had Satan’s Tango on its weekly program) thus becomes a ‘frame’, enabling us to step out to our reality, and then back to this overwhelming medium experience. In this film, the strong presence of the medium overflows the form and appears, as Joachim Paech puts it, as a parasitic third, a noise disabling the creation of a perfect illusionary world. The form processes are continuously interrupted by breaks, gaps caused by the extremely long shots and slow camera movements. Moreover, our motionless bodies in the dark screening room are perfect analogies of those of the characters who are sitting in the dark interiors, staring helplessly at the window, the only source of light. Then, there is another dance: the film, as an adaptation of Laszlo Krasznahorkai’s homonymous novel,10 also marks a relevant mediatic turn, another border-crossing, strongly connected to the socio-political changes induced by the fall of the communist regimes in the Eastern Bloc. Krasznahorkai’s acclaimed novel appeared in 1987, shortly before the events of 1989, and it thematizes, among other things, the ultimate responsibility of the writer to document the events witnessed. In the iconophobic era of communism, in which all images of reality were banned except those that were constructed, writing and literature became the medium able to confront the interdiction, in this case disguised in a parable about a group of people forgotten on a farm, waiting for the arrival of a false prophet, who then takes them to the town and turns them into informers for the secret police. Tarr was immediately interested in adapting the novel, but it took him almost eight years to realize this project, partly due to a lack of institutional and material support persisting after the change of the political regime. Turning the parable into the reality of film images, shot on an abandoned farm, thus using the medium as an unveiling manifesto, Tarr was one of the first film directors in Hungary to take a position towards the recent past and the present. As a film, Satan’s Tango became as thoroughly intertwined with the socio-political message of its time as had the novel. The medium is the message: it mirrors the most actual preoccupations, needs and aspirations of its time. The message is not only the content: it is identical with the inseparable unity of form and content. While the novel thematized the responsibility of writing, its adaptation reflectively represents the observer, spectator and director. Moreover, and

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Hajnal Király 203

most excitingly, the emblematic positions of the writer and director, representing the two different media, are here melded in the character of the doctor. This proves to be a self-reflective gesture, which mirrors the intentional annihilation of medium borders in this adaptation. The doctor switches deliberately from observing or watching (not surprisingly, the only point-of-view shots in the film are his) to taking notes. Watching or observing and writing or reading are shown as complementary activities: in this respect, his name – ‘the doctor’ – (we know all the names of the other characters) appears to be symbolic, as he is ‘healing’ the old aversions between poesis and visual arts. This constant stepping back and forth or intermediality as a dance or tango is, at the same time, a figurative representation of the relationship between the writer Krasznahorkai and the film-director in making this film. In this respect, this adaptation is more than a simple case of what Joachim Paech calls material intermediality – a technical transposition – but is also a symbolic one, due to this double representation (the picture shows itself), or mise en abyme. On the other hand, we see the doctor writing the novel, the adaptation of which the film is: a reinforcement of Marshal McLuhan’s principle that the old medium is always becoming the content of the new one. A further gesture denies a delimitation of the two media involved, and thus excludes the ‘ut pictura poesis’ comparative method: nothing in the film’s credentials alludes to adaptation or to any transformational works undergone. Moreover, the writer of the original novel appears, along with Ágnes Hranitzky and director Béla Tarr, as the author of the movie. This gesture overtly replaces the notion of the author, related to the concept of artwork, with that of author function, defined by Barthes and Foucault: this function, instead of granting the integrity of one specific text – the novel or the film – is responsible for the connections between different texts, and ensures the functioning and status of a discursive set in a given society and culture.11 It is the principle of cohesion in an intertextual web. Thus, this collective authorship participates in a collective, intermedial ‘meaning-making’.

Another tango: space and time, description and narration in the novel Lászlo Krasznahorkai’s novel already challenges the limits of the poetic medium and its temporality due to an original approach to description and narration. The central narrative feature of Satan’s Tango is the ‘fight’ between these two: the world of silent, static objects is repeatedly overtaking that of human action, which freezes all the time into stasis. The kitchen is setting off, like a car, and the silent objects around the protagonists suddenly start a nervous dialogue. The temporality of poetry and all literature, so ferociously defended by the classic discourse of ut pictura poesis seems to

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

204 Mediations and Transformations of Media

be overturned by this kind of narrative, celebrating the spatial effect of the freezing movement and the temporary extension of the description. The floating character of the rigorously detailed description and the tendency of the action to freeze in a tableau vivant is characteristic of what literary criticism calls – in connection with the work of Gabriel García Márquez – ‘magic realism’, often cited as reference for the narrative style of Lászlo Krasznahorkai. This magic realism appears as an uncanny fusion between the preference for detail and materialism of the nineteenth-century narrative and the visionary images of the ancient myths. On this basis the narrative of Satan’s Tango may be called chronotopic: the transmediatic notion of the bakhtinian chronotope12 is a concrete, substantial unity of spatial and temporal characteristics. Time becomes form and space extends into a temporal continuity, space and time become reflections of each other. The characteristics of time are revealed by space and vice versa, space is measured and filled with content by time. The house, the threshold, the road are considered typically chronotopic, transmediatic forms, susceptible to such narrative categories as strange world, public life, encounter, time of adventure and such semantic contents as knowledge, power and desire. In Satan’s Tango, the mobility of characters is reduced to the house (the space of the observer) where they sit all day long, watching ‘how the damned life goes on’ and the muddy road (space of action), flowing away as a river (of time) after the autumn rains, makes it impossible to leave the house. Thus, the moving, acting character is replaced here by the observer, who participates, as Greimas puts it in his structural semantics, in the ‘spectacle of knowledge’.13 The News that They are Coming, Knowing Something, The Perspective, when from the Front, The Perspective, when from Behind, The Circle Closes – as these titles from the table of contents of the novel show, news and knowledge become central semantic contents, together with the chronotopic perspective, a major compositional element from the time of Renaissance painting (in which spatial and temporal aspects are intertwined) and also a symbolic form to express longing and desire.

Reading out the picture: ‘meaning making’ in the film In Tarr’s film, perspective becomes a symbolic correspondent to the semantic content of knowing something, ‘thinking in perspective’, making plans or longing. Accordingly, all roads in the film are represented in perspective and the characters moving on them are shown from behind, except for the last trip of the doctor to the church, when we see him coming back, from the front, as he refuses to leave the farm: the circle closes. The house and the road also become basic chronotopic components of a systematic language, as different frames – windows, doors, thresholds – and ‘liminas’ between interior spaces and exterior world, powerful visual symbols expressing the protagonists’ cognitive modalities, their ability to ‘see

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Hajnal Király 205

through’ (that is, to understand their situation) and, consequently, to ‘step through’ (to change their lives). Through these visual elements all specific mediatic features such as the role of point-of-view in literary and film narration seem to dissolve into a more general language of visual media and visual anthropology. Thus, the semantic contents of understanding and power are not connected to the points-of-view of certain characters, but are represented through their relationship to frames and thresholds. This pictorial convention is represented, for example, by seventeenth-century Dutch painting, one of the most prominent examples being the series by Pieter de Hooch, showing the relationship of different characters of the household (a child, a maid) and to the outer world, in accordance with contemporary social codes. As Wolfgang Kemp has demonstrated, this convention is already present in the works of Giotto and his contemporaries.14 Similarly, in Satan’s Tango, the characters’ relationships to frames or thresholds represents their ability or inability to understand the satanic plan of Irimias (allusion to prophet Jeremiah) and, accordingly, to step out of this trap. Greimas’s semiotic square seems to be a model capable of systematizing these different attitudes, which are responsible for the dramatic turns in the plot. This model, as well as the actantial system, is organized around the key notions of ‘knowledge’, ‘power’ and ‘desire’, and explores the ‘meaning of human actions’ and ‘how man and the world are’. Claude Gandelman, in his Reading Pictures, Viewing Texts uses the semiotic square to demonstrate the cognitive and passage modalities identifiable in seventeenth-century Dutch paintings.15 According to this model, the distribution of the characters on the semiotic square ‘to be able to see (through)’ would be as illustrated in Figure 12. To be able to see (Freedom) Irimias, the doctor, Futaki, Essie After her flight from home

To be not able not to see (Obedience) The informers: The Schmidts The Halics family, the teacher

To be able not to see (Independence) The doctor

To be not able to see (Powerlessness) Essie on No man’s land The farm’s inhabitants before their exodus

Figure 12 Greimas’s semiotic square of cognitive modalities, from A. Greimas (1987) ‘The Interaction of Semiotic Constraints’ in On Meaning: Selected Writings in Semiotic Theory. Trans. P. J. Perron and F. H. Collins. (Minneapolis: University of Minnesota Press)

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

206 Mediations and Transformations of Media

Figure 13 Frames from Satan’s Tango by Béla Tarr (1994). Window- and doorframes, ‘liminas’: visual schemata expressing the semantic contents of ‘knowing something’ and ‘passing through’

Similarly, the semiotic square of passage modalities would appear as follows: ‘To be able to pass’ or freedom (Irimias, the doctor, Futaki, Essie after her flight from home), ‘to be able not to pass’ or independence (the Doctor and Essie), ‘to be not able not to pass’ or obedience (the Schmidts, the Halics family, the teacher, Essie on the no man’s land) and ‘to be not able to pass’ or powerlessness (Essie on the no man’s land and the farm’s inhabitants before their exodus) (see Figure 13). This model not only ensures a categorization of different attitudes, but it is also able to draw the turning points of the narration, which, according to its classical definition, always has to include development. As depicted here, the majority of characters remain blindly obedient (unable not to see and not to step out, or more precisely, even though they step out in the end, nothing changes), while the diabolic freedom of the false prophets and the independence of the outsider doctor remain unchanged. Intriguingly, the only development occurs in the case of Essie, the little girl considered mentally disabled: she manages to step out from her obedient position and gain her independence and absolute freedom through suicide and a miraculous ascension, but this narration is cut short by the middle of both the novel and the film. This is as if to emphasize that narrativity is not an inherent specificity of the film medium, but a learned one. The tension is not created on the level of the story, but is due to a continuous struggle between action (connected to the road) and observation (associated with the house), narration and description, and (de)monstration. The drama of hesitation – Satan’s Tango – is not only that of characters, but evidently that of the ‘two faces’ of the medium: showing (or opening a window to) reality (the documentary tradition established by the Lumière brothers) and telling a story, creating a diegesis, the illusion, the magic of another world often associated with the pioneering work of Meliès. The film chooses the ‘descriptive mode’, thus creating or modelling a visionary world. According to Panofsky, the visionary has been drawn into images by perspective: ‘Perspective seals off religious art from the realm of the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Hajnal Király 207

magical, dogmatic and symbolic and opens it to something entirely new: the realm of the visionary, where the miraculous becomes a direct experience of the beholder; to the realm of psychological, the miraculous finds its last refuge in the soul of the human being represented in the work of art.’16 At the end of the film the doctor sets off to find out whether the bell he hears ringing is real or just an illusion. This can be conceived of as another statement in favour of the reality paradigm of film theory: films exist – as other works by Tarr exemplify – to show and document reality, not to create an illusion of it. After his return to the house (the place of observation) we see him boarding up the window: the fact that the closing of the window marks the end of the film, presents the medium as another frame erected between us and reality, enabling or not enabling us to see through – the old question of the medium transparency – or to pass into its diegetic world. Again, as Panofsky puts it in his Perspective as symbolic form, it is the perspective that transformed the entire picture into a window, ‘and we are meant to believe we are looking through this window into a space’.17 At the same time, this scene metaphorically reinforces the inseparability of the media involved in the adaptation. Intermediality here appears as a space of close interaction between the writer and director, resulting in a transmediatic chronotopic language, the only kind adequate to present a parable on the human condition. This transmediatic chronotopic and symbolic language has been realized with the involvement of codes from a third medium: painting (such as composition, framing, perspective). As I already mentioned above, this is not an isolated tendency in contemporary film production, especially if we consider the wave of films coming from China, Korea, Hong Kong and Japan, currently conquering European cinemas. Their narratological minimalism and visual elegance – reminding us of the best tradition of European film (the work of Michelangelo Antonioni, Andrej Tarkovskij) – brings a refreshing new view to an exhausted Western production. This seems to reinforce the principle that the return to the beginnings is a sign of maturity of any art, already independent from the necessity to show technological competence. Similarly, the concept of medium is no longer reduced to technology. As Erwin Panofsky summarizes in his essay on perspective: Such reversals, which are often associated with a transfer of artistic ‘leadership’ to a new country or a new genre, create the possibility of a new edifice out of the rubble of the old; they do this precisely by abandoning what has been already achieved, that is, by turning back to apparently more ‘primitive’ modes of representation. These reversals lay the groundwork for a creative reengagement with older problems, precisely by establishing a distance from those problems.18

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

208 Mediations and Transformations of Media

This observation coincides with Rudolf Arnheim’s prophetic conclusion of his desperate essay ‘New Laokoön’, on the incompatibility of film and sound: as these ‘hybrid’ forms are always unstable, there is always a hope of returning to ‘pure forms’.19 As we have seen, the second century of cinema has proven itself promising in terms of ‘mediatic purification’. Narratology tends to lose terrain, and the need of a semiotic mediatics has never been more urgent.

Notes 1. W. J. T. Mitchell (1994) Picture Theory: Essays on Verbal and Visual Representation (Chicago: University of Chicago Press), pp. 16. 2. M. Krieger (1992) Ekphrasis: The Illusion of the Natural Sign (Baltimore MD and London: Johns Hopkins University Press). 3. See, for example, A. C. Danto (1987) The State of the Art (New York: Prentice Hall Press) and more recently, A. C. Danto (1997) After the End of Art: Contemporary Art and the Pale of History (Princeton: Princeton University Press). 4. See, for example, P. Arthur’s essay (2005) ‘The Written Scene: Writers as Figures of Cinematic Redemption’ in R. Stam and A. Raengo (eds) Film and Literature (London: Blackwell), pp. 331–42. 5. I am not dealing here separately with the technical aspects of the medium, as I consider it thoroughly intertwined with the aesthetical, cultural and sociological ones. In fact, all technical improvements (the colour, the size of the screen, the television) have added new aesthetical dimensions to the film. As I point out in my essay, film often tends to ‘deny’ its technical specificity (often identified with ‘medium specificity’) by turning towards the conventions of other media, such as painting or photography. Interestingly enough, in this case, the lack of learned conventions becomes a qualifying factor. 6. He is considered the founder of cognitive film theory, relying on cognitive psychology. See, among others: D. Bordwell (1985) Narration in the Fiction Film (Madison: University of Wisconsin Press); D. Bordwell (1989) Making Meaning: Inference and Rhetoric in the Interpretation of Cinema (Cambridge MA: Harvard University Press); D. Bordwell and K. Thompson (2000) Film History: An Introduction (New York: McGraw-Hill) and most recently, D. Bordwell (2008) Poetics of Cinema (Berkeley CA: University of California Press). 7. J. Paech (2000) Artwork – Text – Medium: Steps en Route to Intermediality, http://www.unikonstanz.de/FuF/Philo/LitWiss/MedienWiss/Texte/interm.html. 8. M. Leverette (2003) Towards an Ecology of Understanding: Semiotics, Medium Theory and the Uses of Meaning, http://www.imageandnarrative.be/mediumtheory/ marclaverette.htm. 9. Béla Tarr (1994) Sátántangó. Writers: László Krasznahorkai and Béla Tarr. Cast: Mihály Víg, Putyi Horváth, László Lugossy, Éva Albert Albert Almássy, János Derzsi, Irén Szajki, Alfréd Járai, Miklós B. Székely, Erika Bók, Peter Berling. 10. Krasznahorkai László (1993) Sátántangó (Budapest: Széphalom Könyvkiadó). Available in French translation: Tango de Satan (2000) translated by Joelle Dufemilly (Paris: Gallimard). 11. See R. Barthes (1993)[1960] ‘Authors and Writers’, in S. Sontag (ed.) A Roland Barthes Reader (London: Vintage), pp. 185–93, and M. Foucault (1991) ‘What is

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Hajnal Király 209

12.

13.

14. 15. 16.

17. 18. 19.

an Author?’ in P. Rabinow (ed.) The Foucault Reader: An Introduction to Foucault’s Thought (London: Penguin Books), pp. 101–20. See M. M. Bakhtin (1984) ‘Forms of Time and of the Chronotope in the Novel: Notes toward a Historical Poetics’ in C. Emerson and M. H. Austin (eds) The Dialogic Imagination: Four Essays (Austin: University of Texas Press) pp. 84–258. A. Greimas (1987) ‘The Interaction of Semiotic Constraints’ in On Meaning: Selected Writings in Semiotic Theory, translated by P. J. Perron and F. H. Collins (Minneapolis: University of Minnesota Press), pp. 48–62. See W. Kemp (1996) Die Räume der Maler: Zur Bilderzählung seit Giotto (München: Verlag C. H. Beck). See C. Gandelman (1991) Reading Pictures, Viewing texts (Bloomington and Indianapolis: Indiana University Press), pp. 14–55. E. Panofsky (1997) Perspective as Symbolic Form, translated by C. S. Wood (New York: Zone Books). Originally published as Die Perspektive als ‘symbolische Form’ in the Vortrage der Bibliothek Warburg 1924–25 (Leipzig and Berlin, 1927), pp. 258– 330. See ibid., p. 27. Ibid., p. 47. R. Arnheim (1957) ‘A New Laocoon: Artistic Composites and the Talking Film’ in Film as Art (Berkeley and Los Angeles CA: University of California Press), pp. 199–230.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

210 Mediations and Transformations of Media

Media in the Cinematic Imagination: Ekphrasis and the Poetics of the In-Between in Jean-Luc Godard’s Cinema Ágnes Peth˝ o

A love of cinema desires only cinema, whereas passion is excessive: it wants cinema but it also wants cinema to become something else, it even longs for the horizon where cinema risks being absorbed by dint of metamorphosis, it opens up its focus onto the unknown. —Serge Daney: The Godard Paradox1

Intermediality, the cinematic ‘in-between’ and ekphrasis The complex mediality of cinema is unique among all other arts in its paradoxes and raises a constant challenge not only to theorists who try to define its characteristics but also to filmmakers who consciously explore its boundaries. On the one hand, cinema is the most transparent or ‘invisible’ medium possible, operating with moving pictures that result in the illusion of reality (we seem to see the things themselves and not their representation), and engaging all our senses in their perception. On the other hand, it is also the most abstract and constructed medium possible that has no palpable material form (all the sensual complexity of the cinematic image being nothing but an illusion). More importantly, from a media theory point of view, the moving picture as a medium can remediate all other media forms used by human communication. The mixed mediality of cinema – although it has often been described in terms of the Gesamtkunstwerk ideal – is not a result of an additive process (a unity of moving pictures, language, sound and so on), but consists of a very unstable set of interrelationships that has undergone many changes in its configuration throughout its technical and stylistic history.2 Cinematic experience itself can be defined by the tensions of being in a state of ‘in-between’: in between reality and fantasy, in between empirical 211 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

14

experience and conscious reflection, in between words and images, in between the different art forms and in between media. The mediality of cinema can always be perceived as intermediality, as its meanings are always generated by the media relations that weave its fabric of significations. Cinema can be defined as an impossible, heterotopic space where intermedial processes take place, and where figurations of medial differences are played out.3 Moreover Foucault’s ‘heterotopia’4 seems adequate to describe not only the way in which the images are being situated in a ‘place without place’, but also more generally, to describe cinema’s ‘place’ itself among the arts and media. Jean-Luc Godard’s films have long been associated with the idea of intermediality, in fact it seems that no theory of cinematic intermediality can be forged without references to his works.5 Godard has discovered in cinema a ‘space’ in which all other forms of representations can be inscribed and all other media can be re-mediated. In his films images are always closely related to words6 and cinema is always conceived in a dynamic relation with the other arts, a relation that connects Godard’s cinema to a more general artistic tradition: the phenomenon known as ekphrasis. Ekphrasis, as we know, is a rhetorical device elaborated in Antiquity consisting in the detailed description of a gallery of paintings or a group of statues, a case where a verbal text is produced in competition with the plastic arts. In essence, it is generally understood to stand for the urge of an artist working in the medium of language to express whatever falls beyond the realm of language, to use linguistic expressivity as a ‘tactile’ or visual sense and thus cross over into the domains of the visible. Ekphrasis has been a much debated question in literature, but its applicability to questions of cinema has not been thoroughly investigated. Nevertheless, in a medium so tied up with all other forms of human expression, questions of media borders are bound to emerge. We can say that certain tendencies in film history undoubtedly have aspects that can be related to what theorists call ‘ekphrastic impulse’, a tendency to challenge cinema’s conventionally established perceptive frames, and therefore it seems that the possibilities of a theory of cinematic ekphrasis are worth exploring. What should be clarified first of all, however, is why this particular term should be taken into consideration, and not the term ‘remediation’ that Bolter and Grusin consider as denoting a very similar process within media relations.7 W. J. T. Mitchell defines ekphrasis as ‘the verbal representation of visual representation’,8 Bolter and Grusin call it ‘the representation of one medium in another remediation’.9 Are these two terms interchangeable, as Bolter and Grusin seem to suggest? Mitchell explains quite clearly that there is a possibility of overgeneralizing the term by considering ekphrasis as the name of an overarching principle, and he finally gives a definition that clearly concentrates on the presence of some kind of representation both as signifier (verbal representation) and as signified (visual representation). Bolter and Grusin in their more general

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

212 Mediations and Transformations of Media

media theoretical framework consider ekphrasis as a case of media being incorporated, repurposed by other media. Taking into consideration both viewpoints, I consider that we should not merge the idea of ekphrasis with the idea of remediation or use the two terms as synonyms, but we should consider them as complementary terms. Given also the fact that the idea of ekphrasis is usually linked more closely not only to the idea of representation but also to the aesthetic value of texts, whenever the relations of cinema and the other arts, or the representations of other arts are involved, the term ‘ekphrasis’ seems more adequate; and whenever we can speak of more general media relations (like the use of written or verbal language within a film, for example), the term ‘remediation’ would be more suitable. So what would be the main characteristics of a cinematic ekphrasis? Do all artworks represented in a film result in an ekphrasis? When can we consider that film attempts to ‘challenge its own boundaries’? To list only a few important aspects, the following conditions for the relevance of the term ‘cinematic ekphrasis’ can be named: (a) A film cannot be called ekphrastic simply whenever it includes an embedded representation of another artwork.10 A condition for interpreting it as ekphrasis is that this embedded art form should go beyond the function of a diegetic representation (for example, a painting on the wall) and should be manifest as a medium that is different from that of the cinematic image in which it is embedded. In short, an ekphrasis requires the perception of intermedial relations, as ‘transformative inscriptions’ or ‘figurations’11 of mediality in a work. (b) Cinema can also be perceived as ekphrastic not merely through the media differences of embedded other media forms, but on a more general level, in any case when cinema explicitly attempts to rival another art form (or style developed in another art form). Expressionist films in which we have the characteristically painted settings would be a good example of such an attempt to transform the moving pictures into a sequence of moving paintings. (c) One of the most important features of cinematic ekphrasis is that in fact, in cinema, we can usually speak of multiple or multidimensional ekphrastic tendencies in which one medium opens up the cinematic expression in order to mediate towards the ekphrastic assimilation of another. In most of the cases when cinema imitates another art form this imitation is not the primary ‘target’ of an ekphrastic impulse, but a vehicle, a ‘mediator’ towards yet another medium, the essence of which is perceived as something ‘beyond’ concrete expression, something ‘infigurable’. Common examples of this are the so-called ‘picto-films’, which have acquired something of the status of a sub-genre among literary adaptations, and in which a sense of ‘literariness’ is conveyed through imitations of paintings or painterly styles.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Ágnes Peth˝ o 213

An investigation into the ekphrastic aspects of Godard’s films seems to be extremely fruitful, as his films can be considered ekphrastic not merely because they often transpose representations from other arts onto the screen and foreground essential features of cinematic intermediality, but also because some of his films include explicit quotations from ekphrastic literature and thus engage in a multiple or meta-ekphrastic cinematic discourse. In what follows from the variety of intermedial relations that can be connected to the principles of ekphrasis in Godard’s films, I will outline four such types: 1. A multiplication of media layers ‘opening up’ towards each other and remediating each other. 2. Ekphrasis as a ‘figure of oblivion’. Ekphrasis via media erasures. 3. The function of ekphrastic metaphors. 4. The ‘museum of memory’ and the deconstruction of ekphrasis in later works.

The vertigo of media: ekphrasis and mise en abyme In Jean-Luc Godard’s films there are instances of cinematic intermediality in which one medium becomes the mirror of the other in some way. In other words we can speak of an intermedial mise en abyme. One of the best known examples of this is Godard’s early masterpiece Vivre sa vie (1962, translated as A Life of her Own/Her Life to Live) which also includes a direct reference to the ekphrastic tradition itself. Here, in the final scene, a young man reads out a fragment from Edgar Allen Poe’s short story ‘The Oval Portrait’, which includes an ekphrasis of a painting. Poe’s story is about a man reading a book about a painter painting a portrait of the beloved woman. The story within Godard’s story is also a story within a story. Moreover, the images are over-codified by the voice-over narration (in fact Godard’s own voice). The woman listening to the story of the painting, Nana (Anna Karina) becomes herself virtually a cinematic painting, losing all connections with ‘real life’ just like Poe’s model, whose life is paradoxically stolen away and transformed into the painted image. Throughout the film the protagonist is shown in a Brechtian split between actress (Anna Karina, casting occasional direct glances at the camera, implicitly at her husband-author) and role (Nana).12 In the ‘Oval Portrait’ sequence we are shown the face of Nana against a blank background, the camera outlining her portrait by nailing her to the wall, rendering her a helpless object of representation. All of this is additionally doubled by the mirrorlike presence of a photograph of Liz Taylor pinned to the wall, while Nana herself is ‘painting’ her own lips in a mirror. In earlier sequences in the film, we see her in turn either as a mirror image of Dreyer’s Jeanne D’Arc, or a variation on the topic of ‘lonely courtesan in a Parisian café’ (so familiar in French impressionist painting) or as an intertextual variation on 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

214 Mediations and Transformations of Media

Hitchcock’s enigmatic multiple identity woman in Vertigo (1958). Godard’s film foregrounds cinema’s ‘ekphrastic impulse’ which aims at rivalling the other arts by remediating traditional forms of portraiture both in the visual arts and in literature. The embedded representations flaunt cinema’s multiple mediality, but they also result in an endless process of signification, an endless attempt at ‘figurating’ the ‘infigurable’ identity and beauty of Nana/Anna Karina. The ultimate image of Nana/Anna Karina that we get is placed somewhere in an impossible space between art and reality, between one medium and another, and in this way the film offers instead of the images of Nana’s life (as the title would suggest), the paradoxes of the life of the images of Nana/Anna Karina. Similarly, in other Godard films the numerous reflections of characters in paintings, posters, comic book drawings, genre film iconography, literary figures and so on, can be seen in parallel with the remediational logic of traditional literary ekphrasis.

Ekphrasis as a ‘figure of oblivion’ Besides this ekphrastic model of multiple remediations we have several instances in Godard’s films in which quotations from post-Romantic French poetry are used both as a reference to a model for a relationship to cultural heritage that Godard adopts and also a means of infusing cinematic language with poetry. The poets referred to by Godard usually belong to a literary phenomenon that Harald Weinrich13 described as the ‘art of oblivion’. The traditional ‘art of memory’ (ars memoria) present in European culture since Antiquity was based on the principle of preservation of knowledge by way of mnemonic devices of visualization (association of images and places, for instance). In contrast, the poetry included in the ‘art of oblivion’ was mainly concerned with the renewal of poetic language, of finding new ways of reinforcing the power of words. The birth of a new poetic language is often allegorized in the poems of Verlaine, Rimbaud, Mallarmé or Valéry in a series of images that emphasize the magic moments of loss of consciousness that break up the process of remembering. Poetry derives its power from purification by way of oblivion paradoxically in the presence of ‘frozen images of the past’. Godard’s Bande à part (A Band of Outsiders, 1964) and Pierrot le fou (Pierrot Gone Mad, 1965) are full of quotations from these poets. The cited texts enrich the image and direct our attention towards another medium (poetry), distancing the image from the real life location and weaving around it a texture of pure imagination. There is no break in the pictorial flow, but another verbal picture hovers over the image that we actually see. The scenes gain an inner vibration. Godard’s ambition equals that of the quoted poets: he hopes to enhance the expressivity of the cinematic image, and thus give birth to a new kind of cinematic ‘language’ by remediating the poetry onto the screen. In A Band of Outsiders there is actually a character whose name is Arthur Rimbaud. One of the first scenes in which he appears, together with his 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Ágnes Peth˝ o 215

opening remark, ‘It’s cold and lonely here’, paraphrases Verlaine’s famous poem, Sentimental dialogue: ‘In the old and frozen park, two figures have just passed’. The words that we hear from his friend as well as Godard’s own voice-over are also a mixture of quotations from Verlaine. The quotations are presented in a Brechtian manner, the words do not sound poetic at all: the poetic images seem to be ‘frozen’ into the text of a casual dialogue and voice-over narrative. At the same time, the image they are looking at resembles a painting, the river and the branches of the trees reflecting in the water, the fog veiling the landscape could well be an ekphrastic paraphrase of a number of poems by Verlaine, Rimbaud or Mallarmé. The spectacular landscape in the background appears as a frozen surface beyond which images of literature or painting can be sensed, but which is ignored by the two men. In a traditional (literary) ekphrasis we have an active, speaking subject who contemplates a passive (and usually silent) object of the gaze.14 In Godard’s film we have an active, verbal component (speech) and a passive visual component (image) which are placed in the same cinematic frame and can both separately be called ekphrastic, but which are not in a direct ekphrastic connection with each other. They become interrelated only on a secondary level, where we recognize the type of poetry quoted here and project the images from this poetry onto the screen. The painterly setting ‘erases’ the medium of language (as these pictures replace the poetic imagery that they ‘translate’). The dialogue and voice-over narrative ‘erases’ the medium of the (poetic) image (the unmarked quotations become depoeticized as they are woven into the casual dialogue). The opening up of the image into ekphrastic dimensions of poetry by way of both quoting its lines and ‘forgetting’ about them – first of all by ‘hiding’ them within the dialogue (that is always primarily decoded in its relevance regarding the diegesis) and also by way of remediating aspects of these poetic texts into the images – is paradoxical but effective. There is another scene in Band of Outsiders that could also be linked to the ekphrastic tradition. Godard includes a concrete defiant gesture in his film that can be interpreted as an erasure of a traditional cultural space that usually hosts ekphrastic meditations. I am referring, of course, to the famous scene at the Louvre. The three young protagonists race through the Louvre in a record time of 9 minutes 45 seconds without looking at the masterpieces hanging there. The scene, which ‘forgets’ about canonized contexts and traditional visual artistry in favour of youthful spontaneity, ultimately expresses nothing else but this: the bursting energy of an act of sheer inspiration – something that could be a driving force behind any work of art. Another example of ekphrastic ‘erasures’ is Pierrot le fou (1965), at the end of which Pierrot paints his face blue and thus makes it resemble both the Picasso painting and the portrait of Rimbaud shown earlier in the film. He is thus literally transformed into a cubist image even before, in the viewers’

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

216 Mediations and Transformations of Media

imagination, his face is blown up into pieces similar to cubist portraits. The scene can also be interpreted as a concealed reference to Verlaine’s poem, ‘Pierrot’, in which a frightening face of a scarecrow seems to be blown up, his eyes sizzling in their hollow sockets. Once more, we have multiple erasures: first Godard erases the filmic character and remediates it as a painted portrait, and when Pierrot blows up his face as an image the scene also screens a literary allusion, opening up the image for literature.

Ekphrastic metaphors pointing to the Other of the filmic image Sometimes the mutual erasure, replacement or displacement of text and image and an opening up towards the infigurable is achieved by a single ekphrastic metaphor in Godard’s films. This is the case with literary quotations of only a few words that are included within Godard’s films time and again without any specific mark or clue. On one level these short quotations which are introduced without reference to their source act as a kind of poetic unconscious of the images. On another level these unmarked inclusions of poetic texts have the effect of what Foucault described in his Las Meninas essay in connection with the interdiction of using proper names: ‘If one wishes to keep the relation of language to vision open, if one wishes to treat their incompatibility as starting point for speech instead of an obstacle to be avoided, so as to stay as close as possible to both, then one must erase those proper names and preserve the infinity of the task.’15 In certain cases, however, in Godard’s films exactly the opposite happens, as Godard randomly drops in references to specific names of authors and characters or titles of whole literary works. At the beginning of Pierrot le fou, for instance, the main character, Pierrot complains vehemently about the fact that people no longer think of Balzac when dialling the area code on their phone. This principle of associating literature by way of a one-word reference with commonplace, everyday phenomena is characteristic of many of Godard’s films.16 In the same Pierrot le fou, for instance, we hear the narrator say: ‘Marianne had the eyes of both Aucassin and Nicolette’. Likewise, in the Band of Outsiders, Godard tells us that the characters ‘stopped at a bookstand and Franz bought the novel which reminded him of Odile’. The reference acts as a sort of ‘ekphrastic metaphor’, as we have one word acting as a metaphor that refers to a whole literary text. It does not suggest one particular image, but points to something too complex to be captured within a single image, therefore ultimately unimaginable (we may either not know the texts referred to or know them and then the meanings generated are virtually infinite). However, this placement into a narrative textual context, this mise en histoire can also parody clichés of narrative cinema, which conventionally works by dissolving images within the process of storytelling. As we have learned from the cognitive theories of cinema, the classical dynamics of filmic narrative always consist in images having the role of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Ágnes Peth˝ o 217

directing the viewer’s attention toward the construction of a coherent story. In this case the concrete image is projected into a void that the viewer is confused about how to fill. Classical ekphrasis operates with the absence of the image as the Other of the text,17 Godard plays with the absence of the text. Such ekphrastic metaphors also work in the direction of painting. In A Band of Outsiders, for instance, we hear this: ‘The Seine resembled a Corot’. In the Le petit soldat (Little Soldier, 1963) the narrator similarly says that the ‘somber blue sky reminded me of a painting by Paul Klee’. The images of the foggy river or of the streets at night that appear throughout the film may resemble a painting in general but the particular image it is spoken over may or may not. The image is nonetheless displaced, the word (Corot or Klee) projects it into an impossible space between cinema and painting, just as the earlier example of Marianne’s eyes being compared to Aucassin and Nicolette’s projected the singular concrete image against medieval narrative literature and the whole myth surrounding it. Also in the Little Soldier there is another piece of conversation with a similar logic. First Bruno tells Veronica that she reminds him of a character in a Giraudoux play and later he muses upon the question of whether the colour of girl’s eyes is Renoir-grey or Velázquez-grey. Susan Sontag considers that such references are effective because the viewer cannot verify them.18 We must add that they cannot be verified not because of the ignorance of the viewer, but because of the structure of the reference: the concrete name is referring to a whole range of possible literary works or paintings. Henk Oosterling compares the experience of intermedial in-between to Barthes’ notion of ‘punctum’. He says: ‘The spectator is hit: affected and moved by the punctum’. He considers that this resembles ‘the impossible experience of the breaks between two media’.19 In all these examples, the intermedial reference not only underscores the medial difference (a radical alterity) between cinema, literature or painting, but also identifies in these Others of cinema something that is beyond perception, yet essential in the filmic image. This intermedial opening up of the image achieves the ideal expressed more explicitly by Godard in his later works that ‘the real cinema is a cinema that you cannot see’.20 Oosterling considers this aspect as characteristic for the reception of intermediality itself, a process that entails that ‘the sensible, as a reflective sensibility, balances between presence and absence: going back and forth from one medium to the other, it is a movement in which positions are articulated in the awareness that they are principally relational and provisional’.21

The ‘Museum of Memory’ and the calligrammatic rewriting of ekphrasis As we have seen in the previous examples there appears to exist a permanent duality in Godard’s cinema: the almost tactile quality of the photographic

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

218 Mediations and Transformations of Media

image in cinema, the sensual presence of things doubled with the absence of the physical reality that the image represents, or in certain cases, a void in the signification that can be pointed out by techniques of intermedial mise en abyme or ekphrastic metaphors mirroring the other arts and thus keeping the relationship between signifier and signified infinitely open, making the cinematic image reach beyond its own media boundaries and into the domain of the unnamable. In earlier examples, the filmic image is placed in the ekphrastic, imaginary space created by fragments of poetic language and visual imagery which activate poetic sensitivity but erase literature or even painting as a directly perceivable medium and the institution of museum as a place for the arts or even as a place for meditation upon the arts. In the cinematic essays of the later period the idea of the museum is revived, but more in the spirit of Malraux’s musée de la memoire, a virtually never-ending flow of texts and reproductions of images that generate an also endless number of associations. In Godard’s later films ekphrasis acts as a generative principle. One of the first examples of this can be seen in Letter to Jane: An Investigation About a Still, a film made in 1972 with the co-authorship of Jean-Pierre Gorin, in which Godard and Gorin meditate upon a photograph of militant actress Jane Fonda seen in the company of Vietnamese people during the Vietnam war. The process works both ways in late Godard films: there is a surge of texts interpreting pictures and pictures anchoring the meanings of texts. There is, however, an important deviation from the principle of ekphrasis involved here, namely, the fact that these texts and images are always conjured up not in each other’s absence, but in each other’s presence. So the underlying principle can be called ekphrastic, but otherwise we witness a more explicit word and image relationship in which the two media come to be mutually overwritten and intertwined. The masterpiece in this respect and the ultimate ekphrastic work of Godard’s is undoubtedly the series of essays entitled Histoire(s) du Cinema (Histories of Cinema, a project that he worked on between 1989 and 1999). The ekphrastic nature of the film was consciously explored by Godard who conceived of the project first as the publication in book form of a series of lectures delivered at the request of the Conservatoire d’Art Cinématographique in Montréal, and then released together with the film version, an art book of reproductions and a boxed set of five CDs containing an edited version of the soundtrack (further multiplying the intermedial ‘trans-forms’ of the cinematic project). Godard’s Histoire(s) du cinema can be considered a meditation upon the archaeology of cinema, discovering in it layers of mediality and culture. From this perspective cinema is not defined by its storytelling capacity, but most of all by its possibilities of transcendence, of mediating first of all reality and/or memory. It is also clear from the beginning that for Godard mediation in the movies means remediation. As Jacques

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Ágnes Peth˝ o 219

Rancière wrote, ‘Histories of Cinema is wholly woven out of . . .“pseudometamorphoses,” . . . imitations of one art by another’.22 Cinema appears as painting in movement, as a musical composition of shadows, forms and colours. Medially speaking, it is cinema deconstructed into a constellation of words and images. But which is the medium that ekphrastically reflects the other? It is not the usual case in which literature is seen through cinema or vice versa, it is not even the case in which a newer medium remediates an older one, but quite the opposite: cinema seen through the filter of a seemingly archaic medium of moving pictures. However, this is a form that was constructed in retrospection, a form that has never existed as such, never existed as a vehicle for cinematic storytelling; it can only be called ‘archaic’ because the techniques used were already available at the earliest stages of cinema. Godard took great care in using the most ‘primitive’ techniques possible: photographic inserts, slow motion, shadows projected on a wall and so on, deliberately avoiding the use of more modern technology, while repeatedly showing an old-fashioned editing table and a typewriter as each other’s metaphors in representing the kind of ekphrastic filmic writing he clearly prefers. Accordingly, text and image become equally important and subjected to the same visual compositional principles of fade, dissolve, superimposition. Text penetrates the image, and similarly rhythm and visuality appear as key aspects of language. Viewed from a closer perspective, the medium of the Histoire(s) is derived on the one hand from photomontage, and on the other hand from calligrammatic writing. A calligram-like shot (in fact a detail of the cover of Samuel Beckett’s book The Image) that we can see in part 1.B of the cycle entitled ‘Une histoire seule’ is perhaps emblematic for Godard’s technique. We see the graphic signs of the word ‘image’ appearing as the pupil of an eye in the midst of a white circle of light. The word seems to concentrate the meaning of the image, or the other way round, we can also say that it is the image, the bright circle of light that reveals the graphic signs as text. Moreover, the gesture of this cinematic découpage of the detail of a book cover is also somewhat ekphrastic, as it tears out the word from its literary context and transposes it onto the screen as an autonomous image, thus it actually performs not merely a decontextualization, but also, a multiple intermedial transfer and plunges the title of a well known author into the realm of common language (‘without proper names’, as indicated by Foucault) and pure visuality. By way of the cinematic calligram the aura of the author (and artwork) is lost, the aura of the word in its infinite possible relations and that of the image – that is ‘worth a thousand words’ – is regained. As a whole, Histoire(s) du cinema accomplishes a uniquely paradoxical fusion of photographic collage and calligrammatic text with the musical and spiritual aspects of cinematic montage, and this intermedium is the one that mirrors what cinema is supposed to stand for in-between the arts.23 By

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

220 Mediations and Transformations of Media

deliberately using techniques that can be labelled as primitive or archaic, Godard presents the ‘tangible’ mediality of cinema as it once was: having its roots in photographic representation and indexicality, with its infinite connections with the arts and culture; the mediality of a cinema that came to be displaced, ‘out of time’, with the dawn of the digital era. No wonder that some of the most important literary quotations in the Histoire(s) come from Proust’s À la recherche du temps perdu, as Godard forces us to experience once more the cinematic image in all its ‘palpable’, sensual glory and mystique. This process of rendering the invisible mediality of cinema itself visible by way of a complex set of ekphrases and (calligrammatic) remediations is in fact a feature that is consistent with the essential principle of any Godard film: in it we experience a cinema coming to terms with its own (inter)medial processes, a cinema that never ceases to open up towards other arts and media in a constant quest to become – to quote one section of Histoire(s) du cinema, borrowing a phrase from Malraux – no less than ‘the currency of the absolute’ (‘la monnaie de l’absolu’).

Notes 1. S. Daney (2004) ‘The Godard Paradox’ in Michael Temple, James S. Williams and Michael Witt (eds) Forever Godard (London Black Dog Publishing), p. 68 (italics mine). 2. Technically we can think of such major changes as the shift from silent films to the talkies, or stylistically we can keep in mind that the medial characteristics of a neorealist film are very different from the multimedial extravaganza of a Peter Greenaway film, for example, which consciously constructs its visual texture as an interart palimpsest. 3. See Joachim Paech’s theory of cinematic intermediality in this respect: (1998) ‘Intermedialität: Mediales Differenzial und transformative Figurationen’ in J. Paech (ed.) Intermedialität: Theorie und Praxis eines interdisziplinären Forschungsgebiets (Berlin: Erich Schmidt), pp. 14–30. Cf. also Müller in the present volume. 4. M. Foucault (1984) ‘Dits et écrits: Des espaces autres’ Architecture, Mouvement, Continuité 5 (octobre), pp. 46–9. 5. Cf. J. E. Müller (1996) Intermedialität: Formen moderner kultureller Kommunikation (Münster: Nodus Publikationen). 6. I have elaborated on the questions of the multiplicity of word and image relations in Godard’s cinema in the essay (2008) ‘The Screen is a Blank Page: Jean-Luc Godard’s Word and Image Plays’ in Ágnes Peth˝ o (ed.) Words and Images on the Screen: Language, Literature, Moving Pictures (Newcastle-upon-Tyne: Cambridge Scholars Publishing), pp. 159–87. 7. Cf. J. D. Bolter and R. Grusin (1999) Remediation: Understanding New Media (Cambridge MA: The MIT Press). 8. W. J. T. Mitchell (1994) Picture Theory: Essays on Verbal and Visual Representation (Chicago and London: University of Chigaco Press), p. 152. 9. Bolter and Grusin (1999) Remediation, p. 45. 10. L. M. Sager Eidt’s book on ekphrasis (2008) Writing and Filming the Painting: Ekphrasis in Literature and Film (Amsterdam and New York: Rodopi), includes

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Ágnes Peth˝ o 221

11. 12.

13. 14.

15. 16.

17.

18. 19. 20. 21. 22. 23.

examples of filmic transpositions of paintings but is far from offering a more nuanced examination of ekphrastic phenomena within cinema. Cf. Paech (1998) ‘Intermedialität’. The role itself, through the name Nana, is a hint at a literary text, Zola’s novel having the same title, Nana (the protagonist of which is also a prostitute who has ambitions of working in show business). Cf. H. Weinrich (1997) Lethe: Kunst und Kritik des Vergessens (Munich: C. H. Beck Verlag). ‘The “self” is understood to be an active, speaking, seeing subject, while the “other” is projected as a passive, seen, and (usually) silent object’: Mitchell (1994) Picture Theory, p. 157. M. Foucault (2002) The Order of Things: An Archaeology of the Human Sciences (London: Routledge), p. 10. It is also consistent with Tamar Yacobi’s views according to which ekphrasis can consist of a single ‘ekphrastic simile’ of no more that one phrase, as this functions as an ‘abbreviated reference to a whole pictorial set of works which silently refers the reader to the original itself for details and extensions’: T. Yacobi (1997) ‘Verbal Frames and Ekphrastic Figuration’ in U.-B. Lagerroth, H. Lund and E. Hedling (eds) Interart Poetics: Essays on the Interrelation of the Arts and Media (Amsterdam and Atlanta GA: Rodopi), p. 42. ‘The ekphrastic image acts, in other words, like a sort of unapproachable and unpresentable “black hole” in the verbal structure, entirely absent from it, just shaping and affecting it in fundamental ways’: Mitchell (1994) Picture Theory, p. 158. S. Sontag (2002) ‘Godard’, Styles of Radical Will (New York: Picador), pp. 147–93. H. Oosterling (2003) ‘Sens(a)ble Intermediality and Interesse: Towards the Ontology of the In-Between’, Intermédialités, 1(Printemps), pp. 37–8. Cf. Histoire(s) du cinéma: Une vague nouvelle. Oosterling (2003) ‘Sens(a)ble Intermediality and Interesse’, p. 43. J. Rancière (2007) The Future of the Image (London: Verso), p. 41. I am not disputing Paech’s argument that the film’s main figuration is the medial difference between video as ‘individual’ medium (as video-graphic ‘writing’, a medium suitable for personal archives) and the dreamlike medium of film, I merely suggest that there are some other figurations that contribute to the medial complexity of Godard’s ekphrastic cinematic language. Cf. J. Paech (2002) ‘Intermediale Figuration – am Beispiel von Jean-Luc Godards Histoire(s) du Cinéma’ in J. Eming, A. Jael Lehmann and I. Maassen (eds) Mediale Performanzen (Freiburg: Rombach), pp. 275–97.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

222 Mediations and Transformations of Media

Part V

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

The Borders of Media Borders

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

This page intentionally left blank

15 Heteromediality

In the first part of this essay1 I will offer a very short description of what has been identified as interart studies and intermediality studies. After this I will propose, by way of Lars Elleström’s model, a new multimodal definition of medium. This, in turn, will lead me to my attempt to rethink the field of intermediality studies by way of a new concept, heteromediality. In the second part of my essay I will describe two future possibilities for intermediality studies: either the field may try to establish itself as an academic discipline by strengthening its formalistic foundations, or, on the other hand, a field of investigation based on a more critical and ideological discourse can be imagined. I shall argue in favour of the second possibility. First, a few pragmatic definitions: when talking about ‘text’ I refer to the semiotic idea of ‘complex signs or sign combinations’;2 ‘art’ refers to the conventionally defined forms of music, painting, literature, architecture and so on. I use ‘ideology’ as a term expressing a relatively coherent value system, though not necessarily organized in a political system (consequently it does not refer to the Marxist idea of ideology as merely ‘false conceptions’). ‘Intermedial’ or ‘intermediality’ refers to objects and phenomena whereas ‘intermediality studies’ refers to the activity of investigating intermedial phenomena. The concept of medium, as mentioned above, I shall try to define below.

Part I Interart studies and intermediality studies Various strands of cultural studies have, since the beginning of the twentieth century, investigated the relations between the arts. One comparative tradition, often called interart studies, with subcategories such as ‘word and image studies’ and ‘music and image studies’ has been an important subfield of both comparative literature and art history. The three arts most often referred to are literature, the visual arts and music, with their presumably basic components of words, images and sound. Interart studies 225 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jørgen Bruhn

basically dealt with the relations between the arts, and the object of research of such studies has been conceptualized along different lines, but often focus was placed on, for instance, studies of ekphrasis, of the so-called artistic Doppelbegabungen, or adaptations from music to poetry. However, under the general impression of both the basic tenets of Cultural Studies from the 1960s and onwards (the critique of the traditional hierarchy of the arts), and the new trends in artistic and technological products (hybrids in the arts such as performance and happenings, the development of new digital media), interart studies has since the 1980s, roughly, been supplanted by ‘intermediality studies’.3 A number of definitions of intermedial studies have been suggested. The shortest is, I believe, that of Mikko Lehtonen, stating that intermediality is ‘intertextuality transgressing media boundaries’.4 Intermediality studies is a rather young field of investigation and consequently the object as well as the theory and methods of the field are still relatively loosely defined, and in conferences as well as in publications, the discussions are still at a fundamental (but often sophisticated) level concerning the basic elements of the field: what is a medium? What is the difference between interartiality and intermediality and multimodality? These are questions which show that it is a discipline still to define its own object and limits. Despite the relatively weak foundations of the field, several schematizations have been suggested, among others by Werner Wolf in The Musicalization of Fiction: A Study in the Theory and History of Intermediality (1999) and Hans Lund in Intermedialitet. Ord, bild och ton i samspel [Intermediality. Word, Image, Sound in Collaboration] (2002).5 Lund suggests a useful division where the field of intermediality is divided into combination (with the subcategories ‘interreference’ and ‘co-existence’), integration and transformation.6 Interart studies and intermedial studies have created very important work. Siglind Bruhn, Claus Clüver, Hans Lund, Werner Wolf and numerous other researchers have shown the necessity of interdisciplinary work as a revolt against inexpedient academic borders, but these researchers also share what I choose to call a ‘formalistic vein’ in their work. Despite the impressive list of results, contemporary intermediality studies suffers from two problems which I think may be solved by a change of direction. First of all, by creating a new multimodal theory of medium, the traditional concept of arts or medium in intermediality studies can be rethought in order to establish a solid ground for future studies. This is my main methodological point. Second, and I discuss this in the second part of my essay, I will argue that in order to avoid intermediality studies remaining a rather formalistically biased field of study, intermediality studies should take notice of the ideologically interested trends of modern cultural thinking and contemporary philosophy. In order to suggest a solution to the first problem (creating a definition of medium suitable for future intermediality studies), I turn to Lars Elleström. In Part II of the essay I return to the question of intermediality and ideology.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

226 The Borders of Media Borders

Jørgen Bruhn

227

The important strategic move from interart studies to intermediality studies consisted, as mentioned above, in the broadening of the field of investigation from the traditional arts to, in principle, all existing media. Therefore, intermediality studies can analyse the entire traditional field of aesthetic objects as well as stamps, corporate logos, advertising, the medieval Christian mass and the opening of the Olympic Games.7 Defining ‘medium’ has proved difficult, however. ‘Curiously’, Werner Wolf has noted, ‘problems of definition and typology have not hindered intermediality research. The most obvious among these is the problem of defining “medium” itself.’8 Wolf suggests a broad concept of medium: not in the restricted sense of a technical or institutional channel of communication but as a conventionally distinct means of communication or expression characterized not only by particular channels (or one channel) for the sending and receiving of messages but also by the use of one or more semiotic systems.9 At first view this is an attractive definition because of the pragmatic idea that medium should be defined by ‘conventions’. The definition tends to be rather conservative, however, because it ends up saying, basically, that media is the same as the arts and that leaves the problem unresolved. Elleström’s definition of media, described in detail elsewhere in this volume, consists of a mixture of modalities. In his model, the problematic essentialism (theatre is x, painting is y) is avoided, and instead an open and mixed construction is proposed. As shown in his contribution to this volume, Elleström operates with four necessary conditions for every medium: (1) Material modality concerning the material manifestation of the medium; (2) Sensorial modality concerning the human sensory channels affected by the medium; (3) Spatiotemporal modality concerning the time-space ratio of the medium and (4) Semiotic modality concerning types of signification using Peirce’s distinction between iconic, symbolic and indexical signification. The four necessary conditions (modalities) constitute what Elleström calls ‘basic media’, such as still image, written words, oral words and organized sound. These ‘basic media’ will, however, enter into more elaborate culturally and aesthetically conventional forms, which Elleström chooses to call ‘qualified media’. The implications of a multimodal concept of medium The idea that every medium consists of a number of elements called basic modalities common to all media (albeit always in specific, concrete constellations) means that these modalities (in a particular combination) also form the basis of other media. Consequently, multimodality is a fact of any conceivable text in any conceivable medium. The idea that texts are mixed is of course banal when dealing with openly mixed media such as the 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Defining medium

mixture of sound, image, words, music in modern cinema or the pictures and words in the picture book. The new and less obvious insight is that the mixed character of texts is also a fact of the texts and media which have traditionally been considered pure, without traces of other media. The main point is that even the apparently monomedial text always consists of several modalities. An example may clarify my point: normally, literature has entered intermediality studies via research on traditional ekphrasis, iconic projection (Hans Lund10 ) adaptation theory or the ‘musicalization of fiction’ (Wolf). My point, developing Elleström’s suggestions, would be that any literary text shares modalities with other media – not only the obvious and important cases mentioned here. Let me exemplify with Iain Banks’ The Wasp Factory (1984), a gothic horror story set in Scotland. Traditional intermedial studies would probably dismiss this text as being out of reach of an intermedial analysis, but a multimodal-media model would be able to identify relevant aspects of such a text. Here I choose not to focus on the (short) descriptions of music in the novel (ekphrasis of music) or the descriptions of the landscape (close to ekphrasis proper): nor will I stress the passages that may have been signs of a ‘filmization’ of literature, or the crucial passages where the protagonist’s father turns out to behave like not only a tyrannical patriarch but also an artist who has chosen to use his own son as, literally, a model to be sculpted. Instead I quote the final lines of the novel: ‘Poor Eric came home to see his brother, only to find (Zap! Pow! Dams burst! Bombs go off! Wasps fry: ttssss!) he’s got a sister’. The prose here would not enter any of the traditional categories of intermediality, but I maintain that the passage by way of its multimodality is necessarily what I will define below as a heteromedial text. In this brief excerpt I would stress the obvious sonoric aspects of the sequence ‘(Zap! Pow! Dams burst! Bombs go off! Wasps fry: ttssss!)’. It is a kind of onomatopoetikon mixed with ‘normal’, non-iconic prose. The literary text (which is supposed to consist of symbolic signs (in Peirce’s systematization)) suddenly approached the sound-iconic state, thus using traits of sonorous art forms such as music, or spoken poetry, or performance and theatre. I would also refer to the parenthesis in this fragment of sounding and silent words: a parenthesis is of course a symbolic sign signifying a kind of double textual dimension inside the text and as such is a familiar trait in (written) literature.11 A pair of parentheses is also an iconic visual sign: we can see how it effectively fences in and bars off the passage from the surrounding text, so that we read the symbolic sign as well as see the iconic sign, probably because the text wishes to operate in two different dimensions at the same time. Even the added italics and repeated consonants of ttssss create a kind of mixture of visual and sonorous and symbolical sign production. This example is meant to illustrate my main point, namely that the pure, distinct medium, and the equivalent to this on the level of specific texts, is a

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

228 The Borders of Media Borders

229

historical as well as an ontological illusion. Such a pure medium or text has never existed, and it even appears to be a logical impossibility. I think that this might be what W. J. T. Mitchell has in mind when he claims that ‘the attempt to grasp the unitary, homogeneous essences of painting, photography, sculpture, poetry, etc., is the real aberration’ and that the conception of purity and unity of media ‘is both impossible and utopian’ and therefore media ought not to be investigated as an existing fact but as the result of ideological construction and evaluation.12 Pure media may be, and has been, an ideal in specific historical periods, and in particular ideological surroundings, but it is never a real, existent phenomenon. Consequently, research – and teaching – should take as its starting point the fact that ‘all arts are “composite” arts (both text and image); all media are mixed media, combining different codes, discursive conventions, channels, sensory and cognitive modes’.13 Thus the implications of a new, multimodal concept of medium is that interartial and intermedial models are transgressed and the meetings of media are no longer reserved for the privileged exceptions but become a condition of every text. Defining ‘heteromediality’? This new, multimodal definition of medium raises not only a number of analytical and epistemological questions but also a basic terminological question: is ‘intermediality’ still the best term to describe the multimodal character of all media and, consequently, the a priori mixed character of all conceivable texts? The term intermediality is too limited to satisfy the demands of the new multimodal theory of medium. Therefore I will suggest a new umbrella term, heteromediality, to describe any conceivable text, whereas I reserve the term intermediality to parts of heteromediality. With ‘heteromediality’ the focus shifts from the comparisons between media and art forms, roughly consisting of the numerous possibilities suggested in a diagram by Lund, for instance, where music represents poetry, novel becomes movie, words and picture combine on the poster and so on, to a method investigating the expanded field of media relations (in Elleström’s terminology: media modalities) inside the text. With heteromedial studies we shift the focus from relations between media (always concretized in forms, in ‘texts’), to medial relations within texts, in other words. Therefore the prefix ‘hetero’ (Greek, ‘different’ or ‘other’) is more suitable than ‘inter’ for these investigations. The term heteromediality raises another terminological problem, though, because the term is already in use in Werner Wolf’s schematization of intermedial relations. Wolf does not use ‘heteromediality’ in The Musicalization of Fiction: A Study in the Theory and History of Intermediality from 1999, but in a more recent article, in the Routledge Encyclopedia of Narrative Theory,14 ‘heteromediality’ occupies a minor (and purely separating) role and is defined as follows: ‘Intermediality, in contrast [to intertextuality], applies

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jørgen Bruhn

in its broadest sense to any transgression of boundaries between media and thus is concerned with “heteromedial” relations between different semiotic complexes or between different parts of a semiotic complex’ (p. 252). Heteromedial in Wolf’s definition can thus be translated simply as ‘different media’ in contrast to traditional text-to-text intertextuality, which Wolf calls a ‘homomedial’ relation (p. 252).15 My definition of the term differs clearly from Wolf’s. With the prefix ‘hetero’ I wish to signal the internal mixed character being a condition of any text, as opposed to ‘inter’ signalling ‘media transgressing borders’. My proposal, in other words, aims at creating a new, universal concept of text, the heteromedial text. However it will be useful to operate with pragmatic subcategories of the heteromedial text, in order not to create one monolithic concept in danger of not explaining anything at all. Therefore, there will still be a point in using the terms intermedial text (examples of which are listed in Lund’s diagram, where media are defined as conventionally distinct forms in contradistinction to the multimodal definition) and intertextuality (the theoretical concept designating the fact that all text are dialogically connected but without in this concept taking the specific media-specificities into consideration). Intertextuality defines the overall phenomenon of texts being mosaics of other texts (according to the now classic definitions of Barthes and Kristeva); heteromediality defines the existence of several medial modalities in all conceivable texts. Intermediality, then, is my term for one particular subgenre of heteromediality, characterized by the traces of more than one medium (either in combination, transformation or integration following Lund). Therefore all texts are heteromedial, and they will always cite and will be cited by other texts (intertextuality); but only part of the immense category of heteromediality is intermedial in the restricted sense of the word. In the following I use the term heteromedial studies for intermedial studies as such, thus piously hoping that my proposed terminology may prove useful; when I use the term intermedial studies or intermediality I refer to current or previous research.

Part II Heteromediality and ideology Until now I have described and discussed the development and inherent problems of interart studies (the hierarchical and traditional system of the arts) and intermediality studies (the problematic definitions of medium and the tendency to diminish the real extension of intermedial relations) and I have suggested a multimodal-media model that might constitute the basis for what I propose to call heteromedial studies. Perhaps the most crucial question for future heteromediality studies, however, concerns the interesting and even necessary discussion around the Erkenntnisinteresse (cognitive 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

230 The Borders of Media Borders

231

interest) of the field: What kind of knowledge does the ‘discipline’ want to produce? When I consider the present state of intermediality studies, it seems to me that two possible alternatives for heteromediality studies arise. On the one hand, I see a rather formalistically focused alternative which might be described as a sophisticated development of the New Criticism idea of the text or the work. Within the frame of this first alternative, heteromediality studies can try to strengthen the analytical foundations of traditional intermediality studies in order to establish itself as a conventional, respectable discipline. This might happen through a sober and scientifically valid construction of a methodology and an acknowledged object of investigation. Such a direction would develop heteromediality studies into a genuinely analytic and historically informed discipline based on solid, formalistic investigative goals. As far as I can see, this has been the underlying ambition of a number of eminent researchers of intermediality studies who combine an impressive philological or musicological training with an intermediality perspective. Their analysis, based on a semiotic framework, is usually directed towards relations between the inner working and the outer surface of specific, relatively autonomously conceived works of art. This first line of research I choose to call the ‘formalistic’ line of intermediality studies.16 Heteromediality studies might choose another direction, however. This will be the case if heteromediality researchers engage with positions in modern cultural theory and philosophy which hold particular interests in the ideological dimensions of cultural production and reception. Investigations into and reflections concerning the ideological and political aspects of form have been crucial to a number of the discussions in literary theory in the twentieth century, with inspiration from German neo-Marxism and British Cultural Studies,17 while more contemporary trends such as New Historicism and post-colonial studies have been important movements concerning ideology and the arts. Both movements have designed sophisticated theories of the text’s relation to the surrounding world, and both tend to focus on the inner contradictions of literary texts. These contradictions are considered to be (often unconscious) expressions of the surrounding society’s ideological tensions and constellations of power. Kiernan Ryan claims that the deepest aim of New Historicism ‘is to dethrone and demystify the privileged literary work: to destroy its immunity to infection by circumstance and other kinds of texts and to rob it of political innocence by exposing its discreet commitments, its subtle collusions in the cultural struggle for power’18 – an apt description for a future heteromedial research strategy, I would say. Behind many of the analytic and theoretical endeavours in Anglo-Saxon theory we find traces of modern French thinking, and this magnificent tradition of considering art as an ideological fact should enter the reflexive space of future heteromediality studies. Heteromedial studies should engage with

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jørgen Bruhn

crucial names in what may be called the (heterogeneous) tradition combining art with a wider ‘critical’ context concerning ideological questions of gender, class, race or struggles between different aesthetic positions. Such a ‘tradition’ may be said to consist of the theoretical writings and the important concepts of the generations following after Jean-Paul Sartre and Georges Bataille. There is a long list of thinkers deeply influenced by psychoanalytical thinking, phenomenology and critical theory, and common to these thinkers is a capacity to work across established academic and epistemological boundaries (psychology, philosophy, literary theory and so on) as well as analysing freely across the traditional boundaries of the arts. Aspects of the work of these thinkers ought to be incorporated into heteromediality studies to such a degree that every reflection on aesthetic questions also entails a critical dimension (in the sense of the word mentioned above) in order to avoid heteromediality becoming a formalistic, descriptive discipline. My suggestion is that the idea of the inherently mixed conception of medium (leading to my concept of heteromediality) must be combined with the ideological aspect. Therefore my credo directing my own way of practicing heteromedial studies is that the particular constellation of mixed media in a text often expresses a tension which in a more or less opaque way relates to the historical context of the text in question. This, I think, is what heteromediality as a critical concept would be able to facilitate in the study of cultural history, and the distinctive traits of heteromediality studies would be the urgent knowledge of the mixedness of media as well as an investigation of the conflicts or tensions creating the particular mixture.

Cook and Mitchell I would like to point to a few fruitful positions (outside the self-confessed intermediality researchers) that might help to establish a sound and strong critical version of future heteromediality studies, namely the work of W. J. T. Mitchell and Nicholas Cook. Mitchell combines the lines I have mentioned above: a multimodality concept of the work of art with an a priori ideological grounding. He works through a number of relations between different media, in particular word and image relations, not in order to compare media but because he wishes to investigate what kinds of non-likeness relations can be established between media, and the ideological implications of these relations. His starting point is that every cultural product is ‘mixed media’ and that this relation expresses a particular ideological and/or philosophical constellation connected to the social environment and history of media. At every moment of history, particular hopes and fears can be detected in the background of media, and according to Mitchell any conceivable art work expresses a tension or a stride between media which mirrors a more comprehensive ideological battle of the social surroundings or the epoch of the work, for

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

232 The Borders of Media Borders

233

instance, connected to ideological conflicts concerning gender, class, ethnicity or nationality. Mitchell offers a reconsideration of the often claimed essential differences between different media and art forms and he manages to show that the concept of ‘borders’ between ‘pure’ media is a question of ideology, not a matter of essential definitions.19 Whereas Mitchell has tried to construe an analytical strategy (but never a strict methodology) based on avoiding the lazy comparison and the invention of a more fruitful relation between verbal and visual elements of texts, Nicholas Cook has – from a musicologist’s point of view – made a comparable strategic move.20 Cook wishes to contribute to ‘the current reformulation of music theory in a manner that loosens the grip of the ideology of musical autonomy – the compulsory (and compulsive) cult of what Peter Kivy calls “music alone” ’.21 Consequently, Cook wants to tear down the purist idea of ‘pure’ or ‘absolute’ music. Interestingly, he tries to drive through his argument by way of focusing on aspects that seem to lie as far away as possible from ‘absolute’ music, but when doing this he still manages to reach his main thesis: that absolute music cannot be absolutely pure. Even the purest musical text is a mixed text, to rephrase Cook’s idea in Mitchell’s terms. Analysis of the covers of compact discs, television advertisements of cars and music videos underlines the way that music always, directly or indirectly, bears traces of the multimodal. Thus, as I would state, Cook demonstrates the real but suppressed heteromedial quality of musical products. Cook is fascinated with ‘a hierarchy [of media] whose levels are at war with one another’.22 A thorough analysis of a music video of the pop singer Madonna (‘where the effect is to destabilize the hierarchy of media’)23 can in, Cook’s analysis, achieve far reaching results.

Conclusion: the centrality of marginality In this essay I hope to have shown how a new, multimodal version of medium may imply a substantial rethinking of the discipline of intermediality studies. I propose the term heteromediality to signify that ‘all media are mixed media’ and that the particular mixing of media has its roots in wider contexts (ideological, historical, aesthetic and so on). Furthermore, I have put forward my own suggestion as to how heteromedial studies might be conducted, namely by way of letting French thinking from the period after World War II balance the otherwise formalistic tendency in much of the research that has been done in intermedial studies. The points that I have been trying to make lead me to the following concluding remarks concerning the ‘position’ of heteromediality studies in the future landscape of the humanities. Since the mid-twentieth century, research of the marginalized, the Other(s), the liminal and the suppressed has, paradoxically, occupied a central role in the humanities. The privileged perspective of the marginalized

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jørgen Bruhn

has been expressed by a number of different scholars, or rather, some scholars show how the marginal may produce important insights into the centre of a research field. Nicholas Cook’s efficient if also slightly surprising take on this question, in Analyzing Musical Multimedia (1998), consists, as mentioned above, in using an advertisement for a Volvo car or a Madonna music video as a lever to create a kind of deconstruction of the high-cultural evaluation and understanding of ‘pure’ music. The centrality of the marginal is almost a trademark of the work of W. J. T. Mitchell. I would like to mention two instances of this rhetorical and epistemological tendency in his work, and both examples have gained a central position in academic discussions, partly following Mitchell’s lead. First I turn to his discussion of ekphrasis in ‘Ekphrasis and the Other’ from Picture Theory. In this text Mitchell claims, and he might be right, that research into ekphrasis, despite the voluminous size, is still in a ‘minority’ position and that there is a certain amount of obscurity attached to the term and the research field.24 Despite this, or should I say because of this, the term and concept of ekphrasis might form the foundation for a new theory of description in literary studies, nothing less. The reason is that ekphrasis, itself a description, is typical of all other representations of reality. In another essay25 Mitchell chooses the concept ‘metapicture’, a picture that reflects upon itself as pictorial statement and representation, to do the same job. Metapictures are apparently radical pictures in the history of western representations: from Magritte’s pipe picture to Wittgenstein’s beloved duck-rabbit drawing, to mention just a few. Once again it turns out that the marginal has a central position: ‘The metapicture is a piece of movable cultural apparatus, one which may serve a marginal role as illustrative device or a central role as a kind of summary image, what I have called a “hypericon” that encapsulates an entire episteme, a theory of knowledge.’26 The marginal metapicture turns out to be the typical visual representation. From the point of view of heteromediality studies these claims for centrality should cause us to consider, once again, whether heteromediality studies does not deserve a much more privileged position in contemporary, and future, studies in the humanities. If we once again ponder on Hans Lund’s diagram over relations between the arts one is struck by the clarity of the division and the comprehensive examples of intermedial relations, and most teachers of basic courses in intermediality will probably feel rather satisfied with themselves if their students navigate in a reasonable way in this diagram. Nevertheless the scheme has one crucial flaw, I think, namely that it implicitly states that there exist media and art forms that are ‘un-intermedial’. This is not the case, as I have argued above, where I have referred to the dictum ‘all media are mixed media’ and by naming the study of this fact ‘heteromediality studies’. When understood in a fruitful way the idea of heteromediality (that all texts are modally mixed, and that this impure state implies some kind of ideological tension) might influence not

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

234 The Borders of Media Borders

235

only academic research but even institutional divisions and the unhealthy ‘compartmentalization’ of the humanities. In other words, Lund’s schematization establishes intermediality studies as a marginalized field inside the studies of the humanities, whereas I believe that heteromediality studies break loose from this position and become exactly the opposite: the basic, central conceptual scheme underlying all studies in the (aesthetic part of the) humanities.

Notes 1. Parts of this essay were presented at the Imagine Media! conference at Växjö University (from 2010: Linnaeus University), Sweden, in October 2007. An early version appeared in Tidskrift för litteraturvetenskap (2008) 1. A number of ideas put forth here have been developed in the collaborative edition of J. Arvidson, M. Askander, J. Bruhn and H. Führer (eds) (2007) Changing Borders: Contemporary Positions in Intermediality (Lund: Intermedia Studies Press) and some ideas correspond to ideas put forward in the foreword to Changing Borders, pp. 13–19. I also wish to thank Lars Elleström, Henriette Thune and Jan Lundquist for valuable comments on this essay. 2. The succinct definition is from Claus Clüver, ‘Intermediality and Interart Studies’ in Arvidson et al. (2007) Changing Borders, pp. 19–37 at p. 20, note 5. 3. See Claus Clüver’s historical overview, ibid. 4. Mikko Lehtonen (2000) ‘On No Man’s Land: Theses on Intermediality’, Nordicom Review 4, 71–83 at 71. 5. W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History of Intermediality (Amsterdam and Atlanta, GA: Rodopi); H. Lund (ed.) (2002) Intermedialitet: Ord, bild och ton i samspel (Lund: Studentlitterature). 6. Hans Lund (ed.) (2002) Intermedialitet, as translated in Changing Borders, p. 15. 7. The last two examples are Claus Clüver’s. Clüver believes that these are highly significant (but as yet almost un-analysed) examples of intermediality in contrast to interart phenomena. See the final remarks in ‘Intermediality and Interart Studies’, Changing Borders, p. 34. 8. See Werner Wolf’s entry on ‘Intermediality’ in D. Herman, M. Jahn and M.-L. Ryan (eds) (2005) Routledge Encyclopedia of Narrative Theory (London & New York: Routledge). 9. Quoted in I. Rajewsky (2002) Intermedialität (Tübingen & Basel: A. Francke Verlag), p. 7. 10. See Hans Lund (1992) Text as Picture: Studies in the Literary Transformation of Pictures (Lewiston, Ontario: Edwin Mellen Press). 11. A standard definition of the content of the parenthesis is that it can be omitted without altering the overall meaning of the sentence. 12. W. J. T. Mitchell (1994) ‘Beyond Comparison’ in Picture Theory (Chicago: Chicago University Press), pp. 107, 96. 13. Ibid., pp. 94–5. 14. Herman et al. (eds) (2005). 15. In Werner Wolf (1999) The Musicalization of Fiction. Werner Wolf distinguishes between intertextuality (being a text-to-text relation) and intermediality (being a ‘special relation between media’) (p. 46). One may appreciate the clear

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jørgen Bruhn

16.

17.

18. 19.

20. 21. 22. 23. 24. 25. 26.

distinction, but in my definition of medium such a distinction cannot hold, simply because any text is per definition never not only textual. For an overview of this tradition, see Rajewsky (2002) Intermedialität. Perhaps my classification is about to turn into an anachronism: in the present volume both Siglind Bruhn and Claus Clüver do not confine themselves to semiotic or formalistic issues but instead incorporate ideological and religious aspects in their presentations. Recently, Marion Froger and Jürgen E. Müller (2007) edited Intermédialité et socialite: Histoire et gégraphie d’un concept (Münster: Nodus) which, despite its title and the interesting contributions, offers little when it comes to changing the overall direction of intermediality studies (‘socialité’ is more or less synonymous with ‘history’ or ‘context’). See Simon During’s preface in S. During (ed.) (1999) The Cultural Studies Reader, 2nd edn (London & New York: Routledge), for an overview of the problems inherent in the different periods of Cultural Studies. For an attempt to combine Cultural Studies with intermediality studies, see W. Huber, E. Keitel and G. Süss (eds) (2007) Intermedialities (Trier: Wissenschaftlicher Verlag). R. Kiernan (1996) New Historicism and Cultural Materialism (Oxford and New York: Oxford University Press), pp. xiii–xiv. That this method is not beyond criticism is shown in B. F. Scholz’s recent rereading of Lessing – and Mitchell. See B. F. Scholz, ‘ “A Whale that can’t be Cotched”: On Conceptualizing Ekphrasis’ in Arvidson et al. (2007) Changing Borders, p. 294 and note 29. Mitchell and Cook have not, I believe, referred to each other’s work. N. Cook (1998) Analyzing Musical Multimedia (Oxford and New York: Oxford University Press), pp. vi–vii. Ibid., p. 126. Ibid., p. 125. See W. J. T. Mitchell (1994) Picture Theory (Chicago: Chicago University Press), p. 152. See ‘Metapictures’ in Mitchell (1994) Picture Theory. See Mitchell (1994), ‘Beyond Comparison’, p. 49.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

236 The Borders of Media Borders

Intermediality Revisited: Some Reflections about Basic Principles of this Axe de pertinence Jürgen E. Müller

Nowadays the research axis or axe de pertinence of intermediality is keeping numerous scholars busy at universities and research centres all over the globe and, in fact, this concept proves to be a broad field, a ‘weites Feld’1 for the many disciplines it involves. The variety of aspects of the concept of ‘intermediality’ makes it very difficult or almost impossible to present some sort of general overview with regard to all the options without opening a sort of academic bookkeeper discourse on different terminological, theoretical, methodological and historical items. In this article,2 I will not undertake such an enterprise, which – as we all know – has been done by several scholars who offer critical volumes on intermedial research or typologies of different sorts of intermedial studies.3 Instead, I would like to develop some aphorisms on the actual state of affairs of intermedial studies and some perspectives for a historical intermedial approach, which will start from the reflections of one of its almost forgotten ‘fathers’, Marshall McLuhan’s comments on the media in the electronic era.

When is a medium a medium and when is a new medium a new medium? One of the crucial questions – if not the crucial question – of any study of media encounters or of intermediality is the question of how to conceive of a ‘medium’. We know dozens of proposals to define a medium on the basis of different scientific paradigms ranging from philosophical, social, economical, biological, communicational and technological frames to channels of discourse, simulations and patterns of actions or of cognitive processes – to mention just a few items. McLuhan, for example, uses the notions of ‘medium’ and ‘media’ in a very open, sometimes blurred or blurring way. ‘Medium’, ‘media’ or ‘technology’ cover spoken and written words as well as money, clocks, comics, wheels, bicycles, automobiles, telegraphs, phonographs, light, movies, radio, television, weapons, automation – to mention only the most prominent. All these different concepts of media 237 10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

16

and mediality inevitably have strong impacts on any research and work in the field of intermediality. In my eyes a semiological and functional concept of media, relating media to socio-cultural and historical processes, still seems to be the most helpful framing for any sort of intermedial research. It will be open for aspects of materiality as well as for aspects of meaning.4 For the moment and with regard to the objectives and strategies of this essay, however, these hints to a theoretical frame should be sufficient. If one of the most evident and relevant fields of intermedial processes will have to be seen in the encounters5 between old and new media, we will have to ask ourselves: when does a medium become a medium and when does a new medium become a new medium? In both cases, complex social, cultural, technological and generic processes of institutionalization have to take place in order to install something we would like to call a medium or a new medium. I would like to illustrate this thesis by an example of one of the first representations of a so-called new medium which later was about to become the Leitmedium for almost half a century, ‘television’ or ‘farsight’ (to use one of the British terms of the 1920s and 1930s).

Intermediality as a process – or the test case of television Due to its materiality or – to put in Baudry’s terms – to the specific aspects of the dispositif ‘television’, there are very few sources for the first transmissions or ‘screenings’ of the new medium which was still to be developed. Formats and functions of the live medium, ‘TV’, could only be preserved in other media, such as written text, photography and film. One of the rare examples of these beginnings of ‘early television’ is a filmic ‘document’ of one of the first screenings of television in Germany in the early 1930s in a so-called ‘Fernsehstube’.6 In this example, we can see (and hear in the filmic ‘documentary’) what an audience could and should find in the live transmission of ‘mechanical’ television, which – amongst other inventions – was based on the Nipkow disk (see Figure 14).7 Was it really television that the audience was watching? There was a live transmission from some sort of studio, a dark Abtastzelle (scanning cell),

Figure 14

Fernsehstube, hall and audience

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

238 The Borders of Media Borders

which gave enough room for one speaker or ‘entertainer’ to be presented in a kind of American shot to his or her public. On the reception side of the apparatus, there was an audience of some 20 people in a specific spatial arrangement and configuration, in the Fernsehstube (television hall). In this room or small hall, all axes of viewing were centred on the apparatus, called television – in Great Britain in the 1920s and 1930s it was also called ‘Bairder’, ‘onlooker’, ‘ingazer’ or ‘farsight’. The spatial arrangement of the apparatus and the spectators clearly refers to patterns of cinematic or theatrical space. In this historical setting, television is neither a replacement of the open fireplace of our living rooms nor a home surrogate of cinema. It is somewhere in-between the spheres of public and private viewing. However, to put it in McLuhan’s terms, not only cinema and theatre are the ‘contents’ of this new apparatus, there are also telephone and radio. The connection between the (dark and small) studio and the public was guaranteed by wire and the barking and clearly visible surprise guest, Rex, a dachshund, evoking patterns of radio techniques and formats. There is ‘live talk’ between the audience and the speaker or moderator. They have to be convinced of the live character of the ongoing ‘events’ and this specific achievement of the television technology. The apparatus thus allows interactivity between its public and its protagonists, which brings it very close to recent developments of the webcam and the webphone. Obviously, there are few parallels or common patterns between this demonstration of a new dispositif and the concepts of television we have in mind when we are confronted with the traditional ‘cool’ medium ‘television’ which is undergoing crucial changes today. In fact, the medium we have described above is not television, and this not because of the fact that its content does not correspond to our expectations of the content of this medium; it is not television because, except from the screening of its picture on a glass surface and its live quality, it does not convey any links with the ‘cooling down’ medium ‘television’ as we know it in Europe from the 1960s onwards and as it has been conceived and described by McLuhan in his Understanding Media.8 In my view, there are two aspects worth mentioning. First of all, a blunt conclusion at this point might be that we tend to commit the common error (as, for example, Baudry did when he developed the contours of his cinematic ‘dispositif ’ on the basis of a specific historical form of film and cinema, namely of the 1930s)9 to generalize a medium on the basis of a specific, more or less accidental form of this medium, which one can find in many theories of television. For example, many of McLuhan’s comments on television were due to the very poor and blurred quality of the picture of television sets in the 1960s. There is yet another more relevant aspect to which I would like to draw attention: the writing of media histories or intermedial historiologies. A growing number of scholars in this field conceive of intermedial histories as processes of technological developments within certain historical, economic

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jürgen E. Müller 239

and social circumstances. By doing this, they explicitly locate media developments and media encounters within different cultural and technological systems and, naturally, we will have to follow them on this track. However, in many cases, they treat new media or technologies as if they fell more or less as ‘finished and polished entities’ from apparative, technological or social heaven, as is, for example, the case in McLuhan’s Understanding Media. By doing this, their comments on the status and content of new media miss a crucial point, which can be circumscribed by the simple question: when does a new medium become a new medium? Should we regard the live questions asked in our filmic document of ‘early television’ by the audience and transmitted by the technicians of the Deutsche Reichspost as ‘creatively participant response(s)’10 to a new and cold medium? Or should we rather consider them as attempts to grasp the contours and possible patterns of usage of a somewhat loose combination of different, already existing cultural and technological series which might be circumscribed and be located somewhere between telephone, radio, cinema, vaudeville, theatre and others?11 ‘Nobody wants a motorcar till there are motorcars, and nobody is interested in TV until there are TV programs.’12 This remark is absolutely correct, but when and under what technological, social and historical circumstances does a demand for a new technology start? When is a motorcar a motorcar and when is television television, a new medium or a new technology? If, at our specific moment in the 1930s, there was not yet any fixed and stable televisions programming,13 does this imply that the officials of the German post office produced a place for a simulacrum or an imagination of a medium?14 Our example of the so-called early television demonstrates that media theorists and media historians have to ask themselves15 what television really is, when it has become television and what sort of modalities have influenced these processes of transformation. Similar to the history of cinema and of many other so-called new audiovisual media, there is no exact date to be defined in terms of an exact starting point of the new medium ‘television’. Television is the result of a long imaginative and technological process which started at least some two centuries before the screening of the first broadcasts in Germany in the 1930s; it is the result of the meeting of different cultural and technological series or cultural paradigms, as they have been described by Francoeur and Gaudreault, implying different patterns of temporality, of media-encounters, media-interactions and corresponding user activities. [A] polysystem made of several units of signification (literature, painting, arts and popular tradition etc.) which themselves are subsystems of the former and which have in common the characteristics that (1) they stand in continuous interaction with one another (2) inside a hierarchy with successive growth (3) and with a ‘topping’ work or collection of works

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

240 The Borders of Media Borders

Jürgen E. Müller 241

A media history as a network and rhizomatic history, oscillating between the poles of technology, cultural series, historical mentalities and social practices, would thus lead us to an innovative understanding of the processes of media developments. Following Lars Elleström, technical properties (as realizations of material modalities) should be regarded as central factors of the interplay between the ‘senses and the material impact’,17 which I would propose to conceive of primarily in historical terms and categories. In this sense, an intermedial perspective will make it possible to specify and to differentiate McLuhan’s somewhat crude and global statement that the content of new media is the old media. Television in the 1930s, for example, meets the rather well established and developed cinema or telephone and the still expanding radio. It will have to undergo complex intermedial processes of differentiation and institutionalization and will have to recycle and reshape traditional genres and formats before it can become the medium we are inclined to locate within specific borderlines. So McLuhan’s historical perspective would be in need of an expansion and a differentiation in terms of the involved complex processes and modalities which allow the old media to become the content of new media.

The history of the concept of intermediality in the scientific community Intermediality: a completely new approach in humanities and media studies? There might be some doubts whether the concept of intermediality really is a completely new approach, research axis or even theory in the field of humanities. This not only because of the coining of the term ‘intermedium’ by Coleridge, by Lessing’s study of the Laokoon sculpture, by Wagner’s (by the way rather vague) idea of the ‘Gesamtkunstwerk’, by Walzel’s remarks on the ‘wechselseitige Erhellung der Künste’ (the reciprocal illumination of the arts), or in earlier days, by Kristeva’s notions on intertextual dynamics, or the revival of the concept of ‘ekphrasis’ and the development of interart studies by Clüver, to mention just the most prominent.18 All these, and many other, proposals are indicative of the fact that ‘intermediality’ has a long (not only etymologically grounded) pre-history and, as a Suchbegriff 19 or a Such-Konzept, still has to be regarded as work in progress. This ‘work’ implies procedures of theoretical framing (but not of closure) as well as historical studies in the field of media encounters and the interactions between these two axes de pertinence. Following one of the main trajectories of Roland Barthes’ œuvre, I would expect the so-called theoretical approaches to lead

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

that serves as a first principle of structuring (4) which lasts long enough and has a sphere of concision precise enough to produce a demarcation of spatio-temporal coordinates.16

us to history and history to lead us to theory, and by this to bring about a blurring of convenient traditional borderlines of academic activities. In this sense, intermediality is not a completely new academic concept but a reaction to certain historical circumstances in humanities, the media landscape and the arts. It could be conceived of as an answer to the nineteenth-century academic and institutional heritage of our universities and as a starting point for a repositioning of scholars and corresponding research in a situation of decline of the academic world.20 The history of intermediality and of reflections on intermediality reaches much further than the last two decades in discourses of media studies or literature. Nevertheless, let us take a short glance at the beginnings of the academic discussions of intermediality in the 1980s. At that period, the isolating tendencies of media theories and histories and the rather banal fact that no medium could be considered as a ‘monade’ motivated me and also some other scholars to direct our attention towards the intricate and complex processes of media interactions or media encounters. The notion of intermediality was based on the assumption that there are no (?) pure media and that media would integrate structures, procedures, principles, concepts, questions of other media which have been developed in the history of Western media and would play with these elements.21 Consequently, the main objective and challenge of intermedial studies would then have to lie in the reconstruction of these dynamic processes and of their historical and social functions. Departing from these premises, in the late 1980s I proposed the study of intermedial processes with regard to specific media products or oeuvres (if we wish to call them that), of the interactions of different dispositifs22 and the development of an intermedial approach for media history as central axes of research. Today, some years later, we have to ask ourselves which of these axes de pertinence, or research axes, might have to be corrected, re-accentuated or re-orientated, and how many aspects of these claims, that were then – at least to a certain extent – provocative have been realized. Looking back, for me, two items still seem to be pertinent and worth being further developed: the concept and study of media history in terms of dynamic intermedial networks and the orientation of intermedial studies towards the social and historical functions of these processes, as we have already seen in our short ‘footage’ about the interactions and dynamics in the early phase of the cultural series ‘television’. Following this line, the ‘newness’ of the concept of intermediality would then primarily lie in its capacity of being permanently reshaped and of reshaping traditional fields of research. Let us first take some further glances at the history of intermediality. A Short Retrospective into some Theoretical and Methodological Roots The etymology of the notion ‘intermediality’ refers us back to a play with the ‘being in-between’, a play with several values or parameters in terms of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

242 The Borders of Media Borders

materialities, formats or genres and meanings. In this sense, the materiality of media is from the very beginning one of the central components of the concept of intermediality; it will have to be linked to the so-called content matter. Looking back on the methodological realizations of the intermedial axis of research, the question of how to grasp these different sorts of intermedial plays seems to have been neglected to a certain extent. In fact, intermediality proves to be a rather fugitive phenomenon or, better, process which – as Paech has shown – is only accessible via the traces which it has left in audiovisions23 or – as I would like to add – in dispositifs. A search of traces of intermedial dynamics will have to be one of the central methodological options of intermedial studies. In spite of the usefulness of terminological and taxonomical layers, which cannot and should not be denied, such an approach must not necessarily lead to a taxonomy or a coherent descriptive system of all possible intermedial relationships or modalities. The pure algorithmic or mathematical fact that a supposed basis of – let us assume – 50 ‘distinct’ media (what would ‘distinct’ mean in this context?)24 might lead to 2500 combinations of intermedial interactions of two of them or 125,000 combinations of intermedial interactions of three of them gives an idea of the over-ambition or – on the other side of the coin – the over-reduction of such an enterprise. During the last decades, many typologies of intertextualities have been proposed by Plett, Genette, Grivel, Riffaterre and others who – on the one hand – directed our attention towards options of textual interplays, but – on the other hand – led to questions about the gist of the taxonomic results. In this sense, ‘intermediality’ would have to be considered as an axe de pertinence, once again, a Suchbegriff, an axis of research which will not aim at the constitution of a meta-theory or taxonomy of all media systems, it will rather imply thorough historical studies of paradigmatic intermedial processes or encounters on different levels with regard to specific modalities to be differentiated in ‘material’, ‘sensorial’, ‘spatiotemporal’ and ‘semiotic’ categories.25 Thus, les illusions perdues, the lost illusions concerning the concept of intermediality, as they have been described by Eric Méchoulan,26 for example, would have to be seen in the light of great or exaggerated expectations and of ensuing disappointments which have been articulated during the baisse of the term ‘intertextuality’. Nevertheless today, ‘intertextuality’ – at least to me – would remain a valuable and complementary notion in the field of intermedial research. This research will have to reflect its theoretical and methodological basis and will have to be directed towards historical processes of media encounters and their historical functions which have left their traces in the materialities, the media products or other ‘sources’. Before continuing our tour d’horizon of intermediality, let us take the risk of three short asides on the neighbouring notions and concepts of ‘intertextuality’, ‘interartiality’ and ‘hybridity’.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jürgen E. Müller 243

244 The Borders of Media Borders

Intermediality, intertextuality, interartiality, hybridity

Without any doubt, we can draw some parallels between the history of the notions of ‘intertextuality’ and ‘intermediality’. At the beginning of their elaboration, both concepts were confronted with considerable scepticism and reserve in the scientific community, but subsequently they have been widely accepted and enriched by many elements which – in the end – have led to blurring tendencies. In the early phase (and sometimes even today), there were a number of overlaps of denotations and connotations of these two terms. In the 1970s several phenomena later to be described as intermedial processes were denominated as intertextual processes. Kristeva’s famous notion of intertextualité as ‘transformation d’un système de signes dans un autre système’ or Genette’s system of transtextualité and its five subcategories might be mentioned as paradigmatic approaches which tried to include – to a certain extent – intermedial processes in intertextual reflection.27 However, due to a very strong accent on textual and especially literary aspects in the ensuing analyses, the dynamics of intermedial processes have more or less been neglected. In this respect, the notion of intermediality had to overcome the restrictions of literary studies and to reorient the research axis towards interactions and interferences between different audiovisual and not only literary media. By doing this, it refocused on questions of materiality and the making of meaning, on traces of intermedial processes and social functions. Intermediality and interartiality The notion of interartiality can also be seen as a relative of intermediality. It would be going too far to attempt a summary of the many studies which have been conducted or initiated by Claus Clüver and other central scholars within the fields.28 Given the large number of common or different interests or even overlaps, I would like to point to one crucial aspect. In the early phase of intermediality, the art – or media – aspects of the ongoing processes were not always clearly isolated or recognized (at least, this is a criticism I’ve been confronted with as far as my proposals were concerned). Given this fact, the notion of interart or interartiality would help to maintain awareness of the artistic or intermedial accents of our research. An interart axe de pertinence would primarily aim at the reconstruction of the interactions between the arts at stake in the process of artistic production; an intermedial axe de pertinence would also include social, technological and media factors and will have to elaborate on modalities of these processes, the first being closer to a poetological, the latter to a medial tradition. Let us not forget, however, that arts and media do not live on separate planets of our galaxy.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Intermediality and intertextuality

Jürgen E. Müller 245

The hybrid or the meeting of two media is a moment of truth and revelation from which a new form is born. The parallel between two media holds us on the frontiers between forms that snap us out of the Narcissusnarcosis. The moment of the meeting of media is a moment of freedom and release from the ordinary trance and numbness imposed by them on our senses.29 —Marshall McLuhan. During the past 20 years, the terms ‘hybrid’, ‘hybridity’ and ‘hybridization’ seem to have become almost as fashionable as ‘multi or intermediality’. This fact could be regarded as one of the many discoursive reactions to (post-) modern media developments in the second half of the twentieth century which have often been described as ‘heterogeneities’, ‘eclecticisms’, ‘fusions’ or ‘collages’. The mentioned processes seem to be typical phenomena of our society and media landscape. By making use of the notion of ‘hybridity’, theories of society and media try to cope with these processes which – naturally – are not just new, twentieth-century inventions. With regard to this specific historical background there exist, without any doubt, several parallels between the denotational and connotational profiles of ‘hybridity’ and ‘intermediality’. It is, however, strange to see that, in spite of some common aspects or even overlaps, the notion of ‘hybridity’ has never been put clearly into relation to the intermedial axes of research. Some scholars use this term in a more or less synonymous way for intermedial processes, consider ‘hybridity’ to be a subcategory of ‘intermediality’ or put ‘hybrid media’ under the umbrella of ‘plurimedia media’.30 These usages of ‘hybrid’ or ‘hybridity’ are more or less symptomatic of a rather blurred or unspecific way of handling this term within the framework of intermedial research. I shall refrain from an etymological reconstruction of ‘hybrid’ and ‘hybridity’, which would lead us to the Latin ibrida (bastard) and the Greek hubris (excess) and thus to interesting processes of hybridizations of language, but just point to the numerous and heterogeneous usages of this term during the past centuries, especially in the twentieth century. Hybridity is used in natural sciences, biology, cultural studies, postmodern theory and media studies, as we learn from McLuhan’s quote at the beginning of this short chapter. In spite of all allusions to ‘moments of freedom and release from the ordinary trance’, McLuhan’s hybridity still implies more or less fixed frontiers or borderlines between the media. As a meeting or clash of distinct media, ‘hybridity’ would then be closer to multi than to intermediality. Recent research tries to overcome the weaknesses of the rather static catch-all term ‘hybridity’ by opening it to the dynamics of media transformations.31

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Intermediality and hybridity

246 The Borders of Media Borders

(a) The actual state of affairs of intermedial approaches allows far more differentiated synchronic and diachronic studies of media interactions compared to the quite general category of hybridity, and (b) given the fact that the notion of hybridity is nowadays applied to almost all social phenomena and characteristics of postmodern societies, it is in danger of losing its denotational loadings by offering general catch-all categories. Intermediality would also have to include the dimension of social functions of the intermedial processes and this dimension will have to be related to interactions between cultural and technological series of the media landscape. I would not plead for a pan-intermedialization of all social phenomena, however, by labelling movements of migration as intermediality, for example. Let us now return to our intermedial test case of television.

The test case of television revisited As is well known to us, one of the most provocative components of McLuhan’s media-reflexive universe lies in the more or less consequent refusal of the specific content of the media. ‘The content blinds us to the character of the medium.’32 If we reread this position in a historical perspective of theories of media and culture and remember the dominance of hermeneutic or semiotic approaches and textual readings of the world which had lasted for almost a century in the humanities, or so-called ‘Geisteswissenschaften’, McLuhan’s proposal seems to be very convincing and reasonable. However, some 40 years later, as media scholars, we will have to ask ourselves whether it is not time for a rethinking of the dynamics and interactions between media, materiality and contents of the media, by taking into consideration the modalities of these interplays. Let me briefly refer to our short example of the television transmission in the Fernsehstube. Not only in terms of Baudry’s model of the dispositif 33 we could see and hear how technological qualities and elements of a blurred medium, still having to become television, interact with options of developing a content, and of developing possibilities for messages and social uses or functions. The technological patterns interact with the modalities of demonstrating an apparatus which – in Germany – some 20 or 30 years later will have become institutionalized and could then be named television. The spatial and technological dimensions of the very small and dark studio and the flickering light rays fostered a vaudeville-like and spontaneous live-action and reaction, that is, the ‘showing of the dog’ in relation to an absent and present public. This aspect also supports François Jost’s research

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

However, I would still prefer the concept of ‘intermediality’ to ‘hybridity’, because the first opens two relevant options of research:

Jürgen E. Müller 247

When television does not move, it happens often that it reproduces in a studio the decor of a well-known cabaret, such as Le Chat Noir or Le Lapin Agile, on which Radio comments: ‘On the new set in every detail similar to the original, the Lapin Company around Paolo came to revive the unique atmosphere of the cabaret famous from the past.’34 Tempo-spatial configurations, the framing of the camera, just to mention two elements, did not allow other sorts of ‘performances’ or ‘contents’. In McLuhan’s terms this fact might be considered as proof of the thesis that only the (new) medium matters, that its fascination is grounded on its media characteristics and that its content is constituted by other old media. However, if we follow this argument, we – automatically – miss a sequence of complex interactions between media and cultural series. These interactions will lead after at least two decades of experimentation on the technological and format level to a phenomenon called ‘television’. I would like to follow Gumbrecht in that the shift from hermeneutics or semiotics to media or materialities of communication is an indication of the loss of control of the paradigm of the ‘readable world’ over the humanities.35 Nevertheless, this loss of control must not imply a complete refusal of the content of media and of the interpretation of their formats: the humanities would miss a perhaps unique opportunity of intellectual complexification if they simply tried to replace the traditionally exclusive concentration on meaning and interpretation through an equally exclusive concentration on media and materialities.36 Gumbrecht rightly concludes that for the humanities – and I would like to put media studies under this umbrella too – it will be crucial and imperative to avoid any return to a monistic paradigm. McLuhan’s position and also the position of several other scholars might thus bring about an unnecessary reduction of complexity which misses the contemporary opportunity for the humanities of reaching a higher level of complexity. For McLuhan, the traditional semantic content or (possible) meaning of a medium – to be distinguished by its effects – is like the ‘juicy piece of meat carried by the burglar to distract the watchdog of the mind’.37 However, as watchdogs or media scholars, we must also remain aware of these pieces of beef hidden in the material of our media hamburgers. In this perspective, recently developed axes of research, such as intermedial studies, will be able to differentiate McLuhan’s rather global proposals, primarily in terms of thorough historical studies. As our discussion of the documentary of a quasi television apparatus in a Fernsehstube has shown, these

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

on the beginnings of television where various types of interaction between cabaret and the developing dispositif can be reconstructed.

axes will lead us to some level of intellectual complexity which conceives the relation between ‘sense’ and ‘materiality’, between ‘meaning’ and ‘media’, as a relation of tension or of oscillation – and not as a ‘relation of complementarity or as a relation of mutual exclusiveness’38 and which could be characterized in terms of historical modalities. A study of the history of television as a history of encounters of different technological and cultural series and of corresponding social functions will thus link physical and spatial conditions of media to the meaning productions of their formats, as, for example, Hickethier has demonstrated for the history of German television from the 1950s to the 1990s.39 Following this line, an intermedial and cultural history of the media will have to include the dimension of making meaning understood as a ‘complex relationship with several dimensions of materiality’.40 McLuhan’s remarks would then lead us to a network cultural history of media and media functions including social processes of the production of meaning. Without any doubt, the digital media constitute the greatest challenge for such a network history. That is why, having almost arrived at the end of our short tour d’horizon, it might be useful to develop some heuristic perspectives and questions with regard to intermediality in the digital era.

When the intermedial meets the digital Even if – after several years of excitement and hype in the digitalized world – the phenomenon of Second Life seems to have lost a lot of its attraction, it still proves to be an intriguing test case for the applicability and relevance of an intermedial research axis in the era of Web 2.0. In order to give a rudimentary idea of its pictures, below is a still taken from a documentary on Second Life (see Figure 15).41 It is interesting to see and to learn that the makers and/or the avatars of Second Life not only lead a double life in the virtual tempo-spatial configurations of the platform (which would be a rather banal statement) but that one of the central elements of this play with (and in) virtual worlds seems to consist in various types of usage of filmic or other audiovisual medial and generic patterns in Second Life. In other words: we can find many cases where the makers/producers/users prepare filmic or television sets with corresponding options for actional and narrative patterns. For example, a married couple ‘doubles’ its world in a bright and luxurious Second Life existence (see Figure 15). At first glance, the interference of ‘real’ and ‘second worlds’ and the media powers of Second Life for a unifying digital representation of multimodal patterns proves not to be of primary relevance with regard to our intermedial axe de pertinence, but several other processes seem to be very challenging. For example, we should tackle the question of the analysis of the material and semiotic modalities of the interplay between ‘virtual materiality’ and

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

248 The Borders of Media Borders

Figure 15

A ‘married couple’ in First and Second Life

the ‘content/meaning’ of the actions of different avatars in/or between different sites of the second world. Following this line, intermedial studies will also have to analyse the role and function of the digital nature of the pictures in relation to the so-called live character and to the ‘interactivity’ of the dispositif. For example, what happens to narrative structures and elements of literary, cinematic or television genres when they are transferred into the dynamic virtual spaces and narratives of Second Life? What might be the social and cultural functions related to these and other intermedial processes and how could we distinguish and study the historical functions of certain modalities? These are just some initial questions we might ask with regard to the intermedial processes in games or virtual worlds.42 Our search would not have to stop with the (blunt) answer that – in terms of materiality – the digital era will lead to new stratifications and multimodal combinations of formerly separated media in terms of the ‘unifying’ immateriality of digital codes. It will have to ask the question, in which way have the traditional audiovisual media and/or analogue sounds and pictures left their traces in these digital words, what sort of modalities might be reconstructed and what might be the resulting social functions of these processes for the user of the so-called new media? In this sense, the digital era forms the greatest challenge for intermedial research.

Concluding remarks – or what’s the beef of intermedial studies? Let us return to the starting point of this essay and draw a provisional conclusion concerning the relevance of an intermedial approach. In spite of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jürgen E. Müller 249

some disillusions or even disappointments with regard to the options of the concept or the concepts of intermediality, which – sometimes – can be very curative, an intermedial axis of research still seems to be very promising. Such an axis would not be a convenient motorway or Autobahn for all sorts of theoretical and methodological travels in the fields of media encounters delivering a broad system of systems or a general methodology or taxonomy of modalities for all sorts of media relationships. It would rather be a narrow and winding path in the jungles of intermedial processes, demanding thorough – especially historical – research. In this sense, the elaboration of an intermedial or network history of media would be my preferred option. I hope that we could learn from our test cases of television, film and videogames that such an intermedial history or archaeology will also have to include the reconstruction of social functions of intermedial processes. A network cultural history of television, for example, has to cope with these functional aspects, which become more and more complex with the introduction of camcorders, video recorders, personal computers and MUDs (Multi User Dungeons). A study of these processes will lead us to technological interferences of the dispositifs as well as to interferences between different media genres. In the framework of this essay I could only highlight some of the central aspects and had to restrict myself to a few selected examples. We have to be well aware of the risks of this procedure. There are obviously several other ways of writing a history of intermediality, some of which could and should be added to my proposed network. In spite of the risks taken in this presentation, nevertheless, I hope that the objectives, the outlines and the relevance of an intermedial cultural history of media have become clear on the basis of my theses. The opportunities of such a history have to be seen in case studies of specific innovative ‘thrusts’ or ‘refractions’ of media (such as the introduction of new technologies) and of specific audiovisual productions which will lead us to the social functions of the intermedial processes and to the interactions between socio-historical, technical and other aspects of the processes of the dispositif. The historical study of the theories of intermediality might thus help us to develop profiles of modalities and of historical functions of intermedial processes, ranging from specific patterns of aesthetic experience of the recipient to patterns of action and behaviour of individuals or social groups related to intermedial processes of certain dispositifs.

Notes 1. Here, I follow Werner Wolf. See for example his article of 2002, ‘Intermedialität – ein weites Feld und eine Herausforderung für die Literaturwissenschaft’ in H. Foltinek and C. Leitgeb (eds) Literaturwissenschaft – intermedial – interdisziplinär (Vienna: Verlag der Österreichischen Akademie der Wissenschaften), pp. 163–92.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

250 The Borders of Media Borders

2. I would like to thank very much Charles Nouledo, Bayreuth, for his commitment in the editorial processing of this article. 3. See, for example, M. Mertens (2000) ‘Intermedialiät’: Kommentierungen und Bibliographie (Hannover: Revonnah); or the very useful volume by Joachim Paech and Jens Schröter (eds) (2008) Intermedialität: Analog/Digital (Munich: Wilhelm Fink). 4. As Claus Clüver points out in this volume, such a semiological approach still seems to be useful for many types of (inter)media studies. 5. See the articles in J. E. Müller (ed.) (2008) Media Encounters and Media Theories (Münster: Nodus). 6. With regard to the notion and concept of the Fernsehstube cf., for example, S. Zielinski (1999) Audiovisions: Cinema and Television as Entr’actes in History (Amsterdam: Amsterdam University Press); T. Steinmaurer (1999) Tele-Visionen: Zur Theorie und Geschichte des Fernsehempfangs (Innsbruck: Studienverlag). 7. Cf. J. E. Müller (2008) ‘Perspectives for an Intermedia History of the Social Functions of Television’ in J. E. Müller (ed.) Media Encounters and Media Theories (Münster: Nodus), pp. 203–17. 8. All quotes are from the following edition: Marshall McLuhan (1999) Understanding Media: The Extensions of Man (Cambridge MA and London: MIT Press); here p. 28. Naturally, the ‘definition’ of television as a ‘cool’ medium has to be questioned. 9. See J.-L. Baudry (1978) L’effet cinéma (Paris: Éditions Albatros). 10. McLuhan (1999) Understanding Media, p. 336. 11. As far as this aspect is concerned, see J. E. Müller (2006) ‘Wege einer vernetzten Mediengeschichte: Zur intermedialen Funktions-Geschichte der Television’ in U. Felten, M. Lommel, I. Maurer Quiepo, N. Rissler-Pipka and G. Wild (eds) ‘Esta locura por los sueños’: Traumdiskurs und Intermedialität in der romanischen Literaturund Mediengeschichte. Festschrift für Volker Roloff (Heidelberg: Universitätsverlag Winter), pp. 407–32. 12. McLuhan (1999) Understanding Media, p. 67. 13. In Germany, the first regular television broadcasts and programs started some 3–4 years later, especially in connection with the Olympics in Berlin in 1936. 14. This also seems to be the case in the short advertising film Wer fuhr II A 2992? where some possible options of the still to be developed medium of television are explained to potential customers and to an ‘illiterate’ public. 15. See W. Urichio (2001) What is Television? (manuscript, University of Utrecht). 16. L. Francoeur (1985) Les signes s’envolent: Pour une sémiotique des actes de langage culturels (Québec: Presses de l’Université de Laval), pp. 69–70. 17. Lars Elleström, this volume. 18. See my remarks in J. E. Müller (2006) ‘Wege einer vernetzten Mediengeschichte’. 19. Here, I follow Walter Moser. See, for example, his 2007 article, ‘L’interartialité: pour une archéologie de l’intermédialité’ in M. Froger and J. E. Müller (eds) Intermédialité et socialité: Histoire et géographie d’un concept (Münster: Nodus), pp. 69–92; esp. pp. 69ff. 20. Cf. J. Cisneros (2007) ‘Remains to be Seen: Intermediality, Ekphrasis, and Institution’ in M. Froger and J. E. Müller (eds) Intermédialité et socialité: Histoire et géographie d’un concept (Münster: Nodus), pp. 59–67. 21. J. E. Müller (1996) Intermedialität: Formen moderner kultureller Kommunikation, (Münster: Nodus), pp. 70–1.; see also idem (2000) ‘L’intermédialité, une nouvelle approche interdisciplinaire: perspectives théoriques et pratiques à l’exemple de la vision de la télévision’, Cinémas 10(2–3), 105–34.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Jürgen E. Müller 251

22. I use this term in the sense of Baudry’s work. I prefer the original French term to the rather misleading Anglo-American translation ‘apparatus’. 23. Cf. J. Paech (1998) ‘Intermedialität: Mediales Differenzial und transformative Figurationen’ in J. Paech (ed.) Intermedialität: Theorie und Praxis eines interdisziplinären Forschungsgebiets (Berlin: Erich Schmidt Verlag), pp. 14–30; idem (2008) ‘Intermedialität als Methode und Verfahren’ in Jürgen E. Müller (ed.) Media Encounters and Media Theories (Münster: Nodus), pp. 57–75. 24. In her contribution to this volume, Irina Rajewsky points to the fact that media borderlines always have to be considered as historical categories and as (relevant, but nevertheless) imagined ideal types or constructs. 25. Cf. Lars Elleström, this volume. 26. É. Méchoulan (2003) ‘Intermédialités: Le temps des illusions perdues’, Intermédialités 1, 9–27; at 11 and 15. 27. See Müller (1996) Intermedialität, pp. 93ff. 28. See the essay by Claus Clüver in this volume. 29. McLuhan (1999) Understanding Media, p. 55. 30. Irina Rajewsky (2002) Intermedialität (Tübingen and Basel: A. Francke Verlag), p. 197. 31. For example, P. M. Spangenberg (1997) ‘ “... and my Eyes are only Holograms”: Formen operierender Kontingenz in hybriden Medien’ in I. Schneider and C. W. Thomsen (eds) Hybridkultur Medien Netze Künste (Cologne: Wienand Medien), pp. 7–12 at p.7. 32. McLuhan (1999) Understanding Media, p. 9. 33. With regard to recent discussions of Baudry’s notion and concept, cf. R. Riesinger (ed.) (2003) Der kinematographische Apparat: Geschichte und Gegenwart einer interdisziplinären Debatte (Münster: Nodus). 34. F. Jost (2008) ‘Quand la télévision était un cabaret’ in J. E. Müller (ed.) Media Encounters and Media Theories, pp. 307–20, at p. 310. 35. H.-U. Gumbrecht (2003) ‘Why Intermediality – if at all?’, Intermédialités 2, ‘Raconter’, 173–8, at p. 175. 36. Ibid. 37. McLuhan (1999) Understanding Media, p. 18. 38. Gumbrecht (2003) ‘Why Intermediality – if at all?’, p. 175. 39. See K. Hickethier (1994) Vom Autor zum Nutzer. Handlungsrollen im Fernsehen (Munich: W. Fink) (Vol. 5 of Geschichte und Ästhetik des bundesdeutschen Fernsehens). 40. See Gumbrecht (2003) ‘Why Intermediality – if at all?’, p. 177. 41. Mein wunderbares Ich (Film) (2007) (Susanne Jäger, Director, Germany, WDR-TV). 42. With regard to some preliminary answers to these questions, cf. J. E. Müller (2009) ‘Remediationen in sekundären (und primären) Welten: Zur gattungsspezifischen Paratextualität digitaler Spiele’ in A. Gwozdz (ed.) Film als Baustelle: Das Kino und seine Paratexte. Cinema and Its Paratexts (Marburg: Schüren).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

252 The Borders of Media Borders

Adaptation, 200 Adorno, T. W., 77 A fényképezésröl, 197n26 After the End of Art: Contemporary Art and the Pale of History, 209n3 aleatory intertextuality, 158 Allemann, B., 80n22 Allen, E., 110n18, 214 allusions, 89–90, 153 Ambrose, J. P., 164, 172n6 Amos ou introduction à la métagraphologie, 133n5, 133n11, 133n18 ‘Anabasis’, 76–7 Analyzing Musical Multimedia, 234, 236n21 Anderson, L., 40n5, 81–2, 83, 86, 87, 88, 89–90, 93, 94n1, 94n3 ‘An Eye for an Ear’, 100–1 Ansichten einer künftigen Medienwissenschaft, 186n18 anthropocentrism, 114 anthropomorphism, 114 Antonioni, M., 208 Apocalypse Now, 89 apostrophes, 126 Apter, E., 198n41 Architecture, Mouvement, Continuité, 221n4 Aristotle, 80n13 Arnheim, R., 42n18, 201, 209, 210n19 Ars (art and handicraft) notion, 101 Art and Beauty in the Middle Ages, 109n5 artistic triangles, options at creator’s angle, 138–9 arts arts of time and arts of space, 11 comprehension in, 137–49 intentionality in, 137–49 intermedial relations between, 11 limits of perception, obstacles at beholder’s angle, 139–40 and media, boundaries between, 11 mediation in, 137–49

Trinity and ‘artistic triangle’, relation between, 138 various artistic triangles, options at creator’s angle, 138–9 Arts and the Art of Criticism, The, 48n57 Arts and Their Interrelations, The, 42n14, 42n17 Artwork – Text – Medium: Steps en Route to Intermediality, 197n10, 209n7 Arvidson, J., 39n3, 186n17, 235n1, 236n19 Askander, M., 39n3, 186n17, 235n1 Aspden, S., 172n9 ‘associative’, category, 156 association, differential model, 152 Audiovisions: Cinema and Television as Entr’actes in History, 251n6 ‘auditory text’, 29 avant-garde context, 169 axe de pertinence of intermediality, 237–52 in digitalized world, 248–9 dispositifs, 239–44 dynamic intermedial networks, 242 hybridity, 245–6 interartiality, 244 intermediality as a process, 238–41 intertextuality, 244 medium, 237–8 new medium, 237–8 in scientific community, 241–3 television (test case), 238–41, 246–8; making meaning, 246–8; materiality, 246–8; media encounters, 246–8 Backman, E. L., 108, 110n18 Bakhtin, M., 106, 210n12 ballad, 99–110 see also Medieval ballad and Romanesque church 253

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index

Bal, M., 41n9, 48n57, 85, 95n16, 166, 173n25 Band of Outsiders, A, 215–18 Banks, I., 228 Barricelli, J.-P., 80n28 Barthes, R., 190, 191, 193, 197n19, 197n22, 197n27, 197n34, 198n38, 204, 209n11, 218, 230, 241 basic media, 5, 12, 27–31 Basic Writings, 109n11, 134n25, 134n29 Bataille, G., 232 Bateman, J. A., 40n8 Baudry, J.-L., 238, 239, 246, 251n9, 252n22, 252n33 Bayreuth, 251n2 Bazin, A., 191, 197n21, 197n25 Beckett, S., 220 Benjamin, W., 40n5, 42n18, 95n22, 121n1, 165, 172n18, 192 Berggren, D., 76 Berio, L., 91 Berlin Alexanderplatz, 81, 86, 94n2, 95n23 Bernhart, W., 80n15, 121n2, 172n9 Bernsen, N. O., 40n7 Bertrand, A., 69, 73, 74, 75 Beukes, M., 42n18 Biblia Pauperum, 104 biopoetry of Eduardo Kac, 175–86 electronic music, 182 Genesis, 175–7, 184 intermediality in, 184 ‘medial transposition’, 183 ‘sensorial modality’, 182 ‘spatiotemporal modality’, 182 verbal language medium, 178 written languages, 180 Birders: Tales of a Tribe, 122n20 Birds and Light: The Art of Lars Jonsson, 123n21 Birds in Literature, 122n16 Birds of Eastern and Central North America, 116 Bishara, N., 95n26 Black, M., 69, 70, 72, 76, 77, 79n2, 80n21 Bladen, R., 2, 3 Bleckmann, U., 40n3 Blind Ossian, 162n11 Block, F. W., 185n16

Blush, 56 Bodies, 57, 59–60 Bohn, R., 180, 186n18 Bolter, J. D., 47n52, 47n53, 212, 221n7, 221n9 Book of Mutations and In Our Own Image, The, 185n13 borders, media, 28 between different medial forms of articulation, 64 conventionalization in, 61–2 in current debate about intermediality, problematic status of, 51–68 frames, 64 habitualization in, 61–2 ‘individual media’, 53–4, 62, 64 intermedial references, 62–3 intramedial references, 62 modal, 28–30 qualified, 28–30 Bordwell, D., 200, 209n6 Bourcier, P., 110n16 Breton, A., 144 Broich, U., 161n8 Bruhn, J., 6, 8, 39n3, 48n60, 132n2, 133n17, 161n2, 172n13, 186n17, 225–36 Bruhn, S., 47n54, 73, 74, 75, 80n15, 137–49, 172n12, 236n16 Burks, W., 94n6 Burns, R., 155, 156 Byrman, G., 110n14 calligrammatic rewriting of ekphrasis, 218–21 Calum Colvin: Fragments of Ancient Poetry, 162n14 Camera Lucida, 190 Cantos of Ezra Pound, The, 171n2 categories and medial difference, 159–61 Celan, P., 69, 76, 77, 78, 80n22, 80n23, 80n24, 80n28 Changing Borders: Contemporary Positions in Intermediality, 39n3, 186n17, 235n1, 235n2, 235nn6–7, 236n19 Changizi, M., 95n13 Chapple, F., 45n43 Chew, S., 166, 173n24

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

254 Index

Choros as an ecclesiastical medium, 107–9 Religious Dances in the Christian Church and in Popular Medicine, 108 Christos Pantokrator, 103 churches early medieval church as mass-medium, 102–4 Romanesque church, 99–110 cinema, 25 Cinémas, 251n21 cinematic ekphrasis, 213 cinematic imagination, media in, 211–12 calligrammatic rewriting of ekphrasis, 218–21 heterotopic space, cinema as, 212 intermediality, the cinematic ‘in-between’ and ekphrasis, 211–14 Life of her Own/Her Life to Live, A, 214 mise en abyme, 214–15 ‘Museum of Memory’, 218–21 ‘picto-films’, 213 ‘The Oval Portrait’, 214 vertigo of media, 214–15 Vivre sa vie, 214 Cisneros, J., 251n20 Classical Music and Postmodern Knowledge, 79n1, 80n19 Clüver, C., 7, 39n3, 40n4, 45n42, 46n48, 67n24, 149n7, 172n12, 175–86, 226, 235n2, 235n3, 235n7, 236n16, 241, 244, 251n4, 252n28 Cock and a Bull Story, A, 200 Cocker, M., 122n20 cognitive modalities, Greimas’s semiotic square of, 206 Colapietro, V., 95n10, 95n19 Coleridge, S. T., 241 Collected Papers of Charles Sanders Peirce, 43n26 Collins Bird Guide, 116 Collins, F. H., 116, 121n3, 149n3, 149n4, 206, 210n13 Colors of Rhetoric: Problems in the Relation between Modern Literature and Painting, The, 41n11 Colvin, C., 153, 154, 155, 156, 157, 158, 159, 160, 161n11, 162n11, 162n14, 162n17, 162n19, 162n26

Complete Poetical Works of Amy Lowell, The, 172n5 Complete Works of Aristotle, The, 80n13 Conradie, J., 42n18 contextual qualifying aspect of media, 24 contiguity (indexical signs), semiotic modality, 22–3 convention (symbolic signs), semiotic modality, 22–3 conventionalization, in medial borders, 61–2 Convergence, 44n35 Cook, N., 232–3 Coppola, F. F., 89 Corot, C., 218 Cowell, H., 133n21 Critical Inquiry, 197n14 Cultural Functions of Intermedial Exploration, 172n13 Cultural Studies Reader, The, 236n17 Curry-Lindahl, C., 122n19 Dahlstrom, D. O., 41n11 Dalhaus, C., 80n11 dance, 26 of intermediality, 199–210, see also under film Danielson, E., 110n15 Dansk litteraturhistorie, 109n7 Dante, Alighieri, 101, 109n6, 133n10 Danto, A. C., 199, 209n3 Deane, S. W., 109n11 de Behar, L. B., 95n21 Deely, J., 95n19 de Hooch, P., 206 Dencker, K. P., 185n16 Derrida, J., 70 de Troyes, C., 109n12 De vulgari eloquentia: Über das Dichten in der Muttersprache, 101, 109n6 Den omsnudde verda: Ein studie i dei norske skjemteballadane, 109n14 deontologization, 127–8 Der kinematographische Apparat: Geschichte und Gegenwart einer interdisziplinären, 252n33 Description: In Literature and Other Media, 121n2

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 255

Dialogic Imagination: Four Essays, The, 210n12 ‘dialogicity’, 156 Dickinson, E. C., 121n3 Die Perspektive als ‘symbolische Form’, 210n16 Die Räume der Maler: Zur Bilderzählung seit Giotto, 210n14 differential model, for intermediality, 151–3 ‘allusion’, 153 association, 152–3 ‘indeterminate marking’, 153 referentiality, 152–3 re-presentation, 152–3 ‘selectivity’, 153 ‘structurality’, 153 digitalized world, intermediality in, 248–9 Discourse, Vision, and Cognition, 121n1 discursive remedy or the (critical) intervention, 194–6 dispositifs, 239–44 Divine Madness: On Interpreting Literature, Music, and the Visual Arts Ironically, 42n18, 47n57, 139, 149n2 Döblin, A., 81–2, 86, 87, 88, 89, 90, 93, 94n4, 95n23 Dogville, 61 Doppelbegabungen, 226 Dreyer, C., 109n4 Dreyer, C. T., 214 During, S., 236n17 early medieval church as mass-medium, 102–4 Christos Pantokrator, 103 Majestas Domini, 103 presbyterium, 103 Eckstein, L., 172n13 ecocriticism, 119–21 Ecocriticism Reader: Landmarks in Literary Ecology, The, 123n29 Eco, U., 109n5 Eduardo Kac: Genesis, 185n8 Eichner, T., 40n3

ekphrasis, 34, 109n13, 124, 160, 214–15 ‘art of memory’, 215 ‘art of oblivion’, 215 Band of Outsiders, A, 215–16 calligrammatic rewriting of, 218–21; Histoire(s) du Cinema, 219; Letter to Jane: An Investigation About a Still, 219 Ekphrasis: The Illusion of the Natural Sign, 209n2 ekphrastic metaphors pointing to the other of filmic image, 217–18 as a ‘figure of oblivion’, 215–17 and poetics of in-between in Jean-Luc Godard’s cinema, 212–22, see also cinematic imagination, media in Sentimental dialogue, 216 W. J. T. Mitchell defining, 212 Eliot, T. S., 165, 172n17 Elleström, L., 11–48, 66n7, 79n7, 99, 111, 112, 120, 124, 132n2, 134n26, 139, 149n2, 160, 161n2, 164, 169, 172n8, 174n36, 179, 182, 183, 225, 226, 227, 228, 229, 235n1, 241, 251n17, 252n25 Éluard, P., 144 Eming, J., 222n23 Encryption Stones, 179, 185n8 English Literary History, 133n15 Englund, A., 5, 69–80, 173n30 Erich Doflein: Festschrift zum 70. Geburtstag, 173n34 Erkenntnisinteresse, 230 Escher, M. C., 137 Estes, R., 67n22 Europas fåglar: En fälthandbok, 121n3, 122n12 exteroceptors, 18 ‘extracompositional intermediality’, 56 Fågelguiden: Europas och Medelhavsområdets fåglar i fält, 122n3 Fågelliv betyder orden: Vandringar i Erik Rosenbergs riken, 122n19 Fåglar i Europa med Nordafrika och Mellanöstern, 122n3, 122n7, 122n10, 122n14, 122n17, 123n21, 123n23, 123n26, 123n28 Fåglar i Sverige, 115–16, 122n3, 122n8, 122n13, 123n22, 123n24, 123n27

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

256 Index

Fassbinder, R. W., 81, 86, 87, 94n1, 94n2 Felten, U., 251n11 Festskrift tillägnad Erik Rosenberg på 50-årsdagen 19 17/8 52, 122n19 Fetishism as Cultural Discourse, 198n41 fetishization, 191–192 Field Guide to the Birds of Eastern and Central North America, A, 122n6, 122n7, 122n10, 122n11, 122n12 Field Guide to the Birds of South-East Asia, A, 121n3, 122n6 field guides, intermediality of, 111–23 bird sounds, transcription of, 115 Birds of Eastern and Central North America, 116 Collins Bird Guide, 116 contents of, 112 ecocriticism, 119–21 Fåglar i Sverige, 115–16 image, 113–14 images compared, 116–17 keys in, 112 mediated aspects, 112 text, 114 ‘figure of oblivion’, ekphrasis as, 215–17 film, 199–210 Film als Baustelle: Das Kino und seine Paratexte: Cinema and Its Paratexts, 252n42 Film and Literature, 209n4 Film as Art, 210n19 Film History: An Introduction, 209n6 medium specificity and intermediality in, semiotic approach, 199–210; conceptuality, 200; defining (inter)mediality, tango metaphor, 202–4; Image and Narrative, 201; ‘semiotic mediatics’, 201–2; space and time, description and narration, 204–5; ‘writer’s movies’, 200 see also ‘meaning making’ in film Finlay, I. H., 3 Fischer-Lichte, E., 52, 66n9, 95n17, 95n18 Fischer, O., 40n5, 42n18, 95n17, 95n18, 95n22 fixed sequentiality, 19 Foltinek, H., 67n26, 250n1 Fonda, J., 219

Forman, R. K. C., 134n24 Fortuina, 91 Fossil Folds, 185n13 Foucault, M., 204, 209n11, 217, 220, 221n4, 222n15 Foucault Reader: An Introduction to Foucault’s Thought, The, 210n11 Fragments of Ancient Poetry, 153–7, 162n11 ‘associative’, category, 156 ‘dialogicity’, 156 ‘re-presentative’ category, 156 Francoeur, L., 240, 251n16 Froger, M., 236n16, 251n19, 251n20 Führer, H., 39n3, 186n17, 235n1 Future of the Image, The, 222n22 Gadamer, H.-G., 139 Gale, I., 154, 162n16, 162n21, 162n24 Gandelman, C., 206, 210n15 Gandhi, M., 138 García Márquez, G., 205 Garncarz, J., 44n31 Gaspard de la nuit, 73 Gaudreault, A., 44n35, 240 Geistliche Gesangswerk: Werkgruppe 3, 80n27 Gems: A Censored Anthology, 133n19 Genesis, 7, 175–7, 185n13, 186n19 ‘genetic’ conception of intermediality, 56 Genette, G., 72, 80n12, 243, 244 Gesammelte Werke in Sieben Bänden, 80n22, 80n28 Gesamtkunstwerk, 211, 241 GFP Bunny, 175 Gibaldi, J., 80n29 Giotto di Bondone, 206, 210n14 Glaser, S. A., 186n26 Glass, P., 138 Godard, J.-L., 8, 66n13, 200, 211–22 Goodman, N., 71, 79n9, 160, 162n33 Gorin, J.-P., 219 Gothic period, 104 Götrick, K., 45n42 Gould, S. J., 123n32 Granström, B., 40n7 Grant, P. J., 122n3 Greber, E., 66n12 Greenaway, P., 47n55, 153, 200, 221n2

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 257

Greene, T. M., 48n57 Greimas, A., 202, 205, 206, 210n13 Grivel, C., 65n1, 243 Grosz, E., 194, 198n41 Grusin, R., 47n52, 47n53, 212, 221n7, 221n9 Guide to Bird Song: Descriptions and Diagrams of the Songs and Singing Habits of Land Birds and Selected Species of Shore Birds, A, 122n15 Gumbrecht, H.-U., 247, 252n35, 252n38, 252n40 Gyáni, G., 196, 198n48 habitualization, in medial borders, 61–2 Handbook of Semiotics, 94n6 Hanslick, E., 76 Hao, Y., 95n13 Harald Wiberg: En konstnär, 122n18 Harawi, 144–5 Hatfield, G., 42n17 Heart of Darkness, 89 Hedling, E., 67n24, 149n7, 172n12, 172n13, 222n16 Heffernan, J. A. W., 41n11, 43n20, 143, 149n7 Heidegger, M., 129, 130–1, 134n25, 134n29 Helbig, J., 39n2, 44n31 Herman, D., 66n17, 235n8, 235n13 Hermenetka, 91–2, 95n34 ‘cyberart ritual’, 93 mappings, 91–2 heteromediality, 8, 225–36 centrality of marginality, 233–5 defining, 229–30 heteromedial text, 228 ideology and, 230–2 interart studies and intermediality studies, 225–7 Nicholas Cook, 232–3 W. J. T. Mitchell, 232–3 heterotopic space, cinema as, 212 Hickethier, K., 248, 252n39 Higgins, D., 67n24, 159, 162n30, 173n25 Histoire de la dance en Occident, 110n16 Histoire(s) du Cinema, 219–21, 222n23 Hodnett, E., 123n31 Hölderlin, F., 78 Hollom, P. A. D., 121n3

Holšánová, J., 121n1 Home of the Brave, 94n3 Horizons: The Poetics and Theory of the Intermedia, 67n24 House, D., 40n7 Hranitzky, Á., 204 Huber, W., 236n17 Human Nature, 186n19 humanities, intermediality approach in, 241–3 Hunt, D., 185n8 hybridity concept, 67n30 and intermediality, 244 Hybridkultur Medien Netze Künste, 252n31 hypergraphics, 127–8, 130, 131 iconotextual, 131 notionless, 131 iconic development in Romanesque period, essential features of, 104 Iconicity in Language and Literature, 40n5, 42n18 Iconology: Image, Text, Ideology, 41n9 iconotext, 127 iconotextual hypergraphics, 130, 131 Icons, Texts, Iconotexts: Essays on Ekphrasis and Intermediality, 132n1, 133n20, 161n3 Idea of Spatial Form, The, 43n19 ideology and heteromediality, 230–2 Igor Strawinsky: Trois Pièces Pour Quatuor à Cordes: Analyse und Deutung, 173n32 image, in field guides, 113–14 images compared, 116–17 ‘mug shot’, 113 Image and Code, 42n12 Image and Narrative, 201 Image and Text: Studies in the Illustration of English Literature, 123n31 Imaged Words &Worded Images, 185n16 ‘imagination’, 163–4 Imagining Language: An Anthology, 133n19 ‘in-between in’ Jean-Luc Godard’s cinema, ekphrasis and poetics of, 212–22 intermediality, 211–14 ‘indeterminate marking’, 153 ‘individual media’, 53–4, 62, 64, 66n7 Insistent Images, 40n5, 95n22

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

258 Index

interart and intermediality studies, 225–7 combination, 226 integration, 226 medium, defining, 227 transformation, 226 Interart Poetics: Essays on the Interrelation of the Arts and Media, 67n24, 149n7, 172n12, 222n16 Inter-Art/Interart Studies, 66n9 ‘intertextuality’ notion, 151, 154, 160 interartiality and intermediality, 244 ‘interlingual’ translation, 166 intermedia configurations, 67n24 intermedial, restrictions of, 126–9 intermedial artwork, 153–6 Fragments of Ancient Poetry, 153–5 ‘intertextuality’ notion, 154 intermedial exchanges, in ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’, 163–74 see also individual entry intermedial ‘imagination’, 163–4 intermedial references, 62 of intermediality, 56 intermedial relations, model for understanding, 11–48 intermedial self-reference, 89 Intermedial topography, 69–80 see also metaphorical interaction/metaphoricity intermedial transformations, 34 intermedial references, 34 medial transposition, 34 transmediality, 34 intermedial translation, see translation, intermedial Intermedialität, 34, 48n58, 65n4, 172n10, 252n27, 252n30 Intermedialität: Analog/Digital, 39n3, 47n53, 251n3 Intermedialität: Das System Peter Greenaway, 47n55 Intermedialität: Formen moderner kultureller Kommunikation, 46n47, 221n5, 251n21 ‘Intermedialiät’: Kommentierungen und Bibliographiee, 251n3

Intermedialität: Theorie und Praxis eines interdisziplinären Forschungsgebiets, 221n3, 252n23 Intermedialität: Vom Bild zum Text, 40n3 Intermédialité et socialite: Histoire et gégraphie d’un concept, 236n16, 251n19 Intermédialités, 48n59, 186n25, 222n19, 252n26, 252n35 Intermedialites, 236n17 Intermedialitet, 95n14, 235n6 intermediality, 11, 27–35, 37, 45n40 borders in, 28–9 dance theatre, 56 debate about, 51–68, see borders, media definition, 46, 51–2; the tango metaphor, 202–4 ‘extracompositional intermediality’, 56 ‘genetic’ conception of, 56 groups of phenomena in, 55; in narrower sense of intermedial references, 55; in narrower sense of media combination, 55; in narrower sense of medial transposition, 55 Intermediality in Theatre and Performance, 45n43 Intermediality: The Teachers’ Handbook of Critical Media Literacy, 40n4 ‘intracompositional intermediality’, 56 ‘moving image’ and ‘still image’, relations between, 28 multimodality and, 4 and performativity, 85–8 Peter Wagner definition, 124 plurimediality, 56 qualitative conceptions of, 55 and self-referentiality, 88–94 sub-medium, 29 intermediality, pragmatic approach to, 150–62 aleatory intertextuality, 158 categories and medial difference, 159–61 differential model, 151–3, see also individual entry Fragments, intertextual components of, 156–7

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 259

260 Index

Jahn, M., 235n8 Jakobson, R., 7, 165–6, 173n22 James, W., 81, 143, 161n1, 175, 176 Jamme, C., 80n24 Jansson, S.-B., 110n17 Jean-Luc Godard’s cinema, 212–22 ekphrasis and poetics of in-between in, 212–22, see also cinematic imagination, media in Jersild, M., 110n17 Jia, Zhang Ke, 200 Johansen, J. D., 95n10 Johansson, C., 41n10 Jones, S., 44n35 Jongeneel, E., 161n5, 162n22 Jonsson, B. R., 110n15, 110n17 Jonsson, L., 116, 117, 118, 119, 122n3, 122n7, 122n10, 122n11, 122n14,

122n17, 123n21, 123n23, 123n26, 123n28 Jonze, S., 200 Jost, F., 252n34 Joyce, J., 172n4 Kac, E., 7, 175–86 Kant, I., 42n17 Karina, A., 214, 215 Karlsson, I., 40n7 Karnicky, J., 122n4, 122n20 Kaspersen, S., 109n7 Kattenbelt, C., 45n43 Kaufman, P., 200 Keitel, E., 236n17 Kemp, W., 206, 210n14 Kermode, F., 172n17 Kiernan, R., 231, 236n18 King, B., 102, 121n3, 122n6, 154, 175, 176 Király, H., 7, 8, 44n36, 199, 201, 203, 205, 207, 209 Kivy, P., 233 Klee, P., 192, 218 Kneale, J. D., 133n15 Kostelanetz, R., 185n16 Kramer, L., 79n1, 80n19 Krämer, S., 54, 66n14 Krasznahorkai, L., 204, 205, 209n9, 209n10 Kress, G., 14, 40n8, 47n53 Kristeva, J., 230 Kuklick, B., 161n1 Kværndrup, S., 6, 99, 101, 103, 105, 107, 109n7, 109n13 La créatique ou la novatique, 133n10 La dictature lettriste: Cahiers d’un nouveau régime artistique, 133n9, 133n23 Lagerroth, U.-B., 67n24, 149n7, 172n12, 172n13, 222n16 Lakoff, G., 161n6 Langer, S. K., 39n1, 43n23 language, 125 anticonceptual language, 125 Isou’s definition, 125 Language of Images, The, 42n18, 43n22 Languages of Art: An Approach to the Theory of Symbols, 79n9, 162n33 and music, relation between, 76–7

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

intermediality, pragmatic approach to – continued intermedial operations, categorizing, 151–2 ‘intertextuality’, 151 intertextuality? intermediality? multimediality?, 157–9 ‘multimedial intermediality’, 160 obligatory intertextuality, 158 interoceptors, 18 ‘intersemiotic translation’, 7, 166 Interrelations of Literature, 80n29 intertextual self-reference, 89 Intertextualität: Formen, Funktionen, anglistische Fallstudien, 161n8 intertextuality, 230 and intermediality, 244 Intertextuality: Research in Text Theory, 161n4 Intertextuality: Theory and Practices, 162n27 ‘intracompositional intermediality’, 56 ‘intralingual’ translation, 166 intramedial references, 62 Introduction à une nouvelle poésie et à une nouvelle musique, 133n6, 133n8, 134n32 Isou, I., 124–5, 127, 129, 132, 133n5, 133n6, 133n7, 133n8, 133n9, 133n10, 133n11, 133n18, 133n19, 133n23, 134n32

Laokoön: An Essay on the Limits of Painting and Poetry, 11, 39n1, 43n21, 44n34 Lapham, L. H., 40n6 Las Meninas essay, 217 Laverette, M., 201 Lawson, J., 162n12 Leão, L., 83, 86, 91, 92, 95n34 Le chevalier au lion, 109n12 L’effet cinéma, 251n9 ‘Le gibet’, 74 Lehmann, A. J., 66n13, 222n23 Lehtonen, M., 40n4, 226, 235n4 Leigh, M., 200 Leitgeb, C., 67n26, 250n1 Le lettrisme, les creations et les createurs, 133n13 Le petit soldat, 218 Lesbian Fetishism, 194, 198n41 Les signes s’envolent: Pour une sémiotique des actes de langage culturels, 251n16 Lessing, G. E., 11, 25, 39n1, 43n21, 44n34, 236n19 Les Noces, 174n37 Lethe: Kunst und Kritik des Vergessens, 222n13 L’être et le néant: Essai d’ontologie phenomenologique, 134n28 Letter to Jane: An Investigation About a Still, 219 Lettrism, visual apostrophes in, 124–34 anticonceptual language, 125 comparables, 130 discernible levels of, 130 iconotextual hypergraphics, 130–1 intermediality in, 126–9; restrictions, 126–9 language, 125 lettrist writing, 125–6 ‘meaningless’ visual apostrophes, 131 nothing with words, getting, 129–32 notionless hypergraphics, 131 signification in Lettrist writing, 129 visual apostrophes, 126–7 visuality, 125 written and the visual, tension between, 125 ‘writing nothingness’, 129 Levy, K., 149n5 Life of her Own/Her Life to Live, A, 214 Lina, P., 87

Lindemann, U., 186n17 List of Recent Holarctic Bird Species, 122n5 Listening In: Music, Mind, and the Modernist Narrative, 79n1 Literary Intermediality: The Transit of Literature through the Media Circuit, 47n57 Literary Modernism and Musical Aesthetics: Pater, Pound, Joyce, and Stein, 172n4 Literature, Transcendence, Avant-Garde, 133n4 Literaturwissenschaft – intermedial – interdisziplinär, 67n26, 250n1 Ljungberg, C., 5, 28, 40n5, 42n18, 45n40, 81–95 Lodato, S. M., 172n9 Lommel, M., 251n11 Lönnroth, L., 109n7 Lowell, Amy ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’, 163–74, see also individual entry Lüdeke, R., 66n12 Lüders, O., 87 Luhmann, N., 53 Lund, H., 39n3, 45n42, 67n24, 95n14, 149n7, 157, 162n22, 172n12, 186n17, 222n16, 226, 230, 235n1, 235n6, 235n9 Lundquist, J., 235n1 Lutwack, L., 122n15, 122n16 Luz & Letra: ensaios de arte, literatura e comunicação, 185n2 Lyon, T. J., 120, 123n29 Maassen, I., 66n13, 222n23 MacPherson, J., 153, 154, 155, 156, 157, 158 Madden, J., 200 Madonna, 233, 234 Magritte, R., 234 Magyar Lettre Internationalee, 197n3 Majestas Domini, 103 Making Meaning: Inference and Rhetoric in the Interpretation of Cinema, 209n6 Mallarmé, S., 215, 216 Manifesto, 125 mappings in Hermenetka, 91–2 Maria, J., 103, 107, 108 Marion, P., 44n35, 236n16

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 261

Mary of Egypt, 140–3, 149n3, 149n4 mass media, 31 material modality, 15, 17 materiality, 15 ‘auditory’ material, 15 immateriality and, 15 ‘visual’ material, 15 Matrizes da linguagem e do pensamento, 94n8 Maurer, I., 251n11 McCormick, E. A., 39n1 McLuhan, M., 31, 40n6, 43n29, 47n51, 47n53, 90, 99, 100, 101, 102, 109n2, 109n8, 237, 239, 247, 251n8, 251n10, 251n12, 252n29, 252n32, 252n37 ‘meaning making’ in film, 205–9 cognitive modalities, Greimas’s semiotic square of, 206 freedom, 207 independence, 207 medium concept, 208 obedience, 207 power, 206 powerlessness, 207 Reading Pictures, Viewing Texts, 206 showing medium, 207 systematic language, 205 telling medium, 207 transmediatic chronotopic language, 208 understanding, 206 Méchoulan, E., 243, 252n26 media basic media, 27, 36 borders, 28 combination and integration of, 28, 45n42 ‘form’ and ‘content’ of, connection between, 30 Elleström’s conditions for, 227; material modality, 227; semiotic modality, 227; sensorial modality, 227; spatiotemporal modality, 227 Elleström’s definition of, 227 from a hermeneutical point of view, 13 mass media, 31 media combination category, of intermediality, 56

mediation and transformation of, 28, 45n42 modalities of, 11–48 modes of, 36 qualified media, 27 qualifying aspects of, 24–7, 35 remediation, 31 spatiality levels in, 20 technical medium, 30, 36 temporality in, 20–1 see also qualifying aspects of media Media Encounters and Media Theories, 39, 44n37, 251n5, 251n7, 252n23, 252n34 Media inter Media: Essays in Honor of Claus Clüver, 186n26 Mediale Performanzen, 222n23 medial transposition category, of intermediality, 55–6 media poetry, 179 Media Poetry: An International Anthology, 185n1 New Media Poetry, 179 see also biopoetry of Eduardo Kac media specificities, genre specificities and, comparison, 63 media studies, intermediality approach in, 241–3 mediation, 32–3 medieval ballad and Romanesque church, 99–110 Ars (art and handicraft) notion, 101 art in the High Middle Ages, 101 choir (heaven) and nave (Earth), relation between, 106 Choros as an ecclesiastical medium, 107–9 and cultural interpretation problem, 106–9; ‘ballads of chivalry’, 106 early medieval church as mass medium, 102–4 feudal society, 102 Gothic period, 104 iconic development, essential features of, 104 McLuhan’s method, 102 medieval media, understanding, 99–100 ‘metaphor’ meaning of, 102 numinous mode, 104–6

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

262 Index

sublimes, 104–6 Understanding Media, 99–101 writing and the law, 100–2 ‘Mediterranean’ concept, 91 medium, 11, 82 and art forms, comparisons, 14–15 definition in Wolf, 67n26 description, 13–27, 35 dissimilar notions of, 12 as extensions of man, 13 and ‘form’, 53 Lars Elleström definition, 124 materiality and perception, comparison, 15 McLuhan’s sociological theory, 13–14 and ‘medial configuration’, 53 multimodal concept of, implications, 227–9 notions of, 12 in Peircean semiotics, 82 in semiotics, 82 Medium Theory and the Uses of Meaning, 209n8 Mein wunderbares Ich, 252n41 Meisel, L. K., 66n21 Memory, History, Forgetting, 197n5, 198n52 meontologization, 6, 126–9 Mertens, M., 251n3 Messiaen, O., 133n21, 144, 145, 146, 147 metaphor conception, 69 Aristotelian definition of, 73 Black’s view, 70 ‘interaction view’ of, 69 Metaphor and Musical Thought, 79n1 musico-literary metaphor, 70–1 subjects of, 69; principal subject, 69; subsidiary subject, 69; tension between, 71 metaphorical interaction/metaphoricity, 69–80, 80n20 between music and literature, 71–7; Maurice Ravel’s piano piece Gaspard de la nuit, 73–4; as transmedialized, 75 between music and poetry, 74; ‘Le gibet’, 74, 76; paratexts, 74 in Black’s sense, 77 intermedial relations in terms of, 69

Mozart’s Exsultate, Jubilate, 78 simultaneity notion, 79 ‘transmediality’, 75 wandering phenomena, 75 Middle Ages, ‘medium’ in, 102 Midwest Studies in Philosophy, 43n24 Mitchell, W. J. T., 11, 14–15, 24, 29, 41n9, 41n11, 42n18, 43n22, 43n30, 45n45, 45n46, 46n46, 65n7, 84, 90, 95n9, 95n12, 95n33, 127, 133n16, 188, 190–91, 197n7, 197n20, 197n28, 198n43, 198n51, 199, 200, 202, 209n1, 212, 221n8, 222n13, 222n16, 229, 232–4, 235n12, 236n19, 236n20, 236n24, 236n25, 236n26 mixed media, 45n46 mixed-media configurations, 67n24 ‘modality’, 14 material modality, 15, 17 of media, 11–48 sensorial modality, 15, 17–18 spatiotemporal modality, 15, 19 semiotic modality, 15, 21 ‘tones’, 16 modal media border, 28 ‘mode’, 14 Models and Metaphors, 79n2 Mohr, J., 191 ‘monomediality’, 66n67 Morgan, R. P., 42n18 Morra, J., 166, 173n23, 173n25 Moser, S., 40n5, 95n22, 251n19 Mother Thekla, 140 Mountfort, G., 121n3 Mozart, W. A., 69, 76, 77, 78, 80n24, 80n27 Mullarney, K., 122n3 Müller, J. E., 8, 26, 39n2, 44n37, 46n47, 48n58, 160, 161n2, 180, 186n18, 221n3, 221n5, 236n16, 237–52 multimedia art, intermedial strategies in, 81–95 performativity, 85–8 see also sign, medium as multimedia configurations, 67n24 multimodality, 13–14, 37 multimodal concept of medium, 227–9; implications, 227–9

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 263

multimodality – continued Multimodal Discourse: The Modes and Media of Contemporary Communication, 40n8 Multimodality and Genre: A Foundation for the Systemic Analysis of Multimodal Documents, 40n8 Multimodality in Language and Speech Systems, 40n7 Munro, T., 42n14, 42n17 ‘Museum of Memory’, 218–21 Museum of Unconditional Surrender, The, 7, 187–98, 196n1, 197n4, 197n6, 197n13, 197n18, 197n23, 197n31, 198n35, 198n39, 198n46, 198n49 coda, 196 discursive remedy or the (critical) intervention, 194–6 ‘fetish’, rewriting, 191–2 intermediality, 188 language conceptualization, 194 narration of past displays, 195 ‘an old yellowing photograph’, 190 photograph at the centre, 190–4 photographic medium, themetizing, 187 photography and the photo/graphic, 192–4 ambivalence, 193 diving into the picture, 192–4 ekphrastic description, 192 looking at, 192–4 as a medium of anamnesis, 193 reading, 192–4 position and lens of camera, 188 remediation, 194–6 scopic experience, themetizing, 187 single photograph, 188 ‘symbolic’ intermediality, 188 textual anamnesis, 196 Museum of Words: The Poetics of Ekphrasis from Homer to Ashbery, 149n7 music, 25, 27 and literature, metaphorical interaction between, 71–4 Musical Ekphrasis: Composers Responding to Poetry and Painting, 47n53 musical iconicity, 43n28

Musicalization of Fiction: A Study in the Theory and History of Intermediality, The, 40n3, 41n10, 44n33, 45n40, 45n42, 65n1, 67n24, 67n26, 80n14, 162n32, 173n28, 226, 229, 235n15 musico-literary artefact as transmedialized, 75 musico-literary relations as metaphorical interaction, 71–6 musico-poetic experiment, Three Pieces “Grotesques”, for String Quartet’ poem, 167–168 ‘mutual-exclusivity’, 161n10 Nana, 222n12 Nänny, M., 95n14 Narration in the Fiction Film, 209n6 Narrative Across Media: The Languages of Storytelling, 47n57 New Grove Dictionary of Music and Musicians, The, 149n5 New Historicism and Cultural Materialism, 236n18 New Literary History, 122n9 New Media Poetry: Poetic Innovation and New Technologies, 185n1 ‘new speech’, 125 non-fixed sequentiality, 19 Normand, T., 154, 162 nothingness Heidegger’s notion of, 130 media on the edge of, 124–34, see also Lettrism, visual apostrophes in as ‘a slipping away of the whole’, 131 nothing with words, getting, 129–32 Nöth, W., 82, 94n5, 94n7, 95n11, 95n21, 95n26, 95n28, 109n4, 159, 162n31 notionless hypergraphics, 131 Nouledo, C., 251n2 numinous mode and sublime, in Romanesque period, 104–6 choir (heaven) and nave (Earth), relation between, 106 obligatory intertextuality, 158 Ochsner, B., 65 Oldörp, A., 59 Olofsson, T., 109–10 Olsen, T. D., 109n7

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

264 Index

‘On Linguistic Aspects of Translation’, 166 On Meaning: Selected Writings in Semiotic Theory, 206, 210n13 On Metaphor, 79n2 Ontology, 162n29 Oosterling, H., 218, 222n19, 222n21 operational qualifying aspect of media, 25 oral interpretation, 89 Order of Things: An Archaeology of the Human Sciences, The, 222n15 ‘organized non-verbal sound’, 29 Paech, J., 39n3, 44n37, 47n53, 53, 65n2, 66n13, 188, 197n10, 201, 203, 204, 209n7, 221n3, 222n11, 222n23, 243, 251n3, 252n23 Pailliotet, A. W., 40n4 Palearctic Birds, 122n5 Panofsky, E., 201, 207, 208, 210n16 paratexts, 74, 76 Paratexts: Thresholds of Interpretation, 80n12 partially fixed sequentiality, 19 Pater, W., 172n3, 172n4 Peignot, J., 186n16 Peirce and the Notion of Representation, 95n15 Peirce, C. S., 21, 22, 43n26, 83, 85, 94n6, 95n15 Penrose, Roger, ‘seeing is believing’, 137–49 ‘impossible triangle’, 137, see also under arts musical signifiers enriching an ekphrastic poem (Case study 2), 143–8 sensually or intellectually inaccessible spiritual signifiers (Case study 1), 140–3 performativity, 85–8 intermediality and, 85–8 kinds of, 85 performance and, 86–7 Performativity: A Paradigm for the Studies of Art and Culture, 95n18 radical sense, 85

strong sense, 85 weak sense, 85 White Lily, 86–8 Perron, P. J., 206, 210n13 Perspective as Symbolic Form, 210n16 Peterson, R. T., 121n3, 122n6 Peterson, V. M., 122n6, 122n7, 122n9, 122n10, 122n11, 122n12 Peth, Á., 8, 211–22 Pfister, M., 161n7, 161n8 Phillips, D., 120, 122n9, 123n30 Philosophical Writings, 41n11 photo/graphic traces, 187–98 thematizing photography, 188 see also Museum of Unconditional Surrender, The ‘photorealism’, 66n21 photorealistic painting, 58–60 ‘picto-films’, 213 Pictor, A., 108, 110n19 Picture Theory: Essays on Verbal and Visual Representation, 43n30, 66n67, 133n16, 197n7, 197n20, 197n28, 198n43, 198n51, 199, 200, 209n1, 221n8, 222n14, 222n16, 234, 235n12, 236n24, 236n25 Pictures Into Words: Theoretical and Descriptive Approaches to Ekphrasis, 161n5, 162n22 Pierrot le fou, 215–17 Plett, H., 151, 157, 160, 161n4, 243 plurimediality, 56 Poems and Translations, 80n26 Poetic Origins and the Ballad, 109n1 Poetics, 40n5 Poetics of Cinema, 209n6 Pöggeler, O., 80n24 Pope Benedict XVI., 107 Pound, E., 77, 80n26, 171n2, 172n4 Pound, L., 109n1 Pragmatism, 161n1 presbyterium, 103 Prieto, E., 79n1 primitivism, 170–1 Problem of Pure Consciousness: Mysticism and Philosophy, The, 134n24 proprioceptors, 18 Prospero’s Books, 153 Punzi, M. P., 47n57

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 265

Qiong, Z., 95n13 qualified media, 5, 12, 27–31 border, 28 qualifying aspects of media, 24–7 aesthetic and communicative characteristics, 25–6 cinema, 25 contextual, 24, 33 dance, 26 music, 25, 27 operational, 25, 33 Quiepo, N., 251n11 Quills, 200 quotations, 89–90 Rajewsky, I. O., 5, 30, 34, 39n3, 44n38, 45n41, 46n49, 47n56, 48n59, 51, 53, 55, 57, 59, 61, 63, 65n2, 65n4, 65n5, 66n12, 66n15, 66n20, 67n25, 67n29, 71, 75, 80n10, 80n18, 80n25, 83, 111, 133n3, 133n12, 172n10, 172n11, 183, 186n25, 235n9, 252n24 Rancière, J., 220, 222n22 Ratzinger, Bishop J., 107 Ravel, M., 69, 74 Reading Images: The Grammar of Visual Design, 40n8 Reading Pictures, Viewing Texts, 206, 210n15 Reading Rembrandt: Beyond theWord-Image Opposition, 41n9, 48n57 Realists and Nominalists: An Introduction to Medieval Views of Knowledge, 109n9 Re/collecting Gendered Memory, 197n16, 197n29, 198n44, 198n47 Redefining Literary Semiotics, 95n10 referentiality, 74 differential model, 152 Regard du Fils sur le Fils, 149n10 Regard du silence, 149n10 Reichert, S., 80n22 Reinfandt, C., 172n13 Religious Dances in the Christian Church and in Popular Medicine, 108 remediation, 31, 194–6, 213 Remediation: Understanding New Media, 47n52, 47n53, 221n7, 221n9

Renaissance: Studies in Art and Poetry, The, 172n3 Renard, 174n37 ‘repleteness’, 71 representation, 32, 153, 160 differential model, 152 resemblance (iconic signs), semiotic modality, 22–3 Reutersvärd, O., 137 Reverdy, P., 144 Ricoeur, P., 76, 79n2, 80n20, 187, 197n5, 198n52 Riesinger, R., 252n33 Riffaterre, M., 158, 162n28, 243 Rilke, R. M., 192 Rimbaud, A., 215, 216 Rissler-Pipka, N., 251n11 Robillard, V., 6, 7, 150–62 Rodriguez, La Mala, 91 Roland Barthes Reader, A, 209n11 Roland Penrose: The Friendly Surrealist: A Memoir, 149n8 Role of Comparative Literature in the Sharing of Knowledge and in the Preservation of Cultural Diversity, The, 95n21 Romanesque church, 99–110 see also Medieval ballad and Romanesque church Rosch, E., 161n6 Rosenberg, E., 115, 116, 117, 118, 119, 122n3, 122n8, 122n13, 122n19, 123n22, 123n24, 123n27 Rothberg, M., 189, 197n14 Rothenberg, D., 122n15 Routledge Encyclopedia of Narrative Theory, The, 66n17 Ruppert, R., 180, 186n18 Ryan, K., 231 Ryan, M.-L., 47n57, 66n17, 235n8 Sabatier, R., 133n13 Sacks, S., 79n2 Sadie, S., 149n5 Sager, L. M., 221n10 Said, E., 194 Saint Anselm, 105, 109n11 Sandgren, H., 111–23 Sándor, K., 7, 133n3, 134n31, 187–98 Santaella, L., 94n8, 95n21 Sartre, J.-P., 134n28, 232

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

266 Index

Satan’s Tango, 7, 202–7, 209n9, 209n10 see also ‘meaning making’ in film Satié, A., 126, 130, 131 Saunders, A., 122n15 Schapiro, M., 154, 162n13 Scher, S. P., 78, 80n15, 172n9 Schmidt, E., 44n31, 206, 207, 221n3, 252n23 Schmitz-Emans, M., 186n17 Schneider, F. H., 172n7 Schneider, I., 68n30, 252n31 Schober, R., 7, 47n56, 163–74 Schoenmakers, H., 65n7 Scholz, B. F., 236n19 Schröter, J., 39n3, 44n37, 47n53, 65n2, 251n3 scientific community, intermediality concept in, 241–3 Scott, W., 155, 156, 157 Second Life, 248–9 Selected Prose of T. S. Eliot, 172n17 ‘selectivity’, 153 self-referentiality allusions, 89–90 ‘braiding’, 90–1 ‘cyberart ritual’, 93 Gesamtkunstwerk, 94 Hermenetka, 94 hybrid forms of art and media and, 90 intermedial self-reference, 89 intermediality and, 88–94 intertextual self-reference, 89 ‘Mediterranean’ concept, 91 oral interpretation, 89 quotations, 89–90 Self-Reference in the Media, 95n26 synesthesia phenomenon, 90 in terms of iconicity, 88 various media integrated into an intermedial whole, 90 virtual reality, 90 White Lily, 91–4 Semali, L. M., 40n4 Seminar: A Journal of Germanic Studies, 80n semiosis, sign generation, 83 ‘semiotic mediatics’, 201–2 semiotic modality, 15, 21 contiguity (indexical signs), 22–3 convention (symbolic signs), 22–3

creation of meaning, 22 pictorial representations, 22 propositional representations, 22 resemblance (iconic signs), 22–3 Semiotics of Poetry, 162n28 Semiotics of the Media: State of the Art, Projects and Perspectives, 95n27, 109n4 sensorial modality, 15, 17–18 exteroceptors, 18 interoceptors, 18 levels of, 17 modes of, 17 proprioceptors, 18 receptors level, 18 sensation level, 18 sense-data level of, 17–19 Sentimental dialogue, 216 Shakespeare in Love, 200 Shinsuke, S., 95n13 Sichtbares und Sagbares Text-Bild-Verhältnisse, 66n7 Sievers, R., 167, 173n32 sign, medium as, 83–4 C. S. Peirce doctrine of, 88 iconic, 83 indexical, 83–4 intermediality, 84 ‘narrative painting’, 84 ‘pure painting’, 84 semiosis, sign generation, 83 symbolic, 83–4 signification in Lettrist writing, 129 ‘simulation’, 67n23 simultaneity notion, 79 singing flames, 59 Sjöberg, S., 6, 124–34 Solberg, O., 109n14 Solheim, S., 110n15 Sonic Transformations of Literary Texts: From Program Music to Musical Ekphrasis, 47n54 Sontag, S., 189, 191, 197n15, 209n11, 218, 222n18 Sound Art, 61 Sowa, J. F., 162n29 Space, Time, Image, Sign: Essays on Literature and the Visual Arts, 41n11, 43n20 Spangenberg, P. M., 252n31

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 267

spatial thinking, 42n18 spatiotemporal modality, 15, 17–24 dimensions of, 19; depth, 19; height, 19; time, 19; width, 19 fixed sequentiality, 19 non-fixed sequentiality, 19 partially fixed sequentiality, 19 represented state, 21 Spielmann, Y., 44n35, 47n55 Spinks, C. W., 95n19 Spitzer, M., 79n1 Stead, A., 166, 173n24 Steiner, W., 41n11, 42n12, 43n24 Stein, G., 172n4 Steinmaurer, T., 251n6 Stephan, R., 168, 173n34 Stewart, M., 149n4 State of the Art, The, 209n3 Still, J., 162n27 Storm, I., 122n18 Stravinsky: A Creative Spring: Russia and France, 174n37 Stravinsky and the Russian Traditions: A Biography of the Works through Mavra, 174n37 Stravinsky, I., 7, 163–74 ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’ poem, 163–74 avant-garde context, 169 folkloristic elements, 169 intermedial ‘imagination’, 163–4 intermedial translation, 164–5, see also translation, intermedial musico-poetic experiment, 167–8 ‘primitive’ concept, 170–1 self-referentiality, 168 tonal colour in, 168 ‘structurality’, 153 Styles of Radical Will, 222n18 sublimes, in Romanesque period, 104–6 Suchbegriff, 241, 244 Sukorski, W., 91 Superrealist Painting & Sculpture, 66n21 Süss, G., 236n17 Svensson, L., 122n3 Synesthesia and Intersenses: Intermedia, 162n30 synesthesia phenomenon, 90

Tabakowska, E., 40n5, 95n22 Taha, R., 91 Tarkovskij, A., 208 Tarr, B., 7, 202–4, 207–8, 209n9 Taruskin, R., 174n37 Tavener, J., 140, 141, 142, 148, 149n3, 149n4 Taylor, L., 214 technical media, 5, 12, 16, 30 and basic media, relation between, 33 and qualified media, relation between, 33 Telepresence & Bio Art: Networking Humans, Rabbits, & Robots, 175, 185n3, 185n4, 186n20 television, 238–41 Tele-Visionen: Zur Theorie und Geschichte des Fernsehempfangs, 251n6 Temiz, O., 91 text, in field guides, 114 anthropocentrism, 114 anthropomorphism, 114 ‘Field marks’, 114 narratives, 114 short notes, 114 text comparison, 117–19 Text as Picture: Studies in the Literary Transformation of Pictures, 45n42, 235n10 Text-Bilder: Visuelle Poesie international: Von der Antike bis zur Gegenwart, 185n16 Text Production, 162n28 Theater und Medien/Theater and the Media: Grundlagen – Analysen – Perspektiven: Eine Bestandsaufnahme, 65n7 theatre, 29 Theatre, Sacrifice, Ritual: Exploring Forms of Political Theatre, 95n17 thematizing photography, 188 Thomsen, C. W., 68n30, 252n31 Thor, C., 122n19 Three Pieces for String Quartet, 7, 174n37 Thune, H., 235n1 Tides of Manaunaun, The, 133n21 Tidskrift för litteraturvetenskap, 235n1 topographical model, 69–80 meta-reflective element, 79, see also metaphorical interaction/metaphoricity

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

268 Index

‘transcription’, 7 ‘Transgenic Art’, 175–86 see also biopoetry of Eduardo Kac Transcription Jewels, 185n13 transcription of bird sounds, 115 Transfutur: Visuelle Poesie aus der Sowjetunion, Brasilien und deutschsprachigen Ländern, 185n16 Transkriptionen, 65n7 Translating Life: Studies in Transpositional Aesthetics, 173n24 Translation and the Languages of Modernism: Gender, Politics, Language, 172n14 Translation Studies Reader, The, 172n18, 173n22 translation, intermedial, 164–5 definition, 165 ‘interlingual’, 166 ‘intersemiotic’, 166 ‘intralingual’, 166 ‘On Linguistic Aspects of Translation’, 166 in Modernist literary production, 165 term, 173n25 ‘translation’ term, 172n20 transmedialization, 75 Travelling Concepts in the Humanities: A Rough Guide, 95n16 Trinity and ‘artistic triangle’, relation between, 138 Hindu trinity, 138 Tupsy-Turvy, 200 Types of the Scandinavian Medieval Ballad (TSB), The, 110n15 Typoésie, 186n16 Ugreši´c, Dubravka, 187–98 see also Museum of Unconditional Surrender, The Ulysses, 81 Understanding Media: The Extensions of Man, 40n6, , 43n29, 47n51, 99–101, 109n2, 109n8, 239–40, 251n8, 251n10, 251n12, 252n29, 252n32, 252n37 Urichio, W., 251n15 Valéry, P., 215 Vandekeybus, W., 56

van Leeuwen, T., 14, 40n8, 47n53 Veivo, H., 95n10 Venuti, L., 172n18, 173n22 Verlaine, P., 215, 216, 217 vertigo of cinema media, 214–15 Vertigo, 215 Video: The Reflexive Medium, 44n35 Világoskamra, 197n19, 197n22, 197n27, 197n34, 198n38 virtual reality, 90 virtual space notion, 20 virtual time notion, 20–1, 43n23 Visible Language, 185n1, 185n15 visual apostrophes, 6, 124–34 see also Lettrism, visual apostrophes in Visual Thinking, 42n18 visuality, 26, 37, 47n55, 125, 129, 199, 200, 220 Vitruvian Man, 139 Vivre sa vie, 214 Voßkamp, W., 65n7 Vom Klang der Bilder, 172n4 von Trier, L., 61 Voous, K. H., 122n5 Vos, E., 67n24 Wagner, P., 124, 127, 132n1, 133n20, 151, 161n3 Walsh, S., 174n37 Waltz, S., 57, 58, 62 Walzel, O., 241 wandering phenomena, 75 Wasp Factory, The, 228 Weingart, B., 65n7 Weinrich, H., 215, 222n13 Welle, A., 44n35 Wenders, W., 192 What do PicturesWant? The Lives and Loves of Images, 43n30 What is Cinema?, 197n21, 197n25 What is Television?, 251n15 White Lily, 40n5, 81, 86–9 Why Birds Sing, 122n16 Wiberg, H., 115, 116, 122n18 Wild, G., 251n11 William, W. P., 150, 161, 198n41 Winterbottom, M., 200 Wirkungsgeschichte, 70 Wittgenstein, L., 234

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

Index 269

Wolf, W., 28, 40n3, 41n10, 44n33, 45n40, 45n42, 56, 57, 65n3, 66n17, 66n18, 67n24, 67n26, 72, 80n14, 121n2, 160, 162n32, 172n9, 173n28, 206, 226, 227, 228, 229, 230, 235n8, 235n15, 250n1 Women, Fire, and Dangerous Things: What Categories Reveal About the Mind, 161n6 Wonderful Life: The Burgess Shale and the Nature of History, 123n32 Woodcock, M. W., 121n3 Word and Music Studies: Defining the Field, 80n15 Word and Music Studies: Essays in Honor of Steven Paul Scher on Cultural Identity and the Musical Stage, 172n9 words nothing with words, getting, 129–32 see also Lettrism, visual apostrophes in

Words and Images on the Screen: Language, Literature, Moving Pictures, 221n6 Words and Pictures: On the Literal and the Symbolic in the Illustration of a Text, 162n13 World History of the Dance, 110n16 Worton, M., 162n27 Writing and Filming the Painting: Ekphrasis in Literature and Film, 221n10 ‘writing nothingness’, 129 Yao, S. G., 172n14 Zara, E., 91, 124 Zeitschrift für Französische Sprache und Literatur, 67n29 Zetterström, D., 122n3 Zielinski, S., 251n6 Zohn, H., 172n18 Zola, E., 222n12

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

270 Index