Hyper-Narrative Interactive Cinema: Problems and Solutions

Hyper-Narrative Interactive Cinema

Consciousness Liter ture the Arts

&

18 General Editor:

Daniel Meyer-Dinkgräfe Editorial Board:

Anna Bonshek, Per Brask, John Danvers, William S. Haney II, Amy Ione, Michael Mangan, Arthur Versluis, Christopher Webster, Ralph Yarrow

Hyper-Narrative Interactive Cinema Problems and Solutions

Nitzan Ben Shaul

Amsterdam - New York, NY 2008

Cover Design: Aart Jan Bergshoeff The paper on which this book is printed meets the requirements of “ISO 9706:1994, Information and documentation - Paper for documents Requirements for permanence”. ISBN: 978-90-420-2461-8 ISSN: 1573-2193 ©Editions Rodopi B.V., Amsterdam - New York, NY 2008 Printed in the Netherlands

Table of Contents Introduction

7

Chapter 1: Hyper-Narratives 1.1 What is a Film Narrative? 1.2 Postmodern Narrative Film Theory 1.3 Postmodern Hyper-Narrative Theory 1.4 Cognitive Constructivist Hyper-Narrative Theory 1.4.1 Hyper-narrative Option restriction 1.4.2 Strategies for Devising Coherence between Narrative Threads 1.4.3 Strategies for Devising Dramatic Succession and Closure in Hyper-narratives

15 15 18 23 30 30

Chapter 2: Interaction 2.1 The Interactor as Film Protagonist Fallacy 2.2 From Film Viewer to Hyper-Narrative Film Interactor 2.3 From Computer Game Player to Hyper-Narrative Film Interactor 2.4 A Cognitive-Constructivist Model for Hyper-narrative Interaction 2.4.1 Dramatic Succession 2.4.2 Narrative Thread Restriction 2.4.3 Strategies for Devising Coherence between Narrative Threads 2.4.4 Strategies for Devising Dramatic Succession and Closure in Interactive Hyper-narratives

39 39

Chapter 3: Audiovisuality & Interfacing 3.1 Immersion Hyper-Narrative Film theories 3.2 The Immersion Fallacy in 3-D Movies 3.3 Hyper-reality, Morphing, Digital Replication and Split Screens 3.4 Engaging Interfaces

59 60 66 70 75

Conclusion: Engaging Hyper-narrative Interactive Cinema

83

Bibliography Index of Names and Titles Index of Topics

87 91 94

31 33

43 45 51 51 52 53 55

Introduction The digital imaging revolution is generating far-reaching changes in the structure of narrative audiovisual texts as we know them today in film and television. Theoretical writings have offered a variety of hypotheses concerning the intersections of this revolution with larger cultural shifts (e.g., post-modernism) and their implications for issues of origin, creativity, coherence, identification, affectivity and attention. The general direction in these writings concerns notions of narrative segmentation, depthlessness, lesser affectivity, disorientation, split identity, cyborgization1 and simulacrization.2 At the intersection of technology and creation there is intense experimentation with, and research of the digital revolution’s emergent new media. There have been tremendous advancements in indexing software such as Vsoft, Media Base, MATE, Oracle, Synopsus, etc. A leading creative/technological research direction encapsulating the thrust and promise of these new media deals with what can be termed hyper-narrative interactive cinema. By this term I am referring to a variety of hypotheses and actual works whose common denominator is their focus upon a computer-mediated interaction between users or ‘interactors’ and moving audio-visual texts that strives, through the use cinematic strategies, to offer the interactor an option to change at predetermined points the course of action by shifting to other predetermined options. The Oz project at Carnegie Melon University3 tried to dwell upon cinematic narrative strategies to construct a synthetic environment populated by autonomous animated ‘emotional agents’ that react in a ‘believable’ way to users’ actions. This trajectory has been followed by works such as Façade,4 a synthetic interactive environment consisting of a single drama where the player interacts with a computer generated couple, and by The Virtual Storyteller The preparation of this book for publication was aided by NDS and the Netvision Institute for Internet Studies (NIIS)

8

Introduction

research group at Twente University, Holland.5 The Hyperplex model suggested at MIT Media Laboratory6 strived to create a dynamic large screened presentation of simultaneous films so as to allow the user to shift among them. The Media Lab at MIT also developed automated storytelling systems and methods for analyzing moving images based on indexed media bases.7 Experimental interactive films such as Bob Gale’s Mr. Payback: an Interactive Movie (1995) offered parallel narrative trajectories allowing viewers to shift among up to sixteen narrative developments. The concept of hypertext, which allows links at different points to a variety of intertextual relations8 is employed for educational purposes9 but also in multimedia works such as the CDROMs developed at the USC “Labyrinth” project or in Chris Marker’s Immemory (1999). Salient are multi-narrative cinematic constructs projected to audiences such as Bob Bejan’s film I'm your Man (1998), in which the audience chooses by a majority ‘vote’ (through pressing a button), which course of action the film will take; Guy Vardi’s Point of View: An Experiment in Linear Hypervideo (2000), which offers an interactive perspective-shifting experience where the user chooses from which character’s perspective he views the ongoing action; Glorianna Davenport’s documentary data-based Interactive Cinema;10 Kevin Brooks’ multi-narrative film script authoring tool Metalinear Cinematic Narrative;11 Ruth Aylett’s experiments with emergent narratives;12 and the European consortium Inscape Storytelling, an unfocused ambitious project aimed at “developing a software suite for authoring, publishing and experiencing interactive, multimedia stories and contents”.13 While the aims of these works differ, most of them try to apply cinematic strategies and aesthetics to the creation of such interactive environments.14 This has been intended not only to evoke but also to enhance the cognitive, affective and sensual engagement offered by the cinema. Such enhancement is perceived as promising given the apparent higher potential in interactivity to engage the user’s attention towards the work. However, this intense research and tool development has resulted in little output and meager success insofar as deep, attentive, sustained interaction is concerned. The promise of such developments to exchange or surpass the present mass popular cultural media works of film and television has failed to materialize. In recent years we witness a growing body of research aimed at debating the reasons for the present situation, offering possible

Introduction

9

solutions. Manovich for example, claims that digital works’ relation to film is not one of constructing cinematic interactive narratives at all, but rather in these works’ ‘return’ to the early cinematic paradigm of discontinuous ‘proto-cinematic practices’.15 Lunenfeld on his part suggests that the notion of cinematic interactivity as a text inherently designed to be interactive is ill defined. Films are ‘interactively’ neutral since interactivity is something users perform upon a given film and any film can be altered by fans who ‘poach’ already produced films and create and circulate their altered versions.16 These solutions actually imply an irretrievable loss of narrative cinema’s ability to engage and sustain its viewers’ attention with no comparable alternative. In fact, what Manovich offers is the narrowranged and short-term engagement typical of attraction. Lunenfeld’s suggestion also implies an irretrievable loss of narrative cinema’s ability for deep engagement since his contention that interaction consists of lay people arranging something different out of a previous artistically preconceived work entails the narrow-ranging engagement characterizing social games and enhances the incoherence, non-closure and decentring characterizing postmodern conceptions. Whereas these solutions offer alternative paradigms to narrative cinema, others try to expand its potential towards interactive, computerbased films by trying to overcome the problems entailed in adding interaction to narrative cinema. The Oz Project mentioned above, tried to apply Aristotelian narrative structures, devise interactive believable fictional characters, and apply to interactive texts traditional cinematic audio-visual continuity editing strategies so as to evoke an interactor’s ‘suspension of disbelief’ and consequent ‘immersion’. Likewise, Murray, Brooks and Kinder,17 mentioning the hypertext medium’s infancy, claimed that interactive, engaging hyper-narratives are the future form of storytelling, feasible because of the calculative and storage power of computers and postmodern shifts in perception that expand rather than break away from the inherent faculties of narrative. Kinder, for example, believes that the shift in passing from physical passive reception of narrative films to behavioural active interaction with them can generate deep engagement given the possibility for a successful combination of engagement strategies employed by interactive computer games with certain cinematic narratives. While these approaches to interactive hyper-narratives offer viable models consonant with computer materiality and the human craving for

10

Introduction

narrative arrangements, it is claimed here that these models are bound to fail so long as they are predicated upon misguided assumptions concerning the relation between narrative and cognitive-affective human faculties. As dual-coding theory explains and narrative films show, the correlation of moving images and sounds can engage an audience’s sustained attention more than sounds, still images or moving images alone because when images and sounds are presented the viewer/ listener builds up verbal and visual representations and connects between them.18 Hence, film, television and computerized audiovisual works promise to engage such deep attention. However, splitattention overloading may be abundant in such complex audio-visual articulations. As cognitive-constructivist narrative film theory explains, sound films deeply engage and sustain attention if they allow spectators to construct coherent narratives, characters and audio-visual formations out of the flow of shifting sounds and images. Therefore an overall, continuous editing style, synchronized or otherwise cohering audiovisual formations, character focalized narrative development, and narrative re-centring and closure have become tropes of popular films and television programs. It should be noted that the evolution of the language of cinema and television has devised complex narrative structures based upon such principles, which are not confined to the linear development wrongly identified with Hollywood productions. The reason for narrative cinema’s complex engagement is not due to its simple linearity but to the films’ rewarding play with the spectators’ strive for coherence. As will be shown, many of the projects mentioned above fail because they are based upon misguided conceptual frameworks such as the “immersion” or “postmodern” film perception models. The immersion model wrongly presumes a passive spectator who is manipulated into the irrational belief that the film he is watching is real or that she authors it due chiefly to her identification with the camera’s point of view. As will be claimed, this conception is wrong because as shown by cognitive-constructivism, viewers are cognitively active and aware when watching films. Contrary to immersion models, it is suggested following Carroll,19 that belief in a film’s reality is not a needed factor to explain our deep engagement in fictional situations. Our imagination’s ability to present to itself fictional situations as probable can be just as engaging.

Introduction

11

Likewise, the postmodern desirable decentred, closure-less, ‘open interactive process’ oriented text wrongly presumes a spectator capable of being deeply attentive while splitting her attention. Non-cohering, decentred and closure-less narratives and audio-visual formations often frustrate or distract by demanding that the spectators simultaneously be equally attentive to a flow of several audio-visual occurrences unrelated in space or non-consecutive in time. Another misguided assumption driving many interactive hypernarrative experiments is that the engagement evident in gaming interactivity can be used to enhance hyper-narrative interactive engagement. Countering this claim it is suggested here that gaming interactivity, focused upon skill acquisition and scoring or upon intermittent puzzle-solving interaction, is counterproductive for the generation of hyper-narrative interactive engagement. This is because the main engagement in narratives stems from its ambiguous dramatic succession that allows the viewer to construct the film in his mind from the audiovisual cues the film provides, and which engender her cumulative cognitive-emotional engagement. Finally, countering the misguided assumption that the data base or hyper-textual nature of computers technologically determines the nature of computer cultural creation, it is suggested that while hyper-narrative interactive cinema can and should use peculiar features of computers, these features should be considered in respect of what engages in narratives rather than arbitrarily changing the definition of narrative and of how people perceive narratives in order to comply with one or another computer characteristic. Based upon dual coding theory,20 cognitive load theory,21 and cognitive-constructivist narrative film-viewing theories22 I contend that for non-game, hyper-narrative audio-visual interactive texts to be deeply engaging the computerized ‘hyperlinked nature’ from which these works spring has to be consonant with, rather than alien to human cognitive, affective and sensual faculties. This is because uncritical use of its characterizing audiovisual tropes such as morphing and split-screens, of hyper-narrative parallelism or simultaneity, and of interactivity, easily translates into a perception of de-centeredness and closure-less incoherence. The presumptions about perception and interaction implied in postmodernism, immersion theory, games-as-narrative approach and computer technological determinism, in their incompatibility

12

Introduction

with deeply ingrained and necessary cognitive, affective and sensual human faculties lead to what Jameson metaphorically described as “schizophrenic” reception.23 Understanding these faculties may help us understand why narrative cinema is able to engage deep, sustained attention, whereas present hyper-narrative interactive film experiments often lead to frustration, distraction or shallow engagement. This book suggests hypotheses about the factors required in order to make hyper-narrative interactive cinema deeply engaging and provides some suggestions on how deep engagement might be achieved. For interactive hyper-narratives to sustain deep engagement rather than the current shallow distraction, hyper-narrative structures, interaction and audiovisual design should manage the multi-tasking splitattention problems these constructs engender and – most importantly – use this multi-tasking to enhance rather than reduce engagement. The book is divided into three chapters. The first chapter deals with the concept of hyper-narrative. Offering a definition of narrative based on the cognitive-constructivist approach, it shows how postmodern notions of narrative and of hyper-narrative lead to split attention problems deriving from a conception of hyper-narratives as noncoherent, failing to provide something comparable to the deep narrative engagement offered by films. It then follows Bordwell24 in suggesting a series of hyper-narrative principles that allow for the construction of engaging, coherent hyper-narratives. The second chapter concerns interaction. It rejects postmodern, gaming and identification based approaches to hyper-narrative interaction on the grounds that such approaches engender split attention problems stemming from the multi-tasking of both cognitively constructing the narrative and behaviorally affecting its course. It then suggests a cognitive-constructivist approach that is aimed at engaging the interactor through an effective management of such multi-tasking, based on a hyper-narrative successive flow oriented complementary combination of cognitive and behavioral interaction. The third chapter addresses split attention problems inhering in peculiar interactive hyper-narrative audiovisual configurations and interfaces. Rejecting both immersion oriented audiovisuality and technological determinist accusations of hyper-real audiovisual tropes as inherently shallow, it is shown how the computer-associated tropes of morphing, digital replication and split-screens can be successfully used to deepen the engagement of interactors with hyper-narratives.

Introduction

13

Finally, some examples of interfacing are given that are based on the principle of a reciprocal relation between interface action and hypernarrative evolution, suggesting that such reciprocity can further help manage split attention problems inhering in multi-tasking. The book concludes with a recapitulation of the major principles suggested across the book for the design of a deeply engaging hypernarrative interactive cinema.

1

Donna Haraway, ‘A Cyborg Manifesto: Science, Technology, and SocialistFeminism in the Late Twentieth Century’, in Simians, Cyborgs and Women: The Reinvention of Nature (New York: Routledge, 1991). 2 Jean Baudrillard, ‘Simulacra and Simulations’, in M. Poster (ed.), Selected Writings (Stanford: Stanford University Press, 1988). 3 Joseph Bates, ‘Virtual Reality, Art, and Entertainment’ in Presence: The Journal of Teleoperators and Virtual Environments, 1: 1 (1992), pp. 133–38. 4 M. Mateas and A. Stern, Façade, http://www.interactivestory.net/ (2005). 5 E.g., M. Theune, S. Faas, D. Heylen and A. Nijholt, ‘The virtual storyteller: Story creation by intelligent agents’, in S. G. Bel, N. Braun, U. Spierling, J. Dechau and H. Diener (eds), E 03: Technologies for Interactive Digital Storytelling and Entertainment, Fraunhofer IRB: Verlag (2003). 6 Flavia Sparacino et al., ‘HyperPlex: a World of 3D Interactive Digital Movies’, MIT Media Laboratory (1996). 7 E. Chalom and V. M. Bove Jr., ‘Segmentation of an Image Sequence Using Multidimensional Image Attributes’ Proceedings of IEEE ICP (1996); Glorianna Davenport and M. Murtaugh ‘ConText: Towards the Evolving Documentary,’ ACM, Multimedia '95, November (1995); Glorianna Davenport and B. Bradley, ‘Seeking Dynamic, Adaptive Story Environments’, IEEE Multimedia, Vol. 1, No. 3, fall (1994); G. Ivengar et. Al., ‘Models for Classification of Video Sequences,’ Proceedings of SPIE Storage and Retrieval ‘98, San Jose, California, January (1998); Andrew Lippman, ‘Coding Image Sequences for Interactive Retrieval’, Communications and the ACM, Vol. 32, No. 7, July (1989); Galyean A. Tinsley, Narrative Guidance of Interactivity, Ph.D. Thesis, MIT, 1995. 8 David J. Bolter, Writing Space: The Computer, Hypertext, and the History of Writing, (Hillsdale, NJ: L. Erlbaum Associates, 1991). 9 Roger Schank, ‘Case-Based Teaching: Four Experiences in Educational Software Design’, Technical Report, Northwestern University (1991); Roger Schank, Script, Plans, Goals and Understanding (NJ: L. Erlbaum Associates, 1977). 10 Glorianna Davenport and M. Murtaugh, ‘Automatist Storyteller Systems and the Shifting Sands of Story’, IBM Systems Journal, Vol. 36, No. 3 (1997) 11 Kevin Brooks, Metalinear Cinematic Narrative: Theory, Process, and Tool, doctoral dissertation (Cambridge, MA: MIT, 1999).

14

Introduction

12 Ruth Aylett, ‘And they both lived happily ever after? Digital stories and learning’, in G. Dettori, T. Giannetti, A. Paiva and A. Vaz (Eds) Technology-mediated narrative environments for learning (Amsterdam: Sense Publishers, 2006). 13 Inscape Storytelling: http://www.inscapers.com/. 14 E.g. Marsha Kinder, ‘Narrative Equivocations between Movies and Games’, in D. Harries (ed.) The New Media Book, London: BFI Publishing, 2002), pp. 119–32; Lev Manovich, The Language of New Media (Cambridge, MA: MIT Press, 2001); Janet Murray, Hamlet on the Holodeck: the Future of Narrative in Cyberspace (New York: The Free Press, 1997). 15 Manovich, The Language of New Media, pp. 293–296. 16 Peter Lunenfeld, ‘The Myths of Interactive Cinema’, in D. Harries (ed.) The New Media Book, London: BFI Publishing, 2002), pp. 144–54. 17 Murray, Hamlet on the Holodeck; Brooks, Metalinear Cinematic Narrative; Kinder, ‘Narrative Equivocations’ 18 R. Mayer and R. Moreno, ‘Aids to Computer-Based Multimedia Learning’, Learning and Instruction, 12 (2002), pp. 107–19. 19 Noel Carroll, Mystifying Movies: Fads and Fallacies in Contemporary Film Theory, (Princeton: Princeton University Press, 1988), pp. 93–173. 20 J. M. Clark and A. Paivio, ‘Dual Coding Theory and Education’, Educational Psychology Review, 3 (1991), pp. 149–210. 21 P. Chandler and J. Sweller, ‘Cognitive Load Theory and the Format of Instruction’, Cognition and Instruction, 8 (1991), pp. 293–332. 22 Carroll, Mystifying Movies; David Bordwell, Narration in the Fiction Film (Madison: University of Wisconsin Press, 1985); Edward Brannigan, Narrative Comprehension and Film (London: Routledge, 1992) 23 Fredric Jameson, Postmodernism or, the Cultural Logic of Late Capitalism (Durham, NC: Duke University Press, 2001), p. 26. 24 David Bordwell, ‘Film Futures’, SubStance – Issue 97 (Volume 31, Number 1) (2002), pp. 88–104.

Chapter 1: Hyper-Narratives Hyper narrative interactive cinema refers to the possibility for users or “interactors” to shift at different points in an evolving film narrative to other film narrative trajectories. There is a widespread contention that such interactive hyper-narrative film constructs are the future form of narrative films. It is claimed that this contemporary culturally favored form of film narration will emerge thanks to the calculative and storage power of computers and that hyper-narrative films will expand, rather than break away from the potential inherent faculties of film narrative.1 While computers may fulfill the promise embedded in the concept of hyper-narrative films, it is arguable whether the different hypernarrative film theories forwarded today share a similar conception of film narrative and whether they mean the same thing when talking about the expansion of narrative film potential into hyper-narrative films. I would like to suggest that the conception informing some hyper-narrative film theorists implies a lamentable break with the most rewarding aspects of film narrative with no rewardingly engaging alternative. Others, however, whose comprehension of film narrative and its rewarding faculties is similar to mine, do not really address the major stumbling blocks on the road to expand the rewarding aspects of film narrative towards hyper-narrative films. These latter theorists, rather than expanding film narrative’s rewarding potential into hypernarratives often expand their definition of film narrative until it is almost incomprehensible.

1.1 What is a Film Narrative? Edward Brannigan has suggested a simple and fruitful definition of (film) narrative: “narrative is a way of organizing spatial and temporal data into a cause-effect chain of events with a beginning, middle and end”.2 Beyond the descriptive power this definition carries, underlying

16

Chapter 1: Hyper-Narratives

it are several presumptions about the way humans make their world intelligible, presumptions that explain why narrative films are popular. Hence, as some have noticed, narrative is a particular form of organizing experience and for making sense of the world. Its overall schematic formation has been found to be understood by five year olds. David Bordwell has summarized the findings of psychologists and linguists who discovered that people tend to arrange events that are out of temporal order into causal chains.3 As Brannigan states, “narrative is one of the fundamental ways used by human beings to think about the world”4 and Noel Carroll has claimed that narrative films are popular not because they consist of a set of arbitrary cultural conventions but because they are a cultural invention, easily fitting our cognitive, perceptive, affective and sensual faculties.5 Beyond the relative ease with which we understand film sounds and images, the narrative spatial and temporal organization of audiovisual stimuli into a cause-effect chain appeals to humans once we presume, as Carroll does, that our cognitive-perceptual mode of making the world intelligible is through the framing of questions about phenomena and the raising of “what if” hypotheses to answer those questions. Narrative films are particularly appealing to us because, unlike real life situations, such films use framing, composition and editing to raise clear questions, allow for the viewer’s arousal of imaginary ‘what if’ hypotheses, and confirm or reject these hypotheses by providing full answers.6 Allowing the viewer the possibility of arranging a cause-effect chain out of the narrative audiovisual film-flow, implies an overall trajectory leading from a beginning to an end. Bordwell, who bases his narrative film theory upon the widely accepted constructivist theory of psychological activity,7 maintains that since “perceiving and thinking are active, goal oriented processes”,8 as viewers we strive to construct a cohering, intelligible, goal oriented trajectory out of the flow of sound and images. In other words, we strive to construct out of a given audiovisual flow a causal goal oriented trajectory that starts at some point and reaches somewhere. Bordwell delineated the strategies and procedures that allow active and aware spectators to construct the film narrative in their mind. His presumption was that spectators strive to construct in their mind a goal oriented story out of the film screened before them by constantly forwarding perceptual and cognitive hypotheses (based upon schemes they have in their minds) and trying to fit the film data into them.9 Film

Chapter 1: Hyper-Narratives

17

art in his mind resides in this construction process. The filmmaker presupposes it and construes surprises, distractions, diversions and postponements, which enhance the process of hypothesis arousal, verification or refutation. From this derives the film’s appeal to spectators. Film, like other art forms, address the cognitive faculties of their spectators and strive to allow them to build a world and a story by realizing these faculties, a process they are hardly aware of, or which is hardly satisfied in real life. He maintained that the interaction between spectator and film narrative is based upon an overarching narrative template the spectator has in his/her mind and consisting of ‘exposition, complication, resolution’. The spectator hypothesizes this template and tries to fit the film data into it. Further narrative film-schemes the spectator raises as hypotheses include curiosity hypotheses relating to what has happened before the film started, or expectation hypotheses dealing with what will happen next. In general, viewers strive to verify their hypotheses as the film evolves, and tend to hold on to them or refute them according to their plausibility. Viewers use different strategies concerning the verification or refutation of hypotheses such as a ‘wait and see’ strategy when evidence is inconclusive. The film, on its part uses its own devices so as to encourage the viewer to raise hypotheses. Suspense films for instance, encourage the viewer to raise wrong hypotheses or suspend the verification of a hypothesis through intermediary material, raising the level of suspense. Cognitive constructivist narrative film theory maintains that narrative films deeply engage and sustain the attention of viewers/listeners since they allow them to construct coherent narratives leading to closure. That is why an overall continuous editing style, synchronized or otherwise cohering audiovisual formations, spatial constructions arranged around the logic of the narrative succession, narrative re-centering and closure have become tropes of popular mass film artifacts. It should be noted, however, that the evolution of the language of cinema and of television has devised complex narrative structures based upon such principles, which are not confined to the one-linear development identified with most Hollywood productions. More often than not popular narrative films offer multi-linear and coherently intersecting trajectories leading to closure.10 Hence, the reason for narrative cinema’s complex engagement is not due to its simple linearity, but to the fact that simple or complex films rewardingly play

18

Chapter 1: Hyper-Narratives

with, rather than frustrate, the viewer/listener strive for coherence, goals and closure. These films offer a constantly re-established cohering, audiovisual, spatial, temporal and narrative formation from which the viewer constructs a beginning, a middle and an end. Film narrative caters to the viewer’s cognitive-constructivist process by rewardingly playing with this strive of viewers. Hence, “when the narrative delays satisfying an expectation, the withholding of knowledge can arouse keener interest…The mixture of anticipation, fulfillment, and blocked or retarded or twisted consequences can exercise great emotional power”.11 In fact, sustained suspense (as opposed to momentary suspense), one of the major engaging and rewarding trajectories of narrative cinema, cannot be thought of without the anticipation aroused in viewers for the coming about of favored goals that are endangered, an anticipation which is impossible without the tracing both forward and in retrospect of causal chains that may lead to the favored resolution. Particularly important is that such sustained narrative suspense is also cumulative and cannot be maintained without a sense of an internal causal logic underlying the procession of events unfolding before our eyes.12 If we accept such definition of narrative and the suggested cognitive constructivist process of narrative formation, it becomes plainly clear why some hyper-narrative film theorists do not actually refer to film narrative when they think about hyper-narrative films. It also allows for an outline of the major stumbling blocks facing a transition from narrative films to hyper-narrative films and may therefore offer viable ways to overcome these. There is a paradigmatic hyper-narrative film theory whose confounded comprehension of what motivates the engagement of viewers in narrative films translates into audiovisual hyper-narrative theories that inform shallow or non-engaging practices. This is the postmodern narrative film theory.

1.2 Postmodern Narrative Film Theory The post-modern episteme has generated changes in the structure of some contemporary narrative films13 characterized by the scrambling of temporal order, the mixing of genres, a pastiche of quotes taken from previous films or other cultural forms, and a leveling of light and heavy

Chapter 1: Hyper-Narratives

19

discourses. Many have noticed the dire implications of this episteme for issues of origin, creativity, coherence, closure and identity, along with textual depthless-ness, loss of affectivity and viewer disorientation.14 Postmodernists questioned the value and legitimacy of the basic narrative categories of coherence and closure. Jacques Derrida for example,15 reached the conclusion that the category narrative is nothing but a fictional construct artificially imposing temporality and meaning upon an endless chain of signifiers. He considered these signifiers to potentially emanate indefinite arrangements and interpretative possibilities. Julia Kristeva and Roland Barthes (in his later writings), dismantled the attempts to ground textual analyses on Saussure-inspired semiology considering its strive to discern objective rules determining textual production futile and even harmful since they expressed ideological attempts to control cultural production. Postmodernists turned to the dismantling of textual fixtures. They premised that language and other forms of communication are polysemic and multidirectional. Any attempt to fix, stabilize or systematize the process of signification was in their minds an attempt to control human and textual freedom and creativity. Kristeva’s and Barthes’s postmodern notions of textual and human freedom were based upon a conception of a mutating text and individual: “a divided subject, even a pluralized subject that occupies not a place of enunciation, but permutable, multiple, and mobile places”.16 Following Michail Bakhtin’s literary research, particularly his notion that the meaning of a word in a literary text is not fixed but results from its dialogical interaction with various voices and positions within the text, between texts, and in the reader’s mind, Kristeva premised that “The text, is therefore a productivity, and this means: first, that its relationship to the language in which it is situated is redistributive (destructive-constructive),…and second, that it is a permutation of texts, an intertextuality: in the space of a given text, several utterances, taken from other texts, intersect and neutralize one another’.17 Barthes also turned away from classic semiology and began to view the text as “experienced only in an activity of production…its constitutive movement is that of cutting across…it cannot be contained in a hierarchy…”.18 The text as inter-text began to be viewed as an open set of textual intersections and relations differently realized in each interrelation with a reader or viewer, themselves conceived as having split, shifting identities and entertaining varying positions towards the text. This radical revolutionary-spirited dismantling of narrative for the

20

Chapter 1: Hyper-Narratives

sake of textual mutability, while very productive in the opening up and articulation of unnoticed or silenced textual meanings, nevertheless wrenched away any comprehension of the evident order or disorder inhering in a text. Defining texts to begin with as multi-directional, shifting and polysemic on all levels renders any attribution of this or that interrelation between texts or textual segments as arbitrary. This is perhaps why Barthes described the intertextual process in terms of a non-obliging game while other postmodernists gave up altogether upon notions of textual truth-search and objectivity.19 In general postmodernists failed to notice the deep engagement resulting from the interaction between narrative films’ intended textually embedded narrative and the cognitively and emotionally active spectators, deeply engaged by the films’ modes of narrative delivery in satisfyingly reconstructing the narrative in their minds. Overall, postmodern approaches to texts as boundless and decentered, generates a theoretical and practical labyrinth. This labyrinth emerges when the impulse that led to deconstruction necessarily attempts to re-construct the concepts it had earlier dismantled. Hence, postmodern attempts at narrative re-construction are paradoxical. This is simply because an attempt to define textual narrative trajectories as open-ended or de-centered logically demands that there be trajectories that are then made open or de-centered. But according to the postmodern definition of texts, there aren’t such trajectories to begin with since texts are inherently and simultaneously open to shifting destructions and reconstructions. Hence, how can something that is defined to begin with as de-centered and open be turned de-centered and open? Thus, any perception of texts as constantly shifting configurations of variables is self-contradictory in that it cannot do away at every given point with determining invariables, simply because one cannot specify a difference unless there is something constant against which to measure it. Indeed, postmodern theories and films always posit some such invariable.20 However, given the basic premises of textual boundlessness and decentralization, theoreticians have no good reason to justify their invariables while postmodern filmmakers end up offering temporal variations upon a constant for otherwise they will construe an incomprehensible, confusing, distracting and disengaging scrambled temporality. Actually, textual boundlessness and decentralization are themselves unjustified invariables.

Chapter 1: Hyper-Narratives

21

I would like to suggest that the post-modern cinematic strategies of de-centering and non-closure through non-cohering narrative threads are strategies that engender viewers’ distraction rather than deep engagement. It is my contention that the use of such strategies wrongly presumes a subject capable of being attentive while splitting his/her attention.21 Actually, few, if any real humans are capable of effectively dealing with many of the cognitive tasks demanding split attention.22 Post-modern film narrative theorists seem to overlook this problem or presume that splitting the viewer’s attention is the characterizing and desired form of perception in our age. They entertain a confounded belief that the rewarding narrative and audio-visual coherence of the popular cinematic ‘end product’ may not be lost in what they see as a necessary or desirable decentred, closure-less, process-oriented film narrative. This notion is confounded in my view because closure and cohering strategies are the sine qua non component that enables the deep, cognitive, affective and sensual rewarding engagement of spectators in narrative cinema. Conversely, non-closure and decenteredness, when posited by postmodern textual theorists as a text’s basic premise,23 frustrate the reader’s strive for coherence and often lead to distraction and loss of interest. In a sense, the whole notion of narration is meaningless if the viewer’s aspiration for closure is frustrated to begin with.24 It is only because texts offer you a notion that they are going somewhere that you are willing to follow.25 Hence, postmodern narratives imply a distracted or frustrated response. The film Adaptation (Spike Lonze, 2002) can serve us here as emblematic of the split narrative formation characterizing post-modern films. The film starts as a deeply engaging and complex work aptly articulating the schizophrenic situation of the postmodern man. It tells the story of Charlie Kaufman (Nicolas Cage), a screen writer who is repelled by the way he looks and is extremely unsure of himself both in respect of his writing and of women. He has a twin brother, Donald, who is in many ways his opposite. Donald, as presented from his brother’s point of view is ridiculously self assured but to his brother’s surprise, has success with women and when he decides to also become a screen writer like his brother, he comes out with a script loved by Charlie’s film agent. Whereas insecure, repelling Charlie is interested in a non-eventful film, Donald, ridiculed by his brother, is interested in the generic, causal, conventional development of a story with protagonists who evolve and change throughout the film following their overcoming a difficult task

22

Chapter 1: Hyper-Narratives

involving action, drama and tension. As the film develops Charlie is contracted to adapt into film a book on orchids written by a New Yorker magazine journalist (Merrill Streep) who lives with a boring husband and longs to be in a state of fascination from something (be it a flower or a man). She henceforth inquires into the mentality of orchid flower collectors and starts following a mysterious orchid hunter and collector searching for the rare “ghost orchid”. Charlie decides to adapt the book into film so that it maintains the book’s unique, non-eventful, “life-like” and original stream of conscience style. At one point, however, Charlie gets lost in the labyrinth demanded by his type of film and asks Donald to rescue his script. What the film cleverly causes the spectator to be aware of is that the script being written within the film by Charlie is also the actual film the spectators are watching (a film about a scriptwriter trying to write the film the spectators are watching). True to form, once Charlie asks his brother to help him with the script he is writing (a script that the spectators are watching) and Donald starts taking things into his perspective, the film watched suddenly turns into a Donald action film, filled with drugs, chases and murders. Hence, once the film shifts to Donald’s perspective we become aware that the orchid hunter has an affair with the reporter and that he is actually a drug dealer extracting drugs from the orchids he collects. We then see Donald and Charlie follow the reporter to the orchid hunter’s house, where Charlie, peeping through their window finds out about their drug dealings. He is spotted by the couple, caught by them and once they realize he knows about their drug dealings and love affair they decide to kill him. Eventually they chase both brothers and end up killing Donald while Charlie manages to escape, gets to finish his script and even find love. As can be noticed, Adaptation is not only split into two twin protagonists but also into two respective thematic and stylistic developments. However, when this clever film shifts from Charlie’s stream of consciousnesstype film to his brother’s action type film, the result is frustrating. This is because spectators are pulled out from the depth and involvement they were in when following the stream of consciousness film, and pushed towards an action film that starts all of a sudden without serious earlier development and out of materials that have been differently contextualized and are alien to the action film (e.g., the hunter’s search for the unique, rarely found fascinating orchid suddenly turns into a factory for drugs filled with such orchids). This perspective shift, which if coherently construed could have deeply involve us in the textual

Chapter 1: Hyper-Narratives

23

or life and death implications that are at stake in the film’s relativity of perspectives, ends up neutralizing the impact of both views due to its split narrative construction, engendering a gaming distraction and frustration rather than deep emotional-cognitive engagement.26 In a way these post-modern films seem to reference the modernist avant-garde films of the 1960s and 1970s as their precursors. However, whereas some of those films were predicated on de-centering and nonclosure (e.g., films by Godard or Antonioni), they were in their time aesthetic and ideological (often obscure) searches for truth through challenges to established perceptions, challenges to which committed viewers were expected to respond by reflection upon their lives or by attempts at re-assessment and re-construction. Hence, Antonioni in The Red Desert (1964) used alienating strategies such as long, inconsequential shots and de-centering of protagonists, to deflate and deconstruct dramatic meaningful moments such as the love for a child or extramarital affairs. Through these strategies he projected his explicit existentialist alienated vision of the world,27 lamented alienation and strived for truth. Antonioni’s Red Dessert insinuated that while alienation may be a natural condition of the human being it is also the result of the capitalist mode of production, which generated the alienated mentality of the North Italian bourgeoisie he depicted in the film. Post-modernists, however, seem to presume that split attentiveness is the favored culturally determined state of cognition and reception of their viewers. The resulting subjectification of truth and its conception as neutrally relative is perceived as a blissful end to itself. In other words, avant-garde modernists presumed a centered self whose confidence they wished to reassess through alienating worldviews and respective textual strategies, whereas post-modernists often presume a computer determined de-centered and split self, whose non-confidence, distraction and alienation they reassure through neutralized split narratives projecting neutralized and inconsequential split worldviews.

1.3 Postmodern Hyper-Narrative Theory The digital revolution’s development of the hypertext has more often than not rendered a conception of hypertext theory and production as determined by the consonance between the postmodern episteme and the technological characteristics of the computer. Many hyper-narrative

24

Chapter 1: Hyper-Narratives

theorists not only envision a future form of hypertext-based narrative interactive films enabled by the calculative and storage power of computers, but view this development as oriented by the consonance between the computer’s data-base nature and presumed postmodernstyled shifts in cognition, perception and social conduct. Manovich for example, suggests that audiovisual hypertexts model their works not upon cinematic narratives but upon the early cinematic paradigm of discontinuity, which he finds exemplified in Vertov’s cinema, since these forms correspond for him to the data-base and algorithmic functions characterizing computer-based works: “(narrative) is only one aspect of cinema that is neither unique nor… essential to it…the distinct logic of a digital moving image … subordinates the photographic and the cinematic to the painterly and the graphic”.28 Likewise Lunenfeld, analyzing the Blair Witch Project (H. Donahue, J. Leonard and M. Williams, USA, 1999) distribution strategies and responses, suggests that the current notion of cinematic interactivity as a text inherently designed to be interactive is ill defined. Films are “interactive” neutral since interactivity is something users perform upon a given film. Hence, any film can be altered (and some are) by fans who ‘poach’ already produced films and create and circulate their altered versions.29 These positions conflate technological and formal characteristics of computer-based interaction with postmodern notions. Hence, Manovich maintains that such discontinuity inherently stems from the computer mode of operation, characterized by numerical representation, modularity, automation, variability and transcoding,30 while Lunenfeld complements Manovich’s technological determinist view with the widespread postmodern claim about contemporary shifts in human perception and social conduct, writing about “digital media’s aesthetic of unfinish…” and about the “rhizomatic and dynamic interlinked communicative community…”.31 These solutions actually imply an irretrievable loss of narrative cinema’s ability to engage and sustain its viewers’ cognitive, affective and sensual (audiovisual) attention, with no comparable engaging-wise alternative. In fact, what Manovich offers is the narrow-ranged and short-term engagement typical of attraction.32 Experience for example the game-like restless split attentiveness generated by artistic interactive CD-ROMs such as Chris Marker’s Immemory (1999). When interacting with this work your attention splits between still images, bits of images in motion, written comments, and the game-like arrows and hot spots

Chapter 1: Hyper-Narratives

25

that lure you to impatiently jump back and forth. Manovich extends his notion of attraction to a misguided comprehension of Dziga Vertov’s film Man with a Movie Camera (USSR, 1929). He does offer an interesting comparison between Vertov’s project and the potential use of computer data-bases, in that Vertov’s film shows the process of tracing trajectories (comparable to algorithms according to Manovich) across a data base of the documentary images he has gathered. However, Manovich bases his comparison upon a misguided comprehension of narrative, which he conceives of as a trajectory lacking closure.33 Based upon this he suggests Vertov’s major achievement to be “how to merge database and narrative into a new form”. However, he sees Vertov’s narrative as the open-ended narrative of the discovery of cinematic effects, “an untamed, and apparently endless, unwinding of techniques…”34 He is thus oblivious to Vertov’s major ideological and aesthetic goal of aspiring to a communist decoding of the total social process, through cohering closure strategies. These are found in the film’s overlapping of a film screening process from beginning to end, with a full morning to evening cycle of a city’s life, and a spatial expansion explicating the interrelations of the various forces and means of production in Soviet society at the time.35 Manovich’s convenient underestimation of Vertov’s cinematic implementation of his aspiration for a total comprehension of the mode of production, cardinal to Vertov’s communist ideology, allows him to anachronistically place Vertov on the side of a postmodern closure-less episteme. Although Manovich does conclude his description of Vertov’s project as “a gradual process of discovery” constituting the ‘film’s main narrative,’36 his lack of awareness of Vertov’s aspiration for closure, a sine qua non requirement for a notion of ‘gradual discovery,’ renders his conclusion incomprehensible. Lunenfeld’s suggestion also implies an irretrievable loss of narrative cinema’s ability for deep engagement. His contention that interaction consists of lay people arranging something different out of a previous artistically pre-conceived work and sending it to others (emblematic are the TV series Star Trek fans)37 entails the social simplified reduction of cultural production. It enhances the non-coherence, non-closure and de-centering that characterize post-modern conceptions, failing by definition to achieve narrative cinema’s deep sustained and wide-ranged engagement and offering instead the social narrow-ranged engagement characterizing social games.

26

Chapter 1: Hyper-Narratives

Whereas Manovich and Lunenfeld offer alternative paradigms to narrative cinema, others, like Murray, Brooks or Kinder try to expand its potential towards interactive, computer-based films. These researchers, however, while aware of the difference between cinematic narrative continuity and the potential for multi-narrative and interactivity inhering in computerized interactive films, nevertheless fail to effectively bridge the transition from narrative cinema to deep, sustained, engaging hypernarrative interactive cinema. This is primarily due to their confounded belief that there is some bridge between the rewarding narrative and audiovisual coherence of the popular cinematic ‘end product,’ and what they see as a necessary or desirable de-centered, closure-less, computer based ‘interactive process’– oriented product. As mentioned, this notion is confounded, because closure and cohering strategies are the sine qua non component that enables the deep cognitive, affective and sensual rewarding engagement of the viewers in narrative cinema. Conversely, non-closure and de-centeredness, when considered the basic premise of a text, as maintained by different post-modern textual theories, frustrate the reader’s cohering thrust and often lead to distraction and loss of interest. Kinder’s comprehension of what she terms (following Manovich) ‘database narratives’, described by her as “those narratives, whether in novels, films or games, whose structure exposes the dual processes of selection and combination that lie at the heart of all stories and that are crucial to language”38 exemplifies this problematic attempt to bridge the gap between narrative ‘end products’ and open, ‘interactive process’ products. Although her analysis of films structured around games such as Alain Resnais’ Last Year at Marienbad (1961) or Agnes Varda’s Cleo from 9–7(1961) is impressive and may offer a viable model for interactive hyper-narrative cinema (as discussed in the following section), it fails because of her post-modern premises about narrative. Hence, at the core of her argument on narrative she claims that “Openness lies at the center of the narrative illusion”.39 She supports this claim of narrative openness by her mentioning Certeau’s notion of ‘text poaching’, whereby text readers, rather than following a text’s intended meanings, actively appropriate the text for their own interests. Hence, she is implying that the reason text poachers can appropriate narrative films is the inherent openness of narratives. What Kinder overlooks is that the resulting approach to the text is reduced to gaming rather than elevated to deep complex engagement. Kinder herself inadvertently exposes

Chapter 1: Hyper-Narratives

27

the probable results of open textual configurations. Hence, she writes that data-base narratives “reveal the arbitrariness of the particular choices made, and the possibility of making other combinations, which would create alternative stories.”40 It is this ‘arbitrariness’ that exposes the probable results of open-ended interactive narratives, as also evidenced in the aesthetic and narrative arbitrariness of text poachers’ compilations. Kinder neglects, however, the fact that in the films she analyzes, the various options are carefully interrelated and construed as interchangeable yet feasible on their own, not because “openness lies at the center of the narrative illusion” but primarily because they aspire to coherence and closure both among the different options and within each option, offering concrete optional closures rather than open-endedness. She herself contradicts her narrative theory when emphasizing the structural containment of the films by their game structure and the cohering narrative trajectories outlined in these films. Hence, in her analysis of Agnes Varda’s film Cleo from 5 to 7 (1961) for example, a film segmented by Tarot Cards she writes: “Although Cleo at first seems as stereotypical as the avatars in the tarot deck, once she moves into the open narrative field of the city…the avatar becomes humanized and the temporal countdown a reminder that we are documenting her movements in ‘real time’. In this space she meets a young soldier on leave from Algeria who is also facing death, a chance encounter that locates her in history and redirects…towards an open ending that the driving endgames of tarot, cancer and melodrama did not lead us to predict.”41 Despite her attempts to describe this film as “open ended” and arbitrary in its narrative turns, her own analysis suggests that “tarot, cancer and melodrama” are coherent “endgames” whereas the final thread leading to life is coherently related to the previous “endgame” in that it also revolves around death (the soldier’s and Cleo’s) and the meaning of its indetermination (life) is not “arbitrary” but results from its binary opposition to the previous over-determination (death). What misleads most current attempts to offer non-narrative or expanded narrative cinematic paradigms, are these farfetched postmodern presumptions about human perception, coupled with misleading presumptions about the extent of determination of human cognition by the materiality and ‘nature’ of different media. My claim is that post-modern and consonant media determinist assumptions about narrative comprehension run against deeply ingrained and necessary cognitive, affective and sensual human faculties. Jameson’s apt use of

28

Chapter 1: Hyper-Narratives

the metaphor of ‘schizophrenia’ to describe this type of comprehension, points to the incompatibility and inherent gap between human-ingrained types of narrative reception, and the arbitrary type of narrative reception posited by post-modern cultural theory and some media determinists. Having discussed the fallacies of postmodernism I would like to focus briefly on the fallacies of postmodern oriented media determinists. Neither the arbitrary nature of abstract cultural sign systems (e.g., language or computer data-base), nor its sign’s material qualities are in and of themselves determinant of the manner of their reception, as is often claimed by media determinists.42 Rather, it is the other way around. It is only because we can cognitively and perceptually manipulate signs in order to forge conventions that will render non-arbitrary articulations comprehensible, that sign systems serve us so well.43 Hence, Manovich’s claim that the ‘nature’ and ability of computers to contain a large and easily accessible data-base will probably lead computer-based art works to resist narrative tendencies, or the implication of media determinists that it will forge users’ modes of perception and cognition, is fallacious. This is because people exploit those potentialities that best reward their cognitive, affective or sensual faculties. Computers, despite their data-base characteristic do allow the tracing of narrative trajectories. Hence, as long as narratives are cognitively, affectively or sensually engaging, this computer potentiality will be exploited. While its present narrative exploitation has failed to materialize, it will probably evolve with time as was the case of cinema. Indeed, an expectation similar to that of Manovich was elicited by Walter Benjamin concerning the inherent, material faculties of film. However, film effectively took the road of engaging narration rather than that of distraction, disjunction, or short-termed shocking stimulation that Benjamin predicted it could and should take by its very nature.44 This does not mean, however, that the hyper-textual ‘nature’ of computers has no bearing upon the art works it allows to be produced, or that computer-based narrative audiovisual texts must or can replicate narrative cinema in order to achieve the latter’s engaging attention. A similar process occurred with television. Witness the failure of television to deeply engage attention when screening feature films. This is probably due to the television’s screen size, its conditions of reception (at home), and its multi-channel transmission. This does not mean, however, that television has failed to deeply engage attention through narrative. It means that its narrative strategies had to change in order to

Chapter 1: Hyper-Narratives

29

adapt to television’s peculiar faculties. Thus, television has devised 20 to 40 minute programs with multi-threaded, interlacing narratives such as ER, Seinfeld, or 24,45 more fitting its ‘nature’ yet consonant with the cognitive faculties of its audience. Just like television, computerized interactive films need not replicate cinematic narrative strategies to engage attention. In order for such works to deeply engage attention their peculiar nature has to be used in ways that correspond to human cognitive faculties. Hence, while critical of Murray’s, Brooks’ and Kinder’s theoretical comprehension of interactive hyper-narratives, I think they offer viable models for such works. I also share their view that hyper-narrative interactive cinema is the major potential form consonant with computer materiality and the human craving for narrative arrangements. However, for this form to generate deep cognitive, affective and sensual engagement rather than shallow distraction, the cognitive faculties that audio-visual narratives engage must be taken into account. Based upon constructivist narrative film viewing theories,46 I contend that for non-game, complex, multi-narrative aspiring, audiovisual interactive texts to be deeply engaging, the computerized ‘hyper-linked nature’ from which these works spring has to be consonant with, rather than alien to human cognitive, affective and sensual faculties. This is because uncritical use of its characterizing tropes easily translates into a perception of de-centeredness and closure-less incoherence. As mentioned, the uncritical use of such tropes, not to mention their encouraged use as such by post-modern theory, presumes a subject capable of being attentive while splitting his/her attention, a presumption bound to generate interactor frustration or distraction. Just like television, the digital revolution can be taken to use cohering narrative strategies to engage the audience’s attention while adapting these to the computer’s peculiar nature, so long as its peculiar nature is used in ways that correspond to human cognitive faculties. For digital based films to generate deep cognitive, affective and sensual engagement rather than shallow distraction, the human cognitive strive for coherence must be taken into account. Hence, the main problem with some of the most fruitful hypernarrative models suggested by Kinder, Murray and Brooks concerns their inadvertent disregard for narrative dramatic causal succession, coherence and closure. In what follows I will outline some of the

30

Chapter 1: Hyper-Narratives

principles offered by cognitive constructivism to manage split attention in non-interactive hyper-narratives and deeply engage viewers experiencing hyper-narratives. These principles address the major hyper-narrative split-attention stumbling blocks: non-restriction of narrative threads, incoherent transitions within and between different narrative threads and non resolution of multi-threaded narratives. In what follows I will focus on the non-interactive hyper-narrative aspect of these problems. However, the principles outlined below will help address such problems as they pertain to modes of computer designed interaction with hyper-narratives, a subject dealt with in the following chapter.

1.4 Cognitive Constructivist Hyper-Narrative Theory David Bordwell who has most powerfully introduced the constructivist approach to film narrative has also broken ground towards the development of an audiovisual constructivist hyper-narrative theory. In “Film Futures” Bordwell outlines some of the narrative constructs that allow a film viewer to comprehensibly follow what he terms the forking path narrative, that is, hyper-narrative films that revolve around optional narrative threads that the same character can undertake and which lead him into different destinies.47 In this article by Bordwell we can find guiding principles to deal with the major hyper-narrative split-attention stumbling blocks: nonrestriction of narrative threads, incoherent transitions between different narrative threads and non resolution of multi-threaded narratives. 1.4.1 Hyper-narrative Option restriction Bordwell’s description of hyper-narrative films as forking path narratives (e.g., Run Lola Run, Tom Tykwer’s, 1998 or Sliding Doors, Peter Howitt’s, 1998) is borrowed from Borges’ story “The Garden of Forking Paths,” often mentioned by hyper-narrative theorists as one of its foundational stories. Bordwell starts by questioning the feasibility of constructing a film based on Borges’ suggested labyrinth story consisting of “an infinite series of times, a growing, dizzying web of divergent, convergent, and parallel times…that approach one another, fork, are snipped off…”.48

Chapter 1: Hyper-Narratives

31

He reaches the conclusion that while such story might be imagined it cannot be made into a film a viewer can follow and comprehend and that “In fiction, alternative futures seem pretty limited affairs.”49 I believe this claim is fundamental if we are to devise hyper-narratives that address humans rather than computers. In other words, while the calculative power of computers may generate endless forking story possibilities, and while the philosophical, social or scientific implications of such narrative idea are intriguing, attempts to actually devise such a labyrinth will divert the viewer’s or interactors’ attention towards memorizing and puzzle-solving cognitive activities. These activities, however, have little to do with the satisfying deep engagement offered by narrative. They are also needless given that games like chess offer a much better and more satisfying memory or puzzle solving alternative than the frustrating and confusing attempt to try and figure out endlessly branching stories. Finally, the different intriguing implications of such narrative idea can be fruitfully explored in a deeply cognitively and emotionally engaging way through the suggestion of its implications by a restricted, contained, cognitively manageable optional hyper-narrative film construction. Having suggested option restriction in hyper-narratives, Bordwell noticed that in hyper-narratives “narrative patterning obligingly highlights a single crucial incident and traces out its inevitable implications. Instead of each moment being equally pregnant with numerous futures, one becomes far more consequential than the others, and those consequences will follow strictly from it. Such linearity helps make these plots intelligible, yielding two or three stories that illustrate, literally, alternative but integral courses of events...”50 This means that for hyper-narratives to be comprehensible, coherence within narrative threads and between them must be maintained. Here again, Bordwell has done groundbreaking work in deciphering cohering strategies used by hyper-narrative films, some of which I wish to briefly elaborate upon in what follows. 1.4.2 Strategies for Devising Coherence between Narrative Threads Rather than the unfeasible possibility of shifting among narrative threads at many different points, a project bound to lead to gaming distraction or cognitive confusion, an engaging hyper-narrative needs to

32

Chapter 1: Hyper-Narratives

constrain the possibility of transition to crucial points. The best crucial points are those that are construed in such manner that they clearly evoke in the viewer his/her ‘what if she did that’ or ‘if only had he done that’ hypothetical conjectures. These are usually decision points ripe with moral, survival or emotional consequences (e.g., to shoot or not a threatening punk, to betray or not a loving husband, etc.). However, the immanent crucial decision-making embedded in such crucial points does not suffice in itself to turn such points into viewerengaging transitions between narrative threads. This engagement can only be achieved if there is a coherent and dramatic build up towards crucial points and a coherent and dramatic follow up after the crucial point has led to a transition to another narrative thread. Crucial points can be arrived at in the dramatic succession of events (evoking ‘what if’ conjectures) or returned to after having made the viewer retrospectively understand that in such points a decision inadvertently made was detrimental, or that a past event could have turned out differently (evoking ‘if only’ conjectures). An arrival at crucial transitional points will be disengaging to the viewer if they occur in the midst of a ‘wait and see’ expectation on his/her part, while creating such expectation will intensify his/her motivation to hypothesize when faced with a crucial decision (e.g., in Sliding Doors, once Helen misses the train with the potential lover driven away from her, the viewer adopts a ‘wait and see’ strategy expecting whether they will meet again despite her getting involved with a different man. This predisposes the viewer to hypothesize intensely once they do meet again and Helen has to decide what to do when her potential lover starts courting her, now that she is involved with someone else). Likewise, a viewer may be engaged and led to actively search in retrospect for crucial changing points if he/ she is made to understand that a hypothesis he/she raised concerning a favored narrative outcome turned out to be detrimental, thus raising in him/her an ‘if only had she done that’ derived hope or curiosity conjecture towards the past (e.g., in Sliding Doors the viewer wishes to return to the point where Helen missed the train, having witnessed the miserable destiny that missing the train and the potential lover has led her towards). Beyond the need for cohering trajectories leading to, or stemming from crucial transition points, offering radically different narrative threads following each transition and failing to meaningfully evoke interrelations between these threads will probably lead the viewer to

Chapter 1: Hyper-Narratives

33

loose sight or interest in whatever went on before or in what lies ahead. This is bound to engender distraction even if each narrative thread is cohering within its own trajectory, because such radically divergent narrative shifts, even if signposted as different and unrelated stories, frustrate the viewer’s strive for coherence triggered by their being held together in the hyper-narrative flow. However, if the viewer is allowed to construe meaningful narrative interrelations between different threads her engagement can be deepened because it allows her reflection upon overarching thematic or formal concerns. Therefore temporal, spatial, action and character recurrence, parallelism or variation between narrative threads should be encouraged. As Bordwell shows, hypernarrative films are rewarding because of these interrelations and intersections. Hence, Run Lola Run repeats in succession three slightly different options of the same story with the same characters and narrative evolution, whereas Sliding Doors runs two options in parallel forming intersections between the versions. 1.4.3 Strategies for Devising Dramatic Succession and Closure in Hyper-narratives Two of the major and interrelated engaging components in narrative evolution concern the cumulative effect achieved by a temporal dramatic succession of events that presupposes closure. These two components and their interrelation, clearly evident in narrative suspense may be lost when a film shifts between alternative or optional narrative threads that lead to different outcomes or temporal configurations. This is because, as Bordwell states “A narrative, in Meir Sternberg’s formulation, involves telling in time, and as a time-bound process, it calls upon a range of human psychological propensities. What comes earlier shapes our expectations about what follows. What comes later modifies our understanding of what went before; retrospection is often as important as prospection.”51 This leads Bordwell to suggest the setting of repeated temporal constrains, primacy and recency as guiding strategies to generate an engaging notion of succession and closure among different narrative threads. The setting of repeated temporal constrains refers to different narrative threads repeating or returning to shared crucial points that set time constrains. In Run Lola Run for example, the three narrative versions start with a return to the same phone-call by Lola’s

34

Chapter 1: Hyper-Narratives

boyfriend Manny, telling her he lost 100,000 DM that belong to a bad guy and leaving Lola 20 minutes to raise this amount for otherwise he will rob a store to get the money. Likewise, in Sliding Doors Helen misses the train in the first version but later the film returns again to this point, repeating her run towards the train, which she does board this time around. In both films, the repetitive variation upon the same temporal constrain sets a notion of causal and dramatic succession between the different narrative threads even if these are parallel or unrelated in their inner evolving time-frame. The psychological effect of primacy imparts saliency to the first impression and the viewer will frame to a large degree following changes according to it, whether this initial frame is contradicted, enforced or discarded with altogether in consequent threads. Hence, while hyper-narratives may offer different narrative beginnings, the viewer always encounters one of these beginnings first and this is an obligatory entrance into the hyper-narrative. The salience of this obligatory first framework can be exploited for imparting a sense of succession to following optional narrative threads by maintaining some of its components as invariables (e.g., character personality). Complementing primacy is the psychological effect of recency that imparts saliency to the last impression, turning it into the most plausible one among different alternatives. The viewer will retrospectively frame to a large degree previous threads according to it, whether they contradicted, enforced or discarded it. This makes clear that in hypernarratives the frustration film viewers experience when the closure is inconsistent with what went on before is more acute. An effective way to reinforce rather than diminish the sense of closure through the recency effect is by ‘returning’ to an event that happened earlier and was deemed meaningless at the time, but which in retrospect turns out to be a crucial decision point. This short-circuits intermediary narrative threads and helps lead all ending options to multi-consistent closures. Bordwell’s analysis of hyper-narrative films offers an antecedent for such closure device. Hence in his analysis of Sliding Doors, he shows how the film’s “epilogue folds back on its prologue” when the film reenacts a choice event that happened before Helen missed the train by a fraction and was therefore led to a miserable destiny. In this re-enacted choice event we are reminded that in a first encounter in an elevator James picks up for her an earring she dropped but Helen refuses his implied courting, whereas this time around she accepts his courting, turning her consequent missing of the train event irrelevant.

Chapter 1: Hyper-Narratives

35

While the hyper-narrative principles outlined above may engender deep engagement for viewers, disengagement and frustration can result when viewers turn into interactors that can behaviorally affect hypernarratives. The following chapter, drawing upon the principles outlined here, will address such problems as they pertain to modes of interacting with hyper-narratives.

1 2 3 4 5 6

7 8 9 10

11 12

13

Murray, Hamlet on the Holodeck; Brooks, Metalinear Cinematic Narrative; and Kinder, ‘Narrative Equivocations’. Brannigan, Narrative Comprehension, p. 3. Bordwell, Narration in the Fiction Film, p. 34. Brannigan, Narrative Comprehension, p. xii. Carroll, Mystifying Movies, pp. 138–147. Carroll suggests they do so through three basic strategies: Indexing – drawing attention to something important in the story, as in a close-up showing the spectator that the villain has secretly drawn a gun (why is he doing this?); Scaling – changing the relations between objects, as when the camera moves away to reveal that the villain is standing behind the Sheriff (will he shoot the Sheriff from the back?); and Bracketing – inserting material taken from another context, as when you insert the sound of an off-screen gunshot that kills the villain (who killed him?). In such manner films also frame the answers to the questions raised, as when a consequent shot answers the question ‘who killed the villain?’ by showing up-close the sheriff’s beloved wife holding a smoking gun, followed by a zoom–out re-scaling to reveal her standing on a rooftop overlooking the scene. On cognitive constructivism see in particular Jerome Bruner, ‘On Perceptual Readiness’, Psychological Review 64 (1957), pp. 123–152. Bordwell, Narration in the Fiction Film, p. 31. Ibid, pp. 29–47. Witness for example Robert Altman’s film Short Cuts (USA, 1993), which consists of several coincidentally interlacing narrative trajectories that nevertheless cohere within and in-between themselves. So too do many of the films discussed further on as hyper-narratives, which offer structural and/or multi-narrative coherence. Bordwell, Narration in the Fiction Film, pp. 39–40. Dolf Zillmann, ‘The Psychology of Suspense in Dramatic exposition,’ in P. Vorderer, H. J. Wulff and M. Friedrichsen (eds), Suspense: Conceptualizations, Theoretical Analyses and Empirical Explorations (Mahwah, NJ: Lawrence Erlbaum Associates, 1996), pp. 218–228. As Sobchack, Wolf, Darley and others have noted, narrative cinema, due to the computerization of its production processes, is experiencing a loss of deep engagement and is shifting from depth narrative to shallow spectacle. See Vivian Sobchack, Screening Space (2nd edn.) (New Brunswick, NJ: Rutgers University

36

14

15 16 17 18 19

20

21 22 23 24

25

26

27 28 29 30 31

Chapter 1: Hyper-Narratives Press, 1987); Mark Wolf, Abstracting Reality: Art, Communication, and Cognition in the Digital Age (New York: University Press of America, 2000); Andrew Darley, Visual Digital Culture: Surface Play and Spectacle in New Media Genres (London: Routledge, 2000). See in particular Jameson’s critique of postmodernism where he aptly metaphorically describes as ‘schizophrenic’ the postmodern episteme, referring to the deep sense of alienation and disorientation it forwards. Jaques Derrida, ‘The Law of Genre’, Critical Inquiry, 7.1, 1980. Julia Kristeva, ‘The Bounded Text’, in Desire in Language (New York: Columbia University Press, 1980), p. 111. Ibid, p. 36. Roland Barthes, Image, Music, Text (New York: Hill and Wang, 1978), p. 157. See a critique of this widespread poststructural attitude towards truth and objectivity in Noel Carroll ‘Nonfiction Film and Postmodernist Skepticism’, in David Bordwell and Noel Carroll (eds.) Post-Theory: Reconstructing Film Studies (Madison, Wisconsin: University of Wisconsin Press, 1996), pp.283–303. See also Nitzan Ben-Shaul, Film: The Key Concepts (Oxford: Berg, 2007), pp. 53–59. Kristeva for example, exchanges semiotic and linguistic categories with a typology of logical textual correlations (Kristeva, ‘The Bounded Text,’ pp. 38–40). Likewise, Tarantino’s film Pulp Fiction (1994) presupposes a temporal arrangement before it scrambles it. Mayer and Moreno ‘Aids to Computer-Based Multimedia Learning’. Chandler and Sweller ‘Cognitive Load Theory’. Jaques Derrida, Of Grammatology (Baltimore: John Hopkins University Press, 1976). This claim does not apply to ‘open-ended’ or multi-threaded narratives that are premised on inward, structural coherence and do offer cohering trajectories. The film Run Lola Run (Tom Tykwer, 1999) offers an exemplary case of such structural coherence. This notion concerns the narrative aspect of moving audio-visual texts. These texts can also engage through affective or sensual attachments, but these are usually shortlived attractions if unsupported by narrative. Manovich, who refuses to consider computer-based narratives in the sense outlined here, is therefore led to predict short attachment/attraction-based works. Related to this is also the sensation aesthetic model for new media suggested by Darley and predicated on shallow spectacle on account of narrative depth. See Darley, Visual Digital Culture. For an engaging film approach to the relativization of truth see Kurosawa’s Rashumon (1950). For an engaging film approach to the consequences of wrongly held presumptions see Coppola’s The Conversation (1974). On Antonioni’s alienating cinematic strategies see Nurith Gertz, Motion Fiction (Tel Aviv: Open University, 1993), pp. 159–163. Manovich, Language of New Media, pp. 239–43; 293–296. Lunenfeld, ‘The Myths of Interactive Cinema’. On the phenomenon of user’s text appropriation see Henry Jenkins, Textual Poachers (New York: Routledge, 1992). Manovich, Language of New Media, pp. 18–61. Lunenfeld, The Myths of Interactive Cinema,’ p. 49.

Chapter 1: Hyper-Narratives

37

32 See a similar view in Jay David Bolter and Richard Grusin, Remediation: Understanding New Media (Cambridge, MA: MIT Press, 1999), p. 144. 33 Manovich, The Language of New Media, pp. 227–8. 34 Ibid, 242–43. 35 For a comprehensive account of Vertov’s film see: Annette Michelson, ‘Introduction’, in Annette Michelson (ed.), Kino Eye, The Writings of Dziga Vertov (Berkeley: California University Press, 1984). 36 Manovich, The Language of New Media, p. 143. 37 See on this John Tulloch and Henry Jenkins (eds.) Science Fiction Audiences: Watching Doctor Who and Star Trek (London: Routledge, 1995). 38 Kinder, ‘Narrative Equivocations’, p. 127. 39 Ibid, p. 126. 40 Ibid, p. 127. 41 Ibid, 128. 42 For a good critical discussion of media determinism see Raymond Williams, Television: Technology and Cultural Form (New York: Schoken Books, 1974). 43 On our cognitive ability to manipulate arbitrary signs, see Jean Piaget and B. Inhelder, The Psychology of the Child (New York: Basic Books, 1969). 44 Walter Benjamin, ‘The Work of Art in the Age of Mechanical Reproduction’, in Hanna Arendt (ed.), Illuminations (New York: Shocken Books, 1969), pp. 217–252. 45 ER (NBC, 1994– ), Seinfeld (Sony Pictures TV, 1990–99), 24 (20th Century Fox Home entertainment, 2002– ). 46 Bordwell, Narration in the Fiction Film; Brannigan, Narrative Comprehension and Film; Carroll, Mystifying Movies. 47 Bordwell, ‘Film Futures’. 48 Ibid, p. 88. 49 Ibid, p. 89. 50 Ibid, p. 92. 51 Ibid, pp. 97–8.

Chapter 2: Interaction There are three major widespread and unwarranted assumptions concerning interaction with hyper-narrative films that block attempts at designing engaging works. First, the presumption is made that an interactor can somehow be placed as a leading film protagonist, a notion often based upon the film viewers’ presumed ‘identification’ with film protagonists. Second, it is widely presumed that a seamless transition can be made from a film viewer’s cognitive activity to an interactor’s active behavioral interaction. Third, it is presumed that computer game interactivity can be conflated with hyper-narrative interactivity. In what follows, the problem with each of these assumptions is discussed and a feasible approach to generate engaging interactive hyper-narratives is offered.

2.1 The Interactor as Film Protagonist Fallacy There is a widespread inadvertent presumption that an interactor can somehow be placed within the hyper-narrative as a leading protagonist is placed in a film. This is evident in Kinder’s analysis of what she terms (following Manovich), data-based narratives, referring to films that include hyper-stories structured by a game design. Since her analyses of these films is taken as an example for potential interactive choices allowed by hyper-narratives, Kinder’s constant reference to the choices open to the protagonists within the films imply that the interactor, just like the film protagonist (and the film viewer ‘identifying’ with him/ her) will be similarly involved in these games. Such conflation of film viewer/film protagonist/interactor can be seen in her already mentioned analysis of Agnes Varda’s film Cleo from 5 to 7. Hence, she writes: “Although Cleo at first seems as stereotypical as the avatars in the tarot deck, once she moves into the open narrative field of the city…the avatar becomes humanized and the temporal countdown a reminder that we are documenting her movements in ‘real time’.”1 In this seamless

40

Chapter 2: Interaction

slipping from avatar to protagonist to viewer she insinuates the three can be conceived as one. A clearer evidence of the inadvertent conflation between protagonist and avatar can be found in the loosely designed experiment conducted by Joseph Bates as part of his Oz project. The goal of the experiment was to find out whether an interactor can be made meaningfully engaged in a computer generated dramatic story that allows the interactor to partly intervene in a pre-determined story evolution. Bates and his team scripted a story where a passenger arrives at a bus station followed by the arrival of a blind man and a young punk who starts threatening the blind man. As the threats to the blind man escalate, the passenger is handed a gun by a distressed cashier at the same time that the passenger’s bus arrives. The passenger is led to face the moral dilemma of whether to shoot the punk who has in the meantime drawn a knife at the blind man or hop on the bus and flee the scene. Bates and his team enacted the story on a theatre stage at the University and asked a volunteer to act the role of the passenger without telling him what the story was about. The volunteer was only given the simple goal of purchasing a bus ticket. Three student actors playing the roles of punk, blind man and cashier where directed through headsets to improvise the story situation so that they eventually lead the volunteer playing the passenger to end up with a gun in hand facing the threatening punk. According to Bates’ report, they ran the story twice with two different volunteers and in both cases the volunteer was morally engaged in the situation: one “shot” in the air since he felt he couldn’t shoot the punk and the other left the stage in distress since she was unwilling to take the gun. The aim of Bates’ live experiment was to “consider the presentation by computers of rich, interactive worlds, inhabited by dynamic characters...” in order to see “how does it feel to be immersed in a dramatic virtual world?”2 However, while in the experiment the interactor is the major protagonist to whom things happen, Bates’ presumption that the theatrical improvisation of a dramatic situation figuring a real person can be seamlessly translated as is to a computer generated environment evidences a groundless belief in the possibility to conflate film viewer/ film protagonist and intertactor. 3 The types of interactor figuration presumed in Bates’ experiment and widely used in computer games are either a first-person fictional camera point of view (P.O.V.) as in the first-person shooter computer game Doom (Id Software, 1993) or a totally manipulated avatar representing

Chapter 2: Interaction

41

the interactor in the virtual fiction and responding to his/her command. However, the presumption that such figurations can lead the interactor to experience the situation as if he/she were in-themselves involved in a real simulation is groundless, if only because any such figuration is drastically reductive of the faculties employed by a living person facing a real environment (even a faked theatre stage). Such line of thought, conflating the avatar or the game’s constructed P.O.V. with the interactor, is bound to generate a shallow gaming attitude precisely because it is constantly evident to the interactor that his commanded P.O.V. or avatar are always-already a poor misrepresentation of herself. In fact, the interactor does not consider the game’s commanded P.O.V. or the avatar as representing him, but as a manufactured robotic surrogate obeying the commands it can perform. Through such a type of shallow interaction with an ‘other’ it is impossible to generate deep engagement. I suspect that such unwarranted notions are a legacy from the widespread mistaken presumption in film studies that film viewers are effectively manipulated in films to ‘identify’ with the film camera or with film protagonists. According to this widespread notion, viewers’ identification with the camera leads them to inadvertently believe that not only what they are watching is real but that they are actually the ones looking or even creating it. This identification fallacy recurs also in the presumption that film viewers identify with a film’s protagonist, a concept used to explain why viewers are emotionally involved with characters and implying that this is so because viewers are both predisposed and manipulated by the film into thinking or feeling that they are the characters. Coleridge’s popular phrase about a ‘willing suspension of disbelief,’ or the more elaborated psychoanalyticideological concept of a viewer’s subjection by film are usually brought forth to explain this irrational behavior of viewers. A very elaborated film theory of such manipulation has been developed to explain for example, how is it that viewers remain under such a film spell when it cuts from one point of view shot to a reaction shot, since such transition may awaken the viewers to the fact that the film is not a slice of reality they are experiencing or authoring and that they may not be the characters portrayed, but that the film and protagonists are actually constructed artifacts. 4 Carroll rejected these ideas concerning the viewers’ identification with the film camera’s point of view resulting in viewers believing that

42

Chapter 2: Interaction

what they watch is real or, in more sophisticated theories (which are just as false), that they actually author the movie. How, asked Carroll simply, can spectators who presume they author the film due to their identification with the camera, find themselves in a state of suspense and guessing about future developments?5 In particular, cognitivists rejected the immersion related notion of identification to explain why viewers react emotionally to film protagonists. Explaining that identification in fact means that viewers are manipulated into the irrational belief that they are the protagonists, they asked how can it be that viewers find themselves in suspense when they know the protagonist is in danger, and yet the protagonist with whom they presumably identify, has no knowledge of the danger he/she are in and is therefore not in suspense.6 Carroll suggested instead that the viewers’ emotional response to protagonists derives from their entertaining propositions of the if this was to happen type towards morally favored protagonists, which can lead to emotional responses without the need to believe that what is unfolding before their eyes is really happening to them.7 It also explains why the volunteers in Bates’ live simulation were engaged in the moral dilemma they were led to face. Hence, it seems more reasonable that what particularly engaged the volunteers in the live simulation was not their belief that they were actually involved in a real moral dilemma, but that the character they portrayed and morally favored (a person about to travel home and minding his own business) was involved in such situation, leading them to raise ‘what if’ hypotheses concerning how their character should react to the punk’s behavior rather than believing in the actual possibility of shooting to death the actor facing them. Dolf Zillmann complements Carroll’s proposition by proposing the theory of ‘empathic mediation of involvement’8 of viewers with fictional characters, a theory that offers a much more rational explanation for the viewer’s emotional and cognitive response to protagonists than the irrational notion of identification. The concept of empathy explains away identification theories’ obliviousness to the obvious difference between viewer and protagonist (or even between actor and the character he/she portrays). It also explains how with fictional characters as with friends we are empathic towards we hypothetically fill into their always partial exposure to us, motivational or emotional attributes. This hypothetical procedure opens the possibility for character twists. This character or friend partial exposure, offers another explanation for why identification theories, beyond their irrationality, are unfeasible.

Chapter 2: Interaction

43

This is because our acquaintance with our own hidden motives and emotions will always exceed our acquaintance with those of a friend or a fictional character, implying by definition that identification is bound to fail. Finally, empathy can not only better explain emotional or cognitive engagement with fictional characters, but also innate reflexive empathy of viewers to events happening to characters such as when we ‘hit’ an imaginary break when a character’s car is about to dive into an abyss.9 Rather than falling into the trap laid out by ‘identification’ theories, I suggest we view avatars not as representing the interactor but as the interactors’ fictional friend towards whom she develops empathy. This allows the exploitation of the in-built divide between avatar and interactor to enhance a complex engagement, rather than reduce it to shallow character self-personification by poor acting faculties as evident in Bates’ experiment.10 Thus, as with a friend, we cannot nor (usually) want to have total control upon his decisions and actions, we often expect our friend to tell us of places and experiences we have not experienced, and we know that whatever we suggest to a friend, he may take into consideration and accept or reject it. Furthermore, establishing such interaction opens up the hyper-narrative camera to all the different points of view that an interactor-robot surrogate P.O.V. blocks.11 Positioning the interactor as friend, counselor or even an empathic boss or parent to an independently and hyper-narrativepropelled protagonist, opens up a rich venue for interaction. Hence, such a protagonist may only occasionally respond to our behavioral interventions, may sometimes act against our counseling and be proven right or wrong, and when abiding to our suggestion this friend may find herself in deeper trouble than what we as interactors had hoped for. Hence, while as will be discussed below, Bates’ experiment can be instructive insofar as interactive hyper-narratives are concerned, it could turn out to be so if instead of the narrative revolving around the robotic reduced representation of the interactor’s P.O.V. or avatar, it revolves around a character or protagonist towards whom the interactor is emphatic.

2.2 From Film Viewer to Hyper-Narrative Film Interactor In general, the expectation of interactive film researchers and tool developers that active behavioral engagement through interaction will

44

Chapter 2: Interaction

enhance the user’s sustained attention and deepen his experience, have so far been frustrated. Cognitive constructivist narrative film theory, as Kinder and others have presumed, seems to support the view that interaction will enhance deep attention. This is because it presumes an active viewer engaged in cognitive processes to produce a film’s meaning. This does not mean, however, that by making viewers behaviorally active their attention will be enhanced, or the meaning they construct will deepen. In fact, combining active behavioral interaction with narrative cognitive constructiveness, demands multitasking that may generate split attention overload.12 This split attention may result from the interactors constant positioning in in-between what they cognitively construct from what is going on in front of them, and their awareness of what may potentially lie at stake in options made available to them through behaviorally changing the course of events. This last point refers to one of the major problems facing the author of an interactive hyper-narrative. It concerns the degree of agency allowed an interactor in such works. While interactive hyper-narratives carry the promise of enhanced engagement due to the interactor’s possibility of intervention in, and change of the narrative course, such promise may not materialize if interaction is not carefully designed. Some hyper-narrative postmodern theorists often confound authoring narratives with narrative interaction when suggesting that the widespread activity of text-poaching by fans is a new form of authoring. However, a look at such works quickly reveals that so far such fan activity reduces both narrative authoring and narrative interaction to shallow social gaming. I believe such meager results stem from the fact that interactive changes to the evolution of a story that are predicated on the suspense, surprise, curiosity or comic effect that such change may bring about are redundant for narrative evolution and offer by definition a poorer alternative if not carefully designed. This is because one of the reasons for which narrative evolution deeply engages the interest of its audience is precisely its offering of unknown and often unexpected narrative twists. In fact, when you let the interactor himself change the course of events ‘at will’, you run the risk of turning the unexpected that so intrigues one in narratives, because of their inner logic and cumulative dramatic succession, into an arbitrary change that offers poor and non-cohering changes. Anyway, beyond restless distraction inhering in the possibility of change at will, why should the interactor care to change ‘at will’ if unpredicted change or the

Chapter 2: Interaction

45

entertainment of ‘what if’ possibilities are what fascinates him/her in non-interactive narratives to begin with? Moreover, narrative change, when pre-meditated by the author, deeply engages the viewer, whereas if change is given into the hands of the interactor or the ‘poacher’, who cannot himself devise complex cohering trajectories, whether for lack of talent, knowledge, experience or time, the result will most certainly be shallow, distracting and game-like. Nevertheless, not every authorial pre-meditated narrative change leads to deep interactor engagement. Authors may themselves generate incoherent and distracting interactive constructs such as Immemory (see its description above) or design multi-narrative or other interactive cinematic constructs that generate more or less coherent narrative transitions but are nevertheless game-like and distracting. This is mostly so because the designed interaction does not effectively regulate the opposing lures of interactive intervention and the cognitive construction of narrative trajectories.13

2.3 From Computer Game Player to Hyper-Narrative Film Interactor Several researchers have tried to solve the problem of managing the opposing lures of interactive intervention and the cognitive construction of narrative trajectories by combining computer game behavioral interactivity with narrative cognitive construction. This line of thought is based upon the notion that since computer-game interactivity is engaging then this must mean that it can be used to enhance interactive hyper-narrative engagement and vise versa. Marsha Kinder for example claims that “At this point in history, any simplistic distinction between active game players and passive movie spectators would be naïve.”14 Refusing to see games and narratives as opposing binaries she suggests that these “can be treated more productively as a continuum.”15 Manovich also has suggested such conflation when claiming that “narratives and games are similar in that the user must uncover their underlying logic while proceeding through them – their algorithm. Just like the game player, the reader of a novel gradually reconstructs the algorithm (here I use the term metaphorically) that the writer used to create the settings, the characters, and the events.”16

46

Chapter 2: Interaction

I contend that such conflation, however, not only reduces the lure of both games and narratives but enhances split attention stemming from multi-tasking. Hence, while it is possible to somehow describe the major action based computer game lure of behavioral skill acquisition so as to survive competition and score higher (e.g., Doom) as narrative tropes, a lot of attention to these activities in interactive hyper-narratives will surely divert attention away from the major narrative lures of aesthetic appreciation and the reflective cognitive constructing of the audiovisual flow and dramatic ambiguity into cohering narrative trajectories. Conversely, when acquiring skills in action based computer games, users tend to cancel their attention to the game’s aesthetic and narrative until they have behaviorally mastered whatever device they have been given to interact with and can easily tackle whatever threatens them. This can be evidenced in computer games where the behaviorally passive consumption of compulsory narrative passages in between action-packed interactions are usually perceived by gamers as boring transitional pieces. In fact, if gamers pay attention in such games to narrative lures while busy surviving, their interest or chance to stay in the game drops. In a way, behavioral gaming interaction seems secondary or even intrusive for hyper-narratives whereas narrative successions are secondary and intrusive for computer games. Audiovisual immersion theory has often been drafted to find some sort of mediation between the lures of gaming interactivity and the lures of hyper-narrative cognitive construction. Henry Jenkins for example, in arguing against game designers’ claims that games and narrative are in conflict mostly because interactivity counters narrative engagement and vice-versa, suggests to “introduce an important third term into this discussion – spatiality – and argue for an understanding of game designers less as storytellers and more as narrative architects.”17 Jenkins suggests that games can incorporate narrative through what Don Carson has described following his analysis of Disney’s theme park attractions as environmental storytelling where The story element is infused into the physical space a guest walks or rides through....Armed only with their own knowledge of the world, and those visions collected from movies and books…the amusement park attraction doesn’t so much reproduce the story of a literary work… as it evokes its atmosphere…If, for example, the attraction centers

Chapter 2: Interaction

47

around pirates, Carson writes, “every texture you use, every sound you play, every turn in the road should reinforce the concept of pirates,” while any contradictory element may shatter the sense of immersion into this narrative universe.18

Beyond evoking for the interactor ‘narrative associations’ through such environmental storytelling, placing the gaming activity within such spatiality allows according to Jenkins the incorporation of further narrative attributes. These include “a staging ground where narrative events are enacted”, as when the interactor picks up a lightsabre “and dispatch Darth Maul in the case of a Star Wars game”; “the embedment of narrative information within the mise-en-scene’ where the interactor searches for, or fights his/her way to discover enclosed episodes or pieces of narrative information thus constructing the story out of the spread narrative streams or segments of information; or ‘provide resources for emergent narratives”, meaning that, as in the Sims game (Will Wright, 2000), the characters have certain prescribed possibilities out of which an interactor can forge a narrative segment. The major fallacy in Jenkins’ proposal is his presumption that audiovisual spatiality can render narrative coherence to a series of discreet, disjointed and embryonic narrative segments and thereby generate deep engagement. Jenkins position stems from a miscomprehension of the major engaging faculty of film narrative, a faculty that has little to do with spatiality. Hence, while audiovisual space may give materiality and tangibility to a narrative, what engages us is primarily the sustained dramatic inner logic embedded in the temporal audiovisual flow towards closure characterizing narrative films. Jenkins propositions imply an irretrievable loss of the sustained engagement offered by narrative films with no comparable alternative. What Jenkins offers at most is the short-term engagement typical of attraction or the long term shallow engagement of puzzle-solving. That is why Jenkins favorably mentions Eisenstein’s notion of attraction referring to “any element within a work which produces a profound emotional impact and [Eisenstein] theorized that the themes of the work could be communicated across and through these discrete elements.”19 However, a close reading of Eisenstein’s theorizing of attractions, reveals that he was particularly concerned with “scrutinizing everything for its value as a transitional, clarifying or factual expression of the basic intentional line of the staging, rather than for its ties to logical [e.g., computer

48

Chapter 2: Interaction

games’ puzzle-solving process], natural [e.g., immersive spatiality] and literary traditional piety.”20 Hence, while Eisenstein’s notion of attraction as that which “brings to light in the spectator those senses or that psychology that influence his experience – every element that can be verified and mathematically calculated to produce certain emotional shocks in a proper order within the totality…”21 may be useful to engage interactors, their discreetness, segmentation and intermittent placement in an expansive spatiality leads to a jumpy, incoherent series of emotional reactions. What Jenkins actually professes is a return to the early cinematic paradigm of discontinuity + spatial support. This is evident in his favorable mention in this respect of the early slapstick comedies that were based on a disconnected chain of gags. Likewise, when Jenkins lauds the films of Bruce Lee or the early musicals that used the pretext of a poorly engaging thin story line representing ‘life’ to loosely chain spectacularly choreographed martial arts segments or dance and song performances, he is lauding the failure of such films to offer cohering narrative trajectories. Indeed, Jenkins’ notion of spatiality foregoes Eisenstein’s dynamically shifting narrative oriented film space, which allowed Eisenstein, through his ‘dialectic montage’ constructs, the double axis dynamization of spectator and story. He foregoes the successful evolution of cinematic narrative towards the fertile use of spectacular performances to forward the evolving narrative rather than using it as an excuse for stunning spectacles. This can be evidenced in the evolution of musicals towards a ‘life is music’ flow as in Chicago (Rob Marshall, 2002) or Dreamgirls (Bill Condon, 2006) and the engaging incorporation into the narrative flow of martial arts virtuosity in films such as Crouching Tiger, Hidden Dragon (Ang Lee, 2000), The Matrix (Andy & Larry Wachowski, 1999) and even Kill Bill: Vol. 1 (Quentin Tarantino, 2003). Jenkins’ giving up upon a goal-oriented successive causal temporal narrative trajectory, results in a series of discreet ‘attractive’ scenes that when planted in a stylishly coherent ‘immersive’ spatial expanse, confine our attention into viewing them as ‘enlivening’ atmospheric variations upon a synchronic spatiality (as do the theme park attractions that inform Jenkin’s notion of game spatiality). Such conflation of discreet micro-temporal narrative segments and expansive spatiality results in shallow interactor engagement due to the lack of satisfying and complex diachronic development on the one hand, and the resulting enhanced interactor awareness of spatial redundancy on the other. Hence, while

Chapter 2: Interaction

49

it is questionable whether micro-narratives ‘enliven games’,22 what is clear is that these interspersed segments, even if planted in a coherent navigable spatiality, cannot deeply engage us in the narrative destiny of complex characters as narrative films do, irrespective of whether they logically add or do not add up to a resolvable puzzle. This is because what sustains our engagement and attention in narratives is a dramatic succession underlined by causal logic and a sense of moving towards a goal rather than spatial immersion.23 In sum, introducing complex narratives into computer games or complex behavioral skill acquisition into hyper-narratives is bound to generate split attention since the interactor is split between mastering the skills needed to perform effective interventions and his/her cognitive construction of the narrative trajectories. One way to master such type of multi-tasking split attention in games and hyper-narratives can be achieved by automating one task. Hence, as mentioned above, in action based computer games, narratives are mere background excuses to which users cancel their attention until they have behaviorally mastered whatever device they are given to interact with and can easily tackle whatever threatens them. Only once these skills are mastered or automated, attention can shift to the non-essential for survival elements in the game. Usually, however, once skills have been mastered in such games the users somewhat lose interest, since these computer games are mostly about mastering skills and scoring as high as possible. This is perhaps why action-based computer games pay little attention to narrative.24 It should be noted that this reductive and nonessential function of narrative for action based computer games is also true of puzzle-type computer games. While games like Myst (Robin and Rand Miller, 1993) try through spatial exploration, deciphering of clues and so forth to turn the very construction of embedded story fragments by the interactor into the game’s aim, this has little to do with narrative engagement. This is because such games are based on intermittent cognitive puzzle-solving type activity and not upon the deep cumulative emotional engagement engendered by the dramatic successive flow offered by narratives. Similarly, an automation model can be suggested for interactive hyper-narratives. In this case what will be automated is skill acquisition as when you automatically drive your car and release attention for other activities. Benjamin for example, who proposed to view films as tactile shocking attacks upon their viewers’ perceptions, suggested that the

50

Chapter 2: Interaction

avoidance of a shocking response could be achieved by the development of habits that could train viewers towards the tasks awaiting them in real life attacks.25 Benjamin’s conception of film, however, more befitting a widespread conception of games, is redundant for hyper-narratives. This is because if the main aspect of film consists in shocking effects (i.e., ‘tactile interactions’), then, as in games, once habits have been developed and shocks managed all other aspects of the film are of minor value. On the other hand, if the aim of interaction is that it won’t be noticed through automation so that the interactor can become a narrative film viewer under non-optimal conditions, why bother with interaction? In other words, when you automate driving to enjoy the scenery or ruminate on your life, the car still gets you somewhere and the scenery changes in relation to your movement. However, what is the point in automating behavioral interaction with hyper-narratives? It therefore seems that conflating hyper-narratives and computer games is counterproductive. This is because while computer games are concerned with the behavioral mastery of rules and skills or with intermittent puzzle-solving, narratives are ambiguous, dramatically successive and mostly require the reflective, emotional-cognitive cohering of audiovisual flows.26 This does not mean, however, that interactivity and narrative need contradict each other as some have suggested. It also does not mean that they must be redundant if multi-tasking split attention is to be managed. It only means that gaming interactivity that is focused on the acquisition of skills and game-survival (e.g., Doom), or with the puzzletype reconstruction of a story out of scattered narrative segments (e.g. Myst), conflicts with deep hyper-narrative engagement. It seems therefore that interactive hyper-narratives should pay little attention to skill acquisition for that enhances a shallow gaming attitude, avoid automating interaction for that enhances redundancy, and refrain from offering intermittent gaming transitions between behavioral interaction and cognitive construction, for that arrests the engagement that dramatic narrative succession engenders. The question is then, what form of interaction is needed in order to generate deep hyper-narrative interactive engagement? The answer to this question must stem from the strategies through which narratives emotionally and cognitively engage people (as opposed to games).

Chapter 2: Interaction

51

I propose that only behavioral interactivity that stems from, and enhances narrative’s specific engagement can be engaging to hypernarrative interactors. Therefore, behavioral interaction should be understood to be another devise aimed at the mutual play of narrative digressions with viewer cognitive-reconstruction. When incorporated in such manner, behavioral intervention can enhance the interactors’ engagement and deepen their experience.

2.4 A Cognitive-Constructivist Model for Hyper-narrative Interaction Given the opposing lures of behavioral interaction and cognitive construction discussed above, the problems we encountered in noninteractive hyper-narratives get complicated. These added stumbling blocks complicate the issue of dramatic narrative succession, the use of various optional narrative threads, changes within a given narrative thread and between different narrative threads, and the prospects for offering resolution to multi-threaded narratives. Nonetheless, the principles we found when dealing with noninteractive hyper-narrative films – dramatic succession, narrative thread restriction, cohering narrative threads towards crucial narrative transition points, maintenance of narrative succession appearance between narrative threads through recurrence, parallelism or variation, and leading hyper-narratives to closure through saliency and recency, should also guide the use of computer faculties when designing behavioral interactions with hyper-narratives. 2.4.1 Dramatic Succession The main objective in designing interaction with hyper-narratives concerns turning the opposing lures of behavioral interaction and cognitive construction into complementary lures that reinforce rather than dismantle the interactor’s engagement. Such complementary engagement can be achieved once behavioral interaction is allowed, required or blocked when it is clearly consonant with the moments in which interactors (rather than protagonists) are cognitively lured by the dramatic narrative succession to want to change the course of events rather than await what’s ahead.

52

Chapter 2: Interaction

Therefore I suggest that the interactor be engaged in one of the following ways: a. The interactor would not want to intervene and will not be meaningfully allowed to intervene (narratively evoked through dramatic ambiguous expectation leading to, or returning to crucial dramatic points). b. The interactor will be compelled and allowed to intervene in such manner that he/she would want (and will have) to quit the intervention as soon as possible (narratively evoked through crucial dramatic turning points generating expectations as to the results of the intervention). c. The interactor would meaningfully want to intervene but will not be allowed to do so (narratively evoked through detrimental consequences causally following the interactor’s previous intervention and which evoke his/her helplessness since blocked from behaviorally intervening).27 Such interactor engagement can be exemplified through comparing it to a parent interested in having her child wander freely while taking the minimal intervention measures to help the child avoid hurting herself. This implies engaging cognitive-emotional suspense and willingness to not intervene through ambiguity and expectation (option a: “my child walks so proudly, but will she fall?”); through behavioral interactive engagement when crucial situations and decisions about to occur or about to be taken are clearly detrimental in the interactor’s mind, thus gearing intervention towards narrative succession (option b: “I will push her a bit so that she does not fall and hurt herself”); and intense ambiguous suspense evoking helplessness when willing to intervene but not allowed to do so (option c: “I thought I helped her but after my push she is falling to the other side, why can’t I help her now? what will happen to her now?”).28 2.4.2 Narrative Thread Restriction The short-memory load that hyper-narratives impose and which lead to the restriction of possible threads in hyper-narrative films is further justified when behavioral intervention and the multi-tasking it involves enters the picture. Hence, while computer calculative power lures hyper-narrative authors to design many intersecting narrative threads, this built-in computer potential should be checked if comprehensive

Chapter 2: Interaction

53

and narrative engaging works are to be produced. While computer calculative powers are rewarding when devising the unbeatable Deep Blue (IBM, 1997) master chess player, are functional in deciphering the DNA, and may even serve narratologists in tracing statistical recurrences and variations among texts, the use of its calculative powers to devise endless intersecting and interactive narratives is bound to lead to interactor cognitive confusion and distraction. Although the computer may serve as an efficient tool for hyper-narrative scriptwriters to keep track of their evolving stories, or suggest to them un-thought of possibilities,29 the process of authoring hyper-textual narratives is quite different from that of interacting with hyper-narratives. 2.4.3 Strategies for Devising Coherence between Narrative Threads Interactive hyper-narratives presuppose the possibility of letting the interactor intervene within an ongoing narrative thread without loosing by that the inner causal logic and sense of closure of the dramatic succession he is engaged with (see 2.4.1. option a). Bates’ loosely designed experiment in interactive drama suggests productive venues for the designing of such interaction once we ‘move’ the interactor from his position as leading protagonist as suggested by Bates and place her instead as advisor, friend or active witness to the events undergone by a fictional protagonist she is emphatic towards.30 As mentioned, the goal of Bates’ experiment was to find out whether an interactor can be made meaningfully engaged in a computer generated focused dramatic story that allows her to partly intervene in a pre-determined story evolution. As Bates reported, the interactors in the experiment felt a deep sense of ‘dramatic presence’ that caused their emotional, cognitive and behavioral engagement. This sense of ‘dramatic presence’ was mostly due to the interactors’ behavioral active help to coherently construct the loosely predetermined narrative, and to the meaningful reactions of the characters that manipulated him/her into the major turning points of the story. This active narrative helpfulness, which supports the cognitive constructivist approach, is evidenced in the interactors’ report that the inconsistency of set and even of characters, once their behavior could be feasibly explained away, did not divert them away from their engagement. This suggests that while a film viewer’s cognitive activity needs strict causal clues to construe the film narrative in her mind, adding

54

Chapter 2: Interaction

behavioral interaction enhances the interactor’s willingness to support the narrative construction by expanding the repertoire of schemes he is willing to attribute to character or environmental behavior, so long as the inner logic and dramatic succession are maintained. Bates’ experiment suggests then that it is possible to construct a semi-determined narrative thread allowing for an interactor’s enhanced deep engagement through intervention. Another important point raised by the interactors in Bates’ experiment was that the moral decision faced by the interactors of whether to shoot the punk or not (see section 2.1. above) was effective only because there was a cumulative interactor ‘wait and see’ narrative suspense that led to the decision point of shooting or not shooting the punk. This suggests that the placement of such turning points and their enhancing ‘what if’ (or ‘if only’) propositions are the best construed dramatic turning points at which a behaviorally activated transition between narrative threads can be made available to interactors (see above in 2.4.1, option b). Hence, as discussed in non-interactive hyper-narratives, rather than the unfeasible possibility of behaviorally shifting among narrative threads at many different points, a project bound to lead to gaming distraction or cognitive confusion, an engaging interactive hyper-narrative needs to constrain the possibility of transition to crucial turning points and design a coherent and dramatic build up towards them, as well as a coherent and dramatic follow up to the transition to another narrative thread. The importance of a dramatically engaging follow-up to a transition point is particularly important in interactive hyper-narratives in order to re-engage the interactor after his behavioral intervention, since it causally plays out the consequences of the interactor’s intervention. A particularly emotional-cognitive re-engaging strategy would be one that eventually leads, contrary to the interactor’s intention, to detrimental consequences for the protagonist he is emphatic towards. This may lead the interactor to want to change again the course of the narrative. Blocking the behavioral interactive option at this stage while generating expectation that despite a most probable disaster something might reverse the results will generate intense ‘wait and see’ suspenseful expectation due to a feeling of helplessness (see above in 2.4.1, option c). Having resolved this trajectory, another crucial turning point can be arrived at, compelling and allowing the interactor to behaviorally intervene again (see above in 2.4.1, backing to option a).

Chapter 2: Interaction

55

As already mentioned in our discussion of non-interactive hypernarratives, beyond the need for cohering trajectories leading to, or stemming from crucial transition points, offering radically different narrative threads following each transition and failing to meaningfully evoke interrelations between these threads will probably lead the interactor to loose sight or interest in whatever went on before, or in what lies ahead. Therefore temporal, spatial, action and character recurrence, parallelism or variation between narrative threads should be encouraged. Offering temporal, spatial, action and character parallels among two or three slightly divergent options also allows for many engaging interactor behavioral transitions, due to the many potentially cohering transitions among the optional narrative threads. This would be the case for example if the hyper-narrative film Run Lola Run were turned interactive so that the three slightly different options appearing in succession in the film are offered simultaneously to the interactor, allowing for many cohering transitions between the different versions. 2.4.4 Strategies for Devising Dramatic Succession and Closure in Interactive Hyper-narratives The threat of hyper-narratives to the cumulative effect achieved by film narrative’s temporal dramatic succession of events that presupposes closure, is enhanced when interactivity is added to hyper-narratives because interactivity brings simultaneity rather than evolution to the fore. Thus the strategies of repeating temporal constraints, of primacy and recency turn out to be particularly important in generating an engaging notion of narrative succession and closure (see chapter 1, section 1.4.3). Hence, while interactive hyper-narratives may offer different beginnings, the interactor should always encounter one of these beginnings first and this should be an obligatory entrance into the hyper-narrative, allowing the use of its primacy effect for consolidating attributes that recur in following options. Also, a repetitive variation upon the same temporal constraint in different narrative threads can set a notion of causal and dramatic succession between the different narrative threads even if these are parallel or unrelated in their inner evolving time-frame. Finally, in interactive hyper-narratives the frustration film viewers experience when the closure is inconsistent with what went on before is more acute. A most effective way to

56

Chapter 2: Interaction

reinforce rather than diminish the sense of closure through the recency effect is by constraining the amount of possible concluding narrative threads and coherently lead within each to a ‘return’ to an identical same choice event that happened in the opening sequence and was deemed meaningless at the time, but which in retrospect turns out to be crucial. This short-circuits intermediary narrative threads and helps lead all ending options to multi-consistent closures. Through the strategies delineated above, the potential distraction engendered by intermittent interaction as well as the split attention potential inhering in multi-tasking of simultaneous cognitive construction and behavioral interaction is not only managed but made to enhance engagement. Disengagement and frustration can result, however, from the interactor’s mode of interfacing with the hypernarrative and from non-cohering audiovisual configurations. These two issues are the subject of our next chapter.

1 2 3

Kinder, ‘Narrative Equivocations’, p. 128. Bates, ‘Virtual Reality, Art, and Entertainment,’ p. 134. Another one of Bates’ fantasies is that computer generated characters will be able to improvise in coordination complex believable reactions to the interactors’ actions. It is unfeasible that in any foreseen future computer generated characters will be able to do that. Such improvisation however, is reduced to a minimum if the protagonist, as is suggested further on, is not the interactor but a pre-scripted protagonist that reacts to pre-scripted events and characters while allowing for minor interventions on the part of the interactor. 4 More on this ‘suture’ theory in chapter 3. 5 Carroll, Mystifying Movies, pp. 53–88. The identification with camera fallacy is further discussed in chapter 3. 6 Noel Carroll, ‘The Paradox of Suspense’, W. F. Brewer, ‘The Nature of Narrative Suspense and the Problem of rereading,’ Dolf Zillmann, ‘The Psychology of Suspense in Dramatic Exposition’, in P. Vorderer, H. J. Wulff, and M. Friedrichsen (eds.), Suspense: Conceptualizations, Theoretical Analyses, and Empirical Explorations (Mahwah, NJ: Lawrence Erlbaum Associates, 1996), pp. 80, 109, 210–214. 7 Carroll, Mystifying Movies, pp. 53–88. 8 Zillmann, ‘The Psychology of Suspense’, pp. 214–218. 9 Ibid, p. 216. 10 Moreover, while Bates’ experiment may have been engaging for the volunteer, as stated by the audience watching the live simulation, the whole thing was extremely boring.

Chapter 2: Interaction

57

11 See chapter 3 for interactive P.O.V. variation options. 12 M. Fracker and C. Wickens, ‘Resources, Confusions and Compatibility in Dual Axing Tracking: Displays, Controls, and Dynamics,’ Journal of Experimental Psychology: Human Perception and Performance 15, 1 (1991), pp. 80–96; W. Gladstones and M. Regan, ‘Division of Attention: the Single-Channel Hypothesis Revisited’, The Quarterly Journal of Experimental Psychology, 4/A (1) (1989), pp. 1–17; H. Pashler, ‘Dual Task Interference in Simple Tasks: Data and Theory’ Psychological Bulletin 116, 2 (1994), pp. 220–244. 13 Experience for example playing around with Bob Bejan’s multi-narrative interactive film I'm your Man (USA, 1998), Graham Weinbren’s interactively perspective shifting Sonata (USA, 1990) or, for that matter, zapping the TV channels between simultaneous screenings of Seinfeld on different channels. In all of these experiences the behavioral option is often activated restlessly, resulting in the user/viewer being neither here nor there. 14 Kinder, ‘Narrative Equivocations,’ p. 123. 15 Ibid, p. 122. 16 Manovich, The Language of New Media, p. 225. 17 Henry Jenkins, ‘Game Design as Narrative Architecture’, In Noah Wardrip-Fruin and Pat Harrington (eds.) First Person: New Media as Story, Performance, Game (Cambridge: MIT Press, 2004), pp. 118–30. Here are some of the quotes brought up by Jenkins and disputed by him: “Interactivity is almost the opposite of narrative; narrative flows under the direction of the author, while interactivity depends on the player for motive power,” (Ernest Adams); “There is a direct, immediate conflict between the demands of a story and the demands of a game. Divergence from a story’s path is likely to make for a less satisfying story; restricting a player’s freedom of action is likely to make for a less satisfying game,” (Greg Costikyan). It should be noted however, that while I agree that gaming interaction counters narrative, interaction in general need not conflict with narrative as will be shown further on. 18 Ibid, p. 121. 19 Ibid, p. 125. 20 Sergei M. Eisenstein, ‘Montage of Attractions,’ in Jay Leida (ed.), The Film Sense (San Diego: HBJ, 1975), p. 233. 21 Eisenstein, ‘Montage of Attractions,’ p. 231. 22 Jenkins himself quotes comments by computer game theorists to that effect: “Computer games are not narratives....Rather the narrative tends to be isolated from or even work against the computer-game-ness of the game” (Jesper Juul); “If I throw a ball at you I don’t expect you to drop it and wait until it starts telling stories.” (Markku Eskelinen). Jenkins, ‘Game Design,’ p. 118. 23 See a fuller discussion of immersion theory in the next chapter. 24 In comparing the game Star Wars to the film that inspired it, Jesper Juul has shown how oblivious are the game designers to narrative in general and to the film’s narrative in particular. As he writes: ‘The primary thing that encourages the player to connect game and movie is the title “Star Wars” on the machine and on the screen. If we imagine the title removed from the game, the connection would not be at all obvious. It would be a game where one should hit an “exhaust port” (or

58

25 26 27

28 29 30

Chapter 2: Interaction simply a square), and the player could note a similarity with a scene in Star Wars, but you would not be able to reconstruct the events in the movie from the game. The prehistory is missing, the rest of the movie, all personal relations. Possibly we are even missing the understanding that we are fighting a death star (whatever that is). Finally the most obvious: If you do not complete the mission, this is unlike the movie; if you complete the mission, another death star appears – which is also unlike the movie.” Jesper Juul, ‘Games Telling stories? A brief note on games and narratives’, The International Journal of Computer Game Research, volume 1, issue 1, July (2001). Benjamin, ‘The Work of Art in the Age of Mechanical Reproduction,’ pp. 239–41. Murray has stressed this difference between games and stories. Murray, Hamlet on the Holodeck, p. 140. On interactor helplessness as a narrative strategy to evoke suspense in computer games and suspense films see Jonathan Frome and Aaron Smuts, ‘Helpless Spectators: Generating Suspense in Videogames and Film’, TEXT Technology, no. 1 (2004), pp. 13–34. See Frome and Smuts, ‘Helpless Spectators: Generating Suspense in Videogames and Film’. See Kevin Brooks’ Metalinear Cinematic Narrative, for an interesting hypernarrative scripting tool. For a description of the experiment and the problems with the interactor as protagonist conception see above, section 2.1.

Chapter 3: Audiovisuality & Interfacing Chapters 1 and 2 dealt respectively with problems of split attention inhering in hyper-narratives and interaction, offering suggestions for its management and the use of strategies to enhance rather than reduce interactor engagement. This chapter deals with the management of split attention resulting from the misuse of audiovisual figurations peculiar to hyper-narratives, and from non-engaging interfaces. Following a critique of immersive and postmodern comprehensions of the narrative function of computer associated audiovisual figurations such as 3-D films and computer generated hyper-real simulations like morphing, digital replication, split screens and interfaces, viable ways will be suggested for their engaging use. Dual-coding theory1 explains, and narrative films show that the correlation of moving images and sounds helps memory and can be more sustained – attention engaging than sounds alone, still images alone or moving images alone. This is because images and sounds are processed differently and along distinct channels in the human mind, creating separate representations for the information processed in each channel. When processed together the mind recalls each other thus deepening memory and comprehension. Both imagined and verbal codes for representing information are used to organize incoming information into knowledge that can be acted upon, stored, and retrieved for subsequent use. It seems that the strive to build connections between images and sounds partakes of the more general tendency of the human mind towards coherence. This strive towards coherence, the focus of gestalt and cognitive-constructivist theories, rests upon the idea that we tend to group, structure or interpret stimuli along lines such as proximity, similarity, simplicity and closure.2 It seems that when images and sounds are presented the viewer/listener tends to build up connections between them guided by these principles. Such interconnections lead to a deeper and more sustained attention than when only sounds or images are presented.3 Furthermore, when sounds and moving images

60

Chapter 3: Audiovisuality & Interfacing

are connected, sustained attention is enhanced since interconnections are enriched. Hence, film, television, multimedia and computerized audiovisual works promise to engage deep attention. Not every combination, however, of moving images and sounds engages and sustains attention. Evidence has shown that when images and sounds are integrative rather than random, memory and attention are enhanced. However, as cognitive load theory explains, split-attention generated by simultaneous non-integrated or complex audiovisual articulations such as sound films may lead to a cognitive load on working memory, which viewers/listeners find difficult or impossible to cope with. This does not mean, however, that the best way of construing photorealistic audiovisual formations consists in the rendering of a spacetime continuum. Such approach to film is based on the unwarranted idea of film immersion. Since the idea of film immersion has also been very influential upon, and detrimental to interactive hyper-narrative theories and practices, a discussion of its fallacies may open the way to a more dynamic and fertile approach to the audiovisual configurations of interactive hyper-narratives.

3.1 Immersion Hyper-Narrative Film theories Immersion narrative film theories presume viewers who, rather than being engaged cognitively and emotionally in the process of narrative construction out of segmented shifting audiovisual flows, are more or less effectively manipulated through the film shot continuity, continuity editing techniques and by the audiovisual narrative succession into what Coleridge termed a ‘willing suspension of disbelief’, leading to the irrational belief that what unfolds before their eyes is actually happening. A more elaborated psychoanalytic-ideological immersion film theory was developed to explain how viewers are subjected by film and led to the irrational belief that not only what they are viewing is real but that they are actually authoring this reality. According to this theory, viewers first and foremost identify with the camera’s point of view. This apparently stems from the structure of the camera lenses and from other devices that create most films’ illusion of a continuous three dimensional imaginary flow. Hence, the camera, based upon linear perspective, creates a centered space converging into the viewer’s eye

Chapter 3: Audiovisuality & Interfacing

61

whereby a simultaneous centering of space and viewer occurs.4 This dual centering is responsible for the viewers’ self perception as origin of the film so long as viewers are not consciously aware of the camera’s presence (this is used to explain, for instance, the dominant fiction-film rule whereby the film protagonists never look into the camera lenses so as to not collapse the viewers’ ‘identification’ with the camera’s point of view).5 As for this theory’s presumed problem of the viewer awakening from such illusion when the film cuts from shot to shot, Daniel Dayan for example suggested that continuity editing transitions, particularly in shot-reverse-shot constructs, are based on the suturing of the viewer’s consciousness into the film. In shot-reverse-shot constructs consisting of a sequence of shots whereby the viewer watches character B from the point of view of character A, followed by a shot showing the reverse angle whereby A is seen from B’s point of view (and so on), Dayan claimed that in the first shot (showing character B) the spectators fully identify with the camera’s point of view and sense themselves as originating the image. However, in the transition to the reverse shot the imaginary continuity may be broken due to the change in camera angle and figuration, potentially raising in the viewers’ mind the question of who is showing them the event, a question threatening to lay bare the film apparatus and destroy the illusion and the spectators’ attendant joyful empowerment. However, the shot-reverse-shot construct according to Dayan, manipulates the potential arousal of such question to serve its needs. This is because the reverse shot offers a retroactive answer to the viewer’s question, whereby it was character A that was watching character B. Thus the film diverts the answer to the viewer’s question (‘who is watching this?’) from the level of the film apparatus to the fictional level, suggesting something like “it was the character that you see know that was watching the character you just saw”. Through such manipulations (termed by Dayan ‘the tutor code of cinema’), viewers’ minds are sutured into the film and their illusionary empowerment is not only reinstated but reinforced.6 In a comprehensive critique on this widespread immersion film theory, Carroll and other cognitivists have shown that not only are such notions unwarranted, but are also uncalled for since there are better explanations for the engagement of viewers with narrative films. Carroll rejected immersion-type ideas by showing that the fact that a shot is centered or even converges in my eye, does not lead to the

62

Chapter 3: Audiovisuality & Interfacing

conclusion that I therefore believe I authored the shot or ideologically forces me to accept the representation as real. Spectators can perceive a centered space and reject the apparent reality represented. The spectator, contended Carroll, is not a simple effect of the film.7 Furthermore, Carroll asked why do spectators who presume they author the film due to their identification with the camera find themselves in a state of suspense and guessing about future developments? Why spectators who irrationally believe that the reality watched is happening remain seated and do not escape the theatre when they see a roaring lion on screen? In response to widespread claims that film immersion is somehow comparable to dream immersion so that viewers do not escape a roaring lion on screen as they do not do so in dreams, Carroll pointed out that the difference between films and dreams is larger than their affinity mostly because film viewers are awake and aware. He rejected, for instance, some theorists’ attempt to compare the immobility of viewers seated in front of the screen to dream motor-paralysis 8 on the verifiable ground that spectators, as opposed to dreamers can look at any moment back towards the projector. Furthermore, if viewers are late, they can slowly walk to their seats while watching the movie, and can even get up and touch the screen in case they have doubts about whether what they are watching is not there. Carroll ridiculed in particular the identification-with-camera derived notion of ‘suture’, claiming that its explanation of the shot/counter-shot construct presupposes a farfetched process whereby the viewer undergoes several different consecutive mental manipulations (see below a cognitive approach to shot-countershot constructs). I would also suggest that similarly farfetched is the comprehension of the convention that characters do not look into the camera as deriving from the psychological/ideological need to maintain the viewer’s ‘identification’ with the camera intact. In fact, this convention has been often broken without necessarily disrupting the viewer’s engagement. Beyond the regular address of viewers by TV news anchors or in film dramas for soliloquies (e.g., Amélie, Jean-Pierre Jeunet, 2001), having characters ‘host’ viewers into a narrative while alternating between directly addressing them through the camera and acting within the narrative is a rather common practice (e.g., Annie Hall, Woody Allen, 1977).9 In sum, there is no good reason to assume that a film showing something which is life-like as continuous or apparently continuous in

Chapter 3: Audiovisuality & Interfacing

63

space and time, leads to the conclusion that it is real. This is simply because we know it is not, as is particularly evident in films that portray apparent realities the viewer ideologically disagrees with. In fact, asked Carroll, can a person willingly suspend his disbelief? Can I make myself willingly believe that 1 + 1 = 3? Presuming a spectator or interactor that is aware and cognitively active in constructing a coherent goal oriented story, offers a more rational and fertile approach to the generation of deep emotional and cognitive engagement than the one offered by the immersion approach. Our evident involvement in films need not presume our belief in the film’s reality. As Carroll suggested, people can get emotionally involved when entertaining ‘what if’ thoughts without needing to believe in the actual existence of such thoughts.10 Hence, I can think and visualize someone dear to me on the brink of jumping-off a cliff and get emotionally agitated without needing to believe that the deed is really unfolding before my eyes. It rather seems that when faced with such appearances we know to be unreal, the more lifelike they seem the more we are either amazed or concerned with questions focused on revealing the trick propelling the illusion as in magic tricks, rather than with more complex emotional and cognitive activities. In a way, immersion typed engagement distracts our attention away from the emotional and cognitive complexities of narrative films. While technological advances in audiovisual verisimilitude initially focus our attention on its magic conjuring of life-like appearances, it is only when such technological wonder stops generating the viewers’ amazement that complex emotional and cognitive processes can be triggered in viewers. Such seems to be the case with the technological breakthroughs in the history of film. Hence it took time for the film medium to artistically and comprehensibly incorporate sound, wide screens or digital morphing into narrative. This does not mean, however, that photorealistic images and sounds do not appeal to us in their underlying documentary import or their verisimilitude to how we perceive the world. The photorealistic grounding of narratives in films supports our imaginary grounding of the tale as feasible or possible. This does not mean, however, that therefore we think or believe that what we are watching is, will or has happened as portrayed. It only supports our imagining the tale as possible. This is probably why when there is a discrepancy between photorealistic spectacular verisimilitude and narrative, our attention is split between the photorealist illusion and

64

Chapter 3: Audiovisuality & Interfacing

our cognitive-emotional strive to construct cohering and goal oriented narratives.11 This potential split does not mean that the rationale for editing transitions has to be an irrational ‘suturing’ of viewers’ minds aimed at expanding their immersive wonder. While viewers may notice or not editing transitions, this does not mean that when they do not it is because they have been unconsciously manipulated into entertaining the irrational chain of thoughts proposed by suture theory. It is more feasible to presume that unnoticed cuts play with bottom-up automatic perceptual propositions made by viewers. Viewers do not pay particular attention to editing in continuity editing films because they infer from a cut a temporal or spatial transition or leap. That is why slicing together two shots of the same object from a less than 30° degree angle towards the object leads the viewer to notice the cut and sense a shot repetition. Likewise, when cutting from a long shot to a medium shot of a moving object without advancing the object in between the shots leads to a notion of backward movement since the presumption that a cut means a spatial transition is not met with. Such noticed cuts reveal that viewers make bottom-up cognitive-perceptual propositions rather than being in a state of unconscious passive receptivity. Films often use viewers’ perceptual propositions to rewardingly advance the viewers’ top-down construction of the narrative or to create narrative twists and cognitive surprises when violating the viewers’ bottom-up perceptual expectations. Hence, in a scene from The Silence of the Lambs (Jonathan Demme, 1991) the viewer perceives several shots figuring FBI agents hiding outside a house were they suspect a serial killer is. These shots are alternated with shots of the serial killer inside a house. This triggers in the viewer the perceptual schemes of parallel edited sequence and of inside/outside editing transition, leading the viewer to presume that the whole sequence occurs at the same time in the same location – the besieged house is the one where the killer is. This leads the viewer to raise the hypothesis that the FBI agents are about to break in and a wait and see strategy ensues concerning the results of the break-in. The viewer’s suspense is raised when one of the officers approaches the door and presses the doorbell, a shot cut to the killer seen reacting to a doorbell ring from inside his house. However, when the killer opens the door, a close-up of his face looking ahead is cut to a shot of Clarice, the film’s protagonist, whom the viewer knows to be nowhere near the

Chapter 3: Audiovisuality & Interfacing

65

house besieged by her FBI peer agents. These are seen in turn breaking into the wrong house. The viewer’s ensuing cognitive surprise results from the film’s frustration of one of the editing schemes it has triggered in the viewer’s mind through the viewer’s shot-counter-shot perceptual scheme. Hence, the film realized through the shot-counter-shot scheme the simultaneity portion of the parallel edited sequence scheme but exchanged the spatial relation implied in the previous inside/outside editing transition scheme. The viewer is surprised because she has presumed that the killer will see the FBI agent whom she has seen previously ring the door bell. At this point the viewer generates a new hypothesis concerning Clarice’s fate, using a new wait and see strategy as to what will happen now that Clarice has to face the serial killer all alone and far away from the FBI officers.12 The above discussed unwarranted immersion based explanations for the complex engagement of viewers in narrative films, resurface in hyper-narrative film theories and practices. These theories wrongly presume that interactive immersive environments, seen as an extension of film immersion, will further enhance the deep engagement viewers experience in narrative films. Encapsulated in the theory and practice of ‘virtual reality’, interactive immersive theories suggest that viewers, often through avatars representing them and with which they presumably identify (a subject discussed in chapter 2), will be able to shift at will and at any point the narrative trajectory while sensually and otherwise immersed in this virtual reality. Star-Treck’s holodeck (Gene Roddenberry, NBC, 1966–69), a science fiction hologram construct where participants can play out their fantasies in a total sensual, emotional, spatial-temporal and cognitive logical responsive virtual environment, encapsulates the immersive promise of such hypernarrative film theories and practices.13 However, this myth of total immersion, propelling film fantasies such as Tron (Steven Lisberger, 1982), Johnny Mnemonic (Robert Longo, 1995) and The Matrix to name but a few, while raising interesting philosophical questions having to do with the nature of reality or the freedom of choice, along with a problematic psychological and ideological desire to re-enter a separate reality or hetero-cosmos and live there rather than here, is not only technologically unfeasible in any foreseen future, but its partial attempts at implementation have so far rendered poor and shallow sensationalist typed engagement that points towards narrative ossification and aimless, closure-less wandering. As

66

Chapter 3: Audiovisuality & Interfacing

I will try to instantiate, this shallow or frustrating type of engagement derives from the misleading assumption that what deeply engages us in film narratives is a spatial-temporal engulfing fullness that makes us believe in the reality portrayed.

3.2 The Immersion Fallacy in 3-D Movies The most direct film descendant of the immersion film paradigm, the 3D movie, has so far managed to generate sensationalist typed engagement. This engagement is derived from the film’s ability to confound our sense of sight into perceiving a highly detailed and palpable illusory three dimensional space. This is achieved in 3-D by simultaneously projecting two slightly differing images that when viewed by spectators through plastic goggles with liquid crystal lenses generate the 3-D illusion. Janet Murray, in her comprehensive and well detailed analysis of present digital creation and in her fertile futuristic propositions for the medium’s capacity for audiovisual narration, has nevertheless let herself be swept away by immersion theories. The following analysis of her carefully detailed sensationalist and amazed experience of 3-D films reveals the fallacy of, and stumbling blocks in immersion theory. Describing 3-D films as ‘magical apparitions’ conveying an exciting sense of presence she tells us: “I held my breath when a little blue bird flew out of the screen and landed right in front of my nose. I and everyone else in the audience reached out a hand to touch the bird…”14 Further on, while claiming such sensationalist diversions may subside once audiences get accustomed to 3-D, Murray nevertheless continues explaining more complex engagements in terms of presence and wonder. Hence, when writing about the 3-D film Across the Sea of Time (Stephen Low, 1995) she explains the momentary emotion she felt in a scene where a hungry boy is given a lunch bag, as derived from her deep immersion in the spectacle and consequent identification with the character, an identification so powerful that when the boy’s hand enters the screen apparently from behind the audience to grab the lunch bag Murray “immediately thought of operating it [the hand] as if it were a cursor in a videogame!”15 While Murray tries through this description to forward the idea that 3-D immersion predisposes us to desire to seamlessly live/participate in the film, the precise description of her

Chapter 3: Audiovisuality & Interfacing

67

experience rather reveals inadvertently the major pitfall on the road of attempts to use an immersive narrative film paradigm to go beyond shallow sensationalist engagement to deep emotional and cognitive engagement. This pitfall consists of the split attention generated in the viewer’s mind between the amazement at the degree of life-likeness and the desire to construct an emotionally and cognitively engaging narration. It surfaces once we understand that when Murray was led to try and ‘operate’ the boy’s hand her emotional involvement subsided and turned into a gaming attitude (hinted at in her exclamation about the ‘cursor in a videogame’), since diverted towards her wonder at how the illusion propelled her to try and direct the boy’s hand. Whereas in the case above the split attention pitfall surfaces because her immersive wonder disrupts her emotional narrative engagement, in another scene analysis Murray’s description reveals the inverse split-attention situation when she tries to explain how the 3-D film’s attempt to generate emotional engagement through narrative construction diverted her attention away from her immersive experience. Hence, in this scene she writes “A couple in what would ordinarily be the background crosses the street. But there is no street, I am there…and I want to follow that couple…”, however, to her dismay the camera relentlessly drags me into a bar…I am uncomfortable at these moments because the three-dimensional photography has put me in a virtual space and has thereby awakened my desire to move through it autonomously, to move away from the camera and discover the world on my own…my immersion was constantly disrupted by the director’s shifting from interior to exterior shots and from one point of view to another, which dizzies…It is only when the camera angle switches that we are unpleasantly jarred from our trance of feeling that we are actually there.16

Again, while Murray uses the immersion paradigm to forward the idea that 3-D immersion points to the direction of spatial continuity and spectator agency, the precise description of her experience rather reveals inadvertently the built-in split attention between immersive life-like sensationalist engagement and narrative cognitive-emotional construction. As we have seen, Murray has articulated her experiences with 3-D so as to point to how it arouses in viewers the desire to have some agency and feel they can go wherever they want within photorealistic simulations,

68

Chapter 3: Audiovisuality & Interfacing

while describing her frustration as deriving from not being able to do so due to cutting and point of view shifts. While it is questionable whether viewing 3-D space evokes our desire to change the course of events or go wherever we want rather than the actual compositional layout and our possibility to do so, a more serious problem concerns the probable ossification of audiovisual hyper-narration implied in her position. This is because the idea of immersive navigation in an engulfing space-time continuum exchanges the possibility of an engaging dynamic interactive dramatic cinematic construction with the probable distraction brought forth by the aimless closure-less wandering that such an immersive line of thought may lead to. Hence, in trying to avoid split attention and maintain the amazed feeling of immersion in 3-D, Murray is led to support the widespread notion that computer generated audiovisual narratives should focus on an expansion and exploration of space on account of temporal evolution.17 While the expansion of such film realist tradition, once disjointed from the immersive motivation may render satisfying aesthetic experiences of spatial crossing, and may even develop into a narrative strain based on continuous scene-sequence segments, when Murray exclaims that “I am there…and I want to follow that couple,” an incidental couple in a film focused on something else, it can be easily envisioned how an interactor, when driven towards space navigation will be drawn to constantly shift his navigational focus to by-passing events on account of following engagingly an evolving narrative. The earlier discussed experiment in interactive drama conducted by Joseph Bates at Carnegie Mellon University suggests that striving for narrative engagement through the interactors’ ‘immersion’ in a spatialtemporal cohering styled continuity is irrelevant to explain interactor narrative engagement. This can be evidenced in the fact that interactors in Bates’ experiment were highly engaged despite the artificial and sloppy stage upon which the drama was enacted, consisting of “an unrealistic theatrical set, a fake gun, actors wearing headphones, a giant microphone in the middle of the ‘bus station’, and much confusion between the actors and the director.”18 This has actually been a long understood fact in theatrical productions as can be evidenced for example in the recent and rather engaging 2006 production of A Chorus Line (Michael Bennett, Original Director and Choreographer) where the characters populate an empty stage with only a white-drawn line crossing it. Consider also for that matter ‘make believe’ games played by children where

Chapter 3: Audiovisuality & Interfacing

69

very few props are needed to ground the imaginary fantasy. Hence, while the interactors and Bates himself used the unwarranted ‘realist immersion’ and ‘suspension of disbelief’ terminology to try and explain their engagement, it seems much more feasible that it was the dramatic succession engendering what Bates termed ‘dramatic presence’ that caused their emotional, cognitive and behavioral engagement.19 This sense of ‘dramatic presence’ was due to the interactors’ active help to coherently construct the loosely predetermined narrative and to the meaningful reactions of the characters that manipulated him/her into the major turning points of the story, rather than the evidently fake spatialtemporal continuous grounding. Indeed, as Bates reported on another interactive drama session they held, such continuity was irrelevant for the interactors who found no disengagement involved in their ‘sudden’ move to a separate room in order to discuss possible modes of action concerning the evolving drama.20 Hence, ‘dramatic presence’ presupposes the interactor’s active imagination and assumption of a ‘what if’ approach to the fictional events rather than her believing in the actual possibility of shooting to death the actor facing her. In sum, what sustains our engagement and attention in narratives is a dramatic succession underlined by causal logic and a sense of moving towards a goal rather than spatial immersion. I suggest we drop the presumption that what deeply engages us in 3-D movies is their making us believe that ‘we are there’, ‘I am that boy’ and that therefore the future of 3-D resides in our magical wonder of an illusion of an expansive spatial-temporal continuity so that ‘I will be able to live there and go wherever I want.’ Once we understand that amazement counters deep emotional and cognitive narrative engagement and generates split attention, we can begin to clearly think about how, once amazement subsides in favor of our enjoying a more satisfying imaginary grounding of the tale as feasible or possible, the challenges posed for interactive hyper-narrative construction by 3-D can be tackled and fruitfully exploited. A similar audiovisual challenge was posed to film narration. It took some time for early filmmakers to understand that action within a statically rendered space in a shot arrested narrative evolution and made it cumbersome to convey complex narrative constructs. While some filmmakers (e.g., Orson Welles in Citizen Kane, 1941) developed the uncut shot-sequence and maintained the space-time continuum as an aesthetic (not ‘immersive’) experience of space-crossing and time-

70

Chapter 3: Audiovisuality & Interfacing

passage, filmmaking in general realized that the dynamization of space through the mobilization of cameras and editing, the determination of spatial compositional organization by the forward moving action, and the film’s focalization through shifting character points of view, offered a variety of ways to maintain viewer space-time orientation. This while articulating complex narrative constructs that productively challenged and rewarded the viewer’s cognitive-constructivist strive to coherence and closure. Eisenstein’s dynamically shifting film space, which allows through ‘dialectic montage’ constructs the double axis dynamization of spectator and story, along with Griffith’s continuity and parallel editing strategies have led to the successful evolution of cinematic narratives that use spectacular audiovisuality to forward the evolving narrative. While it may seem at first that 3-D calls for an expansion of space on account of temporal evolution, a notion derived from immersion theories, may only prove fruitful, however, once it is conceived as a further aesthetic development of realist filmmaking. 3-D can also develop or expand its narrative possibilities by adapting the film’s elaborate modes of spatial-temporal dynamization to 3-D’s enhanced perception of depth and solidity. Hence, what should concern us in the audiovisual configuration of interactive hyper-narratives rendered in 3-D is how to dynamically engage the viewer/interactor into constructing the story in her mind while also behaviorally changing its course. This can be done by expanding the repertoire of audiovisual cues used in films to evoke suspense, expectation or curiosity hypotheses to such computer enhanced interactive photorealist audiovisual works. In what follows I would like to consider in particular four leading computer associated audiovisual figurations – hyper-reality, morphing, digital replication and split screens, – which while often used in shallow sensationalist or cognitively confusing ways, can be put to a particularly effective use when construing hyper-narrative interactive transitions.

3.3 Hyper-reality, Morphing, Digital Replication and Split Screens Countering Murray’s immersion based claim that the spatialtemporal fascination of 3-D does not sit well with ‘jarring’ spatial and temporal shifts, hyper-real computer generated simulations, particularly

Chapter 3: Audiovisuality & Interfacing

71

in animation movies, TV commercials and music video-clips, have designed sweeping spectacles that seamlessly traverse in convincing yet impossible ways faked compilations of spatial, temporal and bodily expanses. These sweeping audiovisual hyper-real simulations are often achieved by digital image synthesis. This process involves, as described by Darley, “…the input of mathematical data to the computer’s memory that effectively describes or models and then stores whatever is to be imaged.”21 Such process reaches an unprecedented convincing level of surface textures. Several researchers, strongly influenced by Baudrillard’s notion of the simulacrum22 have claimed that such hyperreal simulations inherently lead towards stunning sensation-typed shallow spectacles on account of narrative or character depth. Hence, Vivian Sobchack has suggested that the digital device of morphing instantiates Baudrillard’s concept of the simulacrum in that the morph has no origin whatsoever, since that from which it changes from does not ‘cause’ or precede that to which it has changed into. The morph implies seamless reversibility and one image is not more real, original or essentially different from the other. For instance she recapitulates Baudrillard’s ideas in a comparison she makes between the pre-morph film transformations in a montage sequence from All that Jazz (Bob Fosse, 1979), showing a mix of ethnic and gender differentiated dancers’ bodies constituting one single pirouette, to Michael Jackson’s morphing of similar bodies into one another in his Black or White videoclip (John Landis, 1991). She argues that whereas through cut transitions in All that Jazz “we are still aware of their discretion and diference…in Black or White…these racially and ethnically “different” singing heads enjoy no discretion: each is never “itself” but rather a mutable permutation of a single self-similarity …[as Jean Baudrillard writes]…Division has been replaced by mere propagation”.23 However, while there are shallow sensational uses of digital hyperreal imaging, such computer associated audiovisuality in itself need not be deployed in such manner. Technological determinist claims such as Darley’s or Sobchack’s ‘shallowness’ accusation of hyper-reality are fallacious. They recall for example the Russian Formalists 1920s aversion towards the application of dissolves to shots that emphasize three-dimensionality, claiming that such use of dissolves was antiaesthetic because it evoked the eerie sensation-attraction brought forth by the violation of natural law in the dissolve’s apparent commingling of solid three dimensional objects.24 A similar eerie sensation used to

72

Chapter 3: Audiovisuality & Interfacing

be often evoked through morphing because of its apparent violation of natural law by its seamless mutation of solid objects. However, just as with dissolves, morphing and digital replication are being gradually used in aesthetic ways within films, to construct narrative transitions that exploit our depth and solidity perceptual hypotheses in non-sensational or eerie ways, enhancing suspense, creating narrative surprises and twists or articulating conceptual ideas that evoke deep emotional and cognitive engagement. It seems that such creative use of digital imaging is made possible because the eerie amazement evoked by its apparent violation of solidity has subsided. Actually, the very Baudrillardian notion of simulacrum that has been attributed to such digital audiovisuality, whereby it propagates interreferring simulations that deny meaningful categorical or hierarchical differentiations, has been in itself critically and engagingly articulated through an aesthetic non-eerie and non-sensasionalist use of morphing and digital replication. This can be seen for example in The Matrix, where the trope of digital replication and multiplication is in itself used to criticize in an engaging way the problem of computer simulated propagation through the self-replicating figure of Mr. Smith the virus agent that can never be killed. Also, morphing is aesthetically used in this film to powerfully visualize the transition through telephone lines between the simulated and the real worlds. Finally, hyper-real impossible choreographies of floating martial arts fights make tangible the film’s idea concerning the elasticity of virtual reality. In general, the post-modern problematic destabilization of the dividing line between the real and the simulated have been deeply explored through digital imaging. Well known cases include Ridley Scott’s director-cut version of Blade Runner (Ridley Scott, 1982) where Deckard, a ‘blade runner’ in charge of tracking down and terminating replicants turns out to be a replicant himself, and in David Cronenberg’s film eXistenZ (1999), where a game designer creates a virtual-reality game that taps into the players’ body and mind but ultimately leaves them (and us) with the idea that the ‘reality’ from which the film started may have been just another option within the game. Moreover, digital imaging has left the realm of science fiction and is gradually used to forge compelling metaphors such as the digitally designed floating leaf in the opening of Forrest Gump (Robert Zemeckis, 1994) conveying the ‘man as leaf on the wind’ metaphor driving the film, or the floating martial arts fights in Ang Lee’s Crouching Tiger, Hidden Dragon that powerfully convey

Chapter 3: Audiovisuality & Interfacing

73

a non-science fiction worldview were miracles, spirits and gods are credible to humans. While in digital hyper-real simulations it is the seamless composite image that may engender sweeping shallow spectacles that come on account of complex narratives and deep characters, in another computer culture associated trope, the split-screen, it is the postmodern use of the split that often generates frustrating or distracting non-coherence and decenteredness, by demanding that viewers be simultaneously and equally attentive to a flow of several unrelated audiovisual occurrences. A good instantiation of this frustration is Mike Figgis’ film Timecode (2000) so admired by Manovich because it concerns the multi-split screen, one of the major features deriving from the immanent ‘data base’ aesthetics of computers.25 In this film, four simultaneously evolving occurrences are presented on a screen split into four. However, whether it is the same occurrence shot from different positions, or different occurrences, it is impossible for the spectator’s split attention to follow what is going on. The only reason viewers can somehow follow Figgis’ film is because he manages to draw attention away from the disturbing parallel occurrences through different strategies that accentuate one frame over the others such as his use of voice enhancement coming from one of the screens, or the reduction to minimal, recurring and non-interesting movement and action in the non-emphasized screens; by using a sporadically appearing earthquake that affects all the events on the four screens; or, towards the end, the use of a melodramatic shootout of a betrayed lover that correlates and clarifies in simple manner the interrelation between the four screens. There is, however, one interesting use of the split screen, in which we see on one screen the betrayed lover, who has planted a microphone in her lover’s bag, listening to her lover in another screen as she makes love to a man the betrayed lover does not know. Then, within her screen we see her get out of the car she’s in while the man with whom her lover was is seen in the other screen leaving the building he was at, and they bump into each other in their respective screens without knowing each other (this is the man she eventually shoots by the end of the film), so that we see simultaneously the same occurrence from different angles. This scene is interesting because it makes cohering rather than distracting use of the split screen, in that it literalizes through the formal means the implications of betrayal.26 Excluding this cohering use of the split screen (which still materializes only two of the four screens), the question arises as to

74

Chapter 3: Audiovisuality & Interfacing

what is this film’s split-into-four screen added value? It only actually manages to somehow and poorly engage our attention because it in fact contradicts its formation. In any case, the only available response for viewers in this film is distraction coupled with shallow engagement, which approximates a gaming attitude towards the film rather than deep engagement. However, there is comprehensive and constructive use of the potentiality of a split screen. This for example is evidenced in the television series 24. In it, different, simultaneously occurring narrative threads are made interrelated and comprehensible, precisely because of the periodical transfer from a full screen presented development to a four part split screen, where we are reminded about where we are at in each of the absent threads, and through this split screen we are also transferred from one narrative trajectory to another. This is a diametrically opposed use of the four screen potentiality used in Timecode. Whereas Timecode confounds us in its simultaneity, 24 coheres this simultaneity. Further compelling uses of split screens can be found in the films by Iranian born American artist Shirin Neshat. In her work, a similar occurrence is shown simultaneously on two screens from different points of view. However, while Figgis’s film simply offers confounding alternate points of view of the same occurrence, Neshat uses these points of view to effectively enhance the thematic narrative trajectory she follows. Hence, her film Fervor (USA, 2000) tells the story of the constantly frustrated mutual blind desire of a man and a woman. While they do not seem to have ever seen or know each other, they each long for the felt presence of the other. Neshat uses this blind attachment to comment on frustrated desires in fundamentalist, gender-segregated Iran. In one scene for example, we see on each screen only one of the protagonists, whereby despite the fact that they both inhabit the same space at the same time, they are mutually blocked from each other’s view. However, this blocking does not result only from the spatial configuration of the screens, but also because we see each protagonist only in it’s respective screen, as each looks, or rather blindly searches in the direction of the other. In another scene they are seen, each on a different screen, from respectively opposing points of view, as they walk towards each other along the same road. However, they never meet upon mutually arriving in their respective screens, at the presumed place of their encounter, as they would have done, had they been seen together on one screen.27

Chapter 3: Audiovisuality & Interfacing

75

3.4 Engaging Interfaces I would like to suggest that such hyper-real and split screen computer associated audiovisual tropes can offer particularly powerful evocations of ‘what if’ propositions in crucial hyper-narrative transitional points, as well as powerful notions of recurrence, parallelism or variation that allow the interactor a challenging way to solidify the narrative coherence and the notions of dramatic succession and closure when transitioning between different narrative threads. As mentioned, the best hyper-narrative crucial point transitions are those that are construed in such a manner that they clearly evoke in the interactor his/her ‘what if she did that’ or ‘if only had he done that’ hypothetical conjectures. These are usually decision points ripe with moral, survival or emotional consequences. The possibility to offer these crucial points as interfaces that include image variations that cue the interactor to the evolution of different options is a particularly engaging way to evoke ‘what if’ propositions. Morphing, digital variation and split screens allow for the double dynamization of spectator and story through their double axis of kaleidoscopic synchrony and diachronic motion. Hence, cueing different options within an interfaced shot by allowing the interactor’s behavioral mutating from option to option through, morphing, slightly digitally replicated variations, or through splitting the screen into different simultaneous points of view that converge upon the same occurrence, can help the interactor shift among the various options while also keep them in mind once one such option is materialized. For example, the violation of the shot-counter-shot scheme can be effectively used in behavioural interaction between interactor and protagonist. Thus the film can show the protagonist talking to another character, allowing the interactor to determine by himself when to effect the shot-counter-shot transition between the two. This perceptual shotcounter-shot scheme can be violated when in a pre-determined narrative crucial moment the interactor’s cut from the protagonist’s close-up shot to his point of view leads to a surprising and suspenseful narrative twist offering different optional sets and resulting narrative evolutions than the one he was viewing. These options can then be presented in a split screen, encouraging the interactor to choose one of the split-screens options.

76

Chapter 3: Audiovisuality & Interfacing

Beyond cueing different options at crucial narrative points, these devices also allow for a solidification of coherence between narrative threads, while maintaining the narrative cumulative effect achieved by the temporal dramatic succession of events. Morphing is particularly effective in such solidification when simultaneous competing optional narrative threads are involved. This is because of what Sobchack has defined as the morph’s non-origin and non-hierarchic nature. Given the morph’s inherent non-causal spatial correlation, it can mutate between disparate narrative threads and solidify the comprehension of their optionality and parallelism, enhancing the viewer’s engagement because it allows her reflection upon overarching thematic or formal concerns. Likewise, the morph’s non-causal temporal mutation allows for audiovisually cohering transitional flashbacks or flash-forwards to enhance the primacy or recency psychological effects. Finally, digital replication or split screens can audiovisually enhance repetitions of scenes where temporal constrains are introduced comparable to the rewind effect used in Sliding Doors to ‘return’ Helen to the run towards the train she missed the first time in order to have her board the train this second time around. While the interactive audiovisual interfaces outlined above may engender deep engagement for hyper-narrative interactors, disengagement or a gaming attitude can still result when using poorly conceptualized figurations of the interactor’s actions. As we have already discussed in chapter 2, in effective, engaging hyper-narrative interaction the moments at which behavioral interaction is allowed or blocked should be calculated so that behavioral interaction complements rather than disturbs the interactor’s cognitive construction of the hyper-narrative in her mind. Also, we have just suggested that morphing, digital replication and split-screens can be used in hypernarrative transitional points as interface screens that suggest possible options and offer engaging transitions between different narrative threads. We now turn to briefly discuss the design of the relation between the interactor’s behavioral action and its audiovisual figuration so that such interaction is rewarding and engaging rather than distracting or splitting the interactor’s attention due to multi-tasking. This may be achieved if there is a coherent complementary correspondence between the figured audiovisual evolution and the type of behavioral action applied to this figuration.28 In other words, the type of interface intervention must fit

Chapter 3: Audiovisuality & Interfacing

77

the result and be reciprocated in cohering and attractive ways by the evolving audiovisual narrative. At present there already exist different modes of interfacing that unskilled users can have with computers. Computers can respond to sounds, movements and tactile activities. Hence, interactors can use remote control devices, touch the screen, clap their hands, vary the pitch of their voice or move around to interact with the evolving narrative. All these types of interfacing may be used if they are coherently reciprocated by the audiovisual figuration. Game developers have devised different reciprocal interfaces. Thus, in recent racing car games the control of your virtual car is through a wheel-shaped remote control that has the car respond to your wheel turning and which trembles in your hands when your virtual car bumps into the sidewalk. Likewise, in remote controlled tennis games when you swing the remote you get a replicated swing from your avatar. It should be pointed out, however, that in hyper-narrative interaction what concerns us is hyper-narrative engagement rather than gaming engagement. Hence, reciprocal interfacing has to be geared towards cohering and forwarding the narrative engagement or for making your interaction with a protagonist complex, rather than offering stunning or funny gaming effects. An interesting example of interactor-narrative interface reciprocity is described by Jim Bizzocchi in his analysis of the CD-ROM Ceremony of Innocence (A. Mayhew and G. Villon, 1997).29 In this work, the cursor not only figures in different modalities and takes on a character of its own, but also performs functions that fit its results. For example, when clicking the cursor on a figured virtual aquarium holding a goldfish, clicking glass sounds are generated whose accumulation suddenly breaks the virtual aquarium, releasing a gush of water and having the virtual fish swim away. The complementary reciprocal responsiveness between behaviorally clicking a mouse and the resulting audiovisual figuration of glass breaking, along with the audiovisual spectacle that follows (the gush of water) and its functionality to the consequent narrative (the fish carries us further on in the story), instantiates how split attention problems, inherent in interfacing with hyper-narratives, can be managed and even used to enhance engagement. Beyond this example, however, the CDROM Ceremony of Innocence suffers from a variety of split attention problems stemming from narrative incoherence and gaming transitions. The Voodoo Office (2002) on the other hand, does offer a coherently

78

Chapter 3: Audiovisuality & Interfacing

flowing interactive hyper-narrative. Developed by Michael Lew at the MIT Media Lab, Voodoo Office consists of two physical voodoo dolls whose tactile manipulation by the user (e.g., making them hug) changes in a coherent cinematic and narrative way the emotion and form of interaction between two protagonists in a hyper-film screened before the interactor. However, beyond the fact that the story told has no closure and is limited to a coherent variation of simple situations, the major problem with this type of interaction is that the protagonists are under the total control of the interactor. While she may not know the precise way in which the protagonists will react, she does know that when making the dolls hug a sympathetic interaction will result. As earlier discussed (see chapter 2), rather than falling into the trap laid out by ‘identification’ theories, I suggest we do not reduce protagonists to shallow interactor self-personifications or robotic avatars but as the interactors’ fictional friends towards whom she develops empathy. This allows the exploitation of the in-built divide between avatar and interactor to generate suspense, curiosity and expectation that derive from the unexpected reactions of the protagonist to the interactor’s interventions and to the following unexpected narrative development. Furthermore, establishing such interaction opens up the hyper-narrative camera to all the different points of view that an interactor-robot surrogate P.O.V. blocks. An interesting attempt at designing such reciprocity can be evidenced in the interface of One Measure of Happiness (2003), a dynamically updated interactive video produced by Udi Ben Arie and Noam Knoller. While the overall result of this work is confusing, what is of interest is the intent behind the interface reciprocity. Hence, the interface consists of a touch-screen where caressing or alternately scratching the face of a figured protagonist brings each time different memories viewed and heard in the background. These memories, however, were intended to reciprocate the mode of touching the screen in cohering yet complex and unexpected ways.30 While the actual results are confusing because of poorly designed and conceptualized scripting and figuration, there appears to be a possibility where the interactor’s caressing may bring at one time the protagonist’s childhood memories of parental love while at another time it can arouse her childhood memories of sexual harassment. Hence, I would suggest that for reciprocal interfaces to be hypernarrative engaging, they should lead to unexpected protagonist reactions and narrative developments. Using such reciprocal interfaces

Chapter 3: Audiovisuality & Interfacing

79

re-engages the interactor back into the fiction by turning the potential disengagement inhering in behavioral intervention into a re-engagement device.

1

Dual Coding theory was originally developed by Allan Paivio. See Allan Paivio, ‘Dual coding theory: retrospect and current status’, Canadian Journal of Psychology 45 (1991), 255–87; Allan Paivio, Mental Representations: a dual coding approach (Oxford, England: Oxford University Press, 1986). 2 On cognitive constructivism see Jerome Bruner, ‘On Perceptual Readiness’. On Gestalt theory see in particular Rudolph Arnheim, The Power of the Center: A Study of Composition in the Visual Arts (Berkeley and LA: University of California press, 1982); Rudolph Arnheim, Film as Art (Berkeley and LA: University of California press, 1957). 3 Mayer & Moreno, ‘Aids to Computer-Based Multimedia Learning’. 4 E.g., Jean Louis Baudry, ‘Ideological Effects of the Basic Cinematographic Apparatus’, in Bill Nichols (ed.) Movies and Methods, Vol. 2 (Berkeley: University of California Press, 1985), pp. 531–542. 5 E.g., Christian Metz, ‘Story/Discourse: Notes on Two Kinds of Voyeurism’ in Bill Nichols (ed.) Movies and Methods, Vol. 2 (Berkeley: University of California Press, 1985), p. 547. For a critique of this notion, see below. 6 Daniel Dayan, ‘The Tutor Code of Classical Cinema’, in Bill Nichols (ed.), Movies and Methods (Berkeley: University of California Press, 1976), pp. 438–50. 7 Carroll, Mystifying Movies, pp. 53–88. Carroll also rejected the whole notion that linear perspective creates a fraudulent ideological illusion. While it does offer an illusion of three-dimensionality, this illusion is not an ideological convention that colonized culture (a notion implying there could be as easily other forms of representation that fit the human sight or rather, that human sight can be different). Linear perspective says Carroll is a rather useful cultural invention fitted to human sight. It allows us to learn about the real world around us. However, while film shots can lend probability and tangibility to fictional events, this does not mean that because they are centered I believe the events portrayed are real. 8 Charles F. Altman, ‘Psychoanalysis and Cinema: The Imaginary Discourse’, in Bill Nichols (ed.), Movies and Methods Vol. 2 (Berkeley: University of California Press, 1985), pp517–530. 9 Addressing the camera can in fact be a particularly productive device to establish an empathic interaction between protagonist and interactor in hyper-narratives (see chapter 2). Such devise is often used in computer games such as in the Nintendo DS game Contact (Grasshoper Manufacture Group, 2006) or in the role-playing computer game Paper Mario: The Thousand Year Door (Intelligent systems, 2004 10 Carroll, ‘The Paradox of Suspense’, p. 86. 11 This tension between spectacular engagement (within which I include life-like verisimilitude) and narrative has been noticed by Laura Mulvey within the context

80

12 13 14 15 16 17 18 19 20 21 22

23 24

Chapter 3: Audiovisuality & Interfacing of her feminist critique of narrative films. Her claim was that whereas spectacle is shallow and static (usually figuring female protagonists in patriarchal determined films), narrative is dynamic and forward moving (and led by the male protagonist in such films). She also showed how narrative cinema has devised strategies to overcome the arrest of the narrative flow by spectacular images. Hence, she showed how narrative films when arriving in the narrative flow to spectacular images (e.g., figuring females as objects to be looked at and desired), converge the film’s point of view with that of the leading character’s point of view (e.g., the male watching the female figured in the spectacle) thus narrativizing the spectacle by its inclusion within the narrative flow. See Laura Mulvey, ‘Visual Pleasure and Narrative Cinema’, in Bill Nichols (ed.), Movies and Methods Vol. 2 (Berkeley: University of California Press, 1985), pp. 303–314. While I disagree with Mulvey’s implication that the female-spectacle / male-narrative conflation is mandatory and cannot be dissolved (even in ‘patriarchal societies’), I find her discovery of spectacle-narrative transition through P.O.V. correlation to be a productive narrative strategy. Her theory also supports my claim that attempts to enhance narrative engagement through spectacle without considering the tension between both types of film articulation is bound to generate split attention and distraction. See in section 3.4 below a possible use of the shot-counter-shot scheme in interactive hyper-narratives. See in particular Murray’s fascinated description of the holodeck in Hamlet on the Holodeck, pp. 13–26. Ibid, p. 45. Ibid, p. 47. Ibid, p. 48. The notion that digital cultural audiovisual production tends towards spatiality is widely shared. See for example Darley, Visual Digital Culture, pp. 158–9. Bates, ‘Virtual Reality, Art, and Entertainment,’ p. 136. Ibid, p. 138. Ibid, p. 137. Darley, Visual Digital Culture, p. 19. Baudrillard’s notion of simulacra shattered the traditional distinction between the object as origin and the image as a resulting simulation of it. Perceiving this distinction as arbitrary and unwarranted he claimed that the simulated image (e.g. film) does not originate from something beyond it (such as ‘reality’) but precedes or even originates what it presents. The concept of simulacrum implied that film was neither a reproduction of reality nor its artistic abstraction. For Baudrillard reality is not an origin for an image re-presenting it since ‘reality’ is always-already an image or a simulation. This engendered a conception of film as one among other fluid succession of images and sounds whose tagging as ‘documentary’, ‘fictional’ or ‘artistic’ referred to noting else but different and equally valid modes of simulation. See Jean Baudrillard, ‘Simulacra and Simulations’. Vivian Sobchack, ‘At the Still Point of the Turning World’ in V. Sobchack (ed.), Meta-Morphing (Minnesota: Minnesota University Press, 2000), p 142. Tynianov, Jury, ‘the Foundations of Cinema’, in H. Eagle (ed.), Russian Formalist Film Theory (Ann Arbor: Michigan Slavic Publications, 1981), pp. 83–4.

Chapter 3: Audiovisuality & Interfacing

81

25 Lev Manovich, ‘Old Media as New Media: Cinema’, in D. Harries (ed.) The New Media Book (London: BFI Publishing, 2002), p. 209. But see also Vivian Sobchack, ‘Nostalgia for a Digital Object: Regrets on the Quickening of QuickTime’, Millenium Film Journal, No. 34, (Fall 1999), pp. 4–23. As mentioned, Manovich’s postmodern and technological determinist exaltation of split-screen’s computer “natural” “data base” aesthetics are fallacious. 26 This is evidence that split screens, as a peculiar formal strategy enhanced by the digital revolution, in themselves do not need engender split attention. Further cohering and compelling split-screen uses will be discussed below. 27 Further cohering split-screen strategies can be found in the experiments conducted by the Russian revolutionary avant-garde filmmakers, particularly in Eisenstein’s and Vertov’s montage compositions. See Annette Michelson, (ed.), Kino Eye, the Writings of Dziga Vertov (Berkeley: University of California Press, 1984); Sergei M. Eisenstein, ‘A Dialectic Approach to Film Form’, in Jay Leida (ed.), Film Form (New York: HBJ Books, 1949). 28 This strategy re-engages the interactor back into the fiction. It turns the potential disengagement inhering in interactive intervention into a powerful re-engagement device. It is a strategy similar in intent to narrative cinema’s shot-reverse-shot device. 29 See J. Bizzocchi, ‘Ceremony of Innocence and the Subversion of Interface: Cursor Transformations as a Narrative Device’, CD-ROM of Proceedings (Melbourne: Digital Arts & Culture Conference [DAC], 2003). 30 Noam Knoller, ‘InterFace portraits: communicative-expressive interaction with a character’s mind’, Proceedings of the 1st ACM workshop on Story representation, mechanism and context (NY: ACM Press, 2004), pp. 63–66.

Conclusion: Engaging Hyper-narrative Interactive Cinema Hyper narrative interactive cinema refers to the possibility for users or ‘interactors’ to shift at different points in an evolving film narrative to other film narrative trajectories. Such interactive hyper-narrative film constructs may turn out to be the future form of narrative films. This book has suggested that a preliminary requirement for that to happen concerns the need to expand, rather than break away from the potential inherent faculties of film narrative, so that multi-tasking split-attention problems inhering in interactive hyper-narrative constructs can be managed and – most importantly – used to enhance rather than reduce engagement. Based upon a cognitive-constructivist approach to narrative and the viewer’s activity it has been suggested that film narrative is designed in such a manner that it rewardingly plays with the viewers’ strive to construct a cohering, intelligible, goal oriented trajectory out of the film’s audiovisual flow, by introducing surprises, distractions, diversions and postponements along the way. Moreover, cognitive and emotional viewer engagement results from the cumulative effect achieved by the narrative’s temporal dramatic succession of events that leads to closure. The book identified the points at which interactive hyper-narratives threaten narrative coherence, dramatic succession and closure. It then suggested feasible ways to turn these deficiencies into advantages. Hence, rather than allowing for the computer-enabled design of many intersecting narrative threads that lead to interactor cognitive confusion and disengagement, it has been suggested, following Bordwell’s analysis of ‘forking path’ narrative films (e.g., Run Lola Run), that optional narrative threads be restricted, and that transitions between narrative threads be confined to crucial decision points, arrived at after a causal coherent dramatic build-up and followed by a causal coherently built narrative thread. Moreover, the viewer should be allowed to construe meaningful interrelations between different threads, evoked through inter-narrative-thread temporal, spatial, action and character recurrence,

84

Conclusion

parallelism or variation. Also, in order to overcome the disengagement and confusion posed by hyper-narrative transitions between options that lead to different outcomes, it has been suggested that the maintenance of a cumulative effect derived from a narrative temporal dramatic succession that leads to closure, can be achieved through the use of repetition and return to scenes where temporal constrains are introduced, and through using primacy and recency effects. While the hyper-narrative principles outlined above may engender deep engagement for viewers, disengagement, frustration or a shallow gaming attitude can result when viewers turn into interactors that can behaviorally affect hyper-narratives. The book identified potential behavioral interaction disengagement in the forms of interactor figuration within the hyper-narrative, in the opposing lures of cognitive construction and behavioral intervention, in the audiovisual design of narrative transitions and in non-engaging interfaces. Rather than falling into the trap laid out by ‘identification’ theories it was suggested that avatars or hyper-narrative protagonists be conceived not as representing the interactor but as the interactors’ fictional friend towards whom she develops empathy and who reacts in unexpected ways to the interactor’s interventions. This allows the exploitation of the in-built divide between avatar and interactor, offering a complex engagement rather than reducing it to shallow character selfpersonification. It has been suggested for example that such complex interaction can be achieved when the protagonist serves as the interactor’s ‘host’ and the interactor is positioned as the protagonist’s advisor with the protagonist seamlessly alternating between addressing the interactor and acting within the narrative. Furthermore, establishing such interaction opens up the hyper-narrative camera to all the different points of view that an interactor-robot surrogate P.O.V. blocks. Hence, an example was given where the interactor is given the option to manipulate the shot-counter-shot transitions between two characters in dialogue, pointing to the opportunity this allows for creating surprise or suspense when the transition effected by the interactor leads to unexpected locations and story evolutions. The main objective in designing engaging narrative interaction with hyper-narratives concerns turning the opposing lures of behavioral interaction and cognitive construction into complementary lures that reinforce rather than dismantle the interactor’s engagement when transitioning back and forth from cognitively constructing the hyper-

Conclusion

85

narrative in her mind to behaviorally intervening in order to change its course. It was suggested that such complementary engagement can be achieved once behavioral interaction is allowed, required or blocked only when it is clearly consonant with the moments in which interactors (rather than protagonists) are cognitively and emotionally lured by the dramatic narrative succession, audiovisuality and type of interface action to want to change the course of events (due to narrative rather than gaming motivations). As for such luring audiovisuality, it was suggested that hyper-real and split screen computer associated tropes can be particularly powerful in luring interactors to behaviorally intervene at hyper-narrative transitional points, as well as offering powerful notions of recurrence, parallelism or variation allowing the interactor a challenging way to solidify hyper-narrative coherence, dramatic succession and closure when transitioning between different narrative threads. Hence, cueing different options within an interfaced shot that allows the interactor to actively mutate from option to option through morphing, slightly digitally replicated variations, or through splitting the screen into different simultaneous points of view that converge upon the same occurrence, is a particularly powerful way to lure interactors to intervene given the double dynamization of spectator and story that these tropes offer in their double axis of kaleidoscopic synchrony and diachronic motion. Likewise, a rewarding and narrative-engaging design of the relation between the interactor’s behavioral action and its audiovisual figuration may be achieved when there is a coherent reciprocal correspondence between the figured audiovisual evolution and the type of behavioral action applied to this figuration. Such reciprocal interfacing has to be geared towards cohering and forwarding the narrative engagement or for making your interaction with a protagonist complex, rather than offering stunning or funny gaming effects. The example given from the touch-screen interface of One Measure of Happiness (2003) points in that direction, in that the kind of ‘touch’ of a face on a screen can generate different protagonist reactions and narrative evolutions causally deriving from such ‘touch’. Hence, this book has suggested feasible ways to fulfil the promise of a deep cognitive-emotional engagement of interactors interacting with the computer associated emergent form termed here hyper-narrative interactive cinema.

Bibliography Altman, C.F. 1985. ‘Psychoanalysis and Cinema: The Imaginary Discourse’ in Nichols, Bill (ed.) Movies and Methods Vol. 2. Berkeley: University of California Press. Arnheim, Rudolph. 1957. Film as Art. Berkeley: University of California Press. ---. 1982. The Power of the Center: A Study of Composition in the Visual Arts. Berkeley and LA: University of California press. Aylett, R. 2006. ‘And they both lived happily ever after? Digital stories and learning’in Dettori, G., Giannetti, T., Paiva, A., Vaz, A. (eds.) Technology-mediated narrative environments for learning. Amsterdam: Sense Publishers. Barthes, Roland. 1978. Image, Music, Text. New York: Hill and Wang. Bates, Joseph. 1992. ‘Virtual Reality, Art, and Entertainment’ in Presence: The Journal of Teleoperators and Virtual Environments 1 (2): 133–38. Baudrillard, Jean. 1988. ‘Simulacra and Simulations’ in Poster, M. (ed.) Selected Writings. Stanford: Stanford University Press. Baudry, J. L. 1985. ‘Ideological Effects of the Basic Cinematographic Apparatus’ in Nichols, Bill (ed.) Movies and Methods Volume II. Berkeley: University of California Press. Benjamin, Walter. 1969. ‘The Work of Art in the Age of Mechanical Reproduction’ in Arendt, Hanna (ed.) Illuminations. New York: Schocken Books: 217–52. Ben Shaul, Nitzan. 2007. Film: The Key Concepts. Oxford: Berg Publishers. Bizzocchi, Jim. 2003. ‘Ceremony of Innocence and the Subversion of Interface: Cursor Transformation as a Narrative Device’ in CD-ROM Proceedings of the Digital Arts & Culture Conference. Melbourne: Digital Arts & Culture Conference (DAC). Bolter, J.D. and Grusin, R. 1999. Remediation: Understanding New Media, Cambridge, MA: MIT Press. Bolter, J. D. 1991. Writing Space: The Computer, Hypertext, and the History of Writing. Hillsdale, NJ: L. Erlbaum Associates. Bordwell, David. 1985. Narration in the Fiction Film. Madison: University of Wisconsin Press. ---. 2002. ‘Film Futures’ in SubStance 31.1: 88–104. Brannigan, Edward. 1992. Narrative Comprehension and Film. London: Routledge. Brewer, W. F. 1996. ‘The Nature of Narrative Suspense and the Problem of Rereading’ in Vorderer, P. Wulff, H. J. and Friedrichsen, M. (eds.) Suspense: Conceptualizations, Theoretical Analyses, and Empirical Explorations. Mahwah, NJ: Lawrence Erlbaum Associates. Brooks, Kevin. 1999. Metalinear Cinematic Narrative: Theory, Process, and Tool. Doctoral dissertation, Cambridge, MA: MIT. Bruner, Jerome. 1957. ‘On Perceptual Readiness’ in Psychological Review 64: 123–152.

88

Bibliography

Carroll, Noel. 1988. Mystifying Movies: Fads and Fallacies in Contemporary Film Theory. Princeton: Princeton University Press. ---. 1996a. ‘The Paradox of Suspense’ in Vorderer, P. Wulff, H. J. and Friedrichsen, M. (eds.) Suspense: Conceptualizations, Theoretical Analyses, and Empirical Explorations. Mahwah, NJ: Lawrence Erlbaum Associates: 71–91. ---. 1996b. ‘Nonfiction Film and Postmodernist Skepticism’ in Bordwell, David and Carroll, Noel (eds.) Post-Theory: Reconstructing Film Studies. Madison: University of Wisconsin Press. Chalom, E. and Bove Jr. V. M. 1996. ‘Segmentation of an Image Sequence Using Multidimensional Image Attributes’ in Proceedings of IEEE ICP. Chandler, P. and Sweller, J. 1991. ‘Cognitive Load Theory and the Format of Instruction’ in Cognition and Instruction, 8: 293–332. Clark, J.M. and Paivio, A. 1991. ‘Dual Coding Theory and Education’ in Educational Psychology Review, 3: 149–210. Darley, Andrew. 2000. Visual Digital Culture: Surface Play and Spectacle in New Media Genres. London: Routledge. Davenport, G. and Murtaugh, M. 1999. ‘Automatist Storyteller Systems and the Shifting Sands of Story’ in IBM Systems Journal, November, Vol. 36, no. 3: 446–56. ---. 1995. ‘ConText: Towards the Evolving Documentary’ in ACM, Multimedia '95, November. Davenport, G. and Bradley, B. 1994. ‘Seeking Dynamic, Adaptive Story Environments’ in IEEE Multimedia, Vol. 1, No. 3, fall. Dayan, Daniel. 1976. ‘The Tutor Code of Classical Cinema’ in Nichols, Bill (ed.) Movies and Methods. Berkeley: University of California Press. Derrida, Jaques. 1976. Of Grammatology. Baltimore: John Hopkins University Press. ---. 1980. ‘The Law of Genre’ in Critical Inquiry, 7.1 Eisenstein, S. M. 1949. ‘A Dialectic Approach to Film Form’ in Leida, J. (ed.) Film Form. New York: HBJ Books. ---. 1975. ‘Montage of Attractions’ in Leida, J. (ed.) Film Sense. New York: HBJ Books. Fracker, M. and Wickens, C. 1991. ‘Resources, Confusions and Compatibility in Dual Axing Tracking: Displays, Controls, and Dynamics’ in Journal of ExperimentalPsychology: Human Perception and Performance 15 (1): 80–96. Frome, J. and Smuts, A. 2004. ‘Helpless Spectators: Generating Suspense in Videogames and Film’ in TEXT Technology, no. 1: 13–34. Gertz, Nurit. 1993. Motion Fiction. Tel Aviv: Open University. Gladstones, W. and Regan, M. 1989. ‘Division of Attention: The Single-Channel Hypothesis Revisited’ in The Quarterly Journal of Experimental Psychology, 4/A (1): 1–17. Haraway, D. 1991. ‘A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in the Late Twentieth Century’ in Simians, Cyborgs and Women: The Reinvention of Nature. New York: Routledge. Ivengar, G. 1998. ‘Models for Classification of Video Sequences’ in Proceedings of SPIE Storage and Retrieval ‘98, San Jose, California, January. Jameson, Fredric. 2001. Postmodernism or, the Cultural Logic of Late Capitalism, Durham, NC: Duke University Press.

Bibliography

89

Jenkins, Henry. 1992. Textual Poachers. New York: Routledge. ---. 2004. ‘Game Design as Narrative Architecture’ in Noah Wardrip-Fruin and Pat Harrington (eds.) First Person: New Media as Story, Performance, Game. Cambridge: MIT Press: 118–30. Juul, Jasper. 2001. ‘Games Telling stories? A brief note on games and narratives’ in The International Journal of Computer Game Research, volume 1, issue 1, July. Kinder, Marsha. 2002. ‘Narrative Equivocations Between Movies and Games’ in Harries, D. (ed.) The New Media Book. London: BFI Publishing: 119–32. Knoller, Noam. 2004. ‘InterFace portraits: communicative-expressive interaction with a character’s mind’ in Proceedings of the 1st ACM workshop on Story representation, mechanism and context, NY: ACM Press: 63–66. Kristeva, Julia. 1980. ‘The Bounded Text’ in Desire in Language, New York: Columbia University Press. Lew, Michael. 2003. ‘Office Voodoo: a real-time editing engine for an algorithmic sitcom’ in Proceedings of SIGGRAPH 2003. Lippman, A. 1989. ‘Coding Image Sequences for Interactive Retrieval’ Communications and the ACM, Vol. 32, No. 7, July. Lunenfeld, Peter. 2002. ‘The Myths of Interactive Cinema’ in The New Media Book Harries, D. (ed.) The New Media Book. London: BFI Publishing: 144–54. Manovich, L. (2001), The Language of New Media, Cambridge MA: MIT Press. ---. (2002), ‘Old Media as New Media: Cinema’, Harries, D. (ed.) The New Media Book. London: BFI Publishing: 209–18. Mateas, Michael and Stern, Andrew. 2005. Façade in http://www.interactivestory.net/ Mayer, R. and Moreno, R. 2002. ‘Aids to Computer-Based Multimedia Learning’ in Learning and Instruction, 12: 107–19. Metz, Christian. 1976. ‘On the Notion of Cinematographic Language’ in Nichols, Bill (ed.) Movies and Methods. Berkeley: University of California Press. ---. 1985. ‘Story/Discourse: Notes on Two Kinds of Voyeurism’ in Nichols, Bill (ed.) Movies and Methods Vol. 2. Berkeley: University of California Press. Michelson, Annette. 1984. ‘Introduction’ in Michelson, Annette (ed.) Kino Eye, The Writings of Dziga Vertov. Berkeley: University of California Press. Mulvey, Laura. 1985. ‘Visual Pleasure and Narrative Cinema’, in Nichols, Bill (ed.) Movies and Methods Vol. 2. Berkeley: University of California Press. Murray, Janet. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. New York: The Free Press. Paivio, A. 1991. ‘Dual coding theory: retrospect and current status’ in Canadian Journal of Psychology 45: 255–87. ---. 1986. Mental Representations: a dual coding approach. Oxford, England: Oxford University Press. Pashler, H. 1994. ‘Dual Task Interference in Simple Tasks: Data and Theory’ in Psychological Bulletin 116 (2): 220–44. Piaget, J. and Inhelder, B. 1969. The Psychology of the Child. New York: Basic Books. Schank, Robert. 1991. ‘Case-Based Teaching: Four Experiences in Educational Software Design’ in Technical Report, Northwestern University. ---. 1977. Script, Plans, Goals and Understanding. NJ: L. Erlbaum Associates.

90

Bibliography

Sobchack, Vivian. 1987. Screening Space (2nd edn.). New Brunswick, NJ: RutgersUniversity Press. ---. 2000. ‘At the Still Point of the Turning World’ in Sobchack, Vivian (ed.) MetaMorphing. Minnesota: Minnesota University Press. ---. 1999. ‘Nostalgia for a Digital Object: Regrets on the Quickening of QuickTime’ in Millenium Film Journal No. 34: 4–23. Sparacino, Flavia. 1996. ‘HyperPlex: a World of 3D Interactive Digital Movies’ in MIT Media Laboratory. Sweller, J. 1999. Instructional Design in Technical Areas. Camberwell, Australia: ACER Press. Theune, M., Faas, S., Heylen, D.,and Nijholt, A. 2003. ‘The virtual storyteller: Story creation by intelligent agents’ in S. G ‫צ‬bel, N. Braun, U. Spierling, J. Dechau and H. Diener (eds) E 03: Technologies for Interactive Digital Storytelling and Entertainment, Fraunhofer IRB Verlag. Tinsley, G. E. 1995. Narrative Guidance of Interactivity. Ph.D. Thesis, MIT. Tulloch, J. and Jenkins, H. (eds.) 1995. Science Fiction Audiences: Watching Doctor Who and Star Trek. London: Routledge. Tynjanov, J. 1986. ‘The Foundations of Cinema’ in Eagle, Herbert (ed.) Russian Formalist Film Theory. Ann Arbor: Michigan Slavic Publications. Williams, Raymond. 1974. Television: Technology and Cultural Form. New York: Schocken Books. Wolf, Mark. 2000. Abstracting Reality: Art, Communication, and Cognition in the DigitalAge. New York: University Press of America. Zillmann, Dolf. 1996. ‘The Psychology of Suspense in Dramatic exposition’ in Vorderer, P. Wulff, H. J. and Friedrichsen, M. (eds.) Suspense: Conceptualizations, Theoretical Analyses, and Empirical Explorations. Mahwah, NJ: Lawrence Erlbaum Associates: 199–232.

Index of Names and Titles 24, 29, 45, 74 A Chorus Line, 68 Across the Sea of Time, 66 Adams, Ernest, 57 Adaptation, 21–2 All that Jazz, 71 Allen, Woody, 62 Altman, Robert, 35n10 Altman, Charles, F. 79n8, 87 Amélie, 62 Annie Hall, 62 Antonioni, Michaelangelo, 23, 36n27 Arenheim, Rudolph, 79n2, 87 Aylett, Ruth, 8, 14n12, 87 Bakhtin, Michail, 19 Barthes, Roland, 19–20, 36n18, 87 Bates, Joseph, 13n3, 40, 42–3, 53–4, 56n2n3n10, 68–9, 80n18, 87 Baudrillard, Jean, 13n2, 71–2, 80n22, 87 Baudry, Jean Louis, 79n4, 87 Bejan, Bob, 8, 57n13 Ben Arie, Udi, 78 Benjamin, Walter, 28, 37n44, 49–50, 58n25, 87 Bennett, Michael, 68 Ben Shaul, Nitzan, 36n19, 87 Bizzocchi, Jim, 77, 81n29, 87 Black or White, 71 Blade Runner, 72 Blair Witch Project, 24 Bolter, Jay David, 13n8, 37n32, 87 Bordwell, David, 14n22n24, 12, 16, 30, 31, 33, 34, 35n3,n8,n11, 36n19, 37n46n47, 83, 87, 88 Borges, Luis, 30 Bove Jr. V. M., 13n7, 88 Bradley, B., 13n7, 88 Brannigan, Edward, 14n22, 15–16,

35n2n4, 37n46, 87 Brewer, W. F., 56n6, 87 Brooks, Kevin, 8, 9, 13n11, 14n17, 26, 29, 35n1, 58n29, 87 Bruner, Jerome, 35n7, 79n2, 87 Cage, Nicolas, 21 Carroll, Noel, 10, 14n19n22, 16, 35n5n6, 36n19, 37n46, 41, 42, 56n5n6n7, 61–3, 79n7n10, 88 Ceremony of Innocence, 77, 81n29, 87 Certeau de, Michel, 26 Chalom, E., 13n7, 88 Chandler, P., 14n21, 36n22, 88 Chicago, 48 Citizen Kane, 69 Clark, J.M. , 14n20, 88 Cleo from 9–7, 26–7, 39 Coleridge, Samuel Taylor, 41, 60 Condon. Bill, 48 Contact, 79n9 Conversation, The, 36n26 Coppola, Francis Ford, 36n26 Costikyan, Greg, 57n17 Cronenberg, David, 72 Crouching Tiger, Hidden Dragon, 48, 72 Darley, Andrew, 35n13, 36n25, 71, 80n17n20, 88 Davenport, Glorianna, 8, 13n7n10, 88 Dayan, Daniel, 61, 76n6, 88 Deep Blue, 53 Demme, Jonathan, 64 Derrida, Jaques, 19, 36n15n23, 88 Donahue, H., 24 Doom, 40, 46, 50 Dreamgirls, 48 Eisenstein, Sergei, M., 47–8, 57n20,n21, 70, 81n27, 88 ER, 29, 37n45

92 Eskelinen, Markku, 57n22 eXistenZ, 72 Façade, 7, 13n4, 89 Figgis, Mike, 73–4 Forrest Gump, 72 Fosse, Bob, 71 Fracker, M., 57n12, 88 Frome, Johnathan, 58n27n28, 88 Gale, Bob, 8 Gertz, Nurit, 36n27, 88 Gladstones, W., 57n12, 88 Godard, Jean Luc, 23 Griffith, D. W., 70 Grusin, Richard, 37n32, 87 Haraway, Donna, 13n1, 88 Howitt, Peter, 30 Hyperplex, 8, 13n6, 88 I’m your Man, 8, 57n13 Immemory, 8, 24, 45 Inhelder, B., 37n43, 89 Inscape Storytelling, 8, 14n13 Ivengar, G., 13n7, 88 Jackson, Michael, 71 Jameson, Fredric, 12, 14n23, 27, 36n14, 88 Jenkins, Henry, 36n29, 37n37, 46–8, 57n17n22, 89, 90 Jeunet, Jean-Pierre, 62 Johnny Mnemonic, 65 Juul, Jesper, 57n22n24, 89 Kill Bill: Vol. 1, 48 Kinder, Marsha, 14n14n17, 26–7, 29, 35n1, 37n38, 39, 44–5, 56n1, 57n14 Knoller, Noam, 78, 81n30, 89 Kristeva, Julia, 19, 36n16n20, 89 Kurosawa, Akira, 36n26 Labyrinth project, 8 Landis, John , 71 Last Year at Marienbad, 26 Lee. Ang, 48, 72 Lee, Bruce, 48 Leonard, J., 24 Lew, Michael, 78, 89 Lippman, A., 13n7, 89 Lisberger, Steven, 65 Longo, Robert, 65

Index Lonze, Spike, 21 Low, Stephen, 66 Lunenfeld, Peter, 9, 14n16, 24, 26, 36n29. 37n31, 89 Manovich, Lev, 9, 14n14n15, 24–6, 36n25,n28n30, 37n33n36, 39, 45, 57n16, 73, 81n25, 89 Man with a Movie Camera, 25 Marker, Chris, 8, 24 Marshall, Rob, 48 MATE, 7 Mateas, Michael, 13n4, 89 Matrix, The, 48, 65, 72 Mayer, R., 14n18, 36n21, 79n3, 89 Mayhew, A., 77 Media Base, 7 Metz, Christian, 79n5, 89 Michelson, Annette, 37n35, 81n27, 89 Miller, Robin, 49 Miller, Rand, 49 Moreno, R., 14n18, 36n21, 79n3, 89 Mr. Payback: an Interactive Movie, 8 Mulvey, Laura, 79n11, 89 Murray, Janet. 9, 14n14n17, 26, 29, 35n1, 58n26, 66–8, 70, 80n13, 89 Murtaugh, M., 13n7n10, 88 Myst, 49, 50 Neshat, Shirin, 74 One Measure of Happiness, 78, 85 Oracle, 7 Oz project, 7, 9, 40 Paivio, Allan, 14n20, 79n1, 88, 89 Paper Mario: The Thousand Year Door, 79n9 Pashler, H., 57n12, 89 Piaget, Jean, 37n42, 89 Point of View: An Experiment in Linear Hypervideo, 8 Pulp Fiction, 36n20 Rashumon, 36n26 Red Dessert, 23 Regan, M., 57n12, 88 Resnais, Alain, 26 Run Lola Run, 30, 33, 36n24, 55, 83 Saussure de, Ferdinand, 19 Schank, Robert, 13n9, 89

Index Scott, Ridley, 72 Seinfeld, 29, 37n45, 57n13 Short Cuts, 35n10 Silence of the Lambs, The, 64 Sims, 47 Sliding Doors, 30, 32–4, 76 Smuts, Aaron, 58n27n28, 88 Sobchack, Vivian, 35n13, 71, 76, 80n23, 81n25, 90 Sonata, 57n13 Sparacino, Flavia, 13n6, 90 Star Trek, 25, 37n37, 90 Stern, Andrew, 13n4, 89 Sternberg, Meir, 33 Streep, Merrill, 22 Sweller, J., 14n21, 36n22, 88, 90 Synopsus, 7 Tarantino, Quentin, 36n20, 48 Timecode, 73–4 Tinsley, Galyean, A., 13n7, 90 Tron, 65

93 Tulloch, J., 37n36, 90 Tykwer, Tom, 36n24, 30 Tynjanov, Jury, 80n24, 90 Varda, Agnes, 26, 27, 39 Vardi, Guy, 8 Vertov, Dziga, 24–5, 37n35, 81n27, 89 Villon, G., 77 Virtual Storyteller, The, 7, 13n5, 90 Voodoo Office, 77–8, 89 Vsoft, 7 Wachowski, Andy, 48 Wachowski, Larry, 48 Weinbren, Graham, 57n13 Welles, Orson, 69 Wickens, C., 57n12, 88 Williams, M., 24 Williams, Raymond, 37n42, 90 Wright, Will, 47 Wolf, Mark, 35n13, 90 Zemeckis, Robert, 72 Zillmann, Dolf, 35n12, 42, 56n6n8, 90

Index of Topics 3–D, 5, 59, 66–70 ambiguity, 11, 46, 50, 52 attention, 7–11, 17, 28, 29, 35n6, 44, 48–50, 59–60, 63–4, 69. 73–4 audiovisual, 5, 10, 11, 12, 16, 17, 18, 24, 26, 28, 29, 30, 46, 47, 50, 56, 59–60, 63, 66, 68, 75–7, 80n17, 83–5, flow, 10–12, 16, 33, 46–50, 57n17, 60, 73, 78, 80n12, 83 avatar, 27, 77 empathy for, 43, 78, 84, identification with, 39, 41, 43, 65, 78, 84 behavioral interaction, 12, 35, 39, 43–6, 49–56, 57n13, 69, 70, 75–7, 79, 84–5 centering, 10, 17, 23, 79n2, 87 closure, 5, 6, 10, 17–19, 21, 25–7, 33–4, 47, 51, 53, 55–6, 59, 70, 75, 83–5 cognitive-constructivism, 5, 10–12, 17–18, 30, 35n7, 44–6, 49–51, 53, 59, 70, 79n2, 83 cognitive construction/interaction, 12, 53, 63, 76, 84 cognitive load theory, 11, 14n21, 36n22, 60, 83 cognitive-perceptual, 16, 24, 64 coherence, 5, 7, 10, 12, 16–19, 21–2, 25–7, 29, 31–3, 35n10, 36n24, 45–51, 53–6, 59, 63–4, 68–70, 73–8, 81n26n27, 83, 85 composite image, 73 computer, 9,11–12, 13n8, 14n18n15n23, 28–31, 36n21, 51, 60, 71–3, 75, 77, 79n3, 83, 85, 87, 89, algorithm, 24–5, 45, 89, data base, 24–5, 28, generated story/environment/audiovisuals, 7, 10, 12, 26, 29, 35n13, 36n25, 40, 53, 56n3, 59, 68, 70,

see also computer games, interaction, behavioral interaction computer games, 5, 9, 39–40, 45–6, 48–50, 57n22, 58n24n27, 79n9, 81n25, 89 see also game crucial transition point, 31–4, 51–2, 54–5, 75–6, 83 cumulative effect, 11, 18, 33, 44, 49, 54, 55, 76, 83, 84 cyborg, 7, 13n1, 88 de-centering, 9, 11, 20–1, 23, 25, 26, 29, 73 digital replication, 6, 12, 59, 70, 72, 76 distraction, 12, 17, 21, 23, 26, 28, 29, 31, 33, 44, 53, 54, 56, 68, 74, 80n11, 83 dramatic, 23, 40, 47, 53, 54, 68, 83 presence, 53, 69, succession, 5, 6, 11, 29, 32–4, 44, 49, 50–1, 53–5, 69, 75–6, 83, 85–6, suspense, 32, 35n12, 56n6, 90, ambiguity, 46, 52, see also crucial turning points, suspense dual-coding theory, 10, 59 emotion, 7, 11, 18, 20, 23, 31–2, 41–3, 47–50, 52, 53, 54, 60, 63–7, 69, 72, 75, 78, 83 empathy, 42–3, 78, 79n9, 84 game, 11, 14n14, 20, 24, 26–7, 29, 31, 39, 41, 45–50, 57n17n24, 58n26, 66–8, 88 skills, 11, 46, 49, 50, 77, scoring, 11, 49, puzzle solving, 11, 31, 47, 48, 49, 50 social typed, 9, 25, 44 see also computer games helplessness, 52, 54, 58n27n28, 88 hyperlink, 11, 29 hyper-narrative interactive cinema, 5, 11,

Index 12, 29, 83, 85 definition of, 7 hyper-reality, 6, 12, 59, 70–3, 75, 85 hypertext, 8, 9, 11, 13n8, 23, 24, 28, 53, 87 identification, 7, 10, 12, 39, 4–3, 56n5, 60–2, 65, 66, 78, 83, 84 immersion, 6, 9, 10, 11, 12, 40, 42, 46–9, 57n23, 59–70 indexing, 7, 8, 35n6 interface, 6, 12, 13, 59, 75–8, 81n29n30, 84, 85, 87, 89 intertextuality, 8, 19–20 labyrinth, 8, 20, 22, 31, 31 MIT Media Laboratory, 8, 13n6n7n11, 13n14, 37n32, 57n17, 78, 87, 89, 90 morphing, 5, 11, 12, 59, 63, 70–2, 75–6, 80n23, 85, 90 multi-tasking, 12–13, 44, 46, 49, 50, 52, 56, 76, 83 narrative definition, 15–18 new media, 7, 14n14n15n16, 36n13n25n28n30, 37n32n33n36, 57n16n17, 81n25, 87, 88, 89 non-closure, 9, 11, 21, 23, 25, 26, 29, 65, 68, 78 non-coherence, 5, 8, 10, 17, 25, 28, 29, 37n45, 42n37, 57n13, 60, 62, 71, 74, 90 open text, 11, 19, 20, 25, 26, 27, 36n24 optionality, 5, 7, 27, 30, 31, 33, 34, 44, 51, 52–6, 57n11n13, 72, 75–6, 83, 84, 85 point of view (P.O.V.), 8, 10, 21, 40, 41, 43, 57n11, 60, 61, 67, 68, 75, 78, 80n11, 84

95 postmodernism, 5, 7, 9, 10, 11, 12, 14n23, 18–21, 23–29, 36n14 n19, 44, 59, 72, 73, 81n25, 88 primacy effect, 33, 34, 55, 76, 84 protagonist. 5, 21, 22, 23, 51, 56n3, 58n30, 61, 64, 74, 75, 77, 78, 80n11, 84, 85 identification with, 39–42, 53, 84 empathy for, 42–3, 54, 78, 79n9 recency effect, 33, 34, 51, 55, 56, 76, 84 simulacrum (simulation), 7, 13n2, 41, 42, 56n10, 59, 67, 70–3, 80n22, 87 simultaneity, 8, 11, 20, 55, 56, 57n13, 60, 61, 65, 66, 73–6, 85 spectacle, 35n13, 36n25, 48, 63, 66, 70, 71, 73, 77, 79–80n11, 88 split attention, 12, 13, 21, 30, 44, 46, 49–50, 56, 59–60, 67–9, 73, 77, 80n11, 81n26, 83 split screen, 5, 11, 12, 59, 70, 73–6, 81n25n26n27, 85 surprise (in narrative), 17, 44, 64, 65, 72, 83, 84 suspense (in narrative), 17, 18, 33, 35n12, 42, 44, 52, 54, 56n6n8, 58n27n28, 62, 64, 70, 72, 75, 78, 79n10, 84, 87, 88, 90 suture theory, 56n4, 61, 62, 64 technological determinism, 11, 37n42 television (TV), 25, 37n42n45, 60, 62, 71, 74, 90 virtual reality, 13n2, 56n2, 65, 72, 80n18, 87 ‘willing suspension of disbelief’, 9, 41, 60, 69