Metonymical Mov(i)es

Lev Manovich: The Language of New Media. MIT Press: Cambridge, Massachusetts / London, England 2001. $34.95, 7×9, 354 pages, ISBN 0-262-13374-1

Upon reading Lautréamonts Chants de Maldoror (1869) surrealist king pin André Breton took over the author’s famous words “beautiful as the unexpected meeting, on a dissection table, of a sewing machine and an umbrella”, thus coining the Surrealist aesthetic of jarring juxtapositions.

Almost as beautiful as Breton’s observation was another unexpected meeting taking place some years later, namely, the use of punched 35mm movie film in order to control computer programs in the world’s first working digital computer built between 1936 and 1938 by German engineer Konrad Zuse.

This significant event which did not happen on a Surrealist dissecting table but, interestingly, in the appartment of Zuse’s parents in Berlin-Kreuzberg, further approached computing and media technologies – and thus further advanced the gradual entwinement of these two distinct historical trajectories.

It was, metaphorically speaking, this strange superimposition of ‘binary’ over ‘iconic’ code, that, according to Lev Manovich, anticipated the convergence of media and computer that followed about 50 years later: “All existing media are translated into numerical data accessible for the computer. The results: graphics, moving images, sounds, shapes, spaces, and texts become computable, that is, simply sets of computer data. In short media become new media.”(Manovich, Lev: The Language of New Media. MIT Press: Cambridge, Massachusetts / London, England 2001. 25.)

Manovich considers the historical merging of computer and media, symbolized by the superimposition of ‘binary’ code over ‘iconic’ code, so central an event for his argumentation that it also adorns the cover of The Language of New Media (2001) [see .jpg]. Beautiful as this symbol may be, it also represents the limitations of this valuable book: (analogue) media and new (digital) media are generally equated with visual media, in particular cinema.

Although photographic and moving images have contributed to the development of a language of (new) media, in this publication they are made to represent the whole of (new) media. To put it bluntly: Movies metonymically make up the language of new media. This is what one has to bear in mind when reading this insightful and valuable publication.

When asked in an interview about how long he had been writing the book, Moscow-born Lev Manovich, today Associate Professor in the Visual Arts Department at the University of California, San Diego, gives three alternative answers: it’s seven years since the first articles were published in 1992, fifteen years since he began to work with computer graphics around the mid-1980s (he came to New York in 1981), and twenty-five years since be had been studying fine arts, architecture and computer science in Moscow.

His 1993 Ph.D. dissertation in Visual and Cultural Studies, The Engineering of Vision from Contructivism to Computers, traced the origins of computer media, relating it to the avant-garde art of the 1920s.

His Language of New Media, which in many instances is connected to his Ph.D. thesis, is structured according to the principles of a computer: the chapters gradually advance the reader from five very basic principles of the underlying code via the interface, the operations and forms to surface phenomena, literally to the surface of the computer (screen).

The meeting of media and computer, and the computerization of culture as a whole changes the identity of both media and the computer itself – whereby, as Manovich asserts, “the identity of media has changed even more dramatically than that of the computer.” (p. 27) Therefore, the focus of Manovich’s book lies on answering the question of how the shift to computer-based media redefines the natureof static and moving images.

In the first chapter of the book Manovich describes five principles of new media which summarize the differences between old (analogue) and new (digital) media: 1. numerical representation, 2. modularity, 3. automation, 4. variability, 5. transcoding. First, all new media objects are composed of digital code, they are numerical representations. Two key consequences follow from that: new media objects can be described formally, i.e. by using a mathematical function, and they can be subjected to algorithmic manipulation.

Media thus become programmable. Second, all new media objects have a modular structure, i.e. they consist of discrete elements which maintain their independence even when combined into larger objects.

A Word document as well as the World Wide Web consist of discrete objects which can always be accessed on their own. Modularity thus highlights the “fundamentally […] non-hierarchical organization” (p. 31) of all new media objects (this actually holds true as long as you use the terms in a metaphorical way as Manovich does with most of the terms throughout his book.

As soon as you employ them in a literal way, it becomes clear that new media objects can, indeed, despite their principal modularity, be organized in strictly non-hierarchical ways). The numerical coding of media and the modular structure of a media object (i.e. the first two principles) allow, according to Manovich, thirdly, “for the automation of many operations involved in media creation, manipulation, and access.” Thus, “human intentionality can be removed from the creative process, at least in part.” (p. 32) Examples for automation can be found in image editing, chat bots, computer games, search engines, software agents, etc. The fourth principle of new media, deduced from the more basic principles – numerical representation and modularity of information – is variability. New media objects are not “something fixed once and for all, but something that can exist in different, potentially infinite versions.” (p. 36)

Film, for example, whose order of elements is determined once and for all, is diametrically opposed to new media whose order of elements is essentially variable (or, ‘mutable’ and ‘liquid’). Examples for variability would be customization and scalability. The fifth principle, and the “most substantial consequence of the computerization of media” (p. 45), is transcoding.

Transcoding basically means translating something into another format. However, the most important aspect is that the structure of computerized media (which, on the surface still may look like media) “now follows the established conventions of the computer’s organization of data.” (p. 45) Structure-wise, new media objects are compatible to, and transcodable into other computer files.

On a more general (“cultural”) level, the logic of a computer “can be expected to significantly influence the traditional cultural logic of media” (p. 46); that is, we can expect the “computer layer” to affect the “cultural layer”.

In the main chapters of the book Manovich discusses some of these changes (esp. the database as the “new symbolic form”). In the very insightful and entertaining “What New Media is Not” he scrutinizes some of the popularly held notions about new media, discussing the historical (dis)continuities between old and new media.

The Cultural Interfaces chapter analyzes how three cultural forms of printed word, cinema, and a general human-computer interface (HCI) contributed to shaping “cultural interfaces” during the 1990s.

Manovich uses the term ‘cultural interface’ to describe a “human-computer-culture interface – the ways in which computers present and allow us to interact with cultural data.” (p. 70)

Now, according to Manovich’s main thesis, “[r]ather thanbeing merely one cultural language among others, cinema is now becoming the cultural interface […]” (p. 86). Cinematic ways “of seeing the world, of structuring time, of narrating a story, of linking one experience to the next, have become the basic means by which computer users access and interact with all cultural data.” (p. 78f.).

Here, one starts wondering which computer users he is talking about: definitely not about computer users in general. What we are confronted with here is another of Manovich’s metonymical moves: without much notice, Manovich deduces from very special forms of new media, in this case computer games and Virtual Reality (VR), a whole general language of new media.

While one can say that cinematographic approaches to interfacing “cultural data” were typical for the whole VR industry’s discourse in the beginning of the 1990s, cinema can by no means be called “the cultural interface”. Cinema is just one of the possible interfaces to datascapes, among many others.

In the following chapters Manovich meticulously analyses how the shift to computer-based media redefines the nature of static and moving images: “New media may look like media, but this is only the surface.” (p. 48)

He analyses the operations, illusions and forms of new media. According to Manovich, the main operations of new media are selection, compositing, and teleaction. Digital compositing refers to the process of “assembling together a number of elements to create a single seamless object.” (p. 136)

This is what makes it radically different to montage of the 1920s up to the 1980s: it is essentially “anti-montage” (p. 143). While montage “aims to create visual, stylistic, semantic, and emotional dissonance between different elements”, compositing aims to “blend them into a seamless whole, a single gestalt.” (p. 144).

Teleaction, as the third operation of new media, enables to see and act at a distance. Manovich prefers the notion of “teleaction” to “telepresence” exactly because one is not present in the distant location, but one acts at a distance. Teleaction allows the user – given that information can be transmitted in real time – “to manipulate reality through representations” (p. 165), through so-called “image-instruments” which allow the user “not only to represent reality but also to control it” (p. 167).

Here, Manovich includes a great passage on distance and aura, namely, on Benjamin and Virilio, concluding that for both of them, “distance guaranteed by vision preserves the aura of an object […] while the desire ‘to bring things closer‘ destroys objects’ relations to each other, ultimately obliterating the material order altogether and rendering the notions of distance and space meaningless. […] The potential aggressiveness of looking turns out to be rather more innocent than the actual aggression of electronically enabled touch.” (p. 175)

In the “Illusions of new media” chapter Manovich entertains the reader with some very enlightening remarks on the partiality and unevenness of synthetic realism generated by VR engines.

An animator using a particular software can, for instance, “easily create the shape of a human face, but not hair; materials such as plastic or metal, but not cloth or leather; the flight of a bird but not the jumps of a frog.” (p. 193)

This unevenness of synthetic realism not only reflects the range of problem addressed and solved, but als bears witness to the fact that the research of particular problems was “determined by the need of the early sponsors of this research – the Pentagon and Hollywood.” (p. 193) In addition to this sponsor-induced focus on certain areas in research, it is also the researchers themselves who “privilege particular subjects that culturally connote the mastery of illusionistic representation” (p. 195).

Examples for these “icons of mimesis”, or privileged signs of realism, would be, e.g., animations of smoke, fire, sea waves, and moving grass. Also highly amusing is Manovich’s witty comparison between Jurassic Park and Socialist Realism.

His thesis is that both can be understood as synthetic images or constructs pointing to a future event which, in order to be understood by their contemporaries, have to be disguised in ‘sub-optimal’ aesthetics.

While the synthetic film images in Jurassic Park are the “result of a different, more perfect than human, vision”, “the vision of a computer, a cyborg, an automatic missile” (whose images were too perfect and thus for the film had to be degraded quality-wise), it is also, according to Manovich, “a realistic representation of human vision in the future when it will be augmented by computer graphics and cleansed of noise” (p. 202).

Likewise, also Socialist Realism “had to retain enough of then-everyday reality while showing how that reality would look in the future when everybody’s body would be healthly and muscular, every street modern, every face transformed ba the spirituality of communist ideology.” (p. 203) Socialist Realism never depicted this future directly: “The idea was not to make the workers dream about the perfect future while closing their eyes to imperfect reality, but rather to make them see the signs of this future in the reality around them.” (p. 203)

It is here that Manovich makes the connection between the Hollywood movie and Socialist Realism: Just “as Socialist Realist paintings blended the perfect future with the imperfect reality, Jurassic Park blends future supervision of computer graphics with the familiar vision of the film image.” (p. 204)

The most important forms of new media are, according to Manovich, database and navigable space. Self-confidently, Manovich states in the beginning: “After the novel, and subsequently cinema, privileged narrative as the key form of cultural expression of the modern age, the computer age introduces its correlate – the database.” (p. 218).

Databases which Manovich calls the “new symbolic form of the computer age” (p. 219), appear as “collections of items on which the user can perform various operations – view, navigate, search. The user’s experience of such computerized collections is, therefore, quite distinct from reading a narrative or watching a film […]” (p. 219).

The database (a term which Manovich uses metaphorically, i.e. not only strictly for databases, but in a more general sense) presents the world as a list of items which it refuses to order. In contrast, narrative “creates a cause-and-effect trajectory of seemingly unordered items (events).” (p. 225)

While database and narrative seem to be diametrically opposed in the beginning of the chapter, it increasingly becomes clear in the course of Manovich’s argument that linear narrative is just one method of accessing data among many other possible trajectories.

Manovich redefines the concept of narrative: “The ‘user’ of a narrative is traversing a database, following links between its records as established by the database’s creator.

An interactive narrative (which can be also called a hypernarrative in an analogy with hypertext) can then be understood as the sum of multiple trajectories through a database.” (p. 227)

Here, Manovich observes a very interesting change concerning the database logic: In old media, as outlined, e.g. by Roman Jakobson,(C.f. Jakobson, Roman: Linguistik und Poetik [1960]. In: Ders.: Ausgewählte Aufsätze 1921 – 1971. Frankfurt/Main 1993. 83-121. Jakobson, Roman: Der Doppelcharakter der Sprache und die Polarität zwischen Metaphorik und Metonymik[1960]. In: Theorie der Metapher. Hg. v. A. Haverkamp. Darmstadt, 1996. 163-174.) the database of choices from which narrative is constructed is implicit (the paradigm); while the actual narrative is explicit (the syntagm). New media completely reverse this relationship: “Database (the paradigm) is given material existence, while narrative (the syntagm) is dematerialised. Paradigm is privileged, syntagm is downplayed. Paradigm is real; syntagm virtual.” (p. 231)

As historical predecessors Manovich mentions two “database filmmakers” who reconcile database and narrative form: Dziga Vertov and Peter Greenaway. Vertov’s Man with a Movie Camera literally projects the paradigm onto the syntagm.

Therefore, Manovich concludes, Man with a Movie Camera cannot simply be labeled “avant-garde”, exactly because it never arrives at anything like a well-defined language (like all avant-garde films), but, rather, “it proposes an untamed, and apparently endless, unwinding of techniques, or, to use contemporary language, ‘effects’, as cinema’s new way of speaking” (p. 242).

Man with a Movie Camera is a “database of film techniques, and a database of new operations of visual epistemology, but also a database of new interface operations that together aim to go beyond simple human navigation through physical space.” (p. 276)

As Manovich argues, while interactive interfaces foreground the paradigmatic dimension, they are yet still organized along the syntagmatic dimension: “Although the user is making choices at each new screen, the end result is a linear sequence of screens that she follows.” (p. 232).

Why do new media insist on the sequential form, why this persistence on a linear order? Manovich’s hypothesis is that new media follow “the dominant semiological order of the twentieth century – that of cinema” (p. 232):

[C]inema replaced all other modes of narration with sequential narrative, an assembly line of shots that appear on the screen one at a time. For centuries, a spatialized narrative in which all images appear simultaneously dominated European visual culture; in the twentieth century it was relegated to ‘minor’ cultural forms such as comics or technical illustrations. ‘Real’ culture of the twentieth century came to speak in linear chains, aligning itself with the assembly line of the industrial society […]. New media continue this mode, giving the user information one screen at a time. At least this is the case when it tries to become ‘real’ culture (interactive narratives, games); when it simply functions as an interface to information, it is not ashamed to present much more information on the screen at once, whether in the form of tables, normal or pull-down menues, or lists. (p. 232)

While it would be really interesting and necessary to critically discuss Manovich’s notion of “real culture” and of the “cultural interface” (when exactly does an interface become ‘cultural’?

Should not the computer itself be included in the notion of ‘culture’?), he introduces many other notions that would be likewise worth discussing, like “cinegratography”, and the “loop as narrative engine”.

Let’s stop here and try to summarize. Lev Manovich’s The Language of New Media is a very well written book (which can also be used as a database) which guides the reader through its rich contents by always providing short summaries of the chapter s/he just read or s/he is about to read.

The author illustrates his arguments very well, not by providing images (apart from some stills from Man with a Movie Camera there are no illustrations whatsoever), but by always giving a broad range of examples from his own practical working with these new media technologies.

Moreover, many examples he uses to illustrate his arguments are net or media art projects and not Hollywood movies, thus giving a new context to these projects, but also implicitely underlining the avant-garde role of art in the digital realm.

While reading the book I wondered why I could not recognize the world Manovich is describing.

I would claim that one can experience new media without ever being so massively confronted with visuals or cinematic code as Manovich suggests. Manovich writes that “the visual culture of a computer age is cinematographic in its appearance” (p. 180).

If you talk about computer games, or about VR discourses developed over the last ten to twenty years, yes, it is cinematographic plus some other elements. Hollywood’s and Silicon Valley’s language of new media is indeed massively cinematographic.

But, for example, if you talk about net culture, or media art, fields I have been involved in over the last ten years, or even if you talk about practices like chatting or SMS culture, then you just cannot claim that we have to deal with a visual culture which is predominantly cinematographic.

The reader also has to bear in mind that when Manovich speaks about ‘computer culture’ he essentially talks about computer game culture, VR development, and, partly, also about what others have at times called the “Californian Ideology”.(Barbrook, Richard / Cameron, Andy: The Californian Ideology. In: Nettime 1995.)

Similarly, when he speaks about new media, he essentially means those visual cultures that predominantly work with filmic or cinematographic codes. Generally, any attempt to define a field as broad as the “language of new media” has to be welcomed quite enthusiastically.

If one cannot expect an author of such a study to include several historical trajectories (there are, as I would claim, at least two important ones: the trajectory of photography, film, and television, and the trajectory of telegraphy, radio and the Internet, with television and Internet converging at present), then one should at least expect that the author makes it clear that, while writing about the “language of new media” s/he is focusing only on one trajectory. However, by describing in detail, e.g., navigable space, database, and “image-instruments”, he already points to the fact that new media are not indebted to the filmic paradigm only.

Still, Manovich repeatedly comes back to implicitly using the notion of visual media as a metonymy for media. Perhaps, thus, in order to avoid misunderstandings, the book should have been called “The Language of New Visual Media”.

In short: Manovich’s precise observations of operations and forms of new media that can be found throughout the whole book come from his practical experience and make the book a very valuable, sometimes funny and even entertaining source of information on new media.

This is a wonderful example of the fact that whoever writes on new media should also be in the state of using them actively. If one takes into account the points I have mentioned, i.e. Manovich’s focus on the visual, on games and VR and cinema, then reading The Language of New Media is really rewarding.