What new media not is Part of my research is

What new media not is Part of my research is concerned with the language aspects of traditionally and digitally produced cultural text, with visual language. Lev Manovich's work seems to aim in the same direction, but in fact offers me an opportunity to differentiate and fine-tune my position. In “the language of new media” Manovich defines various criteria delineating the nature of new media, as in digital media, like numerical representation, modularity, automation, variability and transcoding. He then moves on to discuss interfaces and HCI, analyse various aspects of interaction operations and returns to conclude his analysis in view of his favourite medium, the cinema.

I do not accept Manovich’s criteria as defining new media conclusively. Before discussing Manovich’s criteria of new media, I look at his definition of object. Throughout the book he uses the term object synonymously with “new media object, product, artwork and interactive media” , i.e. the content and the medium are one, a unity.

On the other hand he uses object in the computer science way to indicate the “modular nature in object oriented programming languages such as C++ and java… ”, i.e. a module of a code structure. This can be confusing as one definition points to visual representation and the other to underlying invisible code. I agree with Mcluhan here and find it necessary to distinguish between content and medium as separate entities and will break down Manovich's explanations accordingly.

If new media were defined by numerical representation, modularity, automation, variability, and transcoding only, a 100-year-old woven paisley rug would be a “new media object”. Lets start with “numerical representation”, as Manovich defines it in terms of digital code, as mathematical form : A woven carpet is defined by a strict grid, by horizontal and vertical threads. This is a binary description as we find it in assembly code, as ’X / 0’, or ‘off/ on’ or ‘one and zero’. Assembly code is a low-level computer language, which can be directly understood by the processor. However few people write computer programs in low level languages, the norm is that programs are written in high level languages, which are close to human languages, compilers then render the high level code into assembly code, or machine code as it is also called. So, if we talk about digital code as in binary code, we talk machine language; if we take digital code as programming language, we need to extend “numerical” to “alpha-numerical”. Strictly binary code only stands for the lowest level of the underlying code structure, it describes the grid of the carrier, the medium or the woven carpet. It does not give us an impression or depiction of the visual representation level.

Picture paisley.jpg as visual and alphanumerical code represenation (jpg opened in MS word) To discuss the “content”, i.e. the displayed pattern, we can look at Manovich’s second definition of numerical representation as in algorithms. Some Persian carpets use as “basis of these designs the head-bent paisley motif common in both Indian and Iranian patterns from older times” in a self-similar fashion. The first recorded use of these patterns in England dates back to 1733 , indicating a much older history of those patterns. Paisley patterns can be described mathematically as “Julia set: a non-Euclidean limit set., z z2 + c when c = 0” . The pattern as visualisation of this non-linear mathematical geometry is achieved by introducing “(…) the smallest possible non-zero value of c and the Julia set gets distorted. As we distort in the inversion pictures, we begin to get the beautiful Paisley patterns (…) . A Julia set is an algorithm that describes chaotic behaviour. Chaos theory, in “its conceptual elements had already been appreciated by Leibnitz in the 17th century and Poincaré in the 19th century, (…) did not become fashionable until the 1980s when scientists began to realise that the phenomenon is widespread in the natural world.” “(…) Non-linearity is known to be a crucial ingredient in chaotic systems” So far my carpet still seems to fit the description of a new media object as it conforms to an underlying binary structure that displays algorithmically organised content.

Furthermore the displayed content follows fractal patterns in terms of modularity and variability, Manovich’s next two criteria for new media objects. Both terms are used in chaos theory and Manovich refers directly to the “fractal structure of new media: Just as a fractal has the same structure on different scales, a new media object has the same modular structure throughout. Media elements, be it images, sounds, shapes or behaviors, are represented as collections of discrete samples (…) but they continue to maintain their separate identity.” Again, this conclusion is only possible because Manovich does not distinguish between content and medium. I think we need to be more specific here. Code, as carrier of content, describes media elements on a technical level mathematically but not necessarily as a formula i.e. an image would be described by its RGB (red – green – blue) values per pixel per grid positions.

However this is still a linear description, even though the values of the parameters might vary. A shape would be described as form (e.g. circle), diameter (e.g. 3 cm) and colour (e.g. pantone 123). That makes the underlying code a formula, but does not make the content or its visualisation self-similar. The execution of the math results in independent instances. So while digital media elements can be seen as fragmented as far as the code that visualises them is concerned, they are not fractal in their visual representation, unlike my earlier example of the paisley carpet. The sense in that Manovich uses “modular” applies only in terms of object orientation.

In object orientated environments various independent components are arranged in a certain fashion, a programming shell or container. Manovich uses as example ‘Macromedia Director’, which is author software: software that creates software. Macromedia director uses two operation modes: a linear timeline offering frames in which all kinds of media elements can be placed for linear playback (i.e. picture one follows picture two, etc) and a programming mode in which the playback order can happen in a non linear fashion, according to user input. The programming language in question is an object oriented high-level language called “lingo”; commands would look similar to English language, i.e. if user clicks button “A”, then play sound “A”. This is also the operation mode Manovich calls “discrete”, as this access fragments the linearity of the playback of continuous media elements (like the frame by frame view of a cinema movie). Consequently, while the overall structure of a new media construct can be object oriented and non-linear, the elements involved are independent and self-contained. The fractal metaphor is inappropriate, however, as for fractals the criterion self-similar modules in various scales needs to be fulfilled. Manovich does include variability in his set of criteria, but uses it synonymously with “copies” , which are “mutable and liquid” and not necessarily intertwined with the modules. Used separately, the terms modularity and variability delineate objects, but not fractals.

Manovich is the only theorist I know of who addresses the loss less reproduction of media products in mass production to “old media”, while “new media” in contrast is characterized by variability. Instead of identical copies a new media object usually gives rise to many different versions.” As example he uses websites, which are created on the fly from databases using a set of templates. For instance like in online news. While I agree that this practice is highly computer specific in terms of speed and temporary use value of the displayed information, I would not accept the products as “variations of each other”. If an object takes content as variable and technicality as fixed template, every painting is a variable of another, as they are all using colour pigments in various quantities, spread over canvas. Or, to return to my carpet example, every woven carpet, that displays different patterns or designs for that matter, it does not even need to be the paisley pattern.

The next criterion Manovich lists to identify new media objects is automation. He distinguishes between “low level automation” and “high level automation” Early computerized low-level automation overlaps in its application greatly with electro-mechanic controls as we find it in factories or domestic appliances, like washing machines: simple parameter control, loop control, status indication. It is generally agreed that the historic starting point for ‘digitally controlled production’ dates “around 1800, (when) J.M. Jacquard invented a loom which was automatically controlled by punched paper cards. The loom was used to weave intricate figurative images, including Jacquard’s portrait.” This fact supports directly my position, as far as low-level automation is concerned the paisley carpet still counts as new media object. Low-level automation in media production usually comprises repetitive tasks like image editing batch processing, i.e. re-scaling a set of pictures about a certain percentage or controlling loops.

As examples of “high level automation” Manovich lists agents, game characters, and avatars, which act on more or less complex underlying AI (artificial intelligence) engines. Here I am in full agreement with Manovich, these kinds of representations are truly unique to user-computer interaction and communication. Agents are anything from filters (e.g. set up my default word file in this document format with this typeface as “normal” font in this style) to customised search engines (find product “A” for this price in this region). An agent is a non-pictorial, conceptual representation of the user via a set of instructions, defined by the user. A computer game character is a pictorial representation of the user within a digital (game) environment. Sometimes the user identifies with a given characters in the game (like in Lara Croft, the player is always Lara, you can not choose to fight Lara), sometimes users can choose between a variety of characters (like in role games). While the design of the visual representation in this interaction is pre-defined, the user always determines the final definition of those characters via the behavior. An avatar “is an interactive, graphical representation of a human being in a virtual reality environment”. In contrast to a game character, where the user identifies with a given character, an avatar actually represents the user in cyberspace. Usually one can design their own avatar, either from a set of design elements or use individual designs, to represent oneself for instance in a cyber chat room.

Agents, game characters and avatars are good demonstrations of various interactive interfaces and therefore an interesting starting point for the discussion of interfaces as such. These examples offer the possibility to contrast computer-human interactivity versus CH - interpassivity which is what I call interfaces that regard interactivity as multiple choice option, e.g. to press one of three offered buttons. I agree with Manovich again in rejecting a definition of interactivity in mechanical terms, “equating it with physical interaction between a user and a media object (pressing a button, choosing a link, moving the body), at the sake of psychological interaction. The psychological process of filling in, (…), recall and identification, which are required for us to comprehend any text or image at all, are mistakenly identified with a objectively existing structure of interactive links” Manovich correctly identifies the current understanding of interactivity where the majority of users are presented with “pre-programmed” solutions while before we would form our own view how to proceed, “follow our own private associations”. Now “interactive media asks us to identify with somebody else’s mental structure.” Bearing in mind that Manovich’s focus as new media practitioner is game production, i.e. full screen pictures form interfaces, interactivity for him is also the metal process involved in consuming and making sense of images of various kinds. Interaction becomes synonymous with interpretation. “All classical, and even more so modern art, was already “interactive” in a number of ways. Ellipses in literary narration, missing details of objects in visual art and other representational shortcuts required the user to fill-in the missing information. This sounds very similar to Mcluhan’s attempt to address various media types as hot and cool media, according to their demand on the user to fill in the gaps, i.e. photography is a hot medium as it is rich in infor-mation and requires little “mental interaction” by the user to get the message while a cartoon is reduced / low resolution or cool, and requires a lot of user interaction to create ‘the full picture’.

Manovich refers directly to Mcluhan’s “revolutionary works in the 1950s” in his chapter about transcoding, the last criterion to identify new media. To “’transcode’ something is to translate it into another format,” i.e. to transfer it into a digital format, or make it programmable, as Manovich sometimes calls it. Again, this sounds similar to Mcluhan’s “the content of any medium is always another medium.” Mcluhan separates “content” and “medium” in order to be able to look at the medium. Manovich also identifies two layers involved in media presentation: “the cultural layer and the computer layer”, with the cultural layer being “cultural data” like “texts, photographs, films, music, multimedia documents, virtual environments;” and the ”computer layer” as databases and its functionalities like searching and ordering. The Internet, in Manovichs view, is “one huge distributed media database.” But here is where the similarities end. Manovich then carries on to outline that the two separate levels: content and interface” are not only “old dichotomies” and “ content – form and content - medium can be re-written as content – interface”, but “content and interface merge into one entity, and no longer can be taken apart.” To support his viewpoint he refers to Bolters and Grusins study of new media in their book “Remediation” in which they define the medium as “that which remediates, repurposes, remedies and even replaces” content during its journey through various media. “New digital media oscillate between immediacy and hypermediacy, between transparency and opacity.” Bolter and Grusin claim that the content of new media makes the medium disappear “and leaves us in the presence of the thing represented” in order to achieve “transparent presentation of the real” This notion of reality creates “immediacy” for the user, furthermore in new media environments “immediacy depends on hypermediacy,” the mosaic view of media: Various media combined, interconnected by random access and collapsed into one window, “our culture wants to multiply its media, and to erase all traces of mediation: ideally it wants to erase its media in the very act of multiplying them.” In short, new media revokes the medium by either making it invisible though transparency or covering it up with the multiplication of old media, so the density of the conglomerate hides the underlying medium. The interface is absorbed and erased in the process.

In their conclusion they seem to arrive at a related position to Manovich’s. “Digital media is best understood through the ways in which they honour, rival, and revise linear-perspective painting, photography, film, television, and print. What is new about new media comes from the particular ways in which they refashion older media and the ways in which older media refashion themselves to answer the challenges of new media.” This sounds similar to “we increasingly interface to cultural data: texts, photographs, films, music, multimedia environments” and consequently extends the definition of HCI (human computer interface) to human computer culture interfaces, which he abbreviates to “cultural interfaces.” However, the study “remediation” investigates the role of reality in media representation and the reality of the hyperreal. It advocates a user-centred approach expecting media to transfer “the experience from one person to another.” Moreover it is concerned with the perception of the user and the “formal relations within and among media as well as the relations of cultural power and prestige” .

In contrast Manovich argues from the expert point of view, taking a production centred position. He discusses digital concepts in the context of construction, but not in terms of presentation.

“Cultural interfaces try to balance the concept of a surface in painting, photography, cinema, and the printed page as something to be looked at, glanced at, (…), without interfering with it with the concept of the surface in a computer interface” . Instead of looking at the effects and meaning of new media experience he returns to explore media production processes in a self contained design area, e.g. computer game production, in view of its linear predecessor, cinema.

For instance while I agree that cinema samples time in a non linear fashion, particularly when montage techniques are used, the implied target group is an audience, not an interactive user; the consumption process is anticipated to be passive and continuous. The presentation collapses into linear flatness, the narrative controls the viewer’s perception. Manovich sets culture synonymous with art and representation of art, information culture can be thought of as visual culture , interactivity is a myth , the user is a consumer. His fascination with the medium cinema leads him to map digital media back to analogue media, the only difference being the format, which is programmable and offers random access.

Random access sounds like something accidental, uncontrolled while it actually means the opposite: succinct controlled access to an object in question, i.e. a sound track on an audio CD or a picture in an encyclopaedic database. Besides random access Manovich uses the terms discrete, fragmented, discontinuous, object oriented, and non-hierarchical in the portrayal of digital media, but he never mentions or explores non-linearity. This is surprising as the programming languages he mentions are object oriented and not structured in a linear manner like “C” or “Basic”. The concept of organising content in a non-linear way must be familiar to him, but he seems to be consumed with the idea that content needs to be arranged in a narrative. He even views “the database and the narrative as natural enemies” in order to maintain his linear pursuits. It is because of the database, that “many new media object do not tell stories; they don’t have an beginning or end; in fact, they don’t have any development, thematically, formally or otherwise…” Technically a database is “defined as a structured collection of data. The data in the database is organised for fast search and retrieval and therefore more but a simple collection of items. (…) Hierarchical databases use treelike structures, object oriented databases store complex data structures, called “objects.” The idea of the database coming “to function as a cultural form of its own” is an intriguing idea, as “an architectural plan and a database present a different model of what a world is like.” It actually forms one of the key ideas I will explore throughout my work.

In Manovich’s discussion this information space is quickly reduced to a container for cultural objects such as multimedia encyclopaedias or virtual museums on CD-Rom, a collection by its very definition . In the example of the internet this scenario is amplified, the unordered collection displays an “open nature” which can not “keep a coherent narrative or any other development trajectory though the material, (as) it keeps changing.” Order is only restored in “computer games, (…) experienced by their player as narratives.“ The rejection of change and temporariness as values in their own right, of non-linear story telling as valid contemporary narrative and of the user defined journey through information landscape as compelling experience strikes me as a very limited conception of digital new media, particularly as it is published in 2001. Or as Scott Lash puts it: “Culture has left its stead as representational an narrative and has become - as Benjamin suspected - architectural.” While I appreciate most people have no means to look beyond the interfaces, i.e. cannot access, design or produce digital code, Manovich can and should. When he states “web pages are open, (…) are computer files which can always be edited” or “a number of different interfaces can be created to the same data” he portrays himself as an expert user and producer of digital media. Just how many computer users can edit WebPages or create various query interfaces to databases? The appreciation of users in Manovich’s discussion oscillates between their anticipation as audience and their embodiment in terms of hard- and software as “a computer program can use the information about a user to (..) automatically customize the site “according to detected “hardware and browser (software).” Again, meaning the visual presentation of the site, not the content.

Manovich’s obsession with the medium cinema might make more sense after consideration of an observation Bolter and Grusin add to the discussion, “Meanwhile, computer game makers hope that their interactive products will someday achieve the status of first-run films, and there is even an attempt to lure film stars to play in these narrative computer productions.” More subject oriented interesting aspects Bolter and Grusins introduce, beside the already mentioned “genealogy of remediation”, are the consideration of “the possibility that the desire for immediacy, at least as expressed in visual technologies of transparency, might itself be an exclusively male desire,” “the notion of interconnectedness of media amongst each other as well as amongst social and economical forces,” and the idea of preserving presence and archiving experience.

While I agree that certain aspects of media objects can be archived, so viewers can get a glance of that experience, I would argue that it mixes and intertwines with their own experience and invariably creates a new experience. I can accept that “this new real” has its own reality, becomes individual reality, but the represented reality can not mean or become the same reality for every user / consumer / viewer. Manovich’s notion of the archive deals with the past instead of the now of reality, envisioning a passive user; the stored media objects in turn become subject to retrieval and consumption. Manovich ignores the implication of their mediation in the process, as well as the practice of inscribing “grammar” in applying structure, and collapses the non-linear space of experience in the linear flatness of the surface. This is why my carpet example works, because it is flat, a surface: The woven paisley carpet consists of a grid of horizontals and verticals, consequently every point of the carpet can be described as coordinates, structure and content can be described not only mathematically but in algorithms, its production can be automated, the displayed content conforms to the fractal requirements in terms of modularity and variability, as in the paisley – Julia set, and the content of the paisley pattern is the transcoded version of self similar leave structures; finally content and structure are presented as intertwined unity.

This differs considerably from my understanding of interfaces and their use: I do not view interfaces on the internet as digital representation of cultural artefacts as listed above, but treat interfaces as an individual layer between the content (pictorial or textual) and the medium, the internet. In techno-culture the production of the technology layer is a design discipline in itself, irrevocably seperating the process of preparing the medium and displaying the content. While the creative skill and technical knowledge of design and production process used to be combined in one person, i.e. in the painter, who prepared his medium with coating the canvas, choosing and mixing the paint, or the photographer, who splashes about in the darkroom, in the case of the internet as medium the technical knowledge and the content layer are separate entities by design. Institutional design that is, as code design and production, i.e. programming, is taught in the departments of computer science while visual design and production is confined to the realm of fine art academies and design technique / craft oriented colleges.

Hence “To say” the medium is the message” is to say that the technology is the content,” sits not in contrast with my intention to separate interface and content, as both layers are subject to technological production: the technology that forms the content, i.e. PhotoShop, image editor or illustrator, and the technology that forms the interface, e.g. html, dhtml, java script, java, etc. Mcluhan’s “we become what we behold, we shape our tools and thereafter they shape us, becomes “technology forms our tools and thereafter forms us” or our perception of the world around us.

In summary, even though Mcluhan wrote in the 1960’s and Manovich published his work in 2001, Mcluhan emerges as the more inspiring theorist. At first glance Manovich appeared to be the perfect starting point and platform for my research as he, like myself, develops his theory based on practical experience. However, in discussing new media objects, his “bottom-up trajectory of the book as a whole” always revolves around and ends up at the surface, with cinema as preferred representation. Even though he understands and explains the nature and structure of networks and its objects, he maps everything back to linearity and the limited “snapshot view” that comes with that. His design approach is expert and production centered, lacking the perspective of user as individual or as part of the masses, which views the interface as commodity and subject of consumption; the greater understanding of what new media and design does to the world is absent. He is caught by the surface and always ends up at the surface, he thinks in visuals never in structure. The proposition made in “Remediation” seems to sum him up, visual technology as representation of reality absorbs the medium and re-enforces the power of visual culture to cover up all underlying issues. Or to view this through Plato’s picture: Manovich, chained in the cage, focuses on the shadows on the wall, even though he intellectually knows they are reflections; he is so caught up in their seductiveness that he does not care or attempt to turn around to look what forms the shadows or what they reflect. Alternative representations of the design engineering process, like site maps, blue prints, wire frame models or prototypes, that shape the things-in-themselves, are not investigated, only its visual representation as standardized mental models.

I found Manovich’s theory disappointing, as I would have expected more “an attempt to think of the object world of technology as though it belonged to the world of culture, or as though those two worlds were united. For the truth is they have been united all along. “ In his scrutiny of interface culture, Stephen Johnson refers to Mcluhan’s assertion “At no period in human culture have men understood the psychic mechanisms involved in invention and technology” This adds the social and cognitive extension level I was longing for in Manovichs discussion. So, following Lash, and Johnson, I will investigate my view of interfaces through Mcluhan’s arguments in the next chapter such as: Contrasting Manovich’s notion of the narrative with Mcluhan’s interest in oral culture, likewise examining Manovich’s reproduction of the expert view with Mcluhan’s dissatisfaction of the expert state, discussing the message and the medium as technological construct on different levels, and exploring visual communication as mosaic view, etc.

Bibliography: Manovich, Lev The Language of new media. The MIT Press, 2001 Mcluhan Eric, Frank Zigrone, ed Essential Mcluhan, the medium is the message, Routledge, 1997 Lash Scott, Critique of information, SAGE, London, 2002 lewis lapham, intro MITt edition, understanding media Bolter, Jay David, and Richard Grusin.

Remediation: understanding new media, The MIT Press, 1999