[This is Sandy Baldwin’s considered ‘final’ text, now including corrected typos, some revision and references. The first version of this was posted 12 October and this corrected text was posted 18 October. Sandy has asked for the early drafts to be removed. His brief bio follows the paper:]

 

Mediacy and Rupture in Protocological Domains

Sandy Baldwin

 

"But it is the machine in them that is dreaming." Sartre

 

The first recorded English usage of the word "mediate" deals with the relations of a vassal to a feudal lord. The Rolls of Parliament in 1454 adapt a Latin legal formulation already in use since at least the 13th century, for the obligation of a tenant to pay rents to the lord both "mediate and immediate." The resulting notion of "mediate holding" refers to property and personhood determined by stratified relations to sovereignty. The mediate is a place of contracts and negotiations between legal persons, but this "place" is opened by work, by production value and its capitalization. Mediacy means paying the feudal lord his due. To speak of media is to maintain the place of mediate holding as a space of recognition and obligation (or as the "holding environment" of psychoanalytical part objects).

 

This mediate holding emerges through protocols. Alex Galloway's _Protocol_ uses this organizing principle and condition of appearance to theorize layered domains of culture and technology. Protocols are frameworks for symbolization and exchange. In the net, TCP/IP core protocols determine basic connections between hosts, defining discrete layers from network access to the application; they further define encryption, filtering, and other communication between layers. There is a topography of TCP/IP and a narrative described in its sending, recognizing, acknowledgment, and transmittal of data. At another level, the 127 characters of ASCII encoding, or for that matter the more recent Unicode standard, provides a code point for every possible character. Unicode is the unique graphic medium for all that can be stored and transmitted in the web. Moreover, since a "character" designates a graphematic unit of information, which in turn underlies particular material glyphs, Unicode also provides the framework for the web's phenomenology, determining appearances and conditions of experience.

 

Media theory can describe these levels and isolate the functioning of cleanly defined symbolic orders. The reflection on levels of protocol is the Nietzschian consolation for media theory, the theory hope of contemporary new media departments, critics, and practitioners. "New media" is a discourse of the symbolic capital of mediate holding, of the formalized negotiations and recognitions of protocol. The objects and descriptions of new media reflect of the logic of protocol. In theory, they describe the structure of net communication. In practice, as Alan Sondheim's work on protocol shows, levels collapse and dissolve into each other. The domain of protocol *itself* is undetermined and transitive. The net can not be located solely in the boxes of plastic and wire on our desktops. In short, the levels of protocol or the spaces of mediate holding are pre-broken domains shot through with the real spaces of human interaction. As Winograd and Flores put it of computers and cognition, all domains "are generated by the space of potential breakdown of action."

 

Protocol tends towards the book, towards inscription and order. The etymology of protocol is in the written formalization of transactions and negotiations. It conventionalizes etiquette, proceduralizes good and civilized behavior. The protocological relation to the book is different from the so-called "remediation" of the book in new media, a process that can be grasped theoretically only within the persistence of the book in the symbolics of protocol. The traditional hermeneutic topos of "the book as symbol" found in the book the means through which human life relations were discernible. If protocol tends towards the book, this is because the book can be described but not theorized: it is an elemental domain, the book beyond the book, in Jabès' sense, and elemental domains can only be subjected to processes.

 

"The Books," an ongoing project of the artistic collaboration of cris cheek and Kirsten Lavers under the name "Things not worth keeping," shifts the symbolics of the book from a means of transmission to a means (co-)composition. In doing so, the order of books is loosened, or rather, its essential fluidity is put into play. "Things Not Worth Keeping" works through or wears away the cultural capital of objects, moving from what they call "value-fixities" to "value-transitions." In describing their work, cheek and Lavers insist on the collective, foregrounding the use of "we," repeating at various stages of their collaboration that "neither of us made this decision." The collaboration's abbreviated name TNWK already condenses the "thing" with the problem of value. The preference for the abbreviation over the full or "true" name keeps the worth of the thing in the suspension or transit of writing. TNWK is a cryptogram of the value.

 

TNWK's method is writing processes as collective making. "The Books" works with books collected over time and through the individual passage of a life. In this, the books stand somewhere between the discard and the prized edition. The volumes may be saved because they mark specific moments, or they may be saved for incidental or even impulsive reasons. The material book as memento separates but remains linked to the book as cultural transmission. As TNWK describes them, these are "odd books," in some way "unpromising" or "thought of as too difficult to tamper with" or "too precious to disturb" or "simply not worth keeping." The simultaneous attraction-repulsion is striking here: if the books at first seem cast aside, fallen from the bestseller list into the dustbin, the "oddity" is their expression of a different economy than the incorporation and accumulation of purchases in the consumptive practice of reading. At the same time, the books are not purely expressions and testaments of the case history of an artist's life. Instead, the collection of books are a symbolic order through which value intensifies and insists.

 

These processes of "The Books" de-invest books of individual attachment. The are processes of releasing value - which may be variously coded as commodity, memory, connoisseurship, etc. - into the intensity of shared work processes. This can not properly be called "exchange value" because the processes and work are unbound and transitive. At most, there is energetics in the medium of the book. This excess value of books is expressed but not determined by their oddity. This excess may be pointed out in the material of the book, in the binding, in the specific semantics of "a Latin primer or an Austrian Cookbook," but none of these are the point. Such concrete moments are local catastrophes of value in the inert and worked over matter of books. TNWK process the books with diverse collaborators and through diverse means. They read, select, transform; in short, they work through the book towards the ground of the book, as Jabès puts it. "The Books" is titled and published in print, online, and in performance. Each instance is part of an ongoing writing in the transitive space of things not worth keeping. It produces a new field of the book in diverse media.

 

The book beyond the book is a paradigm of an "elemental domain." Elemental domains are sites of Bachelardian reverie. You dream through the mediate holding of elemental domains. Consider, the pre-broken net domain of leet or 1337. Often described as a "corruption" of written text, the practice of "leetspeak" or "13375p34k" emerged in BBS's, listservs, and MOOs, and is characterized by replacement of alphabetic characters by similar appearing numerals, but may also involve other replacements or radical abbreviations. Leet is not a strict one-to-one encoding. There is no standard for the practice of leet; rather, its diversity varies with the protocols and group cultures of online writing environments where it appears. One leet usage is "teh," a mis-spelling of "the." It is a common typographical error and automatically corrected by word processing software such as Microsoft Word. On a standard QWERTY keyboard, the letters T and E are typed by adjacent fingers on the left hand and H is typed by the right hand. Coordination of the hands conflicts with the incorporated skill of touch-typing. The auto-correct feature lets you speed past your errors. With "teh," the refusal of proprietary pseudo-intelligent software and the proximity to bodily rhythm - and specifically to repetitive and forceful bodily motions - intensifies "the," so that "that is teh lame" means "that is the lamest." The f/ph replacement in "fone phreaks" is early leet, as is the s/z replacement in "warez." The character "a" can become "@" or "4." Either substitution is based on optical recognition of characters and transposition in terms of visual similarity. Leet is character encoding for eyes that see and a body that experiences. Leet is read for the visual glyph, but also read *through* the glyph to the virtual or invisible *intentional* character suspended behind it. Every glyph or graphematic unit is read by rules of encoding and substitution but also through the phenomenology of address, where the glyph is meant for the reader. The illegible carries the legible within it, and leet is a reverie of the digital character.

 

Theorization of leet only demonstrates its false order, a symbolics of elemental husks eaten from within. The phenomenology of leet is specific to consensual domains. Leet as abbreviation allows for speed and efficiency in communication. The reduction or simplification of messages is possible within the framework of a writing environment such as a listserv or chatroom, in terms of group norms and assumptions. In fact, simplificiation is a function of this framework, and the speed gained is only possible by the accumulation and implication of group knowledge in the environment. As a derivation of the word "elite," leet presupposes identification with a particular habitus and shared cultural capital. The leet writer is part of a group and identifies with that group in the very act of reading and writing leet. As elite, leet references a sociology of hackers and users, of those "in the know." To read leet is to recognize inclusion in this habitus.

 

The cost of recognition, however, is enforcement of the protocological constraints laminated on every string of leet. A common use of leet is for undesirable or illegal communication in monitored or censored environments. Leet allows participants in online gaming to swear without being kicked out by monitoring software. A leet formulation such as v1@gr@ can slip by email content filters. Similarly, wares or cracked software becomes "W4R3Z" and "porn" becomes "pr0n." Of course, some sort of filter could flag these terms using regular expressions, but leet as a practice of corruption means writing until there is no possibility of detection and recovery. While a filter might flag four-letter words beginning with "p" and ending with "n" as a high probability of being the word "porn," there is less chance of catching the expression "spl01tz" as leet for "exploits" or hacks. A related use of leet is obfuscation. In general, code obfuscation is a practice that makes computer programs difficult to read and understand, typically by reducing any text-like formatting or by adding arbitrary formatting. The result compiles and runs on a machine but appears as an unintelligible mess to human eyes. The goal is to conceal information, whether from possible thefts and reverse engineering, or as a means of spamming. There is also a thriving practice of recreational or artistic code obfuscation, such as the Obfuscated PERL Contest and the International Obfuscated C Contest. Leet allows a basic form of obfuscation. For example, leet provides a quick means of generating passwords or user names as unintelligible strings. In online gaming, where administrators or higher level players can eject players with a simple "!kick username" command, the leet-generated username is a simple encryption, a hard to type and easily misrecognized.

 

 

In the other direction, leet remains problematic. A work such as Bjørn Magnhildøen's "Plaintext Performance" participates in the aesthetics of leet, mashing of text from algorithms, human input, and system internals, creating a push or feed forward of work that can not be sorted into source codes. The text is worked over and corrupted. "Corrupt" text implies a cleartext suited for the domain at hand, but clearly leet is a principle of generalized corruption. All text is already encrypted in relation to the interminable working of the net. Leet references the absent body. Its corruption is the internal processing and -jectivity of readers. This means that every character encoding and every string on the net is potentially leet. leet=1337=1111=0000.

 

Mediacy is persona, personhood as form and masks. Mediacy is a crowd in the presence of the king. Character recognition filters leet. Recognition places leet in protocological spaces and smears leet across the filtered space. In the digital, the mediate is set up and through recognition. If the digital takes place and takes the place of the analog location and phenomenological context of the user, then this necessarily occurs through the paradigm of recognition. More than anything else, digital facial recognition is the paradigm of the protocological. From the handkerchief of St. Veronica to the growing use of facial recognition security systems, the face is the paradigm and crux of representation. Digital cameras increasingly include facial recognition software. You no longer need to carefully frame and focus on a person's face, the camera automatically does it for you. In a method similar to most facial recognition systems, the software uses twenty or more different recognition points, based on an eigenface recognition algorithm, to output estimated human face locations.

 

The eigenface algorithm is an approach to face recognition based on possible human faces. The eigenface is the merging and normalization of a large number of actual facial images, into a single image composed of statistically common features of human faces - eigeneyes, eigennoses and eigenmouths - standard ingredients that we share to some degree: perhaps 30% of the ears of this eigenface, perhaps 10% of the nose, perhaps 70% of the chin, and so on. Looking at an eigenface, one sees a vague and hazy blur, apparently a human face but not recognizable as anyone in particular. The eigenvalues of a given eigenface are landmarks for eye fixation but not for recognition of any particular face. The bland vagueness is disturbing. It lacks specificity. It is no one's face but is *like* everyone's face. The eigenface drives out the singularity and otherness of individual faces, until all faces tend towards mess and blur. The eigenface and similar recognition algorithms are protocological tokens of value. The eigenface becomes the value of every face.

 

Alan Sondheim's havingfunhead.mov is a short Quicktime video created at the Virtual Environments Lab at West Virginia University. The video uses facial recognition software such as Face Tracker, which takes a video data stream as input and processes it frame-by-frame to automatically locate and extract faces. In conjunction with 3D modeling software such as Poser, or Geomagic Studio, which automatically generates three dimensional virtual objects from flat streams of data, digital artists use Face Tracker to meticulously craft avatars for video gaming and movies. They view it as reverse engineering faces from parts, from the bits of ears and eyes and noses. From scanning body parts to the final knitting together of complete bodies, these are environments built around production cycles, transforming pieces and bits of the analog into the virtual body. An avatar head is built from individually designed ears, nose, mouth, eyes, and so on. It is then sewn together with 3D modeling software. All virtual heads are motley collections of pieces. All that appears in the digital is pure production value, artful assemblies of dismembered parts.

 

The video is a few minutes long, showing nothing more than a head floating against a blue-green background. The head is odd, disturbingly so. It is apparently made of digital images. The ears are indeed breasts, from an image of a naked woman, as are the eyes, and the limbs of the image, which twine around the chin and the back of the head. Other surfaces of the face are some sort of unreadable map, the details too blurred to be discernible. The head is wrapped with video feeds processed by Face Tracker to automatically form into a face. The twenty or more standard eigenface locations are the criteria for wrapping the head.

 

"Hi, are your breasts your ears?" These words begin the video. A voice questions the head throughout the video, and the head responds, turning, nodding, tilting, but never speaking. In what way are they "your breasts" and "your ears"? At least this is what the voice asks, and the head's responses are far from clear. Is the voice speaking to the head or is it speaking to you? Does the head actually respond? Or do you respond, watching the movie, answering the voice by recognizing the breasts and the ears, recognizing the face of this odd mute head? What is clear is that response, whatever response occurs, is part of self-possession, part of being in this environment. Response places the head and the body parts, places the breast and the ears, places the voice and you, places all loci of address, as direction and intention of an image - of this video - and as the subject of a narrative of desire and possession played out on video. In short, as a problem of reading the spectacle.

 

The head seems to nod but the nodding is a kind of twisting. It is hard to know if the twist is an affirmation. It may not even be a nod. What is a nod? Which is to say: what transactions occur when you nod and agree to my request? The voice asks: "do you want me to touch you?" and the head twists, and the voice asks: "I don't know what you're saying, do you mean yes?" The head is addressed, placed by the voice. At the same time, the voice answers for the head. The voice speaks to the head, naming your breasts and your ears. A mouse pointer follows the head's movements. If at first it seemed that the head moved and responded, it soon becomes clear that the mouse is directing the head. The head is responded, turned, nodded, tilted, and spoken for. In fact, the cursor and voice are multiplexed, working together. Voice and pointer are a single field of action across the visible space of the video. As you watch, the voice seems to come from where you sit; the pointer seems directed by your hand out of sight on the computer's mouse. The head responds to your desire.

 

The voice touches the head. It seems to penetrate the thick, image-laden surface, extruding it from the background. The voice wraps the head. The voice multiplexed with the mouse pulls the surface. More than an image on the surface of digital video, the wrapping seems to push into the space between the pixilated screen and where you view the video. The wrapping thickens and replaces the head. The wrapping becomes a medium absorbing your voice and gaze. The voice shapes the head-body into the "skin sack" or "skin fold" of the Freudian body ego. "Does this make you happy? Do you want me to stop?" asks the voice.

 

The head's wrapping looks like a bondage mask. It covers every part of the head. It holds tightly around every feature, containing any hair or teeth. It references the restraint and containment of sadomasochism. The head is a gimp, the submissive partner in domination relationship. This relation is already established: you watch and control, the head responds. The bondage hood objectifies, removing features and turning the head into nothing but a shape, a toy to be played with. The bondage hood silences, gags any sound. The head has ears only for you (and they are breasts).

 

How do you understand the head's desire? Your desire guides the images, interlacing the voice and the image. The head's desire, the submissive desire of the gimp, is withdrawn and hidden in what the images wrap, hidden beneath the surface of the digital. There is a willingness that draws the voice into the head. At the same time, there is torture, a silent cry of a mouth gagged and covered. As digital media goes, the video seems to offer no interactivity or programming. This is untrue: Sondheim's video is an interactive encounter at the level of desire and drive. In the domination-submission relation, power is already negotiated and circulating. Seeing is the interactivity of mutual cathexis. The eye and even the visible field are trained and controlled.

 

Digital face recognition is an enigma. You seem to see, you seem to be faced by a representation, you seem to perceive the object, you seem presented with the "not you." In fact, the digital face is already recognized and appears in a negotiated space of drives and invested objects. The enigma of the digital face is its paradigmatic role as an uncanny object. You typically use "enigma" figuratively for a mystery or unsolved problem, but it means an obscure and riddling narrative. The digital face generates a narrative of otherness and rupture, a topography of reverie. Keep in mind that facial recognition is basic to seeing. The narrative is produced from the asymmetry of the encounter with the digital face, an asymmetry already present in the otherness of any facial recognition. You think you recognize faces but you recognize *with* faces. Recognition already occurred. The face is already judged and addressed. If you see faces, it is because they already exist in a system of exchange, are already labeled and placed. Seeing occurs within a system of exchanges. It is as if you recognize it, but it is already recognized.

 

The presence of the voice in Sondheim's video is crucial: it situates the video in the virtual space of dream. The voice is the symbolic order, or rather, it is spoken from the channel of the symbolic. Presence of language situates the video in the digital. It ruptures and separates the visible field. On the one hand, a virtual space of mediate holding, where cultural signifiers of gender, power, and body circulate and flicker. This is the space of symbolic exchanges, where you recognize the cultural face of protocol, the eigenface of the blurred other. As Deleuze and Guattari put it: "the face is produced only when the head ceases to be a part of the body, when it ceases to be coded by the body, when it ceases to have a multidimensional, polyvocal corporeal code-when the body, head included, has been decoded and has to be overcoded by something we shall call Face."

 

On the other hand, an inert visual smear, where the face collapses into pixelations and out into a vertical line of intense light. This bi-sects the video, a yellow surface flare where Sondheim reports that the video camera was burned by a laser. This smear is the extimate space of the Kristevan chora. The smear is there, across the surface of the video, but it is also always elsewhere. What you recognize in the semiotic space of the smear is a gesture from and towards the face of the already recognized Levinasian other, the face that is not in front of me but above me, the face beyond the face.

 

The digital object in the space of mediate holding is broken by otherness. All domains are broken and opened towards the other. Consider Capgras delusion. It is understood as a breakdown in facial recognition. You see but do not recognize other people. You see a person and insist that they look like or even look identical to someone, but that they are in fact an imposter, a replacement, or a robot. The breakdown occurs most commonly with recognition of those closest and most intimate. You see a parent, spouse, child, sibling, lover, etc., but insist they are replaced by a robot or other synthetic being. Capgras can extend to the self as well: you look in the mirror or at a photograph of yourself and see someone who is identical to you but is not you. You see a robot imposter. In a variant known as Fregoli delusion, you insist that all other people are replaced by a single imposter. The multitudes you see, from intimates to strangers on the street, are in fact one person who moves around and changes appearance. These delusions are thought to result from brain lesions. You see but do not recognize. You do not recognize the "person in the face," according to Oliver Sacks, who compares the failed face recognition to computer optical scanning. There is capture of details, there is locating surfaces, there is even identification of pieces of the face (an eye, the nose, mouth, skin), but the pieces remain pieces. No more persons, nothing but defacements and dismembering. Perhaps Sacks' analogy to the computer is exactly right. Is this not how you see in the digital? You see the object on screen but you recognize it as pixilated and processed. You recognize the eigenface. The digital is exact, everything addressable, everything antialiased. The image is cool and clean, yet precisely its perfection references otherness. If you read the digital, it is because you read into the digital, into and towards otherness. Capgras and Fregoli delusion are the breakdown of facial recognition into narratives of the encounter with the other.

 

Finally, compare to this the "uncanny valley," a measure of human recognition and emotion when faced with robots and other non-humans. The concept was introduced by the Japanese roboticist Masahiro Mori in 1970, who measured emotional responses of human subjects against the anthropomorphism of a range of robot, including humanoid robots, bunraku puppets, industrial robots, stuffed animals, corpses, prosthetic hands, and zombies. The uncanny valley measures the rupture in our symbolic relations to otherness. It measures a visceral, alarming, even repulsive response to otherness. At the same time, Mori recently described other nonhuman faces as the grounds of empathy, giving the examples of statues of the Buddha: the Miroku Bosatsu in Kyoto and Chuguji, and the Candraprabha in Nara. Between such possibilities, between the robot and zombie, on the one hand, and the Buddha's face on the other, are the grounds of recognition in the mediacy of protocols.

 

References

 

Gilles Deleuze and Felix Guattari. A Thousand Plateaus. University of Minnesota Press. 1987.

Alexander Galloway. Protocol. MIT Press. 2006.

Edmund Jabès, The Book of Questions: Yael, Elya, Aely. Weslyan University Press. 1983.

Oliver Sacks. The Man Who Mistook His Wife for a Hat. Touchstone. 1998.

Terry Winogrand and Fernando Flores. Understanding Computers and Cognition. Addison-Wesley. 1986.

Material on leet from Wikipedia.

Material on the uncanny valley from Proceedings of the Humanoids-2005 Workshop.