option drag

Four monitors are placed in a row on the wall. As you walk closer an exhaled breath is heard, then a mouse click, a sigh. A voice commands, “then drag up”; a different voice, “like this”. Excerpts of Youtube typography tutorials populate the screens, complete with Photoshop, Maya, Illustrator, GIMP, etc. interfaces along with the type that is being carefully constructed. A rhythm emerges, “Rotate left, pull down, move forward, like that”. In this piece, a multimodal digital poem forms from the aural language of making visual language. Fragments of descriptive phrases are heard over looping patterns of mouse clicks, exhales, sighs and keyboard strokes amplifying the language of micro-gestures. The unseen role of the body in the circuit of human-computer interaction is ever present in this installation exposing the analog labor of creating digital type and the articulation of the physical process of making digital words. The work humorously explores the physicality of creating visual communication and calls attention to the human, social and cognitive labor behind the typography we take for granted in our daily lives. This digital poem employs a methodology and software developed over the last year as a Fellow in the Open Documentary Lab at MIT. It is a methodology that enables linguistic analysis of audio and video files for playback and synchronization across multiple monitors. Using a corpus of over two hundred tutorial videos, the software parses user defined complex language patterns, parts of speech analysis and phonetic information, creating new aesthetic possibilities for digital poets working with a large corpus of multimedia files.