Tapping the Mind: Memories Beneath Your Fingers

Memories are deeply rooted in the concrete: in space, gesture, and material objects. The cognitive processes of forgetting and recalling, the latter involving “action-oriented responses from a living subject to material triggers -- physical stimuli from external environment” (van Dijck 2007: 30), have not only been studied by neuroscientists, psychologists, and cognitive theorists, but have been addressed and examined by e-lit writers as well. 

Textual “instruments,” such as the app novella Pry (2015), which are activated by bodily gestures and manipulated like games (Luesebrink 2013), invite readers-players not only to trigger (through touch) the narrator’s, protagonist’s, or principal character’s interior thoughts -- thoughts which are verbalized and represented as the strobing words, or floating or stretched text. They also enable readers to experience, within the course of complex tactile interactions, the struggle of retrieving from explicit memory and from the unconscious, personal flashbacks (in Pry represented in the form of very short videos). However, these Bergsonian image-souvenirs, once registered by senses and stored in the faculty of the mind, are not fixed. Rather, while invoked, they are continually being submitted to (re)creation (Kordys 2006: 143, van Dijck 2007: 30). 

This paper focuses on memory (re)construction apps created within the field of new media art. The analysis will not be limited to the gestural repertoire used to interact with facets of someone else’s memory -- in the second half of the paper, the focus will shift to the reader-player’s own memory, which is subjected to the challenge of having to read a constantly transforming text: for example, the stretchtext technology implemented in Pry allows inserting phrases between already read ones, which changes the meaning of each sentence; similarly, readers of Pry come to realize that it is not possible to retrace sequences of already viewed flashbacks. In this context, I will address the question of whether the concept of haptic memory can be adapted to a ludosemiotic approach (Ensslin 2014: 53).