New Media and the Museum

Hello! Because I identify with physical experience so much, I find the exploration of tangible devices as interfaces a nice acknowledgement of [and ploy to] design for the human-centered physical experience. So I’m excited about Tangibles! This topic is also a nice way to segue discussions around awareness and acknowledgement of heritage to user’s own motivation to engage and explore heritage.

This week’s readings all revolve around Content and Interaction Design decisions by leveraging the direct experience of (or perception) of the place, space, mood, object and/ or social interaction. While the readings portray exciting situated experiences in the intended environment, I wonder how the tangibility aspects carried back into visitor’s everyday life (illustrating that an experience was meaningful). Here I just start to touch on how we can bridge the focus on experience design to the extensive content we have gathered through the course so far, and I plan on taking it a bit further in Tuesday’s class. Also, I’m afraid that my PDQ’s are concoctions of many questions, which can be escalated into much larger, overhanging questions on Tuesday.

———

RR #1: Ciolfi, L., and M. McLoughlin. Physical Keys to Digital Memories: Reflecting on the Role of Tangible Artefacts in Reminisce. In J. Trant and D. Bearman (eds). Museums and the Web 2011: Proceedings. Toronto: Archives & Museum Informatics.

This project designed a role for tangible artefacts to bridge the experience of exploring an open air museum environment with an additional layer of personal and social experience. Tangible artefacts were chosen and woven into a digital ‘layer’ to provide:

• A keepsake that connects to the experience
• A process of collecting personal stories that add a subtle ‘game’ experience to the visit
• A method for sharing the experience both during and beyond the visit (with both friends/ family and strangers)

The focus of the experience was on the affordances of the environment, physical components that could hold meaning and social interactions that both actually happened, and could be imagined in the museum experience. ‘Reminisce’ resulted in a combination of tangible artefacts embedded with RFID tags and QR codes placed throughout the exhibits to encourage visitors to gather information that tied their immediate experience to a tangible object that could travel home with them.

One logistical consideration that the project wanted to design for was to add a layer of personal content (characters, stories, objects) to enhance the natural experience when acting staff were not available. Fictional ‘memories’ were developed around a set of characters and objects that were woven continuously through the entire ‘Reminisce’ experience. These memories were designed as ‘clues’ to other memories, so a subtle game of collecting had to be played to hear them all. QR codes were placed in various locations (on buildings, places and objects) and would provide audio files with the memories that could be accessed by a mobile device. Visitors were also able to upload their own descriptions of their experience at that particular location. Tangible artefacts (or ‘tokens’) were designed to complement the memories relating to the museum experience, then could return home with the visitors (recipes, ball of wool, chunk of turf).  Tokens were also embedded with RFID tags to interact with special memories at the end of the visit. At this time viewers were able to place their token in a basket with another provided object (a book, a photo) and would hear the uploaded descriptions of other visitor’s experiences attached to the location.

The results of the study on the design implementation showed that people felt connections to the memories, tokens and the tokens’s ties to the experience. Because objects could travel home with viewers, they tended to tell family and friends more about their experience, assuming that having the object gave a more ‘live’ feeling to the re-told experience. Viewers also had a social experience while visiting the museum when accessing and creating their own memory content, being provoked to discuss the content based on their situated experience in the museum so far.

I thought this project was a great example of really thoughtfully made design decisions. The authors obviously reflected on their experience well, and while the technical design of ‘Reminisce’ seems quite simple I assume that the design of the memories is much more complex. It would have been interesting to have more information on the design of the memories, because I imagine that it was the memories and the process of ‘collecting’ memories that enhanced the experience and stayed with the visitors more than the tokens or the act of using QR codes.

PDQ 1: Can we identify and discuss the affordances of tokens, situated memories, a motivated process for attaining memories and listening to other visitor’s experiences? How important is visitor’s passive or active engagement to the resonance of the experience?

————

RR #2: Tanenbaum, Joshua, Karen Tanenbaum, and Alissa Antle (2010) The Reading Glove: Designing Interactions for Object-Based Tangible Storytelling. Augmented Human Conference, Megeve, France, 2010. ACM Press.

The Reading Glove project focused on the experience of ‘reading objects’ for their history, or psychometry. To do this, the Tanenbaums explored the connection objects have to personal narratives in order to experience the stories that emerge through the combination of particular personal objects. Twelve objects were embedded with RFID tags and connected to different components of a narrative, which ‘revealed’ the narrative to the user in the order the objects were handled. Compared to prior explorations in designing tangible narrative interactions, the Tanenbaums wanted to increase awareness of the objects themselves as meaningful (semantically present objects). Many prior systems used tangibles as placeholders or containers, connecting the object to the outcome. This process emphasized the function of the object rather than its meaning, and hence a deeper integration into the experience. The objects in the ‘Reading Glove’ were to become artifacts of ‘movable heritage’ through affordances to personal narratives. By exploring meaningful coupling of object to narrative, they were playing with the concept of ‘boundary objects’ as sites of negotiation between the perspectives of the viewer. These perspectives are situated in the immediacy of the experience with one’s own personal histories and perceptions of the object alongside the perspective of the object itself with its own personal history. This refers back to Clifford’s paper on Contact Zones as spaces that are only mediated by our own experience and perceptions (Clifford, 1997). These spaces provoke awareness of our own perspectives in order to broaden (or narrow?) our experiences of others. A later paper (Tanenbaum et al, 2011) discusses ‘boundary objects’ more in depth, by stating that the chosen objects implicitly present a narrative to the participant based on their own personal history and experience that challenges or extends the explicit narrative that is presented while handling the object.

Then, we have another aspect of Heidegger Time! (Yea Jeremy! But- I suppose the focus is not on time, but on awareness)

The project identifies the concepts of Present-at-Hand to Ready-at-Hand. Present-at-Hand refers to the experience of becoming aware of an object when it breaks (the hammer breaking while in use, therefore bringing awareness to it as an object because it is not longer performing the way it is intended). Ready-at-Hand refers to the experience of the object existing as an extension of the user (when the hammer works as intended, the awareness is on the task at hand, not the object performing the task). This is paralleled by a definition of transparent immediacy as present in the moment, focused on the task at hand alongside hypermediated as aware of the devices in place that you are experiencing the present moment through. The Tanenbaums go a step further to ‘create’ the term ‘present-at-mind’, referring to the process of becoming aware of the associated or embedded meanings of the object (their example that the worn handle of the hammer or the carved initials remind them of their father, the original owner of the hammer, and the related stories of the father building). While this is an interesting way to coin a term, I’m not sure I agree that becoming aware of the content of an object is within the scope of Heideggers’ concepts of awareness and ‘being’ in general. However, I do think that affording content to an object, and all the design decisions in this project, problematizes the experience in a way that brings awareness to objects and narrative in an unusual way.

This was by far my favorite paper this week, mainly because the authors really focused on all the design decisions of the entire experience. Objects were selected to be of similar era, style, look and feel. The form of interaction was based on encouraging the agency and motivation of the user. The content of the story was meticulously crafted to function as an engaging narrative that could be configured in any order to make a sense as a story (not an easy task…). The design for pitfalls of an emergent narrative was considered and the physical implementation of the electronics in the glove and the tags on the objects were assessed and considered in evaluations.

PDQ 2: How would this experience design scale up to a whole-museum experience? Would the experience change if objects were scattered through a large space and could travel and adjust their content based on their new location?

What other design considerations could be made to make this a more social activity?

Reference: James Clifford (1997) Museums as Contact Zones. In Routes: Travel and Translation in the Late Twentieth Century. Pp. 188-219. Cambridge and London: Harvard University Press.

————

RR #3: Wakkary, Ron, and Marek Hatala (2006) ec(h)o: Situated Play in a Tangible and Audio Museum Guide. DIS 2006, University Park, Pennsylvania, 2006. ACM Press.

The ec(h)o project focused on designing playfulness and liveness into the museum experience. By focusing on the different cultural ecologies created through the space of the museum (many exhibits that emerge alongside and through each other) this project aimed to explore liminality and engagement. Liminality is the experience of being somewhere that is separated from everyday life (discussed as a spiritual or transformative experience, I wonder if this also applies to engagement in spectacle?), which facilitates engagement. Engagement is described as a state that facilitates both learning and play (I think they mean simultaneously?). Wakkary and Hatala explore the balance between content and physical/ embodied play in a museum context by creating a tangible device to act as a trigger for content in focused exhibits. The visitor also wears an RFID tag to trigger sound files that correspond to the ambient environment at large as the visitor walks between exhibits.

Design decisions hinged on Dewey’s Constructivist Theory: that the visitor’s own construction of knowledge is a primary component to their own experience, engagement and agency in a given scenario. By focusing on the visitor’s own agency and the ‘aesthetics of interaction’ a tactile, motivating and humorous experience was designed to ‘perform’ content in a way that leveraged the visitor’s experience of the museum. The product of this inquiry was a fun colored, ergonomic cube object to use as a pointer to choose stories and the content of the stories themselves (which were designed to be colloquial and humorous for engagement). The stories were designed to sound very different from the Unassailable Voice in presentation (Walsh, 1997), yet the information is still a sort of wolf-in-sheep’s clothing. The content is still a single perspective that is recalled from a database with a one-to-one connection to the exhibit (each selection connects to one story, no content is developing or emerging). The feeling that ec(h)o was going for was that of a ‘virtual cocktail party’. The criticism to this is that the experience seemed to be oriented towards a single person’s experience – while a cocktail party experience would be a social one. This issue is addressed in Wakkary’s later project, Kurio (Wakkary et al, 2009).

The paper on ec(h)o also had extensive documentation on the different ways that visitors held the tangible device, illustrating the playfulness of the device design. It would have been interesting to explore this phenomena more deeply: why was the handling of the device considered a method for making the experience playful? Did certain gestures feel more playful than others? How did playful gestures support the engagement with the audio content?

PDQ 3: How important is the visitor’s own ‘construction of knowledge’ in the museum experience? How can this be designed for? How does ‘playfulness’ feel in a museum context? Is the meaning of playfulness, fun, and engagement different in the context of the traditional museum? (Is the Unassailable Voice ever playful? (Walsh, 1997))

Reference: Peter Walsh (1997) The Web and the Unassailable Voice. In Parry, Ross (ed.) (2010) Museums in a Digital Age. London and New York: Routledge.

————

RR #4: Cafaro, Francesco et al. (2010) RFID Localization for Tangible and Embodied Multi-User Interaction with Museum Exhibits. Proceedings of UbiComp’10, Copenhagen, Denmark. Pp. 397-398.

This project briefly describes a system for collaborative interaction (?) with an ambient (?) display. This system looks at the use of RFID tags either on a handheld device (a picture in a frame) or on a person (hence embodied interaction?) to control an already-existing interactive display of historical immigration patterns in the United States. To throw out some caveats right away, I don’t see what is proactively collaborative or social about this experience beyond the system being able to have multiple inputs at a time (this was designed into the GIS project already). I also would not call an interactive media display ambient, since it is actively drawing attention to its content (and is not passively blending into its surroundings like a wallflower). Personally, I’m also hesitant to call any form of position tracking ‘embodied’, since the user does not actually have any interactive control of the system besides the inputs they have chosen for themselves prior to the interaction. It is unclear if the user can control the content they are viewing by their location in the space and the need to know how tall a user is or how they carry their tag is odd (there doesn’t seem to be any reason for that information).

Issues aside, this project is an interactive information-visualization display that existed prior to the addition of RFID interaction. There seems to be minimal consideration of design decisions given the affordances of the content (patterns of immigration to the US by ethnicity), the tangible object (a picture of an Italian/ Swede/ Persian in a frame) or the embodied experience (either the person themselves or their interactions). One action will trigger the resulting information, which doesn’t warrant the design of explicit user interaction. By working on top of complex, interesting data (immigration patterns separated by time, location and ethnicities) there is a lot of potential for exploring user’s agency, personal connections to the data or interactions for emergent information.

PDQ 4a: Because we have a somewhat open-ended framework of a project here with minimal background information: How do visitors to an exhibit engage on a personal level to screen-based content? How could this experience be better designed to connect to the visitor’s personal history?

PDQ 4b: What are the affordances of a tangible object (other) vs. an embodied interaction (self)? How could these affordances have been used to better explore experience and exploration of content?

  ————

Suggested Readings

SR #1: McCarthy, J., & Ciolfi, L. (2008). Place as Dialogue: Understanding and Supporting the Museum Experience. International Journal of Heritage Studies, 14(3), 247-267.

This paper focuses on visitors’ active sense-making and interpretation in a museum environment by exploring the support interaction design can provide to the experience. The overall goal is to provoke critical reflection of the experience by creating a constant dialogue between the situated place, people and technology. McCarthy and Ciolfi discuss that technology often presents a substitute to a real experience, which promotes passive interaction through one-sided relationships. However, acknowledging every aspect of an experience can provoking active engagement (dialogical relationships) through the consideration of WHOLE experiences. They present a framework to consider when designing technology and interaction in a museum environment:

1. Experience is based on many relationships – exploring many perspectives of people, place, community, etc. can provide information for starting dialogue

2. Openness of museum allows for transfers between affordances of technology (new) and affordances of museums (old), creating new dialogues in established environments

3. Dialogue is situated in our perceptions, our direct experiences and the depth that we hold our experiences. ‘We interpret the situation in terms of our previous experiences and we reflect on our experience and our response to it. These processes give our experiences a narrative quality.’ (pg. 252)

4. Sense-making happens by projecting someone else’s interpretations of an object over our own interpretations

5. Dialogue is sensitive to time and place: other stories and perspectives shift our experience

A case study titled ‘Re-tracing the Past’ is presented to illustrate an exhibit design based on this framework. The exhibits were place in a museum that had an established sense of place: the building was an old customs house that held personal narratives of the original family that owned it and had a strong home/ family feeling to the design of the space and place. Interactive room installations with mysterious objects were designed to provoke dialogue between staff, volunteers and visitors. The mysterious objects in the ‘Study Room’ could be investigated via 3D models of the objects and what they would sound like when touched and a ‘radio’ was available to hear other visitor’s theories about what they are and were used for. When visitors moved to the ‘Room of Opinion’ they could physically handle replicas of the objects and leave their own recordings of personal theories. This project encouraged active exploration and discussion/ debate (social aspect) of the museum experience, resulting in collaborative discussion and reflection. One interesting point was that the docents, who spend regular time as volunteers in the museum, were able to continuously develop their own theories and do their own research. ‘It also showed how many different layers of content can be discussed around the handling of objects: participants commented not only on the material qualities of the artefacts but also on their possible use in the past, the feelings they triggered, and their similarities with present-day objects’ (pg. 261)

I think this is an interesting project because they utilize curiosity so well. By drawing on an environment, place, and space that is very specific, clashing with the histories and perspectives of visitors all in their immediate experience, the project built active interaction into the whole experience. The authors claimed that ‘Making place central draws attention to the sensations and feelings, thoughts and emotions, and the plurality of voices in situated interaction’ (pg. 265). This certainly seemed to be the case, requiring visitors to actively use their imagination first in the Study Room by exploring the 3D model (on a screen) and then later giving them tangible interaction to explore and reflect in two very different experiences of sensing and assessing.

PDQ #5: How important is the temporal order of experience as a device for provoking engagement?