The Design INstallation

Engaging memory through embodied, sensory focal points

This project seeks to redefine this paradigm by surfacing visual, audio, and tactile interactions via sensory focal points. These focal points interact to craft personal memory into a shared social memory via interaction with a shared object and space. These objects and spaces “remember” the past interactions, the sounds of those who were there before, the touch of those in another location, every interaction building on the last.

Video edited and created by Isabel Ngan

Problem statement

How we might connect people who are not physically together across space and time through interconnected objects and environments?

There are certain aspects of the human experience that live below the surface in our own subjective perceptions of reality: the lingering memory of another's presence, the sound of someone in another room, the touch of someone familiar.

Prior to the pandemic, humans created meaningful experiences that lived within the physicality of how we as people move through our world and how we are able to connect with another person. However, the shift from physical to digital made certain aspects of human existence more ever-present when communication was forced to be behind digital screens.

Literature Review

Plotting connections between memory, space, and time

We are approaching this project to plot connections between time, space, and memory in a social format that forms new associations and remembrances on the plurality of experiences built up over the time-space that exists. It combines phenomenological concepts with physical interactions to provide a way for people to tangibly interact with time and space in the context of their memories and those of others.

PHENOMENOLOGICAL INFLUENCES

Our minds understand the landscape, taskscape, and time by what we can see, touch, hear, and taste. In his book Perception of the Environment, Ingold explores how we carry maps in our heads blending our past observations and memories over the present experienced environment [1]. These sensory inputs from our perception of the environment through the process of incorporation into the ideas we already hold about the environment. This blends memories from different moments in time and landscapes with the moment in front of us. The portal creates interactions through the layering of past spatial experiences with the current. This makes this implicit blending of past and present experiences as detailed by Ingold explicit in the environment. The portal seeks to surface the new types of interactions participants might experience in a space where they can see their own actions and those of others layered over the present time period.

Further building on these principles, Benny Shanon explores how the more meaning an experience has on us the more it becomes removed from our experience of time as linear contiguity. Temporality is perhaps the most fundamental constituent of human cognition, but how it is perceived is different for each person based on the experiences that build their internal temporality [2]. Interactions with the portal create a structural atemporality in participants' temporal experiences by encoding what usually lives internally in the sensory focal points of the portal’s experience space. The portal seeks to create an intentional modification of temporality as a means of exploring how to make explicit the temporal connections between spatial and temporal memories between them, the portal, and those that were there before.

INTERACTION  INFLUENCES

Several research projects emphasize the human body’s role as a resource. Roquet and Sas explore how the embodied interactions between the body and novel technologies can be used to create adaptive and generative spaces [3]. These spaces can be used to guide sensory experience to bring sensations in and out of focus. The focal points that define the portal’s interaction space creates experiences that build on each other and allow participants to explore time as a part of the environment. This interaction space only exists when a body is present to experience these focal points.

Vogel and Balakrishnan demonstrate the power of ambient displays to foster new types of interactions in public spaces. Their work explores how to build experiences spaces that give people the choice to move between implicit and explicit interaction phases [4]. They developed an interaction system for interacting and controlling the data present in their display. In a similar vein, the portal uses depth data to determine when to move from a neutral ambient phase to one in which the participant is actively adding to the portal's “memories.” Further iterations of the portal design envision gestural interactions to explicitly control the time and location of the spatial memories of that space.

Memories are something you cannot put your finger on, existing undefined at the edges of your consciousness. Through the Olo Radio, a music player that lets people re-experience digital music they have listened to previously, Odom et al explored how any memory can be turned into a  tangible experience [5]. The radio creates a physical interaction with your memory as if you are putting a knob on your memory. The portal allows for passive and active interactions with current and past time periods. This allows for a participant to explore time both linearly and "perpendicularly" across time periods. This interaction helps establish our exploration of how to overlay present and past experiences in a way that can strengthen the spatial memories of presence in a location.

DESIGN RESEARCH INFLUENCES

In the design of the portal, we are exploring not just the object itself, but the interactions with it over time and how this can create a “material” sense of time as the focus. New types of research methods are vital to explore these interactions between Human and (Tangible) Computer interactions. We developed the portal with a materiality lens in mind. Zhong et al present a precedent for this type of design research in the building of the transTexture Lamp, an interactive light with a deformable lampshade surface placed in domestic spaces [6]. Building off the work of Mikael Wiberg’s The Materiality of Interaction, they outline a methodology for exploring the effect of time and materiality on creating memorable and reflective experiences with technological artifacts. The design of the portal explores the process of layering sensory focal points to further interactions within a space, and how these focal points can blend together and become an interface in their own right. The interaction of these methods creates an ever-changing, atemporal experience of the same environment.

Further, our team drew inspiration from the research prototype process outlined in The Tilting Bowl: Electronic Design for a Research Product [7]. This paper informed our research-through-design and multimodal prototyping approach as the authors outlined key challenges and suggestions when creating a working research prototype at the intersection of hardware and software technologies.

Design Exploration

Investigating ways to make the invisible visible

To apply learnings from our literature review, each member of the team explored unique questions that explored concepts of phenomenology and temporality. I decided to explore designs that answer the question:

How does altering how time is materialized change our idea of time?

When exploring different ideas, I was looking at ways both passive and active ways one could engage in the piece that would lead to a momentary realization or reflection of their experience.

Reflection Exploration

This concept looks at ways in which we can interact with past reflections. When you pass in front of the mirror, you see your reflection and may see a projection of past interactions that have been captured.

Geometric Form and Visual Illusion

Deceiving the human eye is easier than we think. Through the use of light, and pattern, this concept explores ways in which we could transform the frequency of spoken words into areas of light and shadow to create a visual landscape.

Materializing Sounds and Time

Inspired by the Malay word, Pisan Zabra, meaning the time it takes to eat a banana, and this concept looks at how each person perceives time uniquely. One can press and hold a button to their Pisan Zabra and the light peaks would begin to form, first showing the individual’s length of time and slowly revealing other’s Pisan Zabra.

A second iteration was inspired by Rafael Lozano-Hemmer’s Light Array and the light of the installation would reflect the speed and time of the passerby.

Key Drivers

triggering changes in temporality by Engaging Sensory focal points

Through our initial literature review and design exploration, we found experiences that engage multiple senses create moments of meaning, which allow our minds to remove that experience from a linear timeline.

Design Testing

Testing the different modalities of creating meaningful connection

Our team saw a unique opportunity to redefine how people experience temporality and the role this plays in fostering a connection between people. 

When memories are triggered, there is a sense of temporal displacement: when one realizes that they are able to interact with the past. In the act of thinking about one’s past self or a past moment, unconscious connections are being created between the current self and one’s memories.

Our design exploration focuses on ways to foster a connection with fleeting spatial memories through the close coupling of objects, the environment, and time.

Re-materializing Mood And Emotion

CHANGING BEHAVIOR THROUGH CONNECTION

VISUAL AND PERCEPTUAL IMPACT
‍

EXPLORING PERMANENCE OF TIME THROUGH EMOTIONALLY DURABLE DESIGN

Arriving to the a Final Form

Iterating through Prototypes to understand how temporality can be encoded into the environment

Once we established the initial form, we began adapting our design based on existing technologies that could create our three primary interactions.

Because of our installation’s range of sensory-based interactions, we prototyped in small stages. This helped us navigate the new technical knowledge required to create working prototypes. We focused on three main interactions:

1. Visual abstraction and reflection
   
2. Distance-based sound distortions
   
3. Tactile time control on touch
   

We developed the portal to explore how temporality can be encoded into the environment and explore new ways of manifesting presence and memory in material ways. While still in an explorative stage, our portal prototype has the potential to explore how the materiality of the interaction space and sensory experience can create memorable spatial experiences and imbue presence in a space that goes outside of the usual perception of temporality. 

Our prototype’s interaction system allows us to begin breaking down the perception of a participant’s temporal experience in a more focused manner that enables us to continue to explore in more significant depth interactions of this type.

FUTURE EXPLORATIONS

In future iterations of the portal's design, we hope to continue to build towards a greater understanding of the relationship between temporal and spatial memories through technological means.

• Narrowing concept to either exploring interacting with different spaces or interacting with time
  
• Test viewers' perceptions of possible interactions to inform the best interactions and outcomes
  
• Explore how the piece interacts within both public spaces and more intimate settings
  
• Understanding how people with disabilities could interact with our concept, pushing our design to be more accessible
  

Retrospective

Going Beyond the screen

This project continues my passion for designing experiences that change how we understand how we move through the built world. I discovered my passions for creating impactful experiences when I saw how art and dance influenced my life and how it affects others. I found it rewarding to spend time thinking about what meaningful experiences mean in a time where relationships are scaffolded by digital screens.

References

[1] Tim Ingold. 2002. The Perception of the Environment: Essays on Livelihood, Dwelling and Skill (0 ed.). Routledge. DOI:https://doi.org/10.4324/9780203466025

[2] Benny Shannon. 2001. Being Outside the Dominion of Time. In Proceedings of the MAPS Bulletin Autumn 2001 Vol. 11, No. 2: In the future, it will be called Despair. 48-53. Retrieved February 10, 2021 from https://maps.org/news-letters/v11n2/v11n2_48-53.pdf

[3] Claudia Daudén Roquet and Corina Sas. 2020. Body Matters: Exploration of the Human Body as a Resource for the Design of Technologies for Meditation. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. Association for Computing Machinery, New York, NY, USA, 533–546. Retrieved February 10, 2021 from https://doi.org/10.1145/3357236.3395499

[4] Daniel Vogel and Ravin Balakrishnan. 2004. Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST ’04), Association for Computing Machinery, New York, NY, USA, 137–146. DOI:https://doi.org/10.1145/1029632.1029656

[5] William Odom, Minyoung Yoo, Henry Lin, Tijs Duel, Tal Amram, and Amy Yo Sue Chen. 2020. Exploring the Reflective Potentialities of Personal Data with Different Temporal Modalities: A Field Study of Olo Radio. 283–295. DOI:https://doi.org/10.1145/3357236.3395438

[6] Ce Zhong, Ron Wakkary, Xiao Zhang, and Amy Yo Sue Chen. 2020. transTexture Lamp: Understanding Lived Experiences with Deformation Through a Materiality Lens. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20), Association for Computing Machinery, New York, NY, USA, 1–13. DOI:https://doi.org/10.1145/3313831.3376721 

[7] Henry Lin, Ron Wakkary, and Doenja Oogjes. 2019. The Tilting Bowl: Electronic Design for a Research Product. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS ’19), Association for Computing Machinery, New York, NY, USA, 345–357. DOI:https://doi.org/10.1145/3322276.3323701