Reimagined COmmunication

Bringing Time to the forefront of conversation

Playbook not only supports task planning and execution, but now supports situational awareness on three levels: perception of elements, comprehension of the current situation, and projection of future status.

Playbook overview of key features

Video edited and created by Isabel Ngan

Level 1 of Situational awareness: Perception of elements

The ability to perceive the values, function, and status of the system

Playbook now makes connecting disparate information and data easier through topic clustering. Topic clustering refers to how close together connected information is to an item of interest.

Adjustable User INterface

Adjustable views of the chat and the timeline provides flexibility that focuses on the information that they are looking for.

Centerline

Bringing time related details forward while splitting who is on what side of the time-delay.

Custom Marcus Bains Line

The red-horizontal line, the Marcus Bains Line, provides a better way to link information from the timeline and the chat. A desync option is provided for added flexibility.

Level 2 of Situational awareness: Comprehension of the current situation

The way we understand and interpret data based on how it is presented to us

The design highlights how we provide context to conversation by providing additional understanding about an item of interest. Context tells the history and background of information, like metadata, and also hints at what you can do with it or expect going forward, aka, understanding the full situation.

Message Actions

We designed key message actions to allow for clear communication: reply, mark messages obsolete, set reminders, create to-dos, and add quick replies like “copy.”

Trace

Understanding how messages may cross-over by visualising when messages are sent and delivered.

Level 3 of Situational awareness: Prediction of future status

The ability to predict how elements will vary in the near future

Playbook assists in the crew's or mission control center's need to do time math. Time math is a concept which adds another layer of complexity to communicating with latency. Someone does time math when they make a calculation to predict a future state while incorporating a time delay.

Aether

Allowing crew and Mission Control Center to easily identify messages that are in-transit and messages that have been sent.

Playbook in Action

Videos edited and created by Isabel Ngan, compiled by Phil Gase

Problem statement

Since we can’t make anything travel faster than the speed of light (yet), How can a communication tools better support deep-space missions?

The NASA AMES HCI group tasked our team with building on NASA’s existing communication and planning tools, Mission Log and Playbook, to improve and redefine asynchronous communication between crew and ground in deep space missions.

Currently, synchronous crew and ground communications are crucial for successful procedure execution. Ground provides crew real-time assistance for task support, troubleshooting, and more. However, synchronous communication will not be possible in deep space missions due to inevitable communication delays, so there must be increased support for asynchronous communication tools.

Scenario Focus

Finding Opportunities in the most complex situation, extravehicular activities

From our research, we found that there are quite a few different scenarios and tasks that happen on these missions. But the most difficult situations happen during extravehicular activities, also known as EVAs. These are activities that occur whenever astronauts have to leave their base and go outside. 

Discovery

UNDERSTANDING current space communication to better design for future communication

To establish a foundation of knowledge about space missions and the role of communication, I assisted in conducting subject matter expert interviews, helped create and run a simulated experiment mission, and led a literature review of relevant papers to understand the current and future states of both synchronous and asynchronous space communication.

Simulated Mission v1.0

To immerse ourselves in the context of the problem space, we created an analogous domain "Simulated Mission" to get a deeper understanding of asynchronous communication in team-based environments that are similar in key ways to what astronauts and MCC will experience. We started off by simulating a task focused on remote instruction, where our “Mission Control” participants guided the Crew on how to build a lego space shuttle using only time delayed messages.

Focused Exploration

UNDERSTANDING current space communication to better design for future communication

We found that Playbook is currently situated and built around the current state of space operations. Our goal was to imagine and test different futures of space communication. This helped us build a roadmap to the future state, and it’s in the blue sky that we can generate far-term solutions that have the potential for significant impact on mission communication.

Simulated Mission v2.0

To corroborate data from Speed Dating on the viability of our initial solutions, we needed to test our ideas in the context of potential use, which pushed us to create a more thorough idea of a testing ground to use this summer. This expanded simulated mission addresses EV/IV scenario of collecting scientific evidence for ambiguous research goals in coordination with MCC. This scenario allows us to test complex and innovative ideas like AI and Automation.

Insights

Key opportunities to better support the future of space communication

Ideation

UNDERSTAND AND DESIGN FEATURES NECESSARY TO IMPROVE SITUATIONAL AWARENESS WITHIN DELAYED COMMUNICATION

Using our findings, our team determined that positioning Playbook and Mission Log at the center of space communication and improving on a multimedia chat feature will simplify the communication process.

Using our 4 insights as the jumping off point for ideation, I led two ideation sessions. Using the Round Robin method, the ideation session generated over 100 ideas that our team organized into near-term, midterm, and far-term.

New Mental Model

Redefining a new mental model for communication

We created clickable Figma prototypes for usability testing and coded a functional chat prototype to try out in simulated missions. Together, both prototypes helped us validate and translate a new mental model into our final design.

why did we need to understand and design for a new mental model in the first place?

Our team tried applying standards we currently use to communicate, but realized that an asynchronous mental model, like email, and a synchronous mental model, like instant messaging, don't quite fit. 

Currently we experience 2 main mental models for communication: Sequential Model and Tree.

Our new mental model first takes the Tree model and structures messages around topics or replies. Then it takes an adaptation of the sequential model, and considers when a message is sent and delivered. This model doesn’t only consider the past and current aspects of time, but the future too.

Prototyping

Design through Research

To better understand this mental model and test our design ideas, we conducted 20 usability tests with a mixture of participants, both from the general populace and those that more closely resemble our target participants.

Evaluation

Testing our design within a delayed context

To take our research a step further and see how our ideas felt in the real world, we began running experiments we called Simulated Missions.‍

We simulated a time-delayed EVA with a photo scavenger hunt game.

Participants were split into two groups: “Mission Control” and “Crew.”  The crew was composed of Photo Takers, who represented EVs, and helpers, who represented IVs. Photo takers had to take photos of items that matched a certain prompt, such as red things or broken things.

Mission control would then receive those photos and assign points to them based on how well they matched the prompt.  They evaluated photos based on information that we provided to them for their eyes only. This extra knowledge that they had simulated how Mission Control in real life has much more information at their disposal than crew does. 
As for the Helpers, they were the intermediary between Mission Control and Photo Takers, like IVs.

They would pass on photos, messages, and provide assistance to Photo Takers as they needed it. All communication between Crew and Mission Control had a 5 minute delay. In our simulation, Mission Control and Crew had to use our prototype to communicate, share information, and strategize to win the scavenger hunt.

This research protocol was provided to the NASA Ames Research Center for future use.

Future Roadmap

Going beyond 2030

Retrospective

Understanding a paradigm shift

This process pushed our team to immerse ourselves in a situation that only a few experience. Through the creation of our simulated missions, we were able to not only understand the current state of space communication but use it as a platform to test out new design. Our team found success in speaking with different experts from across NASA and inspiration in analogous fields like Formula 1 environment and telehealth.