Aura

An Immersive Mixed Reality Museum Experience that Transcends Time and Space

_Type
_{Project sponsored by the 2017 Microsoft Design Expo at the University of Washington with Branden Keller, Mihn-Anh Nguyen, Maureen McClennon, and Michael Frampton}

_{My Role}
_{Research, Sketching, Storyboarding, Motion Design, Visual Effects, Video Production, Acting (not oscar worthy)}

_Timeline
_{Jan - Mar 2017 (12 weeks)}

Goal

For the 2017 Microsoft Design Expo, our team was tasked with designing a product, service or solution that demonstrates the value and differentiation of Mixed Reality (MR). Our design was meant to demonstrate the best qualities of an MR experience that takes into consideration the environment, objects and people that will be participating in our solution.

Process

Research

Through secondary research, contextual inquiry, and interviews, we sought to understand the role museums played in society today and how that could potentially change in the future with Mixed Reality.

Secondary Research

_{To understand the function of museums in society today, we conducted secondary research to immersive ourselves in the space.}

“ An institution whose core function includes the presentation of public exhibits for the public good”

— Eugene Dillenburg, "What, if Anything, Is a Museum?"

Contextual Inquiry + Interviews

_{To further immerse ourselves in the museum experience, we visited a number of local museums which included the Burke Museum and the Pacific Science Center. There we interviewed and observed how staff and guests interacted with exhibits in these spaces.}

Key Findings

After analyzing and synthesizing the data we collected, we distilled our findings into a series of principles:

Design Principle 1: Personalization and Collaboration

Museums make complex information accessible to people of all ages; however, this information does not take the person or group's existing knowledge base into account.

Design Principle 2: Artifacts and Specimens in Immersive Contexts

Museums use artifacts and specimens as vehicles of communication but can sometimes fail to communicate the context in which these artifacts and specimens existed.

Design Principle 3: Dynamic and Engaging Content

Museum content is static and seldom updated on account of financial constraints.

Ideate

In this stage of the design process, we externalized our findings into a series of sketches, storyboards and key frames to receive feedback from our Microsoft sponsors, instructors and classmates. Because we were portraying a future Mixed Reality scenario, we felt as though it would be best to illustrate this future through storytelling-based visualizations.

Sketching

_{We started by sketching how a Mixed Reality museum of the future might look. In this sketch, I chose to illustrate a dinosaur exhibit because MR tech could show the accurate scale of these creatures and how they used to behave in their prehistorical context.}

Storyboarding

_{I then sketched out a storyboard along with my teammate Branden of our concept that featured people interacting with exhibits that leveraged the power of Mixed Reality. Because MR excels at showing scale, creating rich sensory environments, and visualizing complex ideas, we made sure that scenes in our storyboard embodied these elements. After sharing this storyboard with our sponsors, they recommended the following: 1) Think beyond dinosaurs and 2) Make sure the MR experiences are emotionally accessible ( in particular, the T-Rex scene may be scary for some). We implemented this feedback in our key frames.}

Key Frame Slideshow

_{We went back to the Burke Museum and took a series of photos to outline key frames of two characters interacting with assorted MR exhibits. We then designed MR interfaces and visualizations to helped us inform our video prototype.}

Refine

Refinement was all about taking our story and transforming it into a high fidelity video prototype. This was a rigorous process that involved extensive video production and visual effects work.

Filming

_{Once we decided on our key frames and story, we began filming for our video prototype. To impose interfaces and visualizations onto the world, we had to use a few visual effects techniques such as green screening. Pretending to interact with interfaces and imaginary objects triggered some confused looks from museum goers}

Design Concept

_{Aura is a mixed reality museum experience that allows people to learn about the world in an entirely new way. Rather than reading text placards or static images describing what was, people can experience life-like interactive environments in which artifacts and specimens used to exist. They can hear, see, and feel what it used to be like, making for a rich learning experience.}

_{At the entrance of participating museums, visitors are handed a set of Aura Mixed Reality glasses that allow them to experience artifacts and specimens the way they were meant to be seen.}

_{Once the glasses are activated, the museum space is transformed into a sensory rich learning environment where artifacts and specimens take the center stage.}

_{Check out our video prototype at the bottom of the page to see the end result.}

Visual Effects

_{Docent Egg Scene}
_{Here our team wanted to show that Mixed Reality experiences could be triggered by different collaborative contexts like a docent explaining an artifact to a guest.}
_{Collaborative, Engaging, Personal}

_{Plesiosaur Scene}
_{This scene was all about communicating the scale and context in which a plesiosaur would have resided in prehistoic times. This is one of the biggest strengths of MR. (This particular scene was visually designed by my teammate Branden Keller)}
_{Immersive Context, Engaging}

_{Sea Urchin Scene}
_{Here we wanted to demonstrate how existing exhibits can be enhance to create personal and engaging experiences with interactive 3D content.}
_{Dynamic, Engaging, Personal}

Video Prototype

_{After hours of editing in Premiere Pro and After Effects, our team completed our final video prototype of Aura. We presented the concept to our Sponsors, instructors and the class and recieved very postitive feedback.}

Learnings

Getting the opportunity to think about Mixed Reality from a blue-sky design perspective was a truly fascinating experience. The potential of MR is truly mind-boggling and I'm confident that immersive technologies will change the way we interact with each other and the world in the future, much like the smartphone has done in recent past.

Perhaps my biggest takeaway from this project was that technologies like AR, MR, and VR fall along a spectrum of immersion. AR imposes information onto someone's field of view without necessarily taking the world into account. MR imposes visualizations and data in a real-world context. VR disregards the world and envelops a person's senses to transport them into a new world. Right now, each of these technologies are specialized to create a single level of immersion that is ideal for certain contexts. In the future, I think it would make more sense to create a single device that adjusts this immersion level based on a person's context. This is what we demonstrated with Aura and I think it would make for a more fluid and natural experience as people navigate from context to context.

All in all, this project has inspired me to think beyond the screen and work on immersive applications for AR, VR, and MR to shape the future of human-computer interaction.

Aura | Mixed Reality Museum Experience