Guest lecturer; storyboard layout, interface design, node development and technology support utilizing After Effects, Mettle and WondaVR for use on Samsung GearVR and Google Cardboard within Pat FitzGerald’s ADN 560 Advanced Animation Studio – a focus on experimental multimedia animation, which looks at developing design solutions in concept and execution through the creative process.

[ Note: This first appeared in Emerging Technologies Explored. ]

For the final studio project for students were tasked with creating a design solution for the Gregg Museum of Art & Design using conversational interface and VR. This project built on the previous learning outcomes and allowed the students to focus on their individual strengths. “We were more like a four-person design team, rather than a class,” shares Ehuan. “We each had specific roles based on our interests and strengths, and since there is a specific sequence of a project, it worked to our advantage.”

 

A conversational interface is a computer user interface that mimics human conversation; however, the human has the ability to direct the conversation, rather than imputing commands through a computer prompt, such as clicking on icons or buttons. There are two types of conversational interfaces including voice assistants that the user talks directly to and “chatbots,” which are the most common and require typing such as in an online chat. Voice assistant examples are Siri (Apple) and Echo (Amazon) which can do tasks such as play music and perform a search with only a verbal command by the user.

FitzGerald had previously dabbled in using a conversational interface to explore multiple methods of communication that users could toggle between. Various communications modes provide users with options and allow them to better direct the “conversation.” In the case of a learning application, users can control the learning outcome based on personal interests. Weidner noted the flexibility and customization potential in a conversational interface—“a change in communication methods including video meets you where you [as the participant] are and where you want to go in the realm of a particular topic.”

The Gregg offers a perfect environment for the project. Museum visitors have variable interests, learn differently, and may become interested in seeking unique information associated to an artifact. However, the hours and location of a museum can pose a potential barrier for people who want to learn about a particular artifact or genre.

The problems the design team wanted to solve “addressed the museum, visitors, and potential visitors,” shares Ehuan. “The museum has the issue of space; they can only show a small sampling of their entire collection simultaneously,” says Ehuan. Additionally, visitors can be limited by what is currently on display and the inability to garner additional information about an artifact that interests them. “The solution was to use VR to create a virtual museum, where physical space is not an issue and there are no limitations of showing all the artifacts, or expanding the knowledge base of an artifact,” FitzGerald says.

The student team researched various means to deliver information including VR, video, and audio and more linear aspects using popups and icons that lead to additional learning opportunities. They decided to develop an interactive VR learning app that uses multiple modes of communication to explain museum artifacts and information at varying levels of user interest. Secondary-level information using a hierarchy of content delivers a custom contextual experience for participants. “The options of beginner-, intermediate-, and advanced-level information; the addition of fun facts; and the option to navigate your way through all the levels, skip around, or watch everything available on one object and then move to something else based on your interests, is what makes this so limitless in its application,” adds Ehuan. The app was also created with the intent that it could be expanded upon as new information or artifacts were accumulated.

Associate Professor and Art Historian Russell Flinchum, Ph.D., was asked to participate as the narrator and educator in this project. FitzGerald says they asked Flinchum to participate in the project because of his knowledge base and expertise. “He is an expert communicator that speaks to the audience using verbal and nonverbal cues,” says FitzGerald.

The VR experience begins outside of the Gregg, as a way of introducing the venue. Users can move inside the museum and start to explore and make content choices. Arrows on the floor allow users, through gaze selection, to advance to the next artifact or room. Gaze selection captures eye movement based on direction or duration of time; it will prompt a user to move in a specific direction. Flinchum appears next to an object and offers detailed information about the artifact at the level that the viewer has selected (beginner, intermediate, or advanced). The use of the secondary learning allows the viewer to also customize their experience.

“We have a mostly cohesive tour of the Gregg and are excited about the outcome, and the actualization of the prototype is an excellent proof of concept,” shares Wiedner. “ We did this in a short amount of time and the level of difficulty was actually low,” adds Ehuan. Wiedner and Ehuan both indicate that this project could be easily developed further by adding new artifacts and information. Additionally, it would also be beneficial to apply this to other forms of content topics.

“This can be a growing learning archive that can be posted on the web and all you need is a Google Cardboard to view,” emphasizes FitzGerald. As new content is added, the app just needs to be updated. “There are several factors that make this a powerful mode of communication in a VR setting including immersion, sensory feedback, and interactivity that are all providing this VR experience with sight and sound.”

The team is grateful to be able to use the Gregg in their project. They shared their prototype with Mary Hauser [BID ’01, MID ‘04], associate director and registrar at the Gregg. “The Gregg was so generous in offering the facility and time to us,” FitzGerald says.