Through the Looking VR Glasses
The most novel story you can tell is the one you choose to enter. To push the boundaries of mixed-reality storytelling and interactive gaming, the ImmerX team at Carnegie Mellon University’s (CMU) Entertainment Technology Center (ETC) created Travel-in-X.
Mixed reality (MR) games, like Travel-in-X, blend physical space and the virtual world together right before our eyes. By using the augmented reality (AR) feature of virtual reality (VR) headsets, users can see the surrounding room while interacting with virtual elements.
To create five mixed reality worlds over a 14-week semester, the ImmerX team shared a joy of imagination, welcomed the surprises of experimentation, and was grounded in communication, collaboration, and trust to explore ideas. Team members Linaixuan (Nina) Wang (Producer, Creative Director), Regina Xia (Technical Artist), Yawen Xiao (UXUI & Narrative Designer), Airan Xu (VFX Artist), Helen Yang (3D Artist), Oliver Zhu (Programmer, Sound Designer) with Ruth Comley (Project Instructor), each brought their inspirations and diverse technical and artistic backgrounds.
The Worlds and Where They Come From
Travel-in-X brings together a collection of styles that represent the ImmerX team’s varied creative interests, which range from Japanese anime, giant flower monsters, childhood memories, and meditative scenes, in a magical and unexpected way. “Everyone in our group has creative minds, different roles, and different ideas about the project. And in the end, we agreed on the idea to build and blend different worlds and characters,” said Xiao, ImmerX’s UXUI & Narrative Designer.
As a result, the team wove together inspirations into five different worlds that blend physical and virtual reality naturally, allowing players to travel through and explore these reimagined realities. Open-minded exploration and iteration were essential to the team during playtesting, which is when players interact with in-development games.
“We didn’t expect to use the floor of the space as an interactive element, in addition to the windows. But during playtesting, some players were placing the vision gate (which is a movable virtual window) on the ground to be able to see under the house as well,” said Yang, ImmerX’s 3D artist. As a result, the ImmerX Team updated the game design to hide items under the ground for people to explore.
Players can spend time and explore where they are interested and jump into different worlds. In each world, they are to use different hand interactions to complete a task, collect objects using the collector in the game, and use the travel console to progress to the next world.
The Worlds at Your Fingertips
Intuitive. Simple. Meaningful. The goal was for each mixed-reality interaction and the explorations in the game world to be natural and fun for the player, using Meta Quest 3’s mixed reality/virtual reality (MR/VR) capabilities, which automatically recognize the room you are in, including walls, posters, and furniture. You can play the game anywhere that has at least one window and one door in the space for the game, without needing any other special objects.
Technical limitations during development provided sources of creativity. For example, when you are in augmented reality (AR) and there's a virtual object, your hand is always going to show up behind the object with the Meta Quest 3. “This is a detail we had not imagined,” said Yang, ImmerX’s 3D Artist, “And from this challenge, we added virtual hands to represent the real hand.”
With these virtual elements, people can then explore the mixed-reality world and interact with a variety of motions, such as grab, pinch, and poke. For example, by pinching two fingers together using both hands and drawing them apart, people can enlarge a portal in the wall to enter and explore. And there’s also a way for players to change the size of objects. “It’s like they have the magical power to make a flower, toys, or cake really big or small,” said Wang, ImmerX’s Producer and Creative Director, about the interactions that people shared that they loved about the game.
And how do you explore and interact with the five unique, immersive worlds in Travel-in-X? With a travel machine and portals!
The Sense-of-Wonder Moment
Alice fell down the rabbit hole. Lucy crawled into the wardrobe. You enter Travel-in-X’s different worlds through portals. It all starts in your own space. You put on the VR glasses, and the mixed-reality world appears, layered over your room, where you see a travel console, a collector, and a vision gate.
It’s time to take your first step to explore and travel through the five worlds, where you will encounter a variety of characters and must complete different tasks to add to the collector. “It’s been really magical for people to just like enjoying the world that we were able to open up beyond the physical house that people usually live in, and then we were playing around with,” said Wang, ImmerX’s Producer and Creative Director. The immersive mixed-reality experience of each world is enhanced by music tailored to its different stages.
In one of the worlds, you look out the window and are face-to-face with Jirai Girl, an unexpectedly giant Japanese animated character. For this task, you must prepare food for her by mixing rice with an energy drink and adding mint candy. And then there is the Forest Monster in another world, where your challenge is to replace the falling out eye with a seed. Other worlds to explore feature elements such as an aquarium, balloon art, and meeting the ImmerX team as an Easter egg in an unexpected way.
How about opening a whole world on the ground to enter another world? You can do that with the vision gate. “We provide the potential, the possibility for people to explore, but we also provide enough fun things for people to play with. We give people freedom to be immersed in the world, to explore what interests them, and to figure things out,” said Wang, ImmerX’s Producer and Creative Director.
Choose Your Own Adventure
As a multi-world, mixed reality experience and experimental demo game developed in just five weeks after a seven-week research and developing stage, ImmerX provides a foundation, including documentation of what was done, to inform the development of future CMU ETC projects. With advances in MR/VR headset technology, ideas from new interdisciplinary teams, resources, and time, additional features can be explored, such as multi-room AR experiences.
ImmerX team members have gained an expanded worldview of gaming, which now includes extended reality (XR) and creating virtual worlds for people to experience and explore. Xiao, ImmerX’s UXUI & Narrative Designer, said, “Because of this project, I personally am more interested in extended reality, and after this project, I’ll explore more about XR, like joining hackathons.”
Teams like ImmerX and projects like Travel-in-X,, that blend ideas, embrace collaborative design inspiration and co-authorship with gamers, and allow time and space for the magic of exploration, experimentation, and wonder provide a way for gamers to explore, experiment, and be amazed by the world, both physical and virtual, around them.
With mixed reality as a lens and source of inspiration, the worlds we can imagine and co-create open into limitless possibilities.
Take a voyage into Travel-in-X with ImmerX’s trailer. Source: ImmerX