Lighting the Way with Torchbearer
Imagine standing in a room with a friend, both of you reaching toward the same virtual object hovering in mid-air, each of you seeing it perfectly from your own angle. No headsets. No wires. Just a shared, immersive experience on a curved panoramic screen.
That's the vision the Torchbearer team set out to explore this past spring at Carnegie Mellon University's Entertainment Technology Center (ETC), right here at Hot Metal Labs.
Inside the CAVERN
At the heart of this project is the CAVERN, one of the ETC's most distinctive spaces. Built in 2022 by Steve Audia, the ETC's Director of Information Technology, the CAVERN is a 270-degree environment where three synchronized projectors wrap a stereoscopic image around a 20-foot curved screen. As Audia himself has put it, it's "VR without a helmet."
Using shutter glasses and motion trackers, the system can follow a person's position and adjust the 3D perspective in real time, so virtual objects look like they're truly there as you walk around them. The catch? Until now, that trick only worked for one person at a time.
The Problem with Two Players
When you track one player's perspective, the experience is seamless for them. Add a second player, and the illusion breaks. As the first player moves, virtual objects shift in a way that looks unnatural to anyone else in the room.
An earlier 2022 project called HyCave tried to solve this by splitting the screen in two when players moved apart. It worked but sacrificed the wraparound experience that makes the CAVERN special. Audia had a different idea, and this spring he brought it to a team of six ETC students to find out if it could work.
The key insight was about distance. Virtual objects close to the viewer distort dramatically when the perspective is mismatched. But far-away background elements? The distortion is barely perceptible.
Audia's proposed solution was to treat the scene as two layers. Background elements would render as if every player were standing at the center of the room. Objects inside the space would render independently for each tracked player. "In his own words, he said, 'It could be garbage. I have no idea,'" recalled Torchbearer producer Michael Wong.
Carrying the Torch
The team name came out of a two-hour first-week brainstorm. They wanted something that nodded to the CAVERN without being too literal. "It's sort of like you're taking a torch into the cavern like it's a cave exploration," said Wong, "and we're exploring this idea Steve had."
The team started simple: a gray Unity cube floating in the middle of the CAVERN. They put on the shutter glasses and walked around it. "All of us saw that and were like, okay, this is working," Wong said.
From there they built a demo to test as many multiplayer interactions as they could. The most exciting discovery was something unique to the CAVERN format. Because there's no headset blocking your field of view, you can see the real person beside you. You can hand a virtual object to them, watch their face as they reach for it, and share a moment that's both physical and virtual.
"We could do immersive VR with objects virtually in the space and you could look at the other person's physical body and hand it to them," said Wong. "That sort of interaction was what we were interested in."
On the technical side, programmer Jason Jiang overhauled the rendering system to reuse resources across players, so adding more players no longer caused memory to balloon. The team tested up to four players, limited only by the number of available trackers, but the system is built to scale.
What Comes Next
Torchbearer's work lives on as a software toolkit. Built as an extension to Spelunx, a 2024 ETC project that created a Unity development package for the CAVERN, it's available on the ETC's local Unity package server for any future team to pick up and build on.
Audia's longer-term vision is a cycle of CAVERN projects alternating between experimental semesters and experience-focused ones leading up to the ETC's annual Fall Festival. A separate team of CMU engineering students is currently working on hardware that would let each player see only their own version of virtual objects, which could be the foundation for the next chapter of work in the CAVERN.
For now, Torchbearer has proven the concept and built the tools. The door has now widened even further in the CAVERN’s possibilities.
The Torchbearer team includes Michael Wong (Producer), José Francisco Mireles Macario (Game Designer), Frank Lin (Co-Producer & 3D Artist), Ivy Hu (3D Artist), Jason Jiang (Programmer), and Henry Wang (Programmer). The team was advised by Professor Drew Davidson and Steve Audia.
(Image: Liam Neely)