- In virtual theatre, technology brings performers and audience together
- Augmented reality headsets allow real-time collaboration on the virtual stage
- Standards are necessary for widespread adoption of VR/AR
The dancer is about to go onstage. She jogs in place, stretches, reviews her choreography. She checks her costume one last time and takes a deep breath to steady her nerves. Then she puts on an augmented reality (AR) headset and steps into the motion capture environment.
Hundreds of miles away, her fellow performers do the same. The dancers meet when they step onto the virtual stage to perform for an unseen audience watching from their homes, classrooms, and cafes via their own AR headsets.
In virtual reality theatre, performers and audience are brought together through technology.
Farewell to Dawn, a mixed reality dance performance produced in collaboration between computer scientists at Rochester Institute of Technology (RIT) and the department of theatre and dance at Nazareth College, points to the future of live theatre.
“People are now designing entertainment experiences specifically for virtual reality,” says Joe Geigel, computer science professor at RIT. “I don’t think it’s going away anytime soon.”
Going live
The magic of theatre lies in the fact that it unfolds in real-time. Unlike prerecorded film or television, live performance takes risks — flubbed lines can’t be edited and amazing moments can’t be replayed. The live experience thrives on spontaneous interaction between performers, audience, and crew, creating a unique moment in time and space when anything might happen.
The paradigm between doing stuff in VR and doing it in film and cinema are very different. In VR, the viewer is in control, and I don’t think we have figured out how to use that yet. ~Joe Geigel
But in Farewell to Dawn, each dancer performs within a separate motion capture environment.
Networked Microsoft Kinects capture every chasse and arabesque at 25 joints on each dancer and transmit the information to the virtual stage as 3 floating point numbers per joint, 30 times per second. Those numbers are then transformed into stylized avatars that appear as clouds of light.
The dancers are able to coordinate their movements because they are equipped with head-mounted displays (HMD) that allow them to view both the virtual world and each other from the perspective of their avatar.
A Unity 5 stage server is the heart of the system, using a distributed 3D gaming engine to manage all of the assets, motion, and media involved in the production.
But in order for the magic of live theatre to transpire, all of these disparate elements must come together without delay. Though participants are miles apart, they must feel as if they are in the same space, working together. Geigel and his crew rely on Internet2’s 100 gigabit-per-second research and education network to relay the dancers’ information to and from the virtual stage without latency.
“We’ve been doing things like this since 2004,” says Geigel. “But the key pieces have all been coming into place in the last couple of years, particularly the augmented reality.”
Ready for prime time
With the arrival of faster, more affordable hardware, enthusiasm for VR and AR is growing rapidly.
“You don’t have to have a large research grant anymore,” says Geigel. “The idea that VR is becoming more and more accessible just makes it exciting.”
Geigel and his colleagues recently convened a mini-symposium at RIT to raise awareness of how different disciplines are working with VR/AR. Participants hailed from all over campus, with researchers exploring games, entertainment, film, manufacturing, and hardware aspects of VR.
This groundswell of interest in the virtual experience holds a lot of promise for educators, with the potential to transform the classroom experience. But accessibility and interest is not sufficient to make widespread VR a reality.
“People are still trying to figure out how to effectively tell stories in this space,” says Geigel. “The paradigm between doing stuff in VR and doing it in film and cinema are very different. In VR, the viewer is in control, and I don’t think we have figured out how to use that yet.”
There’s also the problem of standards. While the research and educational community is positioned to lead the way in a new era of collaboration and information sharing, linking different augmented reality environments still presents a problem.
“It’s sort of everybody on their own, doing their own thing. I think working towards standards is very important as we go forward,” says Geigel.
Industry wide standards would ensure inoperability, allowing users to travel seamlessly between virtual environments. Internet2’s Metaverse Working Group is leading the way, pursuing open and non-proprietary standards for the security, identity, and interoperability of distributed environments.
Virtual and augmented reality are poised to take their place on the world’s stage. Researchers like Geigel and the Metaverse Working Group are striving behind the scenes to ensure that when the curtain is raised, VR will be ready for its big debut.
