Mixed Reality Performance

Secret Garden was the first innovative mixed reality performance to immerse our guests in a live music and dance performance to unlock the whimsical adventure. Guests could choose to enjoy a fully immersive performance inside a VR headset, explore the virtual world with iPads or through live streaming.

Concept

As technology becomes an inseparable part of our daily lives, it paves the way for new possibilities in the realm of art. The eternal beauty of nature continues to astound us, and now, through the fusion of art and mixed reality technology, we can unlock and celebrate its magnificence in a truly unique and extraordinary manner.

Concealed within the Laundry Steps and Lower E Hall of Tai Kwun, lies the captivating << Secret Garden >> a composition comprising three AR activated murals and a mesmerising mixed reality live performance. Throughout the 20-minute performance, a virtual environment gradually unfolds, transforming the surroundings into a breathtaking garden backdrop. Leveraging MoveAI, a cost-effective and simplified approach, we have ventured into the realm of virtual avatars, crafting a digital representation of the dancer, foregoing the need for a motion capture device.

Findings

1. Augmented reality visuals always take the forefront, requiring special design considerations to seamlessly integrate virtual elements into the real world. In our performance, we strategically positioned most of the virtual elements above the audience’s line of sight to prevent any interference with the front row and the performers.


2. As no VR headsets with AR functionality were available in the market at the time, this project initially sought to incorporate AR into a VR headset. We modified one of the latest VR headsets to fulfill this purpose. To ensure that the virtual sets align with the physical space, which they were not originally designed for, we had to calibrate each device before the show. In case a device went into sleep mode, we needed to recalibrate it. Consequently, it was crucial to maintain the wakefulness of all devices throughout the performance to prevent any disruptions.


AR Activated Murals

Team

This project would not make possible without the below amazing parties and individuals

Presented by: Perrier Jouet 120th Anniversary Art Program
Curator: Anita Lam, The Collective
Technical Director: Andy Stokes, The Collective
Creative Technologist: Yoki Yao, The Collective
Visual Artist: Neil Wang
3D Animators: Desmond Lo, Bob Tsang
Musicians: The Interzone Collective
Dancers: Alice Ma & Jonathan Hung