It takes 48 cameras to capture every millimeter of our dancer Salif, to track every movement ranging from the twitch of a finger to the smooth glide of a moonwalking foot. That’s the setup at 4D Views in France, where we conducted our second tech demo. Following the live motion capture earlier this year, this time we decided to experiment with a newer technology to see what it can bring to our vision of the music metaverse.
Volumetric capture allows us to create a 3D model of a performer which fans can superimpose onto their own surroundings using mixed reality technology. They could point their phone at an empty space in their kitchen, garden—even go to a beach or forest—and Salif will appear there, dancing and talking as if by magic. They can even take videos of themselves interacting with this digital Salif and share them on social media.
Unlike motion capture technology, which is useful for creating avatars to perform feats which would be impossible in reality, volumetric capture allows us to capture real movements with an unparalleled level of detail, creating a more convincing image. It does this using a bank of cameras that surround the performer in a circular space with bright lights, flanked by curtains of green felt. Since there is no motion capture suit needed, it also means the performer can wear different outfits and the movement of the fabric can be captured easily without needing to be programmed laboriously in post-production.
In a high-tech studio in an unmarked building in Grenoble, flanked by the high alps and fresh mountain air, we start at 9am sharp, with our stylist Axelle unloading four suitcases of designer clothes ranging from Louis Vuitton to Jacquemus to record Salif in various styles. We dreamed up as many ideas as we could for our volumetric captures—how might fans want to interact with digital Salif?
We asked him to imagine an invisible fan alongside him and pose with them for selfies, teach them to dance and even tell the story of his first moonwalk in front of the Arc de Triomphe in Paris, the fateful dance which launched him to viral stardom.
On the other side of the green screen, the effect on the computer was magical. We saw a photorealistic digital version of Salif replicated to the smallest detail. It’s easy to run away with the possibilities of this tech—who might want to act out a scene opposite digital Brad Pitt, or dance alongside Beyoncé? Salif’s session went off without a hitch. Now it’s up to us to dream up all the most exciting ways we can give these captures to the world and his dedicated fans.
Interactivity in Virtual Concerts
In the line for the concert snaking around a residential block in south London, it seemed like I was the only person without blue hair.