A group of Magic Leap alumni are making experiences for 50 pairs of smartglasses at once - and they're working with an world-famous performance artist
- Technologists think that augmented reality is the next big thing that could replace smartphones, TVs, and all the screens in your life.
- Alumni from Magic Leap, a hyped augmented reality startup, are now using the technology to create music and art pieces.
- Their first work stars Marina Abramović and will premiere simultaneously to 50 people in London in February.
Marina Abramović, the renowned performance artist, is going to be the star of a piece premiering February 18 at the Serpentine Theater, in London.
But she's not going to be there in person - or, at least, she's not performing live.
Instead, she's going to be performing in the lenses of 50 augmented reality headsets simultaneously, all of whom will be wearing AR hardware, including the Magic Leap One, a hyped pair of smartglasses that launched earlier this year.
"Up until now it has been impossible to understand what the artist does and the purpose of the event is unless you were in the room at the time of the event," Todd Eckert, founder of Tin Drum, which is producing the piece, told Business Insider.
"As a performance artist, my work is about energy, so the work we've been doing here, what is the chance that I can really keep energy that can radiate to the public, and you can feel it," Abramović said in a video starring her 3D avatar.
"This is a really promising media," Abramović continued.
What is augmented reality?
But Tin Drum, a studio partially staffed by Magic Leap alumni and other technologists, is betting that before augmented reality becomes an everyday technology, it's going to become a new tool for artists and directors to create works that can't be displayed on a traditional, flat screen.
With this most recent piece, "you have the ability to feel what performance is like, not as an artifact of something that already happened, but the energy of what's happening now as if you are actually there at the time of the event," Eckert explained.
Eckert first started thinking about augmented reality and its challenges when he worked for Magic Leap as its director of content development. There, he reached out to a variety of musicians and artists to get their ideas on how to transfer a live performance into a wearable augmented reality experience.
So, after he left Magic Leap, he decided to start a production studio focused on augmented reality projects - and the Abramović piece is Tin Drum's first public exhibition.
There are lots of challenges when recording a person in three dimensions, often called "volumetric capture," he explained. It usually requires an array of cameras, as many as 32, as well as a lot of care and time to put the images together in a way that can be placed into a real-world environment.
Other challenges include how colors are represented in augmented reality - red looks very different than, say, black, inside a headset that's based around displaying graphics inside transparent lenses. And each different augmented reality headset, like Hololens or ODG, also requires tweaks.
"The positioning of cameras and the specific lighting attributes during the time of capture is very specific based upon each project," Eckert explained. To create the Abramović piece, Tin Drum teamed up with 4DViews, a volumetric capture company in France.
I was able to see a short preview of the Abramović piece on a Magic Leap One, although I was nowhere near London. Inside the smartglasses, you could see the artist, and although there were technological limitations including the headset's field-of-view, cutting off parts of the virtual people in frame, I felt her "presence" in the room with me - but I was just in a conference room in New York.
But simply capturing a 3D image of an artist is only the first step. There's lots of post-production that's needed in order to take the raw footage and turn it into a graphic that can be integrated with the real world inside a pair of augmented reality glasses. Then Tin Drum's specialists, who are technical and proficient in software like Unity, tweak the footage in Tin Drum offices until it's perfect.
Tin Drum wants to produce other high-quality mixed-reality experiences, and Eckert said the company is close to raising venture capital so it can continue to develop technologies for volumetric capture.
Going forward, Eckert believes that in a new medium like augmented reality, it's going to be more important to experiment with completely new ways of thinking about media, as opposed to shoehorning older modes of thinking onto new technologies.
"I think people have a tendency of trying to use what they know and in order to kind of embrace whatever they perceive as new," Eckert said. "This is recognizing that the technology can do things beyond anything else that's ever existed."