CNN
 — 

In a studio in Culver City, California, T-Pain is able to placed on a present. The American rapper and singer-songwriter, recognized for popularizing Auto-Tune along with his hits corresponding to “Buy U a Drank” and “Bartender,” is used to performing in entrance of hundreds of screaming followers. But as we speak, there’s no crowd; as a substitute, it’s simply T-Pain, a digital camera crew and a inexperienced display screen.

It’s a departure from the previous, however a nod to the longer term. T-Pain is performing songs that may turn out to be a digital actuality (VR) live performance, delivered on to followers through VR headsets.

“I’m definitely used to bigger stages,” T-Pain (whose actual identify is Faheem Rashad Najm) tells CNN throughout an interview on set, including that he sometimes makes use of suggestions from the group to “really gain my energy … (so) it’s just a lot more awkward.”

That doesn’t imply he isn’t excited concerning the alternative. “I’m just a nerd,” he says, who dabbles in VR and 3D software program as a interest at residence. When AmazeVR, the corporate behind what was billed because the world’s first VR live performance tour with artist Megan Thee Stallion final yr, approached him for a collaboration, the Florida-born artist was able to attempt a brand new expertise.

“The fans are going to be a lot closer,” T-Pain says. “You get to see different angles, you get to see a different kind of performance … AmazeVR is kind of like putting (me) in your living room.”

AmazeVR was based in 2015, and has raised greater than $50 million within the final 4 years, together with a latest enhance in December after Apple announced its Vision Pro mixed reality headset, in line with the corporate.

“(Virtual reality) is a very big shift,” says Kyung Kuk Kim, vice chairman and head of manufacturing for AmazeVR. “It’s a new kind of medium. People are not used to it, but it’s very powerful. It really just brings you right (to the) middle of any scene or any situation.”

The digital live performance course of begins with conferences with the artist to determine on the whole lot from track option to visuals, scripts, wardrobe, and choreography. The efficiency is recorded in a single 12-hour session utilizing an 8K stereo digital camera and motion-controlled crane.

In-house synthetic intelligence (AI) helps improve the 3D video in post-production, Kim says. A mixture of that in-house AI and different platforms corresponding to Unreal Engine are used to create the digital stage, lighting, and particular results.

Post-production sometimes takes six to eight weeks, although launch dates additionally coincide with what works for the person artists. (AmazeVR says it plans to launch the T-Pain live performance this Fall.)

“The biggest technical challenge is that the viewer is the camera,” says Lance Drake, artistic director for the corporate. “Instead of cutting or hiding (parts of the) performance, this has to essentially be a truly live performance. And the viewer is given permission to look wherever they want – they can look and take in the world, they can look at the artist’s shoes, they can look at the artist’s face.”

That means it should be the “perfect” single take, Drake provides, “so when the artist is performing to camera, they’re performing to you. When the artist is looking into camera, they’re looking into the viewer’s eyes.”

Still, the massive query stays: how intently can a digital live performance mimic what followers love about dwell music?

It’s one Dr. Mel Slater is attempting to reply. A researcher who has studied the world of VR and our brains for the reason that early Nineteen Nineties, Slater and his workforce not too long ago conducted a study utilizing a 1983 Dire Straits live performance Slater got here throughout on YouTube.

“I thought, ‘I wish I could go to that concert … so that gave me the idea of reproducing a Dire Straits performance in virtual reality,’” says Slater, distinguished investigator within the Department of Clinical Psychology on the University of Barcelona and co-director of its Event Lab (Experimental Virtual Environments for Neuroscience and Technology).

He was concerned with testing a number of parts, together with beneath what circumstances individuals would have the phantasm they have been at a live performance, and the way to take an previous 2D video and create new 3D parts, he says.

“I thought it was a very innocent study, that you put people in a music concert and let’s see what happens … they should just enjoy it and that’s it,” Slater says. “But the results are very interesting. And it’s always good when you get results you don’t expect – because that’s when you learn something.”

The workforce discovered that many ladies within the viewers really felt uncomfortable. “(They) would say things like, ‘there were these men around me, and I got worried that they were going to come over and start talking to me. And they kept looking at me,’” Slater explains. “This wasn’t actually true – we hadn’t programmed these characters around to be looking at them, but they had the illusion that they were.”

In a means, that was a very good signal for the idea of digital concert events generally, he says, as a result of that meant “there was a strong sense of what we call presence – the illusion of being there and that these events were really happening.”

Another remark was that individuals stated that “in reality, when they go to a concert, they never go alone – they always go with friends and family,” he provides. “So, we built another environment where you can attend the concert, but with other people that you know,” utilizing avatars (digital characters) that appear like them.

That mannequin awaits examine, Slater says, and the objective is to check if this alleviates among the expertise points they noticed. “My intuition is that this will quite radically change how people respond to the overall audience and to the concert itself,” he says.

What additionally takes a digital live performance to the following degree is the dwell filming – a key distinction between the standard of the Dire Straits model (utilizing laptop graphics to show an previous live performance video right into a digital expertise) and AmazeVR’s strategy of getting the musician carry out on to the digital camera in 3D. But what stays difficult is convincing the mind it’s actually occurring proper in entrance of you, in line with Slater.

“We look at television and we go to the movies and everything we see is very realistic. But we know the people in the movies are not seeing us,” he says. “(Virtual video) is not completely effective because some part of your brain is saying, this is video, they’re not seeing me … So there are advantages and disadvantages.”

While the concept of a digital live performance versus the true deal may look like a troublesome promote for hardcore music followers, there are advantages to scaling again within the bodily world.

According to the UN Environment Programme, “live concerts and tours can contribute to the climate crisis, driving up emissions through fan and artist travel, energy consumption and the mass production of merchandise.” Meanwhile, rising ticket costs to a few of today’s most popular concerts, together with Taylor Swift’s “Eras” tour and Beyoncé’s “Renaissance World Tour,” are pricing people out (although additionally boosting local economies).

“A lot of artists don’t have a chance to have a big concert because it needs a big budget, all the production and everything is so complicated,” says AmazeVR’s Kim. “But even small or big artists, (with VR), it doesn’t matter. You can always make your own VR concert and share it to all the fans all over the world, and I think it’s giving a very good, fair opportunity to every artist and every fan.”

AmazeVR’s app debuted on Meta App Lab and SteamVR on July 13, earlier than it releases to all main prolonged actuality (XR) shops, together with Apple Vision Pro. According to AmazeVR, pricing is at present per live performance; it should value $6.99 for one yr of entry to the T-Pain live performance, for instance, after that content material launches.

A collaboration is within the works with Okay-pop group aespa, whereas AmazeVR is “in discussions with many labels and management companies,” with curiosity in VR concert events on the rise since Apple’s headset announcement, the corporate says.

It can be producing and releasing blended actuality (MR) concert events – a mix of digital and augmented actuality – by overlaying virtual elements onto live, real-world video. This will enable followers to “interact with the music artists using hand gestures,” the corporate says, a component that Slater believes can go a good distance in how our brains understand and settle for digital environments.

Regardless of the platform or the venue, it’s nonetheless all concerning the music, says T-Pain.

“Even when you’re not at a concert … kicking it with two friends and putting on my music is kind of like a party,” the rapper provides.

“I think the music kind of does that on its own.”

CNN’s Leif Coorlim contributed to this report.