Creative Dialogue

Immersive Storytelling – Using VR To Create Complex, Nuanced Experiences

Michael Bloomberg is in Bonn, Germany, taking a break from the United Nations’ COP23 conference on climate change to watch Our Home, Our People – a short virtual reality (VR) film co-produced by The World Bank and the Fijian Government.

Outside it’s blowing a gale, but inside the headset Bloomberg has been transported to the shores of Fiji’s Vunisavisavi. Here, Bloomberg is guided by a local family through the village to witness the lasting damage left behind by cyclone Winston – the most intense cyclone to hit the Southern Hemisphere. He’s then taken to the coastline where families go about their daily lives in the aftermath, while statistics from the Fiji Climate Vulnerability Assessment emerge to solidify the massive impact that climate change has, and is still, reaping on the island nation:

30% of Fijians live in areas that are at high risk of extreme weather events such as flooding and cyclones.

 Sea levels across the Asia Pacific could rise by up to 18 centimetres by 2030, bringing larger and more frequent storm surges.

…the cost of significantly reducing Fiji’s climate change vulnerability is US$4.5 billion over the next 10 years.

After eight minutes the video ends and Bloomberg removes the headset and sits there thinking, letting the gravity of Fiji’s situation sink in. Tom Perry, Team Leader for Pacific Communications at The World Bank, is one of the storytellers behind this experience and says this has been a typical reaction.

“The most common reaction you see is a very quiet response. I’m very conscious when I’m presenting this film to give people space to reflect on the experience they’ve just had, and time to think about what it means to them.”

Immersive Storytelling - S1T2 and The World Bank

Just A Gimmick?

With international recognition, film awards and over one million views online, Perry could say that VR was the ‘right’ medium to tell this story. If he’s being honest, though, he admits initially seeing VR as a gimmick – an unpractical new technology. It wasn’t until he teamed up with Tash Tan and Chris Panzetta from creative technology agency S1T2 to create a VR series on social conflict in the Solomon Islands that Perry witnessed VR’s power to inspire empathy in audiences.

For this project the stakes were equally as large. 2018 was the first time a small island nation has taken on the presidency of such a major global climate change meeting as COP23. The target audience were key decision makers on climate change, and there aren’t many individuals more relevant – or intimidating – than billionaire and UN Special Envoy for Climate Change Michael Bloomberg. Then there was the need to do Fiji’s story justice – and still S1T2’s Tash Tan believed VR was the right choice.

“When you talk about the Pacific, this region of the world has done little to cause climate change, but is one of the key countries that will experience its impacts most,” says Tan. “We wanted to capture this message and bring it to a global audience in a way that captured the gravity of emotions and experiences felt by local communities. That’s why VR is so important – the medium lends itself to this empathy and understanding of what the situation really is.”

Slice Of Life

Telling this story was easier said than done. Two flights and a long bus ride are needed just to arrive on site, then there’s navigating the weather and identifying the right people before production can begin. Tan says the largest obstacle to quality footage is gaining the trust of the community, a fact made harder inrural Fiji,where your presence alone requires formal consent from the village chieftain. But this is where VR’s unobtrusive, organic nature supports the storytelling process and allows you to put down the camera down and let life happen.

“We’re not there to create our own story, but to tell theirs, so we need to respect the community that has entrusted us. Our process involves getting to know the people we want to feature and ask their permission to share their story, then we say; take us to these places you talk about in your story – and show me. Live life.”

Forget The Hacks

The hardware has improved dramatically since S1T2’s first VR project with the World Bank in 2016. Footage that was once shot with multiple camera’s strapped together can now be captured via self-contained rigs. Similarly, the software has risen to new heights. Using Adobe Creative Cloud For Teams, Tan’s crew were able to use the VR auto-detection function in Adobe Premiere Pro CC to view footage in real-time – a historically laborious process.

“Before that VR function came out you’d have to export footage, render it, put on the headset and experience it. Only after this you’d realise – OK, there’s an issue here. Let’s do it again,” says Tan.

The crew nailed down a storyboarding process on location by using Adobe Premiere Pro CC to easily dump footage during the day then review each night. In the post-production phase, they used Adobe After Effects CC for the “clean up” – anything from adding ground plates to complete a 360-degree view to stabilising footage from environmental factors (“cheeky farm piglets,” in Tan’s case). There is an optimal subject distance when shooting in VR, so the team also fixed any motions that were incoherently captured across cameras in Adobe After Effects CC.

A surprising feature the team discovered on the job was the ability to warp VR-ready text into the footage – an important factor for this project that leverages insights from the Fiji Climate Vulnerability Assessment.

From shooting to editing to publishing, Tan says the process for VR storytelling is getting much easier – but as a creative this is what he expects.

“This is what it should be. You don’t want to have to do all these hacks to create a great story, even though it’s a beautiful part of the exploratory process. You want to be putting your efforts into telling the story itself and letting the tools make it easier for you to do that. That’s what Adobe is for us – instantaneous innovation. As we innovate, they keep on par and push us. Likewise, that’s what we expect as artists and that’s what we crave.”

The Magic

Perry has since returned to the community of Vunisavisavi to show the people their story. He reflects that storytellers today faces a landscape full of short attention spans, deadlines and conflicting stakeholders, but we should never take our eyes off the prize – giving justice to someone’s story is taking time to add that extra layer of nuance and complexity.

One of Tan’s “magical moments” from Our Home, Our People is an everyday scene captured on a boat with local Rai and his children. Seeing a low-hanging branch ahead, Ray ducks then seemingly tells you – the audience – to watch out also, until you realise he’s speaking to his children 180-degrees behind you. Tan says this ability to immerse audiences and humanise a scene was the key to making Fiji’s climate change story stand out amongst the multitude of others being told today.

“When you talk about the purpose of technology, it always comes back to bringing the humanity back into things. What we wanted to do is distil climate change down to something we could all understand – which is home.”

Creative Dialogue, Design, Inspiration, Video

Join the discussion