For the last years, I have developed many different VR experiences that were displayed inside of buildings and in controlled access areas. Allowing the viewer through physical space has proven to be a challenging task as you have to restrict the range of movement whilst constantly supervise movements in order to prevent accidents.
Placing VR in public transport vehicles like buses, trains, boats and perhaps elevators seems to be a move in the right direction as they tend to be safe and existing VR Roller Coasters show that the use of physical controlled movement complements the immersive illusion by adding the haptic sensation of both physical movement and physical acceleration/deceleration.
The first stage of this research project will involve learning how to capture data form the accelerometer and the gyroscope of the VIVE FOCUS in order to sync the movement of the 3D camera (through digital space) with the physical movement of the viewer.
I have already found some tutorials for Unreal Engine, so I hope to develop my first prototype by the end of April 2019.
From March 2018 I started leading a research project funded by Debora Arango Higher Education School. The project aimed to create a bespoke process for the creation of an immersive story.
For six months I led and worked side by side with a team of professionals from the audiovisual industry i.e audio specialist, camera crew, producer and actors.
The first plan was to create a 3D environment and placing real-life footage of performing actors. So we started shooting actors against a green screen and testing the resulting video inside of very basic 3D environments.
After testing the chromed characters inside of both Unity and Blender, I decided that the most efficient course of action due to the complexity of this project was to produce a Monoscopic 360 video with ambisonic sound instead of building a VR app.
We spent a good amount of working hours trying to figure out the best way to place characters in front of a moving camera. One of the biggest challenges was consistently matching lighting schemes on both ends (physical and digital), Another big challenge was to figure out how to place and reveal characters in the 3D environment preventing them to reveal their flatness once the viewer’s point of view changed.
We managed to overcome all this obstacles and ended up introducing a total of 5 characters, including two that directly interacted with the viewer at a very close range.
Regarding the 3D environment, we discover that buying an existing 3D environment was not an option as we wanted it to reflect specific architectonic features from our culture and the models available online have very different ones, so apparently, depicting a typical Colombian city hasn’t been of interest for 3D artists.
So I decided to start building the entire fragment of the city from scratch.
The first models were quite basic and this was actually pretty useful as the script evolved a lot an we ended up changing the story and characters more than 3 times.
I also started testing two different approaches for building creation, the first one explored the process of building an entire facade by extruding some features of a frontal photo of an existing building.
The process proved to ad a quite realistic look to the scene but, at the same time, added a lot of size to the textures used in the project and demanded a lot of retouching e.g removing cables from the facades or getting rid of shiny materials on doors and windows.
The second option was to build all the locations from scratch using convencional 3D modeling and texturing techniques.
In the end, I decided to mostly do every asset from scratch in order to have enough flexibility to design the 3D environment, in total, it took me over three months to build a fragment of typical Colombian city spread across a 5 by 5 array of blocks.
It is worth mentioning that I did use google maps to navigate the cities of Medellín and Envigado to get inspired and also to get the footage I needed to replicate architectural features of two emblematic buildings.
Finally, to create the ambisonic sound for the video I learned how to integrate Reaper (Digital audio production application) and Facebook 360 workstation, this allowed me to spatialize both sound effects and dialogs.
The following is a 2K version of the project, it took 8 days of continuous rendering using 16 27-inch iMacs to produce a total of 8800 frames.
This is an ongoing project, at the moment I am looking to get more funding to finish adding more characters, 3D animations, enhancing the sound FX and rendering it in at least 4k quality.
This last Wednesday Raum Gallery opened the door for me and I had the opportunity to spend for almost 7 hours sharing my first VR piece with people that was passing by (At some point I used the VR set as a “fish hook” – it worked!).
Although I was not counting, I estimate that at least 50 brave participants sat to experiment my VR experiment. This experience taught me a lot as I had the opportunity to chat with almost everybody about the experience.
One unexpected and wonderful fact about this whole event is that, for more than half of the participants, it was the first time ever they interacted with a VR device.
This is a 360 video that closely depicts the VR experience. Try watching it in 4K.