Mad God VR is a stop-motion animated immersive experience by Phil Tippett. A rare example of a traditional artform transcribed to a new medium, the conceptual framework of central staging theater is inverted, with the viewer placed at the center of a macabre arena. Life-size puppets fill the perimeter while ambisonic audio helps direct the series of events that unfold.
View from the Oculus Rift headset.
Making-of story and interview with Phil Tippett.
Stop-motion puppet frame capture. 24 puppets were animated by hand and photographed in stereo based on a measured distance from the viewer.
Photogrammetry capture rig of the miniature set. The camera spins 360 degress with motor and actuator, capturing a still image every 10 degrees. The resulting raw capture data is then retopologized into layout geometry.
Stereo test with the captured set. The background set was rendered in stereo while the creatures were rendered in stereo on cards that were placed around the viewer in 3D space. This allowed for some parallax and freedom of movement while maintaining the look and feel of stop motion.
View in Unity of the character layout and stereo 360 projection shader. The shader was designed to project left/right eye equirectangular images onto meshes from an arbitrary position in world space, allowing for precise camera alignment.
Testing playback performance and spatialized audio in the Oculus Rift. 24 movies were synced and played back in realtime with the help of the HAP codec and a fast NVMe hard drive. Performance was not fill-rate bound, as is typically the case, but rather I/O bound due to streaming so many movies off disk. Pausing playback for characters outside the view frustum was essential.