"This is Cinerama." What was Cinerama? Cribbing from Wikipedia: "Cinerama is a widescreen process that originally projected images simultaneously from three synchronized 35 mm projectors onto a huge, deeply curved screen, subtending 146° of arc." I had to look up subtending - basically a line joining the endpoints of the screen's arc. We'll keep that in mind as we develop our virtual reality recreation of a Cinerama screen. I recommend reading the wiki article, though in summary the process involved filming with three overlapping cinema cameras to project a panoramic image, an effect that is very similar to Imagineering's Soarin' projection technology and very like the multiple camera 360 pano capture devices that are flooding into public hands.
When we ask "can Hollywood make a VR movie" the simple answer is that they sorta' already did in 1962, and it worked quite well. As the documentary Cinerama Adventure demonstrates, some directors were frustrated by the limitations the format imposed on their work, while others embraced the creative possibilities within those boundaries. Both facets are demonstrated in the epic MGM movie How The West Was Won. As one subject put it, the scene is often front lit, side lit, and back lit. That sounds very like a 360° VR shoot. Since Turner has released a Blu-ray version for which their technicians have done a brilliant computer-stitched version of the three camera's frames and even included border material that most theaters lost in projection, I think it is a perfect opportunity to attempt to recreate the Cinerama experience in VR.
The first step involved creating a screen geometry for projection. Since the Blu-ray material extends the HTWWW screen dimensions to nearly 180°, I used Autodesk Maya to build a partial sphere that covered that angle. Interviews with Cinerama filmmakers have made reference to a "Cinerama zone" where I imagine the sweet spot for viewing takes place, much like any current theater. I found that this was not in the center of the sphere, as this gave a feeling of being too close to the screen. In addition, I found that a cylindrical screen - which is what would have been found in Cinerama theaters - led to an effect of stretching at the top and bottom of the image, probably because these areas are at a greater dimensional distance from the viewer, but was likely a necessary concession to audience members who were not seated in the "zone" and also would have been much less costly to build than a spherical dome/screen. We don't have that limitation in VR, although the VR scene we're using can display on any surface form.
After some testing, the screen was scaled up 15 times. One reason viewers are uncomfortable setting close to a screen might be that their eyes are focused at twenty feet when their mind is seeing objects that are supposed to be at or near infinity. It doesn't seem to work the other way, it is not uncomfortable to focus far away on a subject that seems close to the camera, so I scaled our screen geometry to allow the VR viewer to focus at a distance, making the massive screen seem more comfortable.
Because we're using the Unity engine to display the scene in an HTC Vive, there is no projector, instead the pixels are mapped in the GPU via UV position to points on our screen geometry using the GPU's warping capability. It's just texture-mapping with a movie instead of an image, and Unity is up to the task on my MSI laptop, even after changing the texture import settings to 100% quality. This takes a long time to import, and was frustrating when tweaks needed to be made.
Unity won't play a Blu-ray directly, so I needed to convert the movie to MP4 files, and keep the file size under Unity's two gigabyte texture limitation, so in order to keep the quality pegged I exported into several 1GB chunks and wrote a script to swap the movie textures as needed. This made it easier to jump around within the film for testing as well, after I bought the movie here.
Here are the controls and movie swapping code attached to the 3D screen0 model object in Unity:
