XR Storytelling Resources

We are entering a new era of XR together. As early adopters and creators of XR content, it is up to us to pioneer and influence the evolution of these immersive mediums.

The purpose of this page is to help those new to XR. Here is a vault of resources to facilitate the story development process. We also aim to start developing a common toolset and communication techniques that can translate to everyone involved in the creation process, from the producers, technologists, artists, directors, and storytellers.

We do not require any of these templates, styles, or formats in your submissions; you can submit whichever format you feel is best to communicate your vision.

Developing Your Story

Immersive Writing with the Evette Vargas Immersiveplay

While it's not mandatory to follow this format, if you're curious about formats we recommend Final Draft's Immersiveplay, an immersive writing template. Award winning screenwriter Evete Vargas walks us through what VR storytelling is, along with what it should be and what it should not be.

Story Archive

Your story archive document is a vital aspect to your script, and thanks to cloud-based computing, it also makes collaboration across the entire pipeline much more streamlined. Think of your SA as something like a bible (TV) or narrative design document (games), but optimized for VR/AR

In addition to a basic treatment/synopsis, your Story Archive should include a breakdown of Characters, Locations, Objects, and Mechanics. By building out these sections, you’ll make it far easier for other collaborators to grasp your vision without having to clog up the space of the script itself with information overload.

While the first three are self-explanatory, Mechanics takes a bit of explanation. In essence, we use “Mechanics” as a catch-all for anything interactive: abilities players or other characters have, elements that can be activated in a given space/time, physics/laws of a given environment, or any other notable manner by which the experience facilitates interaction. Since mechanics can be associated with all of the other categories, be sure to list these associations in both the mechanic card itself, but also the respective associated cards.

Immersive Storytelling Tools

Many writers who are less familiar with VR and AR/MR content may be intimidated by the multitude of new tools and technology that make this content possible. We’re here to help.

This is not meant to be a comprehensive overview of the different types of XR, but instead is intended to give a taste of the different types and the tools and capabilities of each. To go more in depth on the different types of content, take a look at our resources page, or better yet, check out as many experiences as you can!

VR Classifications

For our purposes, we will break down VR storytelling into four basic types as proposed by Devon Dolan and Kent Bye in “Redefining the Axiom of Story.”

Observant Passive

“Ghost without Impact”

The user is not a part of the story and has no agency to interact with or have any effect on the narrative. This is most commonly seen in 360 live capture pieces. Example: My Brother's Keeper

Observant Active

“Ghost with Impact”

The user is not a part of the story as a character but can have agency in directing the narrative

Participant Active

“Character with Impact”

In this scenario, the user is a character in the narrative with agency to interact with and effect the narrative. The degree to which the user can interact with the narrative can vary. This is most commonly seen in 6dof experiences.

Participant Passive

“Character without Impact”

The user is a character in the story but has no agency to interact with or direct the narrative.

3DoF

In content with 3 degrees of freedom (3DoF), the user can look all around them from one location in 360 degrees, but cannot physically move around the space or move closer to/further away from objects like in a 6DoF experience. This type of content is generally seen on mobile platforms like Samsung Gear and Google Daydream and can be live action, CG, or a combination of the two. While generally more limited in its an interactivity, it can still have interactive elements like in this piece from Felix and Paul: Miyubi.

While the user cannot physically move around the space, the filmmaker can move the camera similarly to how they would in traditional filmmaking on helmet rigs, rovers, drones, etc.

This type of content can either be monoscopic or stereoscopic (“3D”), which enhances the user’s sense of depth and spatial awareness.

6DoF

Content considered to have 6 degrees of freedom (6DoF) is generally built in game engines like Unity and Unreal and are viewed in PC connected HMD’s like the Oculus Rift and HTC Vive, although stand alone 6DoF HMD’s are in the works. In a 6DoF experience, the user can not only look around in 360 degrees but can also move up and down, forward and back, and side to side, getting closer to and further away from objects in the scene.

These experiences can either be created for users to stand/sit in one location with a limited range of motion, or can be fully room scale, allowing the user to physically walk around the virtual space.

Using controllers or hand tracking tools like Leap Motion the user can even interact with virtual objects in the scene, picking up and throwing objects, pushing buttons, opening doors etc.

In location based experiences from companies like The VOID, real world physical objects are incorporated in the experience; when the user reaches out to touch an object in the virtual experience, they actually feel the object in the real world.

Sound

All of these experiences can benefit from the use of spatial audio, which has three dimensional and positional data to depict audio coming from different directions, allowing the user to detect direction, distance, depth, and movement. As objects in the scene or the user themselves move, the audio reacts accordingly, making the experience more immersive. Spatial sound effects and dialogue can serve as important narrative cues to guide users along the story.

Examples: (headphones required) Warning: Horror - Alien Covenant In Utero from 20th Century Fox and 360 Video Spatial Audio sample from ILMXLAB

AR & MR

Augmented and Mixed Reality differ from VR in that they visually incorporate the physical world. This type of content can be created for mobile platforms or for glasses like the Microsoft HoloLens or Magic Leap Lightwear. AR glasses have not yet had a widespread public release and their potential for narrative storytelling has been less explored, creating an exciting opportunity for storytellers.

While the same four basic types of VR stories noted above can play out in AR, the user can incorporate real world experiences in AR in ways not currently possible in other mediums. One of the most fully fleshed out examples of this is the interactive crime story for the HoloLens, Fragments, in which the user participates in scenes that happen in whatever room they’re currently in.

Creators can also effectively use AR for what it’s name implies by augmenting a location based experience that uses real world actors and sets, like this experience by Here Be Dragons. As hardware improves and major players like Magic Leap release consumer ready headsets, the possibilities for new ways to tell new stories will continue to grow.