January 29, 2018 Justin Chin

Storytelling for 360 Media

THE CHALLENGE

With new emerging platforms today (and the general evolution of new media) we now find ourselves with some interesting narrative challenges.

Specifically, of note; 360 video or virtual reality experiences where your viewer has a choice of what they want to focus on. Experiencing a 360 video can be stunning to the viewer, but telling a story with the medium can be a challenge. There are many ways to tell a story and we are entering a new evolution of what that means with 360 video and the VR world.

So how can we evolve from our current storytelling paradigm into a 360 medium?

360 / VR / FLAT TERMINOLOGY

For this article I’ll be using term “360” as a way to describe both 360 video (360V) as well as virtual reality (VR). 360 video and VR have many uses and applications in their respective platforms but they both also share qualities that will be my focus in this article. Whether it’s a 360 video or a VR or variable POV video game, each allow the users to change their view in any direction they choose—-no matter if they are able to directly interact with the world or not.

When I use the term “flat” that indicates most any medium shown in a fixed frame, and a fixed timeline experience like a traditional feature film or series for the television or computer screen.

TEMPORAL STORY DELIVERY

Primarily what this article will explore is the relationship to plot and the 360 medium. There are plenty of areas in the art of storytelling that can be discussed, but for now I’m focusing on the what it means to deliver the baseline of a story – usually through the method of plot.

Let’s begin by examining how a narrative is delivered to the audience in the traditional flat medium.

Sequential varied framed imagery, over time, with voice and music.

This is a simplistic distillation of the flat storytelling medium, but what is important in the description is how the creator delivers the desired imagery over a certain time period. A series of images can direct the viewer into a very specific (controlled) flow of visual and audio storytelling.

For 360 content, that delivery method changes dramatically. The primary challenge is that the viewer can change their desired view angle at any point.

There are a few important factors to consider when approaching the way a viewer interacts with any experience in 360:

  • A visual landscape that entirely encircles your viewer.
  • The temporal adjustment the viewer must make on each cut/location/scene change.
  • Each cut feels like a move in space therefore the viewers must make an adjustment to orient themselves. This tends to limit your ability when dictating pace (i.e. through traditional editorial means).

No longer can we depend on the visual coda that has been developed over the last century. Linear framed imagery doesn’t exist in VR. If I look to the left, I might miss what’s to the right. If I hear something behind me I might turn my head, but then I might miss something yet again. Additionally, you can’t keep the story going in one direction, that would be a waste of the medium.

Creators are using various techniques to guide the audience with movement and sound cues. Those techniques can help, but they don’t always do what they are supposed to do. They are going to miss something; it’s inevitable. If that something is an important story element and they haven’t seen it, you’re at risk of losing your audience.

So, in short, the challenge is:

If I can’t dictate where the viewer looks, how do I ensure that they see my story?

The truth is that you can’t. Your audience is, in part, a participant in the editing of your story. They will pay attention to what they want to see.

So, my recommendation is to embrace it.

This is a paradigm shift, and with that, you must think differently about what storytelling means in this medium. So let’s break that down to how one might approach this challenge.

ELEMENTS OF STORYTELLING

Visual storytelling can often be driven either by plot or by thematic motives. Plot driven stories have been written in three acts, with a series of obstacles all leading to an eventual resolution. This is has been referred to as a story triangle.

A typical story triangle might look like this:

[fig 1] 3 Act Story Triangle

In 360, your audience might miss some of those specific elements if they’re not watching them. What comes to mind when developing a story for VR with a 360 landscape is an opportunity to deliver the components of a storyline through a permeation of thematic elements—-both in space and time. And as mentioned before, a flat story projects sequential framed images over time, but the 360 world surrounds your viewer, and they have control in what they want to focus on.

Because of this, progressing your story becomes not just about moving the plot (plot driven), but projecting your theme (theme driven). The best 360 experiences have and must heavily build theme in the storytelling. Theme development should be reflected in all aspects of your story details, and more importantly they must permeate the world you’re creating. Remember you’re building a world, not just a frame. Inside that world lives your story and or theme.

When writing any script, you’re most often writing from a character’s point of view. In a flat medium that POV is expressed through the frame: what the protagonist observes, what they can’t observe, what they are not allowed to observe, or what they’re failing to observe. That POV is a large part of framed storytelling, but it’s NOT in 360 video.

Specifically I see the differences as:

FLAT = A character’s POV of the story

360 = The audience’s POV of the world in which the story takes place

This is the crux of the differences in each medium, and in turn where we must evolve our thinking as to how 360 stories are developed, written and executed.

Maslow’s Hierarchy of Needs

In considering how flat storytelling is about the POV of your character, it made me think about Maslow’s Hierarchy of Needs, which is represented in a pyramid (coincidentally also a triangular shape).

[fig 2] Maslow’s Hierarchy of Needs

Maslow’s theory was that a person develops from the bottom to the top of this pyramid. He suggests that at the most basic level one must fulfill one need before they can fully move one to the next (much like the acts or chapters in a story).

Working from bottom to top:

  • Physiological – our basic biological needs – air, food, water, clothing, shelter etc.
  • Safety – personal security, health, financial security etc.
  • Love / Belonging – Friendship, intimacy, family etc.
  • Esteem – mastery, recognition
  • Self-actualization – this is the realization of one’s full potential

Maslow understood that anyone can focus on multiple levels of motivation at any time, but his theory focused on the notion that betterment is achieved through a constant state of change. It’s important to understand that these are all interrelated and not just discrete levels. I think about it this way: the top is my goal, the elements below it allow me to be my best self in order to achieve that goal. It also says that the better you develop the lower needs, the stronger you’ll be in achieving and reaching the levels above it. This is a building, supportive structure.

360 Hierarchy of Story

So what does this have to do with 360/VR and the future of narrative content? If you’re a storyteller, you might find that this sounds like the progression of a character-driven narrative or a character arc.

360 = The audience’s POV of the world in which the story takes place

This statement suggests that we must think at a higher level. To that end we can take the method and shape of Maslow’s Hierarchy of Needs to build our own pyramid that illustrates how 360 storytelling can develop.

Here’s what our hierarchy of a 360 story might look like:

[fig 3] 360 Hierarchy of Story

Working from bottom to top:

  • World – The context of your story, the place, the rules, the tone.
  • Characters – The individuals that will deliver the needs of the story.
  • Thematic Questions – The central question, or questions you’re posing as a storyteller.
  • Motivations – What any of the characters want, at any given time. Motivations can change over time and often do.
  • Answers – Your audience participates in how your thematic question is answered. This level of the pyramid is the sum of the elements below it. This is the end result of what your viewers construct in their minds. They help craft the answer based on what they’ve seen.

Think of this pyramid as a landscape, and not so much as a timeline. The pyramid diagram is more about how your audience mentally and temporally navigates the elements (of the pyramid) they observe. It’s in those observable elements where the audience builds the “story” in their minds. The elements can bounce back and forth within the imagination of the viewers and it is here where they climb the levels of the pyramid and find the meaning of your story (or get closer to it).

A linear representation of the concept 360 hierarchy on a three-shot level:


[fig 4] 360 Hierarchy Example Diagram

If you compare this diagram with the flat diagram [fig 1] you’ll see that a flat medium delivers its story on the basis of plot generally through Act movements, turning points, obstacles and then a resolution. In 360 those plot details can be missed with a turn of a head. The 360 medium can support and even deliver its story through thematic concepts, and most importantly in the subtle repetition of the diagramed “layers”. Your audience will navigate those layers at will. A turn of the head is an opportunity to reveal something thematically meaningful to your story.

It is this thematic development that should be at the core of 360 storytelling. Of course theme is important in all types of storytelling, but deepening and understanding your theme for your 360 world will provide a guidepost to all the details your viewers experience in your story. In a linear, flat medium the viewer (all viewers) receive the same information in the same sequence. In a 360 medium, your viewers may miss details and or “reorder” (making their own order of) the images. It becomes important to provide variants for details needed to describe a scene, action or character.

DAVID LEAN’S LAWRENCE OF ARABIA / 360

I’m all about the examples and I think an important exercise for all 360 storytellers to try is to convert their favorite films to the 360 medium. This exercise allows the creator to take something that has already been thought out and presented, and further develop those concepts in a 360 world.

So therefore, how can you further deepen your thematic question to build a better 360 film, and how will you utilize the 360 canvas to express it?

Here is a simple example using the first 33 shots of David Lean’s Lawrence of Arabia as written by Robert Bolt and Michael Wilson. There are many ways to cover the first moments of this film, and this is a great way to reflect on how one can really deliver a good 360 story; both visually and conceptually. The writing and the shooting of the film’s starting sequence continues to be one of the great setups to a truly epic film. It’s also one that I felt could be staged beautifully in 360, obviously borrowing from director Lean and his cinematographer Freddie Young’s brilliant imagery (not to mention Anne V Coats’ editing).

For context, here’s the original telling from the film.

Lawrence of Arabia (first 33 shots in David Lean’s version)

A few notes on how I compiled these shots: I screen-grabbed the film from the DVD with what appears to be a 1990 remaster of the film. It even includes the four-minute musical overture at the start. For the descriptions I was aided by the original script, and used its wording as much as I could (some of which was different than what was actually shot).


SHOT 00:
Musical overture, 4 minutes and 23 seconds.


SHOT 01:
Title and credit sequence as a MOTOR-CYCLIST readies his motor bicycle.



SHOT 02:
We see the MOTOR-CYCLIST kick-start his motor bicycle and ride out of the farmyard (CU to a tilt and pan to a WIDE).


SHOT 03:
MED of MOTOR-CYCLIST (camera attached to bike) as he rides down the country road.


SHOT 04:
MOTOR-CYCLIST’S POV as he passses some roadwork (on his side of the road).


SHOT 05:
Return back to MED MOTOR-CYCLIST.


SHOT 06:
Back to the POV.


SHOT 07:
After passing the road work, the MOTOR-CYCLIST picks up speed.


SHOT 08:


SHOT 09:
The MOTOR-CYCLIST passes the last house in the village.


SHOT 10:
Now just outside the village, the MOTOR-CYCLIST picks up speed.


SHOT 11
The POV shot builds on the speed.


SHOT 12
Even faster.


SHOT 13
MOTOR-CYCLIST POV.


SHOT 14
Yet faster.


SHOT 15
The open road, and the POV of the speed. Two bicycles are on the wrong side of the road.


SHOT 16
The MOTOR-CYCLIST reacts.


SHOT 17
POV of narrowingly missing the two bicyclists in the road.


SHOT 18
A quick shot as the MOTOR-CYCLIST struggles to keep control.


SHOT 19
POV as we swing from right to left and vear off the road.


SHOT 20
The MOTOR-CYCLIST as he goes off the side of the road.


SHOT 21
Through the bushes and into a field.


SHOT 22
The motor-bicycle jumps a mound.


SHOT 23
ECU as the MOTOR-CYCLIST flies off the motor-bicycle.


SHOT 24
The motor-bicycle, without its rider crashes into the brush.


SHOT 25
The MOTOR-CYCLIST’S goggles dangle from a branch.


SHOT 26
We cut to the bust of T.E. LAWRENCE.


SHOT 27
BRIGHTON responds to cleric, “He was a remarkable chap. By any counts, remarkable.”


SHOT 28
We now see that this memorial is taking place at St. Paul’s Cathedral.

SHOT 29
On the steps of St. Paul’s, a reporter asks ALLENBY about COLONEL LAWRENCE. ALLENBY makes a deliberately formal statement. When the reporter asks for a more personal statement about LAWRENCE, ALLENBY is regretful, and responds “I didn’t know him well you know.” The reporter then darts to another person.


SHOT 30
The reporter finds BENTLEY, “It was my privilege to know him and to make him known to the world: he was a scholar, a poet, and a mighty warrior.” Satisfied, the reporter moves on. BENTLEY then quietly says to his colleague, “He was also the most shameless exhibitionist since Barnum and Bailey.”

A MILITARY GENTLEMAN, overhears BENTLEY’S remarks and chastises him. BENTLEY asks, “Did you know him?” The MILITARY GENTLEMAN responds in the negative, but adds, “I had the honour once to shake his hand in Damascus,”


SHOT 31
We cut to MURRAY, on the same topic of knowing Lawrence he answers, “No I never knew him. He had some minor function on my Staff in Cairo.” CUT TO:


SHOT 32
Close up of a man painting a map.

SHOT 33
This shot reveals that the man is T.E. Lawrence.

The introduction into this epic film is spectacular even today. It’s a great set-up to the mysterious nature of T.E. Lawrence as it takes its time in the beginning (which often works well in the 360 medium) and prepares the audience for what’s to come. The film addresses Lawrence’s death and controversial legacy, but more importantly, it underscores the film’s conclusion.

Lawrence in 360

For this version I used the released film as a guide to how I could further interpret it in a 360 landscape. I also used the shooting script as a way to peek into what might have been some details that could enhance this version.

I think it’s also important to note that much like the Lean film, this version keeps Lawrence obscure at the start. We only see his unobscured face when the film cuts to the memorial bust. It’s clear that this is the intent when in the script he’s not named directly, but only referred to as, as “A MAN” and “MOTOR-CYCLIST”.

SHOT 01:
The camera is just below eye level, inside a garage. On one side of the 360 shot, we see a MOTOR-CYCLIST in his garage doing things, on the other we see his motor-bicycle in the driveway. The MOTOR-CYCLIST walks back and forth preparing his motor-bicycle, and mostly remains in silhouette and unrecognizable. In the garage he puts on his goggles, walks out, starts up his bike and rides off and out of sight.

The elements of the shot are mostly world and character. We don’t see much of the MOTOR-CYCLIST in detail, but his actions are pretty obvious.

  • If the audience only pays attention to the motor-bicycle, they understand he’s preparing for a ride.
  • If the viewers just look at the interior of the garage they’ll see the MOTOR-CYCLIST in his motor-bicycle gear putting on his goggles. The reason to put on the goggles here and not by the motor-bicycle is to allow the audience get the clue that the MOTOR-CYCLIST is going on a ride.
  • The sounds of the motor-bicycle starting and driving off, to clarify his exit to the audience if they missed the visual.
  • If the viewers follow Lawrence as he busily works, they’ll see most of the relevant details.

SHOT 02:
The MOTOR-CYCLIST on his motor-bicycle. The camera extends out from the front handlebars. As in the Lean version he starts out slow, but then picks up speed after passing the road work. The roadway is in the front view, The MOTOR-CYCLIST is in the rear view. After several seconds of riding, he swerves to avoid the bicyclists, we follow every move as the motor-bicycle runs off the road.

  • If the viewers only look at the road ahead, it’ll be obvious that they’re on a motorcycle. They will also see the bicyclists in the road that surprise the MOTOR-CYCLIST.
  • If the viewers turn to see the MOTOR-CYCLIST’S face, they’ll see his goggles, and then his startled reaction as he tries to avoid the bicyclists. The bicyclists appear behind Lawrence as he swerves past them.
  • The tires screeching on the road as well as the chaotic visual of running off the road will make it clear to the viewer that he crashes no matter what direction he or she might be looking.
  • Even if we miss the reason why Lawrence crashed, it is the banality that underscores the end of an amazing figure. If you know his story, it’s a reminder; if you do not, that will be revealed as the film progresses—-the significance may be different but the end result is the same.

SHOT 03:
Camera is situated off to the side of the road. The MOTOR-CYCLIST’S riding goggles hang from a branch, and smoke can be seen in the distance on the other side of the bushes.

  • If the viewer looks to the goggles, this obviously symbolizes the dire situation, behind the goggles we can see the road and the bicyclists rushing toward the accident.
  • If the viewers look the other way, they will only see the smoke, and the wheel of the broken motor-bicycle. Both images suggest that the rider has died.

SHOT 04:
A bust of T.E. LAWRENCE sits on a pedestal before BRIGHTON and a CLERIC. BRIGHTON responds to the CLERIC, “He was a remarkable chap. By any counts, remarkable.” The camera sits between the bust and BRIGHTON as well as the CLERIC.

  • On one side the viewer can see the Bust, and the surrounding halls of St. Paul’s cathedral. On the other side we see BRIGHTON and the CLERIC.
  • The conversation can be heard no matter where you turn.
  • The dialogue defines the theme; the imagery shows legacy. The audience shouldn’t miss the dialogue. The thematic question: Who was Lawrence, and despite his legacy, did we really know him?

SHOT 05:
This next segment is a complex scene, so I made an animated diagram to help describe the action. You can find it at the end of this shot description. The camera sits at about the actor’s eye level on the midpoint landing of St. Paul’s grand stairway.

The key points about this shot are:

  • The first conversations start looking down the stairs and then work around to interactions on the landing. The dialogue can be heard no matter what direction you’re looking, so no matter where you look, you can listen to what’s being said.
  • This is a complex blocking setup as it covers three different conversation areas. The original accomplishes this with three camera setups. I designed a simple single setup that can cover all of it. The scene has a structural arc in the dialogue and it might serve the emotional tone to stretch out the mood and draw out another camera setup.
  • Regardless of what you’re looking at, you’ll hear the dialog. And even though this is the first time we see these characters, it’s not detrimental to the story if the viewer misses them. The dialogue highlights the varied impressions of the iconic T.E. Lawrence. This scene is about how Lawrence remains a mystery even to his closest allies.

First interaction – REPORTER and ALLENBY:

The congregation is leaving. On the steps of St. Paul’s, a reporter asks ALLENBY about COLONEL LAWRENCE. ALLENBY makes a deliberately formal statement. When the REPORTER asks for a more personal statement about LAWRENCE, ALLENBY regretfully responds “I didn’t know him well you know.” The REPORTER then darts to another person.

Second interaction – REPORTER and BENTLEY:

The REPORTER finds BENTLEY, “It was my privilege to know him and to make him known to the world: he was a scholar, a poet, and a mighty warrior.” Satisfied, the reporter moves on.

BENTLEY then quietly says to his COLLEAGUE, “He was also the most shameless exhibitionist since Barnum and Bailey.”

Third interaction – MILITARY GENTLEMAN and BENTLEY:

A MILITARY GENTLEMAN, overhears BENTLEY’S remarks and chastises him. BENTLEY asks, “Did you know him?” The MILITARY GENTLEMAN responds in the negative, but adds, “I had the honour once to shake his hand in Damascus,” (this interaction is even more significant when you see that the MILITARY GENTLEMAN has in fact unknowingly met LAWRENCE twice, the first time thinking he was an Arab and slapping him).

Fourth interaction – MURRAY:

MURRAY, on the same topic of knowing Lawrence, replies, “No I never knew him. He had some minor function on my Staff in Cairo.”

Here’s the diagram that shows the interactions above:

SHOT 06:
The camera is at mid-level to T.E. Lawrence as he puts the finishing touches on his map. The main view is of him and the map as he sits up to review his work—Lawrence’s full face is in view. The camera is above the massive map on the table and as the viewer’s gaze wanders, it remains in view. I would also extend the shot to include more of the scene: a conversation with a SERGEANT and a CORPORAL.

  • You can see the map at any angle, and it represents the world the audience will see as the story progresses.
  • Thematically, we introduce Lawrence’s boredom with being holed up in the “dark room”. The CORPORAL comes in carrying a newspaper with news of an attack on a Turkish stronghold by a Bedouin tribe. Lawrence expresses his dismay over the British upper ranks’ lack of understanding about these events and the future of Arabia.

The scene continues with this shot…

The original film’s 33 shots have effectively been turned into six 360 shots. The decreased number of shots is not as much about economy as it is about the viewer’s experience and ability to take in a minimum of story themes. No matter where you turn, you’ll find some detail to the thematic progression of the story. You might miss something, but you will likely catch one or more of the important details.

So how does this example fulfill the layers in the story pyramid? Working from bottom to the top again:

WORLD

In this example, the details of the world are parcelled out as we lead up to the events in Arabia. If we forgive the notion that this is a historical event, and that the audience might already know the facts in the story, we get a sense of time and place with details like:

  • The style of motorcycle
  • The character context and setting
  • The mention of Damascus – and the transition to it
    The dialogue in “The revolt in the desert”
  • The different dialogues surrounding his memorial that illustrate the complex view of Lawrence
  • The bold music that sets the tone of the adventure and scale
  • The map Lawrence is working on that suggests the scale of our/his story (his position and relation to it, as well as the camera’s position)
CHARACTERS

Despite the unceremonious incident that killed T.E. Lawrence we expect that the story will open up about his life and the events in the desert. It is, after all, called, “Lawrence of Arabia.” Several characters are introduced at the memorial and we expect that they, and their relationship with Lawrence, will be explored in the narrative. The reveal of Lawrence’s face in statue form underscores the mystery.

THEMATIC QUESTIONS

Opening with the death of T.E. Lawrence is significant as it frames the context of the theme: the measure of a man, his deeds, and his place in society (being an illegitimate son of a Knight). The perspective of how the story is told and written explores Lawrence emotionally and, for the most part, stays away from making up its own details for some of the unknown events in the war or as they are described in Lawrence’s own writings. This knowledge about T.E. Lawrence is unnecessary as this introduction sets up the many opinions about him. The introduction of the characters at the memorial hints at the theme of the story; the many sides of the mysterious and controversial hero. More importantly, despite the honor, our protagonist is still an outsider.

MOTIVATIONS

The first time we actually see an unobstructed view of our main character’s face is in the form of his memorial. It’s a terrific way to reveal this iconic man – he’s a mystery and we get to hear what others think of him. Dividing up the 360 stage with various character dyads further builds on the discordant opinion about Lawrence, and his motivations / needs / challenges. It also opens up a question to what may be the (current/past) motivations of the supporting characters as well.

For Lawrence’s motivation we see/hear his knowledge of the what is to become the Arab Revolt, and what he feels is his superiors’ ignorance of its significance. His motivation at this point is to change their minds about how to approach the Arab nation – we will soon learn that his superiors have very different goals that leverage Lawrence’s unexpected success on the battlefield.

ANSWERS

The original film and script does an amazing job of showing the many and often conflicting sides to T.E. Lawrence and his heroism, however misplaced. It shows how more divisive forces leverage the ideals of a young man in search of his identity. It’s a man in search of place, after a life of being saddled with the stigma of being a bastard. By the end of a successful war, fostered by his leadership, there isn’t a homecoming parade for Lawrence. He’s still in limbo, despite having garnered respect in his home country as well as his adopted country of Arabia.

History is made by the youth, but written by the elders. As the story progresses we learn that Lawrence by his pure will becomes a means to an end. In the end his many hard won battles leave him unsatisfied and without the meaning he was searching for.

One final look:

CONCLUSION

Admittedly, this article is only a cursory exploration into the evolution of this relatively new medium. There are so many areas where a deep dive will be necessary, and this only serves as a way to deconstruct how we approach plot in the 360 medium. There are aspects of shot design, story structure, viewer experience, pacing (most literally everything) that must be explored—-every artistic, expressive medium goes through this.

In conclusion, what I hope this article helps reveal is that we can’t just translate linear plot-driven storylines into a 360 medium—-360 stories are naturally more thematic and thus they require an attention to detail. I propose that with 360 storytelling the creator must consider, and in turn, deliver theme on many levels to build what will become the audience’s impression of your story. They will assemble the details in their own order and compile its meaning with those details they’ve experienced. The key for the creator is to fully develop those details that best describe the world that will test and define your thematic question.

So, if you, as a creator, want to control every temporal detail, this likely isn’t the medium for you. Traditional sequential story mediums and their ever-developed codas provide a medium for distinctly defined images. You can be supremely precise in your creation and be relatively sure that everyone in your audience will experience the same visual storytelling.

But, if you are excited about the notion that you can build a world and a web of experiences that can explore a thematic story-view, this might very well be the perfect medium for you. 360 storytelling is certainly challenging as you can’t rely on the traditional cheats that other sequential mediums offer. Your task in 360 isn’t to direct the viewer, it’s to provide a world in which the viewers can utilize your meaning to build their unique experience.

The whole is greater than the sum of its parts.

Comments (2)

  1. Ben

    Great article about an emerging field Justin. I’m dipping my toe in soon. This answered a lot of questions about writing and directing for VR

    • Justin Chin

      Glad it helped, Ben. Can’t wait to see what you’re cooking up!

Comments are closed.