hack text

In which I don't bother to asssign a subtitle.

How can we use order and placement within a multimedia package or interactive story to enrich the narrative?

I’ve been reading through Scott McCloud’s Understanding Comics on and off these last few months and it strikes me just how important this book is to anyone dealing with multimedia, not just comic artists or aficionados. When you think about it, in many ways comics were multimedia before anyone else was. They took an unusual step in completely combining images and words to create unique narratives that spoke in both media with one voice.

In comics, you can’t take any media out and still have a complete story. A comic without the words or without the artwork is not really a comic and may not even be a story. Even without words, there is still a narrative drawing everything together. The same can’t be said for most implementations of multimedia now. Many people who create narratives online do so with separate media in separate places, or create text with audio or video that doesn’t add to the narrative or does so in a way unassociated with the space it is in.

All this is confused even more by the web, with hypertexts and social media it would be relatively easy to have someone destroy the intended sequence for a user.

The question is, in what way can we use sequence to imbue our linked multimedia with narrative, instead of sitting along side it.

Get rid of sidebars?

Sidebars have been a standard practice in print media for a long time and in many sites it seems that the practice has carried over. The sidebar holds related but not integrated content for a narrative. Sometimes it may hold a summery for bored readers. But this practice ignores the possibilities of the web.

• Content can be embedded inside of text with tooltips.
• Rolling over text can reveal hidden elements.
• Separate sequences of text and imagery can be integrated to enrich narrative. But go beyond slideshows, show the sequence, push interaction.

Match beats

There are often different multimedia sequences which share beats with other media related to the narrative. By beats, I mean places where moments in the narrative line up between two media, for example photo and sound. This is why Soundslides is so attractive, it allows you to tell richer stories by combining photos and audio then you would be able to do with those two media sitting separate on the same page.

‘Here sequence is ridiculously important, the sequence of audio clips and photos can be matched up to create a rich story. One that is easy to digest and quite attractive.

• Narrating video with voiceovers
• Put navigation elements next to related content (tagging).
• Integrate related social searches (like Twitter) inline with an article.

Move beyond the paragraph

With the advent of easy typography on the web, considering how your text looks within the body of a website is key. The paragraph itself is a form of punctuation. However, we need to think beyond just the two+ sentence paragraph and into methods that take advantage of all the ways we can manipulate type today. The form and accepted format of the paragraph is something that comes from how we write by hand and is further formalized by the technology behind the printing press. However, the popularity of something like Twitter shows that this is not always the way we think anymore.

Placement of paragraphs, sentences or words can have a unique effect as well, imagine spreading out elements of a story in a way that shows emotion or meaning. We could deform letters or display them one at a time. We can even use words as playing cards. Why should we limit our storytelling to paragraph format?

• Spread and sequence sentences or paragraphs. Many people work on screens that are far wider than a book page. Why not spread out you sentences on one line or consider other formats?
• Place standard text within non-standard places. You can use floating divs or JavaScript to write on top of pictures or even on top of videos. With JavaScript or perhaps HTML5, you’d be able to animate it. Gives a new meaning to ‘voice’ over, huh?
• What about a cloud format? Letting concepts float up and hover around?

Consider left to right, than break it

McCloud notes how we read from left to right and how that changes how we read, what blanks our mind fills in, and where our eyes go next. (I wonder how things change where the society’s primary language goes in the opposite direction.)

What does this mean for web design? Should we throw out the concept of right-side sidebars or the sidebar-content-sidebar format? I’m not sure. I’d love to see some statistics on that if you have them. But what it does mean is that we have an opportunity to direct navigation in an order, not just as the presentation of options. It also means that we can reward users at the right side of a page by presenting them with something a little unusual, a new kind of punctuation.

But if we understand that the eye works this way, we can play with it, exploit it. With subtle strokes we could force it along a path. We could even defy the standard format and give multiple options. Instead of a buffet of every single area on a website, what about a limited menu, or a path that implies a narrative through the design.

• Can website navigation tell a story?
• Create a set of paths through your story or website using links or anchors. What if the selection of one path results in the exclusion of some options, what sort of narrowing down can be done?
• Embed artifacts in your site. Things that enrich the narrative of a site through consumption.

What’s next?

There is a lot to consider here. This doesn’t even get into real interactive options and the idea of narrative web design is something that has been banging away in my head for a while and I feel needs some serious expanding in the future.

However, It is important to start thinking right now about how sequences of events, media or even color can change how an online narrative is consumed.