Robert Pratten 
This article is also available for DOWNLOAD as a PDF or via Slideshare
An Age of Fluid Reality
Stories are the way we make sense of the world. Our minds can’t deal with randomness and we see connections, causes and reasons even where there are none. Whether we are happy or sad, positive or negative, this is often the result of the story we construct – it’s the meaning we attribute to events and things that without human interpretation have no meaning. So powerful is story that the life we lead today is a result of the stories we told ourselves in the past.
We are entering an age of fluid reality.
Today, our mind can be invited to switch between many parallel realities and anyone can choose to live in whichever reality they feel best suited to. This has always been true to a certain extent but now the imagination is not having to work as hard as it once did – and this allows many more people much greater scope to not only “fantasize” but actually feel like they are living the life they always wanted to live.
As digital representations of the real world become abundant and our view of the “real world” becomes ever more mediated through technology, so our faith in the accuracy of the digital world seems to become greater than our faith in the things it claims to represent. This is as troubling as it is enabling. The blending of the real world and virtual representations of it present opportunities to rewrite belief systems and exert influence on an unprecedented scale. We must be sure to use this new power wisely and educate people in its use and misuse.
This year we learned the phrase “hybrid warfare” – a mix of traditional rockets and guns camouflaged with denials and grassroots political activism. Propaganda and miss-information have always been vital weapons of war and suppression but today there’s an over trust of digital information such that social media will cause us to doubt what we see with our own eyes.
This is a battle of realities is not a battle of “reality on the ground” but a battle of “realities of the mind”. Reality lives in the mind; it lives in the imagination.
What’s Shaping Reality?
The digitization and virtualization of the physical world is reshaping our perception of reality. This takes several forms:
- digital mapping and digitally created environments as exemplified by Virtual Reality (VR)
- digitally created objects and characters overlaid on the real world as exemplified by Augmented Reality (AR)
- virtualization of physical things to create process-oriented models of how the real world works facilitated by the Internet of Things (IoT)
- virtualization of people, groups and communities in the form of social media, avatars and online personas.
Virtual Reality (VR) and Augmented Reality (AR) are well known to many but less widely known are the opportunities afforded by virtualization: Alternative Reality (ALR) and Mixed Reality (MR).
Virtual Reality in the form we commonly understand it as an isolating experience that requires a headset with stereoscopic vision (Oculus, GearVR, Google Cardboard) – we block out the real world and replace it with a live action movie or computer-generated environment.
VR is this generation’s ViewMaster. It’s a clever trick and part of the wonder and thrill of wearing the headset is our marvel at how easily our mind can be fooled through our eyes. Looking around and up and down and seeing a virtual world around us is fascinating but for now that’s usually as much agency as the wearer is afforded. Choices and movement through the virtual world in VR is tricky and uncommon at the moment but in time it will be solved and be more common.
VR headsets and 360 degree movies are generating lots of excitement and it’s leading to new discussions about audience agency. Now that the audience (albeit an audience of one in these typically solo player experiences) can look around the virtual space, how do filmmakers tell their story? Now that the frame can’t be controlled as precisely as it can with movies & TV, what’s the new language of VR cinema?
We have had interactive movies for several decades and they never really took off because usually they’re too much work – too disrupted for a satisfying laid-back experience and not enough interaction for a gaming experience. The best examples are those designed with a gaming mindset rather than a filmmaking mindset.
Quite how VR will translate into feature length experiences and multiplayer/multiviewer experiences is yet to play out but this new found alternative reality may open the eyes of some to the possibility that virtual worlds can be all around us too – and not confined to those moments when we wear a headset but alive in real physical spaces.
Augmented reality like VR is again a trick played with our vision. Here though, we view the real world and see images and video overlaid upon it (HoloLens, Meta One, Google Glass). AR is a broader umbrella than VR and doesn’t always require us to wear a headset or use our mobile but it does always require a camera to see the world and a screen to display the augmented world (camera’s view + image or video overlay).
AR is much less isolating and more social than VR but tends to have many of the limitations of choice and movement.
A less well known enhancement to reality is what could be known as Alternative Reality (ALR) – a popular if scary example of which can be seen in the movie, The Game (1997), in which Michael Douglas becomes the audience and player in a personalized experience for only him that blends his real life with a fantasy.
ALR heralds from the Alternate Reality Game (ARG) – a format popularized around the turn of this century with games like The Beast (for the movie AI) and the off-screen story woven around The Blair Witch Project. Here the defining characteristics of the ARG might be considered to be collaborative problem-solving, the wide-ranging agency afforded players and the use of real world artefacts (physical and digital).
ALR doesn’t trick the eyes: it works with the imagination. It uses our mind’s curiosity and its need to connect the dots. ALR is about human experience design in the broadest possible sense because the goal is to allow an active belief in a parallel world that’s interwoven with our everyday world. Nobody is fooled; rather they’re given permission to believe.
Imagine the personal and societal benefits of ALR: It has the potential to rewrite the stories we tell ourselves using the right cues at the right time to stimulate positive interpretations of the world. When a lightbulb blows and the depressive thinks “why is everything in my life going wrong?, a digital intervention could flash to the wristwatch “let’s have a candle-lit dinner? :)” or a phone call “Trisha let’s see the gas lamps on Westminster Bridge! 1807 is going to be a great year for us! I’m certain that we’ll soon all have gas light in our homes at the pull of a lever!”
Mixed Reality (MR) is a combination of all the above. Experiences can start in the real world, dip into a completely virtual world, go to an augmented reality and then finish with alternative reality before, hopefully, passing the participant back into their preferred reality.
The project “Meet Lucy” conceived and developed by Nina Simoes and written by David Varela is an example of MR. Here, participants meet Lucy and her family online, communicating via email, sms and blog posts (ALR) and then can virtually step into Lucy world’s (VR) via the Oculus headset and Unity (the software that generates the virtual world). Coordinating all this is Conducttr which even personalizes the VR experience by telling the Unity engine who and what to render based on the participants online interactions.
What Opportunities Do Alternative & Mixed Reality Offer?
Alternative Reality and Mixed Reality offer an opportunity to create experiences that are connected, personalized, participatory and social. Importantly the experiences fit around the audience and their habits and interests, not the other way around.
Although the possibility to operate commercial services that use AR and MR is revolutionary (and at this time only possible with Conducttr), the audience and consumer behaviours that underpin the opportunities are age old.
Using transmedia storytelling, new opportunities arise for connected, personalized, participatory and social experiences can that reinvent traditional industries like:
- Entertainment
- Advertising
- Education
- Healthcare
Reinventing Entertainment
The problem with the entertainment industry is that it’s still broadcasting and not listening.
The phenomenal rise in the popularity of gaming (including Twitch streaming and eSports); the outpouring of “user-generated content”; the rise of the six-figure-income “YouTuber”; the rise in popularity of escape room experiences; and the popularity of communities like TripAdvisor and Reddit… they all point to a new empowered, self-actualized audience that is mostly ignored, placated or sidestepped by broadcasters.
TV audiences are getting older and younger generations are not watching TV – this is a fact across all continents. This is correspondingly causing problems for advertisers who want to reach that younger generation but can’t do so as well as they once could on TV.
The opportunity for TV is to become the “cut scenes” inside a persistent experiential worlds. Instead of reluctantly adding a website to support a TV show, broadcasters should create massively participatory interactive experiences that add TV content. Just as mobile is now the “first screen”, so transmedia experiences will become the norm. Why? Because this firmly places the audience at the centre of the experience and allows greater scope for a spectrum of revenue opportunities – of which advertising may or may not be the primary source of financing.
Reinventing Advertising
The problem with today’s advertising technology is that it focuses on the needs of the advertiser, not the consumer.
For all the claimed innovation, most adtech is really just about shouting louder or taking the consumer’s watch and telling them the time. In fact the worst offenders are those technologies that suck user-generated content from the community, aggregate it and regurgitate it for the needs of the brand. It’s like arriving at a party and giving the host flowers stolen from their garden.
Similarly, tracking consumers across platforms by using cookies belongs in the 1990’s.
Hopefully nobody stalks their friends across the internet nor in real life and yet we know what to buy them as a birthday gift. This is because we have a relationship. Today’s advertising technology doesn’t seek a relationship, it seeks data and hopes that by analysing enough data an insight into someone’s preferences will be gained. Yet we know that many consumers have multiple online identities and the technology can be defeated or fooled. As education and society shift towards greater media and technology literacy and mistrust of surveillance, tracking with cookies is going to be consigned to the past where it belongs.
What alternative reality and mixed reality experiences offer is the opportunity to form real relationships. These are formed by giving back to the community and insights are formed through willing consumer participation.
Consumer participation in this context means creating brand-sponsored, empowering experiences that invite people to connect with each other and connect to the issues that matter to them. Some brands do this from time-to-time in the form of live events and such like but they tend to be campaign-based which means customers are allowed to get excited for a short period and then they get abandoned. Still, better to have periods of giving than never at all.
See this short demonstration, Connected Brand Experiences, for an example of how things might be done differently.
Reinventing Education
The problem with today’s education is that it was designed centuries ago for people who would go to work in a factory.
As manufacturing industries declined and service industries increased, the lesson topics changed but the process of education didn’t. So now we have schools that feel like prisons. Mobile phones, social media, games and responsive systems of all kinds are all around us in homes, workplaces, shopping malls… but not in schools where phones must be switched off and everyone must sit in silence and listen to teacher. So the students have one of the most powerful computers of all time in their pocket but they’re not shown how to use it wisely.
Today’s students are growing up in a world that is changing faster than at any time in the past and the jobs they’ll do when they leave may not have been invented yet. Many students know this and the uncertainty leads to insecurity and in the worse cases hopelessness and depression and suicide. We have a duty of care to make students more mentally resilient to uncertainty and give them skills they can take to any job: critical thinking, collaborative problem solving, negotiation skills, conflict resolution, mindfulness and so on.
Using mixed reality-based education we can blend formal and informal learning spaces to create an environment of engaging continuous learning. By putting the student at the centre and using transmedia storytelling we create dynamic, team-based, collaborative problem solving, participatory environments that encourage exploration, examination, reasoning, debate and negotiation – vital skills in the modern workplace and that of the future. The teacher too is liberated – no longer expected to be the holder of all knowledge who can never be wrong but a mediator who guides and encourages student exploration so that they find solutions for themselves.
Reinventing Healthcare
The problem with the current trend in digital healthcare is that it’s fixated on patient data and encouraging people to exercise.
Just like the pharmaceutical industry, everyone’s looking for a quick win in the shape of an easy pill to swallow: the premise is that more information will lead to better decisions. And maybe it does in some cases but it’s not the only answer. More information doesn’t address motivation or mood. And leader boards might motivate the competitive but coming last can reinforce low-self-esteem in others.
The stories we tell ourselves affect our mental state and our mind affects our emotional and physical wellbeing. Illnesses like addiction, depression, and obesity are best addressed through mindfulness techniques that rewire our perception of reality to write a new story and replace the corrosive stories.
More data won’t fix a broken heart or a miss-wired brain. Pervasive, mixed reality stories can. Immersive real world experiences that use real-time and behavioral data and simulation techniques – can send positive reinforcement, encouragement and inspire compassion, love and gratitude. The future of healthcare lies in a partnership between the storyteller and the patient in which imagination drives enthusiasm, understanding and better habits.
Conclusion
The ideas presented in this document are intended to inspire and encourage new avenues of inquiry in traditional industries that can be radically improved through the use of alternative and mix reality.
Our unique pervasive entertainment platform, Conducttr, offers a storytelling and gaming layer for the real world which, combined with real-time participatory transmedia storytelling, can enrich lives and better prepare us for it.
Anyone can now sell commercial experiential services using Conducttr as a middleware between their clients/students/patients and the real world.
Make everyone’s life an adventure.
Transmedia Podcast – Episode 1 – StoryNexus