How to use AR and VR to increase visitor engagement and participation
1. Terminology
In this resource, the term ‘audience’ will be used to refer to the people who engage with your heritage organisation; whether they are visitors to your museum, a member of your poetry hub, or a campaign supporter for the protection of local parks. At Limina, we use ‘audience member’ or ‘participant’ rather than ‘user’ to refer to people experiencing immersive content, as this helps humanise our target audience and think about them as individuals actively engaging with the creative content we have to offer, rather than passively ‘using’ it.
In terms of the technology itself, framing this as immersive media that can be used as another creative tool for engagement, rather than reducing it down to the hardware can really help you unlock its creative potential and think innovatively about how you can apply these new technologies to your audience experience. It’s important to take a holistic approach when considering the use of emerging technology, starting with the audience and the creative concept before deciding on a suitable platform ― which may or may not be an immersive technology-based medium.
There is a lot to consider before diving into an immersive media project. Hopefully this resource will provide insights to help clarify some questions you might have around this area.
2. Overview of technology
We define Immersive Media ― otherwise known as XR, extended realities or sometimes the metaverse ― as human imagination and experience bottled and then shared with potentially hundreds of thousands of people.
XR (Extended Reality) is the umbrella term used to refer to a number of different immersive technologies.
AR (Augmented Reality) is when computer-generated content is overlaid onto the real world, usually experienced through a smartphone, tablet or AR head-mounted display (HMD). AR experiences rely on either a camera or GPS to be activated.
These are a few terms you might see used to categorise different types of AR experiences:
- Markerless ― digital objects can be overlaid onto any physical space, for example BLAM app
- Marker-based ― the digital world is anchored to and activated by real-world objects, for example AOL Augmented Art Gallery
- Location-based ― the digital world is anchored to a physical location. Markerless and Marker-based AR can also be location-based, for example Story of the Forest and Pokemon GO
VR (Virtual Reality) a term popularised by Jaron Lanier, virtual reality transports audiences to an entirely different world, separate to our real world surroundings. It is a simulative, computer-generated experience often experienced using a HMD, although dome content is sometimes included in this category.
VR experiences range from linear, 360/180° films, such as these VR experiences by The Met and NASA, to fully interactive, room-scale experiences built in Real Time Games Engines (RTGEs) like Unity and Unreal Engine, such as Modigliani VR: The Ochre Atelier.
In simple terms, VR replaces what is seen, while AR adds to the reality being experienced.
In simple terms, VR replaces what is seen, while AR adds to the reality being experienced.
MR (Mixed Reality) is often used to refer to a blend of AR and VR, real and virtual, whereby computer-generated content is augmented onto real world surroundings so that it interacts and responds to the real world in a more immersive way. Interactive holograms could be classified as MR. For example, a 3D scan of a vase projected as a display item in a museum that visitors can interact with by pressing buttons that turn it and bring up text boxes providing additional information about specific parts of the vase.
Immersive Media terminology can get rather confusing, so finding a common language to use when talking about these kinds of experiences can be really useful in helping break down barriers to the technology.
It is also important to note that there are other kinds of interactive and immersive experiences that take advantage of technology in creative ways, but aren’t necessarily classified as immersive media or VR/AR experiences. For example:
- A Knight’s Peril ― a marker-based ‘choose-your-own-adventure’ audio journey by Splash & Ripple based around Bodiam Castle using RFID (radio frequency identification) and Raspberry Pi technology.
- The Lost Palace ― a GPS (Global Positioning System)-based experience around Whitehall by Historic Royal Palaces which incorporated bespoke handheld devices, binaural 3D sound and haptic technology.
- EnchanTales ― an interactive installation exhibited in Devon libraries as part of Libraries Unlimted’s Evolve programme, which combined projection mapping software and conductive ink technology.
3. Using immersive technology to engage with heritage audiences
Over the past decade, more and more heritage organisations have been taking advantage of immersive media to find new ways to engage their visitors.
There are a number of reasons why heritage organisations might consider introducing this kind of technology to their programme. According to the PEC’s discussion paper on Immersive Experiences in Museums, Galleries and Heritage Sites, ‘immersive encounters’ have the potential to:
(a) increase visibility and contribute to a culture of innovation
(b) appeal to new audiences
(c) allow for more meaningful participation
(d) facilitate better engagement
(e) provide additional revenue
So there are many possible benefits to using this kind of technology, but there is also a lot to consider logistically in order to make any immersive media project a reality.
Let’s take a closer look at the pros and cons of AR and VR, so you can begin to evaluate the suitability of their application within the context of your own organisation. Here are just a few ways in which these technologies could help to make content more engaging and accessible for your audiences:
Visual
Augmenting digital objects and characters into the physical space could help to engage people who find it hard to absorb information from chunks of text and/or visualise what something might have looked like in the past. For example, Tamworth Castle’s AR Explorer had a ‘Memory Keeper’ character as a virtual companion for audiences as they explored the castle by scanning AR Markers located in different rooms which revealed animated ghosts and objects from the collections.
Social
Immersive experiences can produce strong moments of social interaction and help bring people together, encouraging audiences to feel more connected not only to the content, but to the other people engaging with it as well. For example, the Tamworth Castle AR app included a selfie filter which allowed audiences to capture their experience and share it with friends.
Access
Immersive media can give audiences a unique opportunity to access objects/people/places that are impossible to experience in reality, such as buildings which have since been demolished, or people from the past. These kind of experiences can bring things to life in a much more visual, tangible way for audiences compared to other media. For example, Hold the World with David Attenborough, created by Factory 42 and commissioned by Sky VR Studios, allowed audiences to delve into the collections of the Natural History Museum.
Interactive
Gamification is a popular and effective technique to increase people’s ability to absorb information and engage more deeply with the content presented to them. This isn’t news to heritage organisations, who have been using activity trails and treasure hunts to engage families for a long time. Immersive media experiences that include tasks, challenges, and/or games offer a digital, more animated version of this and can enable people to engage with your organisation in a more interactive way.
Gruffalo Spotter
For example, families could collect digital characters they discovered from the Gruffalo around Forestry England sites using the Gruffalo Spotter App in 2017. The success of this project resulted in the development of the Gruffalo Spotter 2 app in 2021.
If there are multiple possible outcomes to the experience, this can also encourage audiences to return and repeat the experience to uncover what they might have missed.
Context
Immersive experiences can provide additional context and highlight areas of interest around a physical location. The benefit of using a digital format means that the location is left undisturbed by physical signs or information boards, and that it can be possible for audiences to experience the place remotely if they are unable to be there in person. Subsequently, this can improve the accessibility and audience reach of heritage sites and experiences.
Bridge Tales App
For example, the Bridge Tales App allows visitors to the Clifton Suspension Bridge in Bristol to learn more about the site and explore the bridge virtually from any location. This can be interesting for both tourists and locals who are already familiar with the bridge by providing contextual information they may not have previously been aware of.
Reconnection
Jonathan Amakawa and Jonathan Westin proposed in 2017 that AR can be used to connect the stories of disenfranchised or underrepresented groups with physical sites, and allow for a greater emphasis on intangible heritages that are often sidelined. Although it is important to note that this should not be considered as a substitute for longer-term programmes of change within institutions to re-balance and challenge existing representations and narratives. For example, The Nomad Project explored the creative use of immersive mixed reality and web-based technology to contextualise archival Somali objects with the people and traditions to which they belong.
Though there are many advantages to introducing this kind of technology to your heritage offer, they also come with a number of challenges. Careful planning will help your project to run smoothly and successfully, and that your audiences leave the experience feeling engaged and inspired rather than underwhelmed and frustrated.
In the next section, we’ll discuss the considerations that need to be made when planning this kind of project in order to overcome these challenges.
4. Planning an XR project
Feasibility
First off, you need to consider the feasibility of introducing emerging technology to your organisation. Look at what resources you have available to help you consider the breadth and scale of your project.
Budget
The cost to exhibit immersive content can vary massively and is entirely dependent upon the kind of experience you want to make. Before undertaking such a project, make sure you are fully aware of all the potential costs involved and that your budget covers it. There are many elements that make up an immersive media project. Here are just a few of the costs to consider:
- VR/AR Developer time
Discuss how many days they will need to spend on the project. Elements which can impact this are whether they will be using existing/developing new software and whether they’ve made similar content in the past. - Hardware
Does your experience require any hardware such as VR headsets, computers, projectors or tablets? Make sure to factor in back-up devices and have a maintenance plan in place; the more regularly the equipment is used, the more often you will have to fix and/or replace it. - Materials
Are there other physical elements to the exhibition that could accompany the digital elements? Think about the audience’s whole experience; what do they encounter leading up to and after the virtual content? The design of the real environment is just as important as the virtual when it comes to immersive media experiences; it can have a huge impact on the audience’s overall enjoyment of the work.
Tips:
- Partnering with university students to create your experience might offer a more affordable option
- 360 video projects often cost less to produce than VR experiences built in games engines like Unity. Click here for more information on the cost of 360 production. According to Volan Media, this is rough guide on how much VR production costs in 2021:
- Low End: £8 – 20K (monoscopic 360 film with limited post-production, visual effects, locations, etc)
- Mid Range: £20 – 30K (average VR project, both monoscopic and stereoscopic 360 film, a mix of post-production + visual effects, more potential for locations, actors + sets)
- High end: Over £30K (interactive VR Apps, require a lot of developer, production + planning time)
Staff time
How much staff time will the experience demand? This includes their involvement in:
- The design process
- User testing phase
- Training sessions ― this includes trying the experience themselves, setting up the experience, briefing audiences, fixing technical issues, etc.
- Supporting audiences’ to engage with the experience ― this needs to be well thought-through, as attempting to use the technology without sufficient support can be leave audiences with a highly frustrating and negative experience
Note: VR headsets should not be left unattended and must be sanitised and reset in between every audience member.
Space
Whether you plan to make your immersive content available online or at a physical location, you must make sure to choose the right kind of experience for the space you have available.
Longevity
How long will the experience be available to the public for? If your experience requires hardware like tablets or VR headsets, this equipment will need regular maintenance, and the longer it’s used for, the more likely you will need to repair and/or replace it. That is why larger, location-based exhibitions involving lots of equipment are often designed to be temporary. Whereas augmented experiences like holograms and AR apps accessed via the audience’s own phone require less maintenance and are thus more suited to long-term engagement.
The speed at which immersive technology is advancing is another factor that can impact the longevity of your experience. It is important to note that that platform you design for and the hardware you invest in might be made obsolete within a couple of years, so reducing the time it is made available can help to mitigate this risk.
Case study
Here is a timeline demonstrating the speed at which VR technology has advanced and how Limina had to adapt to this:
2017― Limina hired VR-ready computers and Oculus Rifts to run VR events. Equipment was large and cumbersome, and headsets had to be tethered to computers, along with having external sensors to detect audience movement.
2018 ― Limina started using Oculus Go headsets for pop-up VR events. This was Oculus’ first standalone headset (no wires/computers needed to run); this was a revelation for VR exhibitors. The headsets were affordable, compact and didn’t require wires / computers to run, meaning transporting equipment to events was easier and audiences were able to experience content without the worry of getting tangled in wires!
2019 ― Limina invested in a fleet of Oculus Go headsets to use at our physical venue which ran for 6 months on Bristol’s harbourside, as well as for pop-up VR events after this time. Soon after investing in the hardware, Oculus released their brand new headset, the Quest.
2020 ― Oculus discontinued the Go, meaning we could no longer replace faulty headsets and creators were no longer producing content designing for that specific headset.
2021 ― Limina have now invested in a set of Oculus Quest 2 headsets and are focussed on research and consultancy; sharing our knowledge to support others and help organisations to identify and seize the opportunities that immersive tech brings.
5. Research
Audience feedback
It’s really important that the experience is user-friendly and engaging for people who aren’t already familiar or confident with technology. This is why it is a good idea to carry out audience research before deciding on a concept. Ask them questions, send out questionnaires. This step will provide the foundation for your ideas, and help to identify whether introducing technology in this way is the right fit for your organisation and what this technology might bring to your audience experience.
Once you’ve developed a prototype, carry out user testing sessions with different groups to ensure the experience is accessible and engaging for your intended audience. Ask them about the content and the format, i.e. whether they enjoyed both the content and the way in which they experienced it.
Continuing to gather audience feedback once the experience is made available to the public will ensure it continues to run smoothly, help assess the impact of introducing this new technology to your organisation, and influence ideas for future immersive media projects.
Try it out
It is important to experience as much immersive and interactive content as you can before taking on your own project. Get to know the possibilities of these new mediums, be critical and take inspiration from other experiences (past and present). Doing this will help you design an innovative experience that really takes advantage of what the technology has to offer and provide a truly unique experience for your audience.
6. Design
Usability
Experiences need to be thought through and well-designed with the audience in mind. A clunky or buggy experience risks doing more harm than good to your audience’s experience. Consider how much training/guidance the audience will require to use the technology and how physically comfortable it is to experience. The more intuitive and comfortable you can make the experience, the more accessible and engaging it will be for a wider range of audiences.
Visibility
If the audience experiences the content using equipment provided by your organisation, think about the way in which the technology is presented and how visible you want the hardware to be. Rather than having a plain tablet or VR headset, explore how it could be encased in something related to the subject matter. For example, Marshmallow Laser Feast’s ‘In the Eyes of the Animal’ was exhibited using forest-themed headset covers.
Interactivity
Think about how the audience interacts with the content and to what extent is it interactive. For example:
- a hologram of an object allows audiences to walk around and view it from different angles
- an AR experience augmenting an object onto the physical space could be manipulated by the person using the device
- a VR representation of an object might allow audiences to ‘pick up’ an object, move it around a virtual environment and see how it interacts with it’s virtual environment.
Format
Identify the most suitable platform and format for your experience. There are a number of reasons why you might choose one technology over the other. For example, as AR overlays content onto a real-world setting, this medium can be effective in enhancing existing objects, exhibits and/or places. Comparatively, as VR completely replaces a person’s reality, it is often presented as a standalone experience or add-on that is designed to work alongside real-world objects, exhibits and/or places. For example, the V&A’s Curious Alice: the VR experience was produced as part of the ‘Alice: Curious and Curiouser’ exhibition, which showcased numerous art works and costumes inspired by the story of Alice in Wonderland.
AR experiences that rely solely on the audience having a smartphone can offer a more viable project for your organisation compared to other technologies like VR, as this kind of experience doesn’t require investing in hardware that needs to be maintained and staffed. However, it does bar access to people without smartphones, so having a selection of dedicated tablets/phones to lend can be a good way to overcome this issue.
If you have the budget, you may consider introducing a number of different kinds of emerging technology to create a multi-layered experience. For example, Nottingham Castle’s Robin Hood Exhibition included a range of different interactive experiences designed to immerse audiences in the world of Robin Hood.
7. Further reading
- Read Limina Immersive’s report on Immersive Content Formats for Future Audiences produced for Digital Catapult in 2018. In this, you can find a wide range of exemplary immersive media content, covering VR, AR and immersive audio experiences.
Browse related resources by smart tags:
AR Augmented Reality Immersive Immersive media Mixed reality MR Virtual reality VR XR
Except where noted and excluding company and organisation logos this work is shared under a Creative Commons Attribution 4.0 (CC BY 4.0) Licence
Please attribute as: "How to use AR and VR to increase visitor engagement and participation (2022) by Emma Hughes supported by The National Lottery Heritage Fund, licensed under CC BY 4.0
