Playhouses may be dark, but the Royal Shakespeare Company is to bring immersive theatre to audiences wherever they are in the world by combining revolutionary virtual reality and video games technologies with traditional live performances.

Real-time animation will be created by actors interacting with a live audience, who will in turn influence a story inspired by A Midsummer Nights Dream.

Mischievous Puck and his fellow fairies will be conjured up by actors whose every movement will be translated by cutting-edge technology into virtual avatars that mirror them as extraordinary creatures.

A fantasy world inspired by Shakespeares imagination will be recreated on an audiences computers and mobile phones in a way that allows them to interact directly with the narrative and music. They could, for example, become an iridescent firefly, lighting up a virtual enchanted forest along with thousands of other fireflies and guiding Puck along the way. Or they could trigger layers of instrumental sound, adding to a symphonic score performed by the London-based Philharmonia Orchestra.

In a pioneering project, to be announced tomorrow, specialists from the worlds of theatre, music and gaming have shared their expertise, developing a prototype that could be adopted for future live performances.

In a groundbreaking RSC production of The Tempest in 2016, actor Mark Quartley wore sensors that captured his movements, rendering Ariel, the sprite, as an animated ethereal character on stage.

It used real-time performance-capture technology that has developed dramatically since then. Now in a production entitled Dream, to be premiered next month motion capture and facial rigging will reproduce the movements of seven RSC performers for virtual avatars that are more detailed, including faces and hands, which could not be achieved for Ariel.

Unlike a regular live stream, this is interactive, showing how the excitement of a live performance can be recreated beyond the confines of a physical location.

Headed by the RSC, the projects key players include the Philharmonia Orchestra, Marshmallow Laser Feast, virtual reality creators, and Epic Games, developers of the Unreal Engine video games software.

The RSCs artistic director, Gregory Doran, spoke of his excitement at the projects potential.

He told the Observer: This is a 21st-century reimagining of Shakespeares play, which is giving us a completely different vocabulary of imagery. Thats extraordinary. When we did The Tempest in 2016, we explored the opportunities that the digital world could give to us.

I really did think then quoting the play this is a brave new world. Theres so much opportunity out there. Its as if technology is now providing us with this amazing paintbox. We have a lot of very exciting talent that can find uses for those new tools.

He added: What I love about this potential for an interactive experience is its not the play of A Midsummer Nights Dream, but its the world of A Midsummer Nights Dream. If that makes you want to read or watch this play, then thats great.

He recalled that in his youth he got hooked on the Greek classics by watching the extraordinary creatures created by Ray Harryhausen, who pioneered animation effects in films such as Jason and the Argonauts, the 1963 masterpiece.

He believes that the technology could inspire interest in the theatre and Shakespeare from young people who are already so familiar with the world of video games: This gives audiences of different ages a shared point of connection across the generations and, if that experience also has Shakespeares imagination behind it, thats got to be a good thing.

Luke Ritchie, the Philharmonias head of innovations, said: Weve never had a collaborative process like this. Its a game-changer. Youll be seeing real-time live animation which is being made live by live performers on a stage, creating virtual beings.

Sarah Ellis, the RSCs director of digital, said: We are expanding our theatre-making toolkit. This is a pioneering piece of stage production. It allows us to look at exciting possibilities.

She added: One of the things we were most proud of with The Tempest was the inter-generational moment where we had young and old audiences responding to it. Through gaming technology were now creating a piece of live performance, not a game and we hope it will once again bring together generations, either through the Shakespeare or the technology.

The production will include two new orchestral pieces by Esa-Pekka Salonen, the Philharmonias principal conductor, and by Swedish composer Jesper Nordin, creator of a groundbreaking interactive music software called Gestrument, which enables Dream to have an interactive musical soundtrack affected by the actors movements.

Dream is an Audience of the Future Demonstrator project, part of the governments Industrial Strategy Challenge Fund. It will be performed between 12 and 20 March. Audiences can either buy a 10 ticket, to interact directly, or view the performance passively for free. Tickets will be available via dream.online

More:

Is this an avatar I see before me? Audience takes to stage in virtual Shakespeare play - The Guardian