Seeing with your ears: Novel acoustics project aims to improve virtual reality, explore ear shape effects on 3-D sound – Phys.Org

June 25, 2017 Screenshot of a visual rendering model of Notre Dame Cathedral, created based on plans, laser scan data and site visits. Credit: Ghost Orchestra Project/LIMSI

Paris' Cathedral of Notre Dame has a ghost orchestra that is always performing, thanks to a sophisticated, multidisciplinary acoustics research project that will be presented during Acoustics '17 Boston, the third joint meeting of the Acoustical Society of America and the European Acoustics Association being held June 25-29, in Boston, Massachusetts.

In the project, computer models use recordings from a live concert held at the cathedral and detailed room acoustic simulations to produce a novel type of audience experience: a virtual recreation of the live performance using spatial audio and virtual reality.

Researchers reproduced the recordings using computerized acoustical data and enhanced it with computer-generated virtual navigation3-D visualizations made with immersive architectural rendering that float the viewer through the complex acoustics of the acclaimed medieval gothic cathedral.

Combined, the multimodal sound and image footage of the ghost orchestra produce a spectral tour to the sounds of the 19th century opera "La Vierge"The Virginperformed live during the 2012-2013 concert season to celebrate the Cathedral's 850th anniversary.

Multimodal virtual reality integration is central to the project's significance, said Brian F.G. Katz, lead investigator and CNRS Research Director at the Institute Jean Le Rond d'Alembert, Pierre and Marie Curie University, in Paris.

"3D-audio is the hot topic today in virtual reality (VR) that is currently a very active subject in both academic and industrial research," Katz said. "With the commercialization of affordable VR systemsthe cheapest allowing for VR on smartphonesspatial audio is rapidly immerging from the laboratory."

The next stage in spatial audio is personalized audio rendering that involves being able to adjust the rendering to match one's individual head and ear details.

"The importance of multimodal interactions, how visual and auditory cues balance in spatial perception, is key to VR and the sense of immersion, of being 'in' the VR world," Katz explained.

He envisions many applications emerging from the investigation.

"For me, spatial audio is a domain on the boundary of physical acoustics, psycho-acoustics, perception and cognition, and digital signal processing."

His work focuses on using each of the fields to learn more about the othersfrom virtual audio scenes exploring how visually impaired people understand and remember architectural spaces, to improving understanding of spatial audition, to refining virtual reality rendering capabilities.

Created in the context of the French funded research projects FUI-BiLi, (Binaural Listening and ANR-ECHO), the acoustical foundation of the project is a 45 channel close-mic recording of the live concert made by the Conservatoire de Paris combined with a detailed geometrical room acoustics computational model that was created and calibrated based on in situ measurements of reverberation and clarity parameters.

Next, the team will apply the methodology to other complex multimodal environments such as theatre simulations.

"Aside from the acoustics in this project, I think we definitely learned a lot about computer graphics and VR content production, which has opened up a lot more dialogue with those communities that we intend to pursue," Katz said.

Explore further: Concert hall acoustics influence the emotional impact of music

More information: For more information about the project: groupeaa.limsi.fr/projets:ghostorch

Aalto University researchers have found that the emotional impact experienced by music listeners depends on the concert hall's acoustics.

Researchers at Aalto University in Finland have developed a method that allows accurate comparisons of concert hall acoustics. The leader of the research group, Associate professor Tapio Lokki, was presented with an International ...

Researchers at Aalborg University, in conjunction with Bang & Olufsen in Denmark, have developed a sound laboratory that can reproduce the acoustics of any environment from cars to concert halls. One goal is better design ...

From the sweeping sounds of strings to the full and hearty woodwinds, live orchestral performances have a way of enveloping listeners within musical compositions. But not all symphonies are created equalmuch of what determines ...

People who are visually impaired will often use a cane to feel out their surroundings. With training and practice, people can learn to use the pitch, loudness and timbre of echoes from the cane or other sounds to navigate ...

Aalto University researchers have found that music is perceived to have greater dynamic range in rectangular, shoebox shaped concert halls than in other types of halls.

Paris' Cathedral of Notre Dame has a ghost orchestra that is always performing, thanks to a sophisticated, multidisciplinary acoustics research project that will be presented during Acoustics '17 Boston, the third joint meeting ...

Microphones, from those in smartphones to hearing aids, are built specifically to hear the human voicehumans can't hear at levels higher than 20 kHz, and microphones max out at around 24 kHz, meaning that microphones only ...

Google said Friday it would stop scanning the contents of Gmail users' inboxes for ad targeting, moving to end a practice that has fueled privacy concerns since the free email service was launched.

Like driving a car despite a glowing check-engine light, large buildings often chug along without maintenance being performed on the building controls designed to keep them running smoothly.

Researchers at The University of Texas at Dallas are getting more out of the sweat they've put into their work on a wearable diagnostic tool that measures three diabetes-related compounds in microscopic amounts of perspiration.

Researchers at the College of Engineering at Carnegie Mellon University have developed a novel design approach for exoskeletons and prosthetic limbs that incorporates direct feedback from the human body. The findings were ...

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

More here:

Seeing with your ears: Novel acoustics project aims to improve virtual reality, explore ear shape effects on 3-D sound - Phys.Org