Four Strings Around the [Virtual] World

The playful notes of “Tango Etude No. 3” by Argentinian composer Astor Piazzolla dance amid a lush backdrop of manicured hedgerows and crimson azaleas at the National Arboretum in Washington, D.C.

Irina Muresanu, an associate professor of violin in the University of Maryland School of Music, adds another burst of color, wearing a floral gown as vibrant as the piece she’s performing directly in front of you.

Yet neither you nor Muresanu is actually there.

What you’re seeing is a hologram of the artist, who—with sound flowing from her 1849 Giuseppe Rocca violin—has been digitally captured and transported to this and other locations that represent musical compositions from different cultures.

This new virtual reality experience is a collaboration between the university’s College of Arts and Humanities and the College of Computer, Mathematical, and Natural Sciences (CMNS). It builds on Muresanu’s “Four Strings Around the World” project, a studio album and series of live concerts that celebrate diverse musical cultures through the unifying voice of the solo violin.

Muresanu said this virtual adaptation of Four Strings offers a trove of possibilities in both musical education and performance.

“We could have only dreamt of something like this several years ago,” she said, giving the example of virtually performing in the exact location that inspired a musical masterpiece. “But that’s what we’re supposed to do as academics—take the dream one step closer toward reality.”

Making the dream happen, however, meant overcoming several technical and logistical challenges.

Television stations and movie productions have long used green screens to superimpose weather anchors in front of forecast maps, or place actors in elaborate settings that are impractical to film. But that technology is good only for two-dimensional viewing on a television or movie screen, or more recently, on a smartphone.

Full immersion, where viewers can experience a scene in 360 degrees as if they were there in person, requires a much higher level of technical proficiency, said Amitabh Varshney, a professor of computer science and dean of CMNS.

The innovation for the violin project came out of the Maryland Blended Reality Center (MBRC), where, among other efforts, researchers are developing new immersive technologies to capture the intricate hand movements of surgeons at the R Adams Cowley Shock Trauma Center in Baltimore. “We’re focused on accurately representing the highest levels of nuances and details, so that it can be used as a teaching tool,” Varshney said.

The MBRC team applied the same technology used to record a surgery to virtually render the rapid hand and bow arm movements of Muresanu’s violin playing. They also incorporated spatial audio, meaning 3D sound that “moves” in tandem as a listener turns his or her head or looks up and down.

After filming Muresanu on a soundstage performing three pieces from Four Strings, Varshney’s team hit the road to film the immersive background settings needed to complement the music.

The National Arboretum in peak bloom represents the joy of Piazzolla’s South American tango. Three majestic New York City cathedrals are the backdrop for Bach’s powerful “Chaconne” from the Partita in D minor.

For composer George Enescu’s “Airs in Romanian Folk Style,” Muresanu had a special request: Growing up in Bucharest, she had always wanted to perform in the Romanian Athenaeum, one of Europe’s grandest concert halls. When Varshney’s technical team was unable to find suitable 360-degree footage from inside the building, Muresanu gained access for a professional European crew to film it.

Now Muresanu and Varshney are seeking private support for the additional technical and staffing resources needed to film an entire virtual concert. They believe the technology will be useful for teaching: Violin students from anywhere in the world can analyze—multiple times at any speed from any angle—the motions of Muresanu’s hands and bow arm.

Virtual reality technology also helps democratize the performing arts, Varshney said. Attending a live concert performance can be expensive and inconvenient, if not impossible for, say, hospital patients, or those with low incomes. “We don’t want anyone to be deprived of these amazing gems of human performances that can lift people up in a very dramatic way,” he said.