Yale-led effort yields Zoom upgrades 

 

Even as a careful return to some in-person learning and teaching is under way, much of what music students will do at the beginning of the fall semester will happen on Zoom. This is the way it’s been since the pandemic arrived and sent everyone online. And while we’ve been able to see and hear one another, music instruction and study have been hampered, even with the digital tools of the day. Until now. Last week, Zoom released a new “high fidelity music mode,” which enables users to turn off some features that hinder music-making while taking advantage of a sampling rate that’s much higher than it was previously. The new functionality was created at the request of more than two-dozen music schools in a Yale-led effort.

When instruction and study moved online in March, faculty pianist Boris Berman and his colleagues were immediately “concerned about this form of teaching in general.” There are certain things that can’t be done online because of latency (a problem that’s currently without a solution), Berman pointed out, not to mention the loss of human interaction. Above all, he said, “the quality of sound itself was a huge concern.” 

Most music teachers, YSM Associate Dean Michael Yaffe said, “were completely unprepared.” Music education, he said, was “a profession in crisis.” 

“How are we going to do this from a technology standpoint?” Matthew LeFevre, YSM’s media-production manager, asked. “There wasn’t a quality way of actually teaching,” he said. Students were recording and sending videos to teachers. “That’s how lessons were working at the beginning,” LeFevre said—the beginning a reference to the arrival of COVID-19.

LeFevre, Berman, and others at YSM began troubleshooting, experimenting with a combination of Zoom (for video) and Cleanfeed (a recording platform, for audio), seeking a sound that’s “closer to reality,” Berman said. While these coupled technologies did provide a “higher veracity of the sound,” Berman said, using a combination of platforms was a little bit “cumbersome.” 

Yaffe had started a conversation with John Barden, Yale University’s associate vice president and chief information officer, about the predicament in which music instruction—provided privately, at community music schools, in pre-K-through-12 classrooms, and in college and university music programs—had suddenly found itself. “Music schools have a unique problem,” Barden agreed. 

Yaffe also reached out to Scott Metcalfe, whom he’d known since they both worked at the Hartt School at the University of Hartford. Metcalfe, chair of the music engineering and technology/recording-arts program at the Peabody Institute of the Johns Hopkins University, had been conducting audio-quality tests with students there and noticed that Zoom’s stereo function wasn’t reliable and that the sound quality in general wasn’t adequate. “Everyone kind of had the same complaint,” Metcalfe said: We can’t use this audio. Just as LeFevre and his colleagues at YSM had tried using Cleanfeed in conjunction with Zoom, Metcalfe and his colleagues at Peabody had started troubleshooting using Listento, a product that allows musicians and producers to listen remotely to a recording session. The music community needed something more user-friendly.

“To have a one-stop shop for both audio and video adds a lot less friction to the process,” LeFevre pointed out. 

We need “as good a possibility as is available to us,” YSM faculty violinist Ani Kavafian said.

Zoom needed to upgrade its service.

Barden suggested leveraging the collective power of higher education. Ted Hanss, associate chief information officer for Yale’s medicine and health divisions, previously worked, on loan from the University of Michigan, for Internet2, a university-led consortium that provides connectivity for the U.S. research and development community. He tapped into that network for more support.

On June 26, Barden, having tapped his own connections, sent a letter to Tain Barzso, Zoom’s product lead for education, and Brendan Ittelson, the company’s chief technology officer, that read, in part: “Over the course of the last several weeks, leaders from a number of music education programs across the country have been in dialog about specific concerns regarding effective online music instruction. … There is a sincere hope by all engaged that we can maintain a singular-provider approach to these issues.” The letter, which was authored by Yaffe with more than two-dozen schools including Yale, Peabody, the Bienen School of Music at Northwestern University, the San Francisco Conservatory, the University of North Texas College of Music, and the University of Southern California’s Thornton School of Music, among others, pointed out that “this issue is of immense interest to anyone involved in music instruction and performance”—including “music programs at colleges and universities,” “K-12 music programs in schools,” and “private music programs.” The letter, which explained that “Zoom’s current audio functionalities are well-suited for human speech but are insufficient to capture the complex harmonic components of singing and instrumental music,” outlined specific technological asks. Barden attached a cover letter and put the request in the mail. 

“We were very careful to ask for reasonable specs,” LeFevre said. The ask, as Hanss explained, sought configuration changes to the existing Opus codec. 

“It wasn’t a huge lift for them to do what we were asking them to do,” Barden said. Barzso agreed, saying the request was “pretty straightforward.” 

“Our engineering team is very agile, very flexible,” Walter Anderson, Zoom’s senior product manager, said. Tweaking the product wasn’t a challenge. The advanced setting “already existed,” he said. It’s just that it only turned off partof the noise cancellation and partof the echo cancellation while continuing to apply sound compression. 

“We actually asked them to take awayfunctionality that was getting in the way of the music,” LeFevre said.

After a follow-up phone call with representatives from Yale, Peabody, the Juilliard School, the University of Rochester’s Eastman School of Music, and Northwestern University’s Bienen School of Music, Zoom’s engineering team reconfigured the codec and redesigned the user interface in the settings menu.  

Explained in a blog post published on the Zoom website, the new functionality allows users “to disable echo cancellation and post-processing and get rid of compression” and increases the “audio codec quality from 22kHz to 48kHz, 96Kbps mono/192Kbps stereo for professional audio transmission in music education and performance applications.” 

“The biggest change,” Anderson said, “was to run the Opus codec at a higher bit rate.”

“We got pretty close to what we were after,” LeFevre said. “The audio quality took a huge jump forward. They got rid of the filters that were negatively impacting live music.”

LeFevre invited Berman to beta-test the enhanced functionality. 

“I found it substantially better than what Zoom used to offer up until now,” Berman said. 

 Metcalfe reached out to a skeptical colleague in the Annapolis area who tested the beta product with a saxophone student in Baltimore. “He went in very skeptical,” Metcalfe said. “By the end of it, he was sold.” Others who did beta testing in and around Peabody “were impressed,” he said. For users who may not be as technologically savvy as he and LeFevre are, using the new functionality is a matter of “clicking a couple of preference settings.” 

The enhanced functionality “is, of course, a positive step in the right direction,” Berman said, a useful tool for teachers and students who are adapting to the moment while looking forward to a return to in-person music-making.