The Silicon Symphony

Professor Frederick Bianchi is at the leading edge of a controversial field of music he calls the Virtual Orchestra. While replacing live performers with computer-generated music has raised the ire of musicians and music reviewers, Bianchi says it will provide new opportunities for technically trained musicians and help create an exciting operatic art form in the United States.

By Ray Bert '93

Let's listen to just a single flute...

Seated before a Macintosh computer, Frederick Bianchi makes several quick keystrokes and selections with the mouse. From the speakers in his office comes the lively sound of a flute playing a familiar melody.

...now we'll add another...then a clarinet

A fuller sound now issues from the speakers as the cursor on the monitor dances across electronic "sheet music."

...now a bassoon...and a French horn

More and more voices join the texture and the sound level rises like the slow build of a stadium chant.

...finally, here's the entire orchestra.

With a flourish, he makes one last keystroke and the full, thumping, swelling, joyous majesty of Mozart's Overture to Figaro pours forth.

Bianchi plays conductor. He relays a tempo to the computer by manually tapping it on a keyboard. At first he accelerates the music to cartoon-chase speed, then drops it into a staggering torpor before returning the overture to its normal stately pace.

Flashing the wide smile of a man playing with a tool of his own creation, Bianchi remarks, half to himself, "There's just so much you can do with this."

Mr. Mozart, welcome to the world of the Virtual Orchestra.

---

Bianchi teaches two popular courses on electronic music composition in WPI's Computer Music Laboratory.

---

Frederick Bianchi has been an associate professor of music at WPI since 1994. He received his doctorate in music theory and composition in 1985 from Ball State University, where he also minored in computer science. From 1986 to 1994 he was on the faculty at the University of Cincinnati's Conservatory of Music. He taught music composition and orchestration classes along with courses in electronic music and music technology. As a composer, he has overseen dozens of major performances of his original works throughout the United States and Europe, including a world premiere at Carnegie Hall.

Bianchi is also part of a sound design team he formed in 1987 with sound designer David B. Smith. While they continue to do sound design work for opera and theatrical productions throughout the country, their most intriguing and controversial projects include the use of their full-scale Virtual Orchestra. The two have been working together to develop and introduce this emerging technology for nearly 10 years, and have established themselves internationally as the leading innovators in their field.

The Virtual Orchestra is a live-performance computer system with extraordinary musical and sonic control. Unlike computer music systems used in production studios for film, television, music recording and multimedia, Bianchi says the Virtual Orchestra "must perform and manipulate complex musical scores and interpret a continuously changing scenario of expression in real time. It does not use traditional music keyboards or standard musical interfaces. All of the musical detail is meticulously programmed, but flexible enough to be transformed and recombined under the control of a conductor during performance. Musical tempo and dynamic fluctuations are examples of nuances that change from performance to performance and from moment to moment. The ability to respond in real time to these artistic and aesthetic nuances is what makes the Virtual Orchestra so versatile."

Most of the "raw sounds" for the Virtual Orchestra come from digital samples recorded by the Prague Symphony Orchestra. These samples comprise thousands of individual notes and combinations of instruments. A musical score is carefully assembled from these samples so that each note and musical phrase blends properly to give the effect of a live musical ensemble. "There are so many acoustical nuances that need to be accounted for when scoring for a production," Bianchi says.

"For instance, a violin sounds slightly different depending on which direction the bow is traveling, its angle, its horizontal position, its pressure, and so forth.

"Our Prague recordings are essentially a collection of 'dumb sounds.' They're transformed into intelligent and expressive music by laboriously tweaking each note, one by one. In some scores there are hundreds of thousands of notes. A typical Virtual Orchestra sound design takes two years - some designs have been three years in the making. So you can see, we're not talking about performers simply sitting down and playing keyboard synthesizers - this is a completely different proposition."

---

Corollary: In Search of a Venue

---

As he describes how he was drawn into the world of virtual orchestras, Bianchi alternates between a rational explanation and a more romantic one. "I've always worked in both mediums - acoustic and electronic. My early experiments began in the late '60s with monophonic tape recorders, record players and, eventually, early analog synthesizers. For the next 20 years I was always at the edge of the development and fortunate to be in the company of some extraordinary artists and technicians. So one day, at an opera rehearsal, when a major logistical problem threatened to close the show, I said to myself, 'I think I have the technology in my studio to solve this problem,' and I stepped forward."

One dilemma was - and continues to be - that there are theatrical and operatic companies in this country that can no longer support the escalating cost of the orchestra. Overall, opera in the United States has fallen on hard times. With cuts in funding from the National Endowment for the Arts, there have been fewer grants to arts organizations, which, in turn, has meant leaner operating budgets. Opera companies have had to pass more of their costs on to the audience in the form of pricey tickets, making it even harder for them to compete for the general public's entertainment dollars and to grow their audience base.

"The whole industry is slowly crumbling because of the economic reality of trying to use a 200-year-old institution (the orchestra) on a modern-day scale," Bianchi says. "It won't work. Not only is it becoming economically impossible, it is irresponsible and visionless for the industry to assume that a generation nurtured on film, television, amplified music and CD-quality sound would relate to an art that lacks any relevant cultural cues. In other words, they're losing their audience, or, at best, boring them."

Virtual orchestras are not without problems, of course. For example, real orchestras are, for the most part, reliable and don't crash. Says Bianchi, "With the Virtual Orchestra, the control and reliability of the music lies in the hands of a few people. Any technical or human error during performance would be catastrophic." He points out that as a result, the people who implement this technology must be technically and musically savvy. Over the past 10 years he says he has witnessed a transformation in the skill and preparation of young sound designers who work on his team and refers to them as "the new breed of technical artists."

"Sound designers can no longer assume the traditional roles of simply running mixing boards and hanging speakers," he says. "They must understand all aspects of the system as well as the theoretical and aesthetic implications of the music. That is so important. They need to know, for example, why the third act begins at a specific tempo or why the composer has voiced the fifth of the chord for the soprano. I spend a good deal of time talking to my sound designers about music and probing their minds. This is all prerequisite knowledge and preparation, because it is inevitable that at some point during a performance, and at a moment's notice, each sound designer will be called upon to make high-level musical decisions that will influence the production. That's a lot of responsibility. These new sound designers must have nerves of steel, an egocentric confidence, and be rock solid night after night - no mistakes."

Perhaps the most important challenge facing the technology is creating convincing music, says Bianchi, who notes that computer-generated sounds must be realistic, not only in a musical sense, but in a physical sense as well. "When you're in the balcony at the opera, if you close your eyes and listen to the music, you can almost feel the size of the hall. The music has a presence that's hard to define. In trying to achieve that effect, we've learned that the realism improves greatly as the sound becomes more distributed, and that means more speakers - 30, 40...maybe 100 speakers. Each sound source needs its own physical space before it becomes acoustically integrated into the entire soundscape."

As Bianchi has found out firsthand, however, there are those in the opera community who feel that the indefinable "presence" of music is inextricably bound to the presence of a live orchestra. Overcoming this attitude is as big a challenge as any technological quandary relating to the system. In addition, the absence of live players has made many in the opera community suspicious or angry at the perceived audacity of replacing the authentic with the synthetic.

In the winter of 1995-96, the Kentucky Opera Company was hoping to mount a production of Hansel and Gretel for the Christmas season. This huge and complex post-Wagnerian score called for 100 musicians - well beyond the physical and financial reach of the Kentucky Opera Orchestra, which had already contracted to play the Nutcracker ballet. Casting about for a way to enable the show to go on, the Kentucky Opera turned to the Virtual Orchestra to provide the musical score. Though the Virtual Orchestra had several productions to its credit, this would be the world's first completely digitized opera by an important professional opera company. That fact alone guaranteed that the production would come under intense scrutiny by the international opera community. For its trouble, the Kentucky Opera was picketed by incensed musicians and union members and witnessed lines of protesters distributing leaflets to patrons decrying the "diminished production."

The opera received a critical drubbing by some (though not all) opera publications. One of those dressings-down was published by Opera News.

---

Bianchi conducts the Virtual Orchestra during a performance of Hansel and Gretel by the Kentucky Opera Company.

---

It read, in part: "Music is an art of communication. Even if the technology improves, and it surely will, an essential ingredient is missing: the vibrant interplay among human beings, an experience that cannot be duplicated by a machine." At one point, the reviewer went as far as to accuse Bianchi of contributing to "...the dehumanization of society."

On the other hand, many reviewers acknowledged the historical significance of the event.

In defense of the technology, Patrick Smith, editor of Opera News, stated: "...there were many who criticized the invention of the automobile and shouted 'get a horse' at the sight of a broken-down vehicle on the side of the road....however, there are no horses on the New Jersey Turnpike!"

The criticism is not lost on Bianchi, whose divided feelings toward his own technology reflect the schism of his two passions. As a music technologist, he is convinced that the Virtual Orchestra is both necessary and effective, so he strives to improve constantly, to bolster the quality of the performances and to quiet the naysayers who insist it can't be done. And yet, as a composer and operagoer, he can't help but sympathize with those who say it shouldn't be done.

"There is quite a conflict that I must continuously resolve concerning the implications of this work, and I have given the philosophical problem more time and thought than any other aspect related to the Virtual Orchestra," he says.

Bianchi is married to Janna Hymes-Bianchi, the resident conductor of the Charlotte Symphony in North Carolina. In contrast to his work, she is involved with live orchestras, the ballet and opera on a daily basis. Says Frederick Bianchi, "So the Virtual Orchestra should not be perceived as the work of some whacked-out technologist living in isolation from the real world. I cannot honestly, and with integrity, make a single move without considering the implications."

Ask him point blank if he would prefer to experience an opera performance with a live orchestra or one with a virtual orchestra, and Bianchi says, "No question. I would prefer a live orchestra. And yes, there are some elements of the theatrical experience that cannot be reproduced by a Virtual Orchestra. But the reality of the situation suggests that if we intend to move forward we must be willing to accept a change. And I believe the industry is in the process of changing."

The conflict and controversy will not disappear anytime soon. Traditional opera cannot reinvent itself, Bianchi says. It will continue to thrive at the major centers for opera - the Metropolitan Opera in New York, the San Francisco Opera, and opera houses in major cities like Chicago and Houston. But smaller companies will, more and more, turn to contemporary opera productions that will help them attract a new and younger audience. For Bianchi, this emphasis on contemporary opera production would present the best possible scenario, allowing him to change his focus from breathing a few last breaths into an ailing institution to helping establish an operatic tradition for a new generation.

"Realizing scores like Hansel and Gretel has been great," he says. "It helped us introduce and refine the technology, but it doesn't explore the potential of the Virtual Orchestra. If you let the Virtual Orchestra function more idiomatically and let it do what it does best, then there is almost no limit to the sonic worlds that can be created."

One such idiomatic use of the Virtual Orchestra was Bianchi's production of The Wizard of Oz. Productions that are less "highbrow" or don't carry the burden of tradition are ideal settings for the technology, he says. They are also the most fun, as evidenced by the glee in Bianchi's eyes as he describes how the Tin Man's music was produced by transforming raw metallic noises into a beautiful musical accompaniment.

"The Virtual Orchestra score to Oz was under development for over three years, and when you consider the amount of automation and special effects synchronization, the number of surroundsound speakers, the degree of interactive control, and the overwhelming technical muscle that was involved, it would certainly rate as one of the most sophisticated sound designs in musical theater to date. And that would include anything done on Broadway or at Disney."

"I would hope," he says, "that in a few years I won't even be involved in replacing the orchestra, but that a whole new aesthetic will have evolved." Until then, he says it will be necessary for the technology to serve primarily as a stand-in for live orchestras. So he continues to improve and refine the Virtual Orchestra, to make it as close to the experience of a live performance as possible. In fact, even as he acknowledges the shortcomings of the technology, he maintains that the lines between authentic and synthetic music are blurring more and more all the time.

"Every four months there's a quantum leap in this technology," he says with enthusiasm. "What we're doing now is miles beyond what we were doing last year."

Bianchi says he foresees a certain future for the Virtual Orchestra. "If opera does reinvent itself, this technology will be right there with it. Traditional opera will certainly continue and will survive as long as there is an audience to support it," he says. "But I would prefer to be a part of the new opera and to have put all my eggs in the basket that says opera will change. I want to take the risk of being on the edge of that change."

Bert is a turbine blade engineer at Howmet Refurbishment Inc.