AI software gives virtual guitars a lifelike sound

AI software gives virtual guitars a lifelike sound

Portland, Ore. - Sibelius Software Ltd. has successfully applied the principles of artificial intelligence to give the performances of its music software a more humanlike sound. By crafting a rule system that simulates a human virtuoso, Sibelius and its new "guitar-only" version, called G7, perform music convincingly enough to turn heads.

"I think we will continue to research ways of making Sibelius' performances sound more human, with an ultimate goal of making it so you couldn't tell-a kind of Turning test, isn't it?" said Ben Finn, co-founder and managing director at Sibelius.

Sibelius began its AI quest with "expressivo"-an expert system embedded into Sibelius 1.0 for varying the dynamics (amplitude) of individual notes as they play, but Sibelius 2.0 and G7 also add "rubato," which slightly changes the tempo (speed) for emphasis and dramatic effect. It also contains an autoarrange feature that extends its AI rule set for music into the realm of orchestration.

"A lot of people using expressivo assumed that it was doing things to the tempo as well as the dynamics, whereas in fact it wasn't-but the human ear is, of course, easily fooled by these things," said Finn. "With rubato we were trying to do something quite a lot harder, which is the tempo getting faster and slower. The reason that is harder, is that there are even fewer known rules about how it should work than there are for the dynamics-in fact, the only known rule was that you should slow down at the end of a piece," said Finn.

No 'mechanical' quality
Computer-generated music has long connoted a stilted, mechanical-sounding performance. Not so anymore, according to Finn, because Sibelius incorporates algorithms gleaned from analysis of the subtle nuances, the slight variations in tempo and pitch, as well as from the different amplitude and attack/release emphases human virtuosos impart to the notes they play.

"Rather like with expressivo, rubato looks for various aspects of the music, such as whether the notes are going up or down, when deciding whether to quicken or slow the tempo" said Finn. "It proved quite hard to get it to work in a number of different styles."

The most sophisticated of the new "humanness" features, rubato performances reveal the feelings of a virtuoso performer as they are allowed to dramatically influence the duration of the note being played. Here not just a single note, but whole groups of notes taken together must be analyzed in real-time to reproduce how a virtuoso performer will dramatize segments, such as the final harmonic resolution of a song.

In practice, rubato changes what musicians call the tempo-engineers call it the number of beats per minute-60 bpm for slow blues up to 120 bpm for fast rock and up to 225 for the speediest jazz tempos. For rubato, the tempo is varied smoothly toward the slower, then smoothly back up toward the faster, always springing back to the assigned tempo of the song after the dramatic passage has passed.

In Sibelius and G7 scores, musicians can explicitly indicate where they want the rubato to begin and end with rit (slow down) and accel (speed up) notations written right on the musical staff for the virtual musicians to "see." The AI rules, however, come into play when you ask the program to interpret rubato on its own, with no markings in the score.

Composers choose from six levels of virtuosity for rubato and expressivo, the part of the program that changes the amplitude of notes. In just the manner that rubato subtly adjusts the tempo like a human, expressivo subtly changes the amplitude given to each note. Virtuoso performers change both the tempo and amplitude of notes in live performances, and by turning on both rubato and expressivo in Sibelius and G7, the musical performances become virtually indistinguishable from those performed by humans.

"I think this is something that can be improved with quite a lot more research-there are a number of different cues that the ear uses and I think that the ear is not particularly good at telling which of these is making it sound real-thus the ear is easily fooled," he said.

Finn said that a natural extension of the autoarrange feature would be to try to emulate individual performers-such as choosing between McCoy Tyner and Ray Charles on piano. However, because many Sibelius users have inexpensive PCs, Sibelius' true capabilities often cannot be appreciated.

"A natural extension would be to try to emulate individual performers, but that would require quite a lot of research. We would not only have to analyze how various well-known performers play, but we would also have to add some quite sophisticated controls to the software so that an advanced user of Sibelius could tweak it," said Finn. "Something we might be able to do in the future is to actually include some high-quality sounds right in Sibelius, because at the moment that is the thing we have least control over. Lots of people have got crummy sound cards that will never sound good."

For now, besides rubato and expressivo, parameters for 16 different "renderings" of the rhythmic feel of a composition are included. For example, "shuffle" or "swing" slightly extends the "on" beats and correspondingly shortens the "off" beats. Or users can choose "pop," "rock," "funk" or "reggae" to subtly change the rhythmic feel of the chosen genre without changing a note in the score.

A control panel allows composers to control all the "human"-style playback functions from a single location in the Sibelius or G7 program. Expressivo, rubato, rhythmic feel plus a variety of more subtle performance parameters, such as how much extra length should be afforded "slurred" notes, can all be tweaked to not only "humanize" the performance, but perhaps to also localize it to a particular kind of musician.

Other improvements in Sibelius and G7 allow composers to notate many subtle musical parameters that until now could only be interpreted by real musicians, such as the "curve" that rit/accel commands follow when changing the tempo. Also for guitarists, many of their special techniques can now be directly input into scores. For instance, rather than describing a guitar string "bend" as an upward change in pitch, as would be necessary with other musical notation programs, in Sibelius and G7 "bend," "slide" or "harmonics" can simply be notated on the score to be implemented in a performance. Composers can even add physical effects to instruments, such as a mute to a trumpet.

Besides more "human" performances, Sibelius and G7 also enable several sophisticated human capabilities that don't work in real-time. For instance, a unique "arrange" feature lets composers try out different instrumentations for their pieces without really trying. A jazz quartet, for example, can be autoarranged for a complete symphonic orchestra in under a minute.

Added intelligence
"Up until now, not just Sibelius, but pretty much all music software were pretty much passive tools. They will help you in lots of ways, but they won't actually write the notes for you, so this is our attempt to add a bit of intelligence," said Finn. "It is not as sophisticated as what a human can do, but it can save arrangers quite a lot of time-it does a perfectly fine transcription, making sure the notes can be played by the instruments and all the basic things that a orchestrator would make."

Composers can also rearrange for a rock, pop, jazz group, high school band, choir or any of over 130 ensembles of various instruments. Such arrangements can include those for high brass and pitched percussion; tenor doubling with soprano down an octave; impressionist orchestration with harp; woodwind and strings; jazz quintet with various lead instruments; baroque wind concerto; or even special-purpose instrumentation, such as film orchestration for an action sequence. The orchestration styles were created by Michael Price, whose credits include the epic "Band of Brothers" TV series produced by Steven Spielberg and Tom Hanks.

What it Means

- System expands AI rule set, changes tempo

- Plays back musical scores by synthesizing sound

- Optical character recognition scans scores into computer

- Musical score for a quartet can be rearranged for a full orchestra

http://www.eet.com