Sound from a Virtual Planet

For some time we’ve taken pitch transposition and time-stretching of audio for granted, with one caveat; only an entire audio signal gets processed. We’ve come to expect programs like Ableton Live and now Pro Tools to treat audio like butter, speeding up and slowing down loops, time correcting sloppy drumming and pitch correcting the wanna-be diva. The Waterloo of this technology has traditionally been isolating events within an audio file. If a singer hits the wrong note, it’s usually not a problem to correct, since it’s a single monophonic event that can be easily isolated and processed. What about when the piano player hits the wrong note in a chord? Well, that’s another take…. We’ve always operated under this assumption, and it was a great reason to record performances as MIDI data, since we had the freedom to freely manipulate individual notes a chord.

All this changed last week at Musikmesse, Germany’s massive musical instrument trade show, when Celemony debuted their Direct Note Access (DNA) technology. DNA can analyze an audio event, isolate individual pitched elements, and freely manipulate them in pitch and time. While I’ve said that changes in music technology products are often evolutionary, not revolutionary, this is a really big one.

Celemony started in 2001 with the initial release of Melodyne, the brainchild of German programming whiz Peter Neubäcker. The whole idea behind the technology was to allow users to edit the pitch and timing of a note graphically. Melodyne does this by analyzing the source and displaying the result as graphic data on a pitch and time grid. From here, the audio properties can be manipulated much like note and controller data in a MIDI sequencer’s graphic editing window. Opcode first introduced this concept in the late 1990’s with their StudioVision sequencer. Here, monophonic audio performances were analyzed and represented as MIDI note and pitchbend data. You simply edited the MIDI data, and rendered the result back to an audio file. Melodyne expands on this, bypassing MIDI altogether and greatly enhancing the resulting sound quality.

Melodyne is available in a variety of products from the entry level Melodyne Uno to the flagship Melodyne Studio. Starting in Fall 2008 with version 2 of the Melodyne plug-in, Direct Note Access will be incorporated into their full line and perhaps inspire some interesting new products.

Peter Neubäcker freely admits that he assumed extending the Melodyne model to individual note events in a chord was not possible. Only after challenging this basic assumption did the algorithms behind this begin to take shape. The DNA acronym works, since Direct Note Access is really about exploring the genome of the harmonic life of music. While the demo video is truly amazing, keep in mind that the types of performances here illustrate what may be the best case scenario for effectively using this technology. There are limits, and one wonders if DNA could root out a wrong note in a dense, orchestral recording. Still, what Celemony has come up with is nothing short of remarkable.