Sound from a Virtual Planet

Perhaps the most anticipated software development this spring was the release of the MAX/MSP/Jitter version 5 from Cycling 74, which came out last week. For those of you who don’t already know, MAX is object oriented programming environment for sound, music, video applications. Wikipedia has a very good overview and history of the program listed under Max (software).

A MAX/MSP Patch

While updates over the years have focused on, new objects and support for new OS technologies, the basic look and feel of MAX hasn’t changed much since it’s initial commercial release from Opcode. (Does anyone remember them?) MAX 5 addresses this with a complete rewrite of the entire underlining code, aligning it with current hardware and OS software platforms. A revised user interface includes a variety of on-screen de-bugging tools. Some of these, like a visual display of signal level at any connection, easily translate into powerful ways to learn about signal flow and processing. Visually, objects are much easier to look at, and a new user interface view separates the underlying patch structure from how it appears to a user/performer. Last year Cycling 74 announced an alliance with Ableton, makers of Live, and entire look of the new release of MAX looks a lot like its German cousin.

Pretty cool… That said, MAX (or Reaktor for that matter) isn’t for everyone. If you’ve got some hacker instincts, you’ll be able to get around MAX after working through the excellent tutorials that come with the program. Plan on a few weekends of focused study and experimentation and you’ll be on your way. Cycling 74 has a very good video introduction that will give you a taste of what the program is all about.

For some time we’ve taken pitch transposition and time-stretching of audio for granted, with one caveat; only an entire audio signal gets processed. We’ve come to expect programs like Ableton Live and now Pro Tools to treat audio like butter, speeding up and slowing down loops, time correcting sloppy drumming and pitch correcting the wanna-be diva. The Waterloo of this technology has traditionally been isolating events within an audio file. If a singer hits the wrong note, it’s usually not a problem to correct, since it’s a single monophonic event that can be easily isolated and processed. What about when the piano player hits the wrong note in a chord? Well, that’s another take…. We’ve always operated under this assumption, and it was a great reason to record performances as MIDI data, since we had the freedom to freely manipulate individual notes a chord.

All this changed last week at Musikmesse, Germany’s massive musical instrument trade show, when Celemony debuted their Direct Note Access (DNA) technology. DNA can analyze an audio event, isolate individual pitched elements, and freely manipulate them in pitch and time. While I’ve said that changes in music technology products are often evolutionary, not revolutionary, this is a really big one.

Celemony started in 2001 with the initial release of Melodyne, the brainchild of German programming whiz Peter Neubäcker. The whole idea behind the technology was to allow users to edit the pitch and timing of a note graphically. Melodyne does this by analyzing the source and displaying the result as graphic data on a pitch and time grid. From here, the audio properties can be manipulated much like note and controller data in a MIDI sequencer’s graphic editing window. Opcode first introduced this concept in the late 1990’s with their StudioVision sequencer. Here, monophonic audio performances were analyzed and represented as MIDI note and pitchbend data. You simply edited the MIDI data, and rendered the result back to an audio file. Melodyne expands on this, bypassing MIDI altogether and greatly enhancing the resulting sound quality.

Melodyne is available in a variety of products from the entry level Melodyne Uno to the flagship Melodyne Studio. Starting in Fall 2008 with version 2 of the Melodyne plug-in, Direct Note Access will be incorporated into their full line and perhaps inspire some interesting new products.

Peter Neubäcker freely admits that he assumed extending the Melodyne model to individual note events in a chord was not possible. Only after challenging this basic assumption did the algorithms behind this begin to take shape. The DNA acronym works, since Direct Note Access is really about exploring the genome of the harmonic life of music. While the demo video is truly amazing, keep in mind that the types of performances here illustrate what may be the best case scenario for effectively using this technology. There are limits, and one wonders if DNA could root out a wrong note in a dense, orchestral recording. Still, what Celemony has come up with is nothing short of remarkable.