Sound from a Virtual Planet

For some time we’ve taken pitch transposition and time-stretching of audio for granted, with one caveat; only an entire audio signal gets processed. We’ve come to expect programs like Ableton Live and now Pro Tools to treat audio like butter, speeding up and slowing down loops, time correcting sloppy drumming and pitch correcting the wanna-be diva. The Waterloo of this technology has traditionally been isolating events within an audio file. If a singer hits the wrong note, it’s usually not a problem to correct, since it’s a single monophonic event that can be easily isolated and processed. What about when the piano player hits the wrong note in a chord? Well, that’s another take…. We’ve always operated under this assumption, and it was a great reason to record performances as MIDI data, since we had the freedom to freely manipulate individual notes a chord.

All this changed last week at Musikmesse, Germany’s massive musical instrument trade show, when Celemony debuted their Direct Note Access (DNA) technology. DNA can analyze an audio event, isolate individual pitched elements, and freely manipulate them in pitch and time. While I’ve said that changes in music technology products are often evolutionary, not revolutionary, this is a really big one.

Celemony started in 2001 with the initial release of Melodyne, the brainchild of German programming whiz Peter Neubäcker. The whole idea behind the technology was to allow users to edit the pitch and timing of a note graphically. Melodyne does this by analyzing the source and displaying the result as graphic data on a pitch and time grid. From here, the audio properties can be manipulated much like note and controller data in a MIDI sequencer’s graphic editing window. Opcode first introduced this concept in the late 1990’s with their StudioVision sequencer. Here, monophonic audio performances were analyzed and represented as MIDI note and pitchbend data. You simply edited the MIDI data, and rendered the result back to an audio file. Melodyne expands on this, bypassing MIDI altogether and greatly enhancing the resulting sound quality.

Melodyne is available in a variety of products from the entry level Melodyne Uno to the flagship Melodyne Studio. Starting in Fall 2008 with version 2 of the Melodyne plug-in, Direct Note Access will be incorporated into their full line and perhaps inspire some interesting new products.

Peter Neubäcker freely admits that he assumed extending the Melodyne model to individual note events in a chord was not possible. Only after challenging this basic assumption did the algorithms behind this begin to take shape. The DNA acronym works, since Direct Note Access is really about exploring the genome of the harmonic life of music. While the demo video is truly amazing, keep in mind that the types of performances here illustrate what may be the best case scenario for effectively using this technology. There are limits, and one wonders if DNA could root out a wrong note in a dense, orchestral recording. Still, what Celemony has come up with is nothing short of remarkable.

The hardest part of getting through a NAMM show is wearing a badge that identifies me with Berklee. Now don’t get me wrong, I’m proud to represent the institution, and while my affiliation opens many doors, there are scores of alums in all aspects of the music industry who love re-connecting with their alma mater. If you want to travel to NAMM incognito, get your badge from Harvard.

The best spokesperson for any product is an artist who uses the product, and uses it well. This year, I was pleasantly surprised to see one of my former students, New York electronic artist Matt Moldover, talking about his work and performing at the Ableton Live booth. While sharing a common school experience with fellow alum Dan Lehrich, profiled in an earlier blog entry, Matt has taken a very different path, establishing a profile as performing artist.

Matt was one of the legion of guitar players that comes to Berklee each year. While most are looking to follow in the footsteps of one fretted deity or another, Matt always wanted to forge his own path, and after getting in the Music Synthesis major, that was combining interactive electronic performance with the guitar. At Berklee he discovered MAX, and soon was on to the idea of extending what he did as a player to sound from electronic sources. Matt didn’t want to play in a band, he wanted to play with sound.

Matt also got turned on to DJ and club culture. Moving to New York after graduation, he found a scene for like-minded electronic performers, and jettisoned his first name, becoming the artist known as Moldover. Being a player and a geek, he was in the right place at the right time when Native Instruments came out with Guitar Rig. The first time I saw him at NAMM, he was the Guitar Rig guy at NI. While he gave knowledgeable and convincing demos, I got the sense a different muse was calling. At a party in LA we had a chance to talk, and I got a glimpse of some of the projects he was working on, the first of which was the Interstellar ReMix Wagon for Burning Man, 2004.

The thing I didn’t quite realize about Moldover was that he was really pretty good at building stuff. His next project was the Octamasher, a performance system fueled by Ableton Live that gave eight “mashers” a tool to communally create a club mix. Social networking and interactive performance might sound like a research project at the MIT Media Lab, but this is a guy with a laptop, hacking a bunch of cheap keyboard controllers and hitting parties…. pretty cool.

Sometime last fall came a new website and the birth of “controllerism.” According the the site, Controllerism is “the art of manipulating sounds and creating music live using computer controllers and software.” Perhaps Matt will be the first to make both YouTube and dictionary.com. But, what I saw from him at the Ableton booth this year was a virtuoso performance that combined electronic music with the spontaneity and inventiveness of a jazz soloist, swapping clips of sound for notes and scales.

Dan Lehrich and Moldover may seem at opposite ends of a very wide playing field, but what really fascinates me is the real passion they both have for creating immersive performance experiences using computers and physical interfaces. While research in the field of interactive music systems continues at the highest levels of academia, it’s really cool to see real innovation happening on the street as well.

Fall 2006 brought a cornucopia of software updates for music production. If you’re a Mac user, that includes the long awaited new operating system, Leopard. A late November release of Live 7 capped a season where we saw the arrival of Logic Studio, Reason 4, NI Komplete 5 and Pro Tools 7.4. This onslaught raises the inevitable question for users of when to upgrade –what works or when will it? Although most of us involved with technology welcome change, but we are periodically reminded of the commitment we make to troubleshooting and learning new features. This past fall, that was a big one.

Out of nowhere, Logic Studio was announced in early September. After months of rumors about what would become of Logic, 10 DVDs held the answer. By this time, there’s a number of really good reviews out of Logic 8, but suffice it to say, this is an evolution, not a revolution. However, with Logic 8 shipping as a software suite with Soundtrack, Compressor, and Mainstage –a new performance application that hosts software synths and processors– at half the price of Logic 7, the update for users is a no-brainer. The install took forever, even without adding the lifetime’s worth of GarageBand loops that are included. But when all was said and done, Logic 8 ran like a clock and played nice with just about all the plug-ins it scanned –again another wait while the AU police did its gig.

I had a beta of Reason 4 over the summer, so when I finally got the release version in October, there were no surprises. Our friends in Stockholm release no software until its time and Reason remains the most stable piece of software I have ever used….period.

By the time Leopard was announced, my attitude was two down, bring it on. The new OS went on sale at 6:00 PM, I had it in my hands by 7:00, and at 9:00 my G5 tower studio computer rebooted to reveal shades of purple. I soon found out it was the color of envy… of all those who had the good sense to leave well enough alone. Leopard brought every single piece of music software to its knees, with the exception of standalone softsynths, and of course Reason. Times like this bring out my dark side…the fearless geek. As with any other vice, indulgence turned into another lost weekend….sorting through plug-ins and general troubleshooting.

OK, I knew Pro Tools wouldn’t work, but when the new Logic 8 crashed on every launch, I got nervous. Some people read mysteries, others chase down software incompatibilities, and it was off to the races for me. As Logic started, things seemed to bog down when I got to the Waves plug-ins. With a quick trip the Waves Website, I found that their line of plug-ins was not yet compatible with Leopard. So, once my Waveshell hit the trash, things got a bit further on start-up, but still no luck.

It seems that I never met a plug-in I didn’t like, and I install just about anything I come across. The problem is, they stay there. After sorting though all the demos and betas, I finally narrowed the field to a few likely suspects. Again, off to the trash; but still, no luck. One of the most reliable ways to start sorting out problem children in the plug-ins folder is to take them all out and open the application. With an empty Components folder, Logic opened without a hitch. The next step is the tedious task of closing the application, adding a plug-in, then opening. As long as Logic opened, I was in the clear. Instead of adding individual plug-ins, I went through families at a time. I was pleasantly surprised that my favorites were not at fault. After a bit of this low-level detective work, I found that the Melodyne Rewire plug-in, one that I had never actually used, was the culprit. Once Logic opened, all the other applications that had previously crashed, ran without a problem. Any program that was an Audio Unit host stalled on that one plug-in. (As of this writing, all current Melodyne plug-ins run under Leopard.

I might add that when installing Leopard, I chose to migrate my applications, settings, and preferences, and thankfully all of installs and the associated labyrinth of copy protection schemes remained intact.

So, was it worth it? Heck yes… Leopard is a really slick OS visually, and despite the hit you might expect the processor would take from the added graphic elements, the system runs smoothly, is very stable, and there is a noticeable improvement in the performance of some applications. At first glance, there doesn’t look to be any changes to Core Audio, but I wouldn’t be surprised if there were some goodies somewhere under the hood.

So, should you upgrade? Well, that depends. Here are a few thoughts on when to upgrade:

Tips for upgrading:

1. Do you need to? If you use a machine for billable work, be very cautious with upgrades. (If you’re a working pro, I probably don’t need to tell you that.) If you are working on projects that have deadlines, don’t do it.

2. If you have two machines, start with one, using it as a test platform, then transition to the other. I started with my studio machine, and since I had no looming deadlines, this made some sense, hence my somewhat cavalier attitude this time out. I use my laptop to run my life and since it’s a newer Intel machine, it can work for just about any project that comes up as a back up. I’ll update it when the dust settles.

3. If you think you’re ready to make the leap to a new OS revision, check manufacturers’ Websites for compatibility. Don’t forget any drivers you may need. Although many are now class compliant and need no additional drivers, this is not always the case. MOTU hardware requires driver software, and they are thankfully pretty quick to update.

4. If you’re updating an OS or a host application such as any DAW, check with the companies that supply the plug-ins you rely on for compatibility.

5. Weed through your plug-ins before running a new OS or software version. A bit of housecleaning will usually ease a transition. I always try to set aside time when I make major upgrades or revisions to clear out software and plug-ins that I don’t use.

6. Back up before you make any changes. Getting a new OS is like getting a heart transplant (or at least a bypass). You’re making a major change to the critical part of your system and stuff can happen. You never know exactly how compatible documents will be with new versions of software that authored them.

Happy New Year and have fun with all the new stuff that’s out there.