Adobe Audition

Markers are the un-sung heroes of audio production. And Audition CS6 adds a bunch of new capabilities that make them even more useful in your day-to-day work. Rather than just focus on new features, we thought it would be helpful to recap a couple of the most common workflows for using markers.

But let’s start with the basics. In Audition CS6, the Markers panel is ganged with the Media Browser and Effects Rack by default, so it’s more visible and easier to get to. One of the new features we added in Audition CS6 is the ability to display markers for all open files. Audition 3 could do something similar (display markers for each file in the Files panel), but this is a much sleeker approach as it both displays the markers and enables you to work with them. In the screenshot below, we’ve opened a bunch of files with a variety of marker types.

Note that the new QuickSearch field in the Markers panel means you can search for markers by name, type, or other attribute.

Markers in Audition can specify a single point in time, or they can identify a range of time—the only difference between the two types is that range markers have a duration, and are particularly useful for identifying the parts of a longer clip that you want to use. When you add a maker, its start point is added at the current playhead position.

Audition supports four marker types; if you click the Add Marker icon on the Markers panel, you get a Cue marker by default—but the fastest way to add makers in either Waveform or Multitrack view is to use the keyboard shortcuts:

We covered the new CD Burning feature and the role of Cart Timers in earlier blog posts. Subclip markers are similar to Cue markers, but are designed to enable Audition to make use of the subclips created when you log media in Adobe Prelude CS6 or Adobe Premiere Pro CS6. Use them a lot? You can always customize your keyboard shortcuts.

One common task where markers can really help is taking a long recording (say, an interview, or a concert) and breaking it into smaller segments which would then either be added to a multitrack session or saved out as separate audio files. Since we know this is something a lot of folks do, we wanted to share some tips:

• Use the M keyboard shortcut to add markers as you listen to the recording in the Waveform view.
• To create range markers, use the I key to set an in point, and the O key to mark an out point, then press command + shift + M to turn each selection into a range marker.
• Convert your point markers into range markers: select the markers you want to create ranges for, then use the Merge Selected Markers option (third button from the left in the Markers panel) to turn the first of each adjacent pair of markers into a range marker with that duration.
• Tweak your markers: Adjust range durations as necessary, or give your markers more descriptive names.
• If you’re prepping a CD, convert Cue points to CD Track markers using the Type drop-down list; you should also consider naming each range marker with the track name.

Once you’ve got your file marked up, you can insert the ranges directly into the Multitrack, or you can export them as separate files. You can also save marker ranges to a Playlist, a topic we’ll cover in more detail in a future blog post.

• Export files from the Waveform view: Select the markers you want to export as files, then click the second button from the right (Export audio of selected range markers to separate files). This exports each range as its own file; you can specify whether to use the marker name, add prefixes or postfix numbers, where to save the file, and so on.
• Insert ranges into a multitrack session: If you select range markers and click the last button on the right (Insert into Multitrack), you’ll be prompted to either add the ranges to an existing session or to create a new one. If you’re in Multitrack view, you can drag and drop marker ranges directly into your session.

One last handy marker-related tip: you can quickly select the time specified by a range marker by hovering over either end of the marker range icon and double-clicking. Sweet! And thus concludes our whirlwind tour…. Comments? Questions? Let us know!

The Audition CS6 Help covers some marker-related features, and Colin Smith has posted a video tutorial that touches on markers as well as Skip Selection, which we covered in an earlier blog post.

Not using Audition CS6 yet? You can download a free 30 day trial or just go ahead and order a full copy or upgrade!

Audition is part of the Adobe Creative Cloud, so you can subscribe once and get access to Audition CS6, Photoshop CS6, and all of Adobe’s other creative tools.

A week or two ago, some Audition users at a major entertainment company (we wish we could say who, but we can’t!) got in touch with us with some questions about whether the new version of Audition could support a legacy workflow. Not surprisingly, the answer was “yes!”—so we thought we’d share the info, in case some of you have similar questions.

These folks create audio that gets played back on a number of different devices, some of which have specialized requirements—and they needed to be able to export raw PCM files from Audition. Audition 3 supported this capability—but Audition CS5.5 did not.

Happily for these customers—and maybe you too—one of the areas we focused on for Audition CS6 was expanding the range of file formats we support for both import and export. Now you can import and export any format supported by the open source libsndfile library (http://www.mega-nerd.com/libsndfile/), which includes a whole BUNCH of audio file formats, from the esoteric to the widely-used:

• *.au
• *.avr
• *.caf
• *.flac
• *.htk
• *.iff
• *.mat
• *.ogg
• *.paf
• *.pcm (There it is! Easy to miss on such a long list.)
• *.pvf
• *.rf64
• *.sd2
• *.sds
• *.sf
• *.voc
• *.vox
• *.w64
• *.wve
• *.xi

Of all of these formats, the ones we got the most requests to add support for were FLAC and OGG—so to make them more visible, we gave them each their own line item in the export format drop down. It’s all the same under the hood, so doesn’t matter which route you choose.

In addition, we added support—for the first time ever—for importing and exporting *.mp2 audio files, which is still widely used as a standard in radio. And we also added support for the fabulously-named Monkey’s Audio format, *.ape.

Need to work with WMA and WMV files? They’re supported through the Dynamic Link Media Server, which is turned off by default to keep performance zippy. To turn it on, go to the Media & Disk Cache pane of the Preferences dialog, and check the Enable DLMS Format Support option. We’ll go into more detail about DLMS in a future blog post about expanded video format support in Audition.

So, let’s say you’ve picked libsoundfile as your export format. The next step is customizing your export settings: click on the Change button next to the Format Settings display area, then choose whichever format you need; you can also choose  from a variety of encoding options.

Questions? Comments? Ideas for topics you’d like to see us cover? Let us know.

Not using Audition CS6 yet? You can download a free 30 day trial, or just go ahead and order a full copy or upgrade!

Audition is part of the Adobe Creative Cloud, so you can subscribe once and get access to Audition CS6, Photoshop CS6, and all of Adobe’s other creative tools.

Brian King is a PhD candidate in Electrical Engineering, and for the last several summers he’s worked as an intern in Adobe’s Creative Technologies Lab, an internal “think tank” where some of the technology that makes our products work like magic is developed. Brian’s research led to the hugely popular Automatic Speech Alignment feature, also known as Rubbadub, which he initially previewed at Adobe’s MAX conference in 2011.

To use the feature, you need two selected clips in a Multitrack session. Choose Clip > Automatic Speech Alignment, confirm the settings in the dialog box (below), and click ok. Using what seems like magic, the replacement audio lines up perfectly with the original.

In this interview, Brian talks about how the technology underpinning this feature works.

Ellen:  Brian, what got you interested in your area of research, digital signal processing?

Brian: I started playing guitar when I was in second grade, and I’ve always loved music and technology. Digital signal processing really fuses my two passions.

Ellen: Since you’re almost finished with your PhD, you must be writing a thesis. What’s the topic?

Brian: It’s on single channel source separation. Let’s say you have a number of people talking in a room at the same time—you can use this algorithm to extract individual voices.

Ellen: That sounds useful!

Brian:  Yeah, definitely.

Ellen: You’ve been working on features in Audition for a couple of releases now; the big one for CS6 was Automatic Speech Alignment. How did you get started solving that problem?

Brian: We wanted to make ADR easier. Let’s say we’re shooting a hit movie and we get a great take in terms of performance, but the audio is really noisy. What studios typically do is bring the actors back into the studio and have them overdub their lines, then manually align the new audio with the original performance—it’s a process known as automatic dialogue replacement.

As you can imagine that can be a really tedious process because if the actors don’t get their timing exactly right it’s not going to line up with the original video and it’s going to look like an old Godzilla film or Kung-Fu movie.

Ellen: Because the lip movements don’t match what you’re hearing?

Brian:  Exactly. Typically the actors have to record each phrase of dialogue dozens of times to get the timing as close to the original performance as possible, and then a poor studio engineer has to take all of these multiple takes and somehow find the best snippets to fuse into a coherent phrase, which is both time-consuming and kind of tedious.

So given a noisy, original dialogue recording that was taken out in the field and a high quality overdub that has no noise but really bad timing, I developed a method for automatically stretching and compressing every instant of the overdub so that it matches the timing of the original recording.

Ellen: You say every instant—how much time is that?

Brian: Right now it’s set to about every 15 milliseconds, so it’s small enough that it’s imperceptible. I think (laughter). A typical video frame corresponds to about 33 milliseconds, so 16 milliseconds is approximately one-half of a video frame.

Ellen:  How do you identify the signal that matters in the original recording with all of the ambient noise?

Brian: What I’ve been working on for my Ph.D. is source separation, but for this I’m using a matrix factorization method, where you learn the important features or spectral characteristics of the audio.

Ellen:  So you build a fingerprint of the existing audio first?

Brian: What we do is identify specific spectral characteristics of the original dialogue. For example, if I say “Hello,” and you say “Hello,” a human will understand we are both saying the same words, but we can tell if it’s me saying it vs. you saying it, because my voice has a lower pitch, or maybe I have slightly different timing or other vocal characteristics.

So this feature learns the vocal characteristics of the speaker or the actor from the overdub clip, and it also learns the timing of that clip. Then when we compare it to the noisy clip, it knows what to look for. It finds the same features in the noisy clip, and for everything else that doesn’t match it says, “Oh, well that’s noise. Let’s ignore that.”

Remember Highlights magazine, which you used to find in dentist offices when you were a kid? You’d see this picture, and there’d be all of these hidden things in it. That’s kind of what the method that I developed is doing: the overdub recording shows you exactly what you’re looking for, kind of the hat in the picture. Then when you get to the noisy recording, it’s as if all of those pictures are still in there but hidden with all sorts of other things. Since you know what you’re looking for, you’re able to find it easily and accurately.

Ellen: Once the pieces that match are identified, what happens?

Brian:  That’s taken care of by a dynamic time warping algorithm, so that’s fairly simple.

Ellen: A dynamic time warping algorithm? That doesn’t sound that simple!

Brian:  Well, it has three stages. The first finds the important features: “These frequency characteristics at this point match these characteristics at this other point.”

Then, it figures out how to align those two points temporally. If the overdub is faster than the original, you want to stretch the overdub to slow it down, but if a word in the overdub took twice as long to say as that same word in the original reference recording, then you would compress it instead.

Then the third part of it is synthesis. Once we know how to shrink, compress and expand each instant, then we simply synthesize a new signal.

Ellen: Any tips for working with different clip lengths?

Brian:  The best way to use this tool is to synchronize two clips at a time, and to work with relatively short, phrase-length snippets that have natural start and end points. You definitely wouldn’t want to take a two hour long movie and then try to align two hours of overdubbed dialogue.

Ellen: What’s the optimal length?

Brian: Anywhere from a few words to a paragraph. In most spoken dialogue there are natural breaks that are really easy to identify. “Here’s the beginning of this piece of dialogue. Here’s the end.” Just use those natural breaks. You don’t have to go mid sentence or anything.

Ellen: So if you have a long clip, your recommendation would be to break the original noisy clip into smaller segments and then replace each small piece of dialogue?

Brian: Yes.

Ellen: What happens if the lengths of the clips aren’t roughly the same length? Say you’ve got a 12 second original piece and a two second replacement clip for a phrase or two of dialogue in the middle of that longer piece.

Brian:  It’s all or nothing at the moment, so if you try to align a very large clip and that doesn’t work then you can try breaking it up in to smaller pieces and playing around with the parameters.

Ellen: What happens if the words don’t match?

Brian: That’s an interesting question. It tries to align them as best as possible. We’ve seen situations where it’s the same actor and the same dialogue, but maybe he said, “This will be great.” Then in the overdub he says, “This’ll be great.” You have that “wi” gone, and in that case when it stretches it says, “Well, all right there are no kind of frames corresponding to the wi sound, so we’re just going to take the wi in the overdub and make it as fast as possible,” and kind of almost shrink it to nothing so that we can kind of get the rest of the alignment to happen as best as possible. In that situation, it can sound kind of funny because you will have a natural cadence and then you’ll have this really short piece when you listen to it and think, “What the heck happened there?” Often in those cases, Audition is trying to match something that wasn’t there in the first place.

Ellen: So this feature doesn’t do anything to help with overdubbing in a completely different language?

Brian: Not yet! But we’ve been looking at it.

Ellen:  Are there any uses for this feature that may not immediately apparent?

Brian: I know some people have used it for aligning different music tracks. I experimented a little bit with just aligning vocals. For example, I overdubbed myself singing a couple of different rap parts, and it was able to align those really well.

We also aligned some different guitar parts that should have the same timing. Instruments that have strong attacks like guitars, cymbals, and high hats work really well. Softer stuff didn’t really work that well, but it could work for aligning back vocals in some cases—it would really depend on the characteristics of the sound.

Wrap-up

We chatted with Brian about a number of other things he’s been working on—but unfortunately, can’t talk about them yet! Between the fabulous developers who work on Audition and our internal research teams, we’ve got a great roadmap for Audition. But we’d love to hear from you about difficult and time-consuming problems you think our research teams might be able to help solve—so please, leave your comments or get in touch by submitting a feature request.

Not using Audition CS6 yet? You can download a free 30 day trial. Or just go ahead and order a full copy or upgrade! 

Audition is part of the Adobe Creative Cloud, so you can subscribe once and get access to Audition CS6, Photoshop CS6, and all of Adobe’s other creative tools.