Porting DAW Bench Projects to Ableton Live

Vin Curigliano's DAW Bench is the standard toolset for measuring digital audio workstation hardware and software performance. While the current DAW Bench suite includes projects for several different DAWs, Ableton Live isn't one of them- which was a problem for me, since I was planning to investigate the performance differences between Live and Cubase for an upcoming post. Rather than ditch DAW Bench and go to the effort of developing my own test suite from the ground up, I chose to port the DAW Bench Cubase sessions to Ableton Live. What follows is a detailed analysis of the original DAW Bench projects, and my account of the surprises and challenges I encountered when porting those projects to Live.

When I first began this effort I thought I was the first person to do such a thing, but I eventually discovered user on a music forum who did his own attempt at a Live port a few months back. His goal was different from mine, and his projects weren't direct ports (he only replicated DAW Bench's "DSP" tests, and he chose a different set of plugins for his projects from the ones used in the official tests), so I continued with my own effort.

It is October 2017 as I'm writing this, so my observations here are based on the 2017 edition of the DAW Bench DSP and VI projects.

Analyzing the DSP Projects

Each DAW Bench DSP project contains hundreds of instances of a given CPU-intensive effects plugin, simulating a very complex mix. The "score" of a DSP test is the number of plugin instances you can enable without encountering any glitches in the audible audio.

• There are four audible tracks of pre-recorded audio, comprising the loop that plays while you enable FX plugins, listening for pops and drops. There are no effects plugins on these tracks.
• There is another "monitor" track which is just a recording of a sine wave. I don't know its purpose, but its track volume is turned all the way down.
• Next there are 40 more tracks of sine wave recordings, each with eight instances of the plugin under test filling the track's insert slots. Each instance is disabled when you first load the project. Track volume for each of these tracks is also turned down all the way.
• All tracks are routed to the main stereo outs, but only the first four audio tracks are audible due to their volume settings.
• There are three different DSP projects, one for each of these free-to-use effects plug-ins: SGA1566 (Shattered Glass Audio), MJUC jr. (Klanghelm), and a special version of ReaXcomp (Cockos) which is actually included with the DSP project download.

The DSP-1566 project on Cubase 9.

Analyzing the VI Projects

While the DSP projects are all about effects plugins, the VI tests only use instances of a virtual instrument plugin: Native Instruments Kontakt 5. The "score" of a VI test is the number of individual Kontakt "voices" you can have playing simultaneously before encountering audio glitches.

• The project contains one "Multi1Orchband" instance of the Kontakt plugin that contains a 16-part multi (one part per MIDI channel) of different Kontakt instruments that comprise most of the audible content in the project.
• Next there are ten "Multi2Poly" instances of Kontakt, each with 16 instances of the same instrument: A layered pad patch named "Light Breaks Through." Due to the patch's two sample layers, a single MIDI note played with this patch uses two Kontakt voices/oscillators. As with the sine tracks in the DSP projects, each instrument part in the Multi2Poly multis is turned down all the way. More on this later.
• The project also contains 16 MIDI tracks, each routed to a different MIDI channel in the Multi1Orchband Kontakt instance.
• Finally there are ten folders containing 16 MIDI tracks each, each one routed to its respective MIDI channel in one of the ten Multi2Poly instances. The MIDI tracks are simple clips that play a 16-bar sustained 10-note chord. So at play time, a single Multi2Poly instance can have up to 320 voices going at a time (16 parts of a 2-voice patch playing 10 notes each).
• There are two flavors of VI projects, "CV" and "NCV." In the CV project. many of the instruments in the "Multi1Orchband" multi have convolution reverb enabled as an insert effect, while that effect is bypassed in the NCV project. Both projects use convolution reverb as a send effect on a few channels. Importantly, the Multi2Poly multi is identical between the CV and NCV versions of the project (no reverb effect actively enabled as an insert or a send), so the results between CV and NCV tests are generally not very significant.
• The VI projects in DAW Bench were initially developed using Kontakt 4, whose factory library had a different layout from the library in Kontakt 5. While the current generation of the DAW Bench projects uses the Kontakt 5 plugin, the multis still use samples from the Kontakt 4 library. I am a licensed owner of Kontakt 4 but I do not currently have that version of the plugin installed anywhere, so I keep a folder containing all the necessary samples on my hard drive for when I need to do tests.

The VI CV project on Cubase 9.

Porting the DSP Projects

• Session view vs Arrangement view: Cubase is a traditional "piano roll" DAW while Live offers both a piano-roll style Arrangement view and the clip/loop-based Session view. I chose to build my projects in Session view because of the looping nature of the tests (it's just a few measures of repeating audio, after all), and because this view offers better access to the plugin slots for every track (after some UI resizing, at least).
• Audio tracks: The DAWBench DSP 2017 folder includes an "Audio" folder that contains the pre-recorded audio content (including sampled sine waves) used by these projects, which I easily imported into Live as stems. I grouped the tracks in the same order as the folders used in the Cubase projects.
• Plugin presets: All instances of the effects plugins in these projects use identical settings, which is important, because some plugin presets can be more CPU-intensive than others. Since plugin settings are embedded in the DAW project, I didn't have preset files I could import into Live, so I noted all of the plugin values used in the Cubase projects and created presets matching those settings for each of the plugins in Live.
• Plugin inserts: Just like with the Cubase projects, I inserted 8 instances of the desired effects plugin on each of the sine tracks. I enabled display of the insert slots in Session view and resized it so all 8 slots were visible at all times.
• Levels and routing: All tracks are routed to Live's Master outs, but all of the "Sine" tracks are set at -inf volume.

The DSP-1566 project on Live 9.

Porting the VI Projects

• MIDI tracks: For the MIDI parts that play into the "Multi1Orchband" instance of Kontakt, I dragged all of the MIDI clips out of Cubase and imported them into Live as MIDI clips. There were 16 MIDI clips for the Orchband multi, and another 16-bar MIDI clip for the polyphonic Sine tracks. This clip simply played a 10-note chord for a full 16 measures. Note: Due to an issue I discovered while performing benchmark testing, I discovered that the original 16-bar Poly tracks could not be used for reliable testing. (I will go into more detail on this in my next post.) Because of this I also created a 1-bar version of the Poly track that plays a sustained chord for the first seven 8ths of the measure. Session view allows me to add both the 1-bar clip and the 16-bar clip to every poly track and switch between them at will. Because of this I also edited separate copies of the original Cubase VI projects to use the same 1-bar loop.
• Audio tracks: The VI projects include one audible pre-recorded audio track (a 2-bar drum loop to accompany the Orchband parts), and 16 sine wave tracks whose volume is turned to -inf/silent. I presume these additional tracks are to help simulate a real music project that is handling both virtual instruments and audio tracks.
• EQ: In Cubase, each of the Sine audio tracks had channel EQ enabled, although it wasn't applying any boosts or cuts. This is not a default track setting, however, so I figured it was intentional. Since Live doesn't have built-in track effects, I instantiated an EQ Eight effect on each of the Sine tracks in Live, and only enabled the first four filter activators, setting them to the same frequencies as those in the Cubase projects, to simulate an equivalent processing load.
• Levels and routing: All audio outputs are routed to the Master outs. The Cubase VI projects use the MIDI volume fader in the inspector to set the volume of each part in the Orchband multi. Live uses envelopes for this sort of thing, so I used the pink noise mixing method to set the MIDI levels of each of the audible MIDI parts. As mentioned before, all Sine audio tracks were set to -inf, and most importantly, all parts in the Multi2Poly instances of Kontakt were also turned down to -inf- which proved to be a problem during testing.
• Kontakt multis:
• I didn't want to have to go through the laborious process of browsing for the correct samples for each of the VI projects when loading them into Live, so in Cubase, I opened both the CV and NCV versions of the DAW Bench projects and saved the Kontakt multis as "monoliths," which include samples. I saved only one copy of the Multi2Poly multi (since it is identical in both flavors of the DAW Bench VI projects), and separate CV/NCV versions of the Multi1Orchband multi. To ensure that my choice of monolith vs. traditional NKI did not adversely affect my tests, I did do a project built that used the traditional sample loading method. I found that RAM consumption was identical whether you used a monolith or not (Kontakt appears to load all samples for all parts when it is able to do so).
• Upon my first playback of my imported MIDI clips and multis, I discovered something very strange. One of the parts in the audible potion of the project was clearly out of tune, and not playing in the same way that it sounded when playing under Cubase. It took a while to debug, but I determined that the problem was ocurring in part 13 of the Orchband multi. It was a patch named "Mini Lead 2." This Kontakt patch is explicitly designed to be monophonic (to only play one note at a time). Strangely, the original Cubase DAW Bench projects play chords into this track and they are audible as chords. I don't know why this works under Cubase (because it's not supposed to). But basically, the patch's monophonic design was affecting how it interpreted the chords being sent to it under Live, resulting in strange and unexpected behavior. In order to make the project sound the same in Live as it does in Cubase, I had to bypass the Unisono behavior on Mini Lead 2.
  
  Here's a demonstration of the Unisono issue with the Mini Lead 2 part. It's a 16-bar track divided into four 4-bar sections: 1) The track as it's supposed to sound, 2) The Mini Lead 2 part correctly playing, soloed, 3) The track when the Mini Lead 2 part is not working correctly, and 4) The out-of-tune Mini Lead 2 part, soloed.
  
  

The VI CV project on Live 9.

The results

See my upcoming post about the DAW Bench results when I compared Live 9 to Cubase 9. I may also publish these project files at some point so others can do their own tests in Live.