Showing posts with label shootouts. Show all posts
Showing posts with label shootouts. Show all posts

Monday, April 2, 2018

Comparing USB Audio Interface Latency on Windows 10

I may never fully understand my love for audio interfaces. At the end of the day they're just devices that do a better job of recording and playing back music than the built-in audio of most computers, but I almost spend more time experimenting with and writing about them than I do using them to make music. I have done interface performance tests several times before, but that was all on Windows 7 (and mostly on a now-10-year-old DAW PC), so I was recently curious how well some of today's USB audio interfaces perform on a newer computer with the latest version of Windows.


About Audio Interface Latency

Audio interfaces are audio streaming devices, and on modern operating systems all streaming is "buffered" or "packeted." Rather than truly sending a constant binary stream of audio data, your computer bundles up tiny chunks of audio into separate data buffers that are reassembled at the destination end of the stream. This buffering introduces some amount of latency; that is, the fact that the audio data is buffered means that there is a small amount of built-in delay between when the audio data is first transmitted by one device (an audio interface) and received by another (your computer).

There is some amount of latency in both an interface's input and output audio path, and round-trip latency (RTL) is the combination of both of those times. RTL is the metric I tested for: What is the total amount of time an interface takes to send and receive audio given certain settings?

A latency measurement is only meaningful if you know two other values: Sample Rate and Buffer Size. The sample rate is the number of samples per second the audio stream is encoded at, and the buffer size is the number of individual samples included in each streaming buffer.

When you record at high sample rates, your computer processes more audio data, which usually requires larger sample buffers in order to handle audio as reliably as at lower sample rates. The buffer provides protection against glitches (pops and drop-outs in the audio stream), and the harder your computer is working, generally the bigger the buffer you need for glitch-free audio.

The trade-off (and the reason we're studying this at all) is that too big of a buffer at a given sample rate can result in such a great delay that it can become difficult or impossible for a musician to keep in time with the rest of the music while attempting to sing or record. When shopping for audio interfaces, it's good to know which devices offer you the lowest reliable round-trip latencies at given sample rates and buffer sizes.

Not all interfaces perform alike. There are many factors that contribute to interface performance, but the most important appears to be driver quality. A driver written for a specific device with efficiency and optimization in mind can significantly outperform a less optimized driver on similar hardware.

For my tests, I used a free tool called RTL Utility, by Oblique Audio. With this tool, you patch your audio interface's outputs to its own inputs, forming an audio loop, and measure the time it takes for a full output->input round trip at a given sample rate/buffer size.

The Devices Under Test

I have bought and sold many interfaces over the years. These are the ones that I still own and used for these tests.

Behringer U-Control UCA222

The U-Control UCA222 is a very low-cost interface with unbalanced RCA-style analog connectors that's designed more for consumer audio applications than for music production. I bought one so my wife could create digital recordings of her old cassette tapes.

  • Like most Behringer interfaces, the UCA222 does not require any special drivers or software; all modern versions of Windows recognize it as an audio recording/playback device. This also means, however, that there is no native ASIO driver for the interface, so I conducted my tests in "Windows Audio" mode for this device only.
  • The default Windows audio drivers also didn't export all of the traditional "powers of two" sample buffer sizes, so for some tests I had to pick the closest available buffer size for comparison.
  • Despite its low cost, this little interface includes a couple nice features like a physical direct monitor switch and an optical S/PDIF port for digital output.

Focusrite Clarett 2Pre USB

The Clarett 2Pre USB is a desktop interface with high-quality preamps and sophisticated routing/mixing technology. Mine serves as an external DAC for a computer I mostly use for multimedia purposes.

  • While the original Clarett 2Pre was a Thunderbolt-only device. The Clarett 2Pre USB is USB 2.0-only, although its connector is the small USB-C type, normally associated with newer standards. It includes the required USB cables, which is good, considering I did not already own any USB-C cables.
  • From what I've read, this device's USB implementation is the same as what's used in Focusrite's "2nd Generation" Scarlett line of interfaces.
  • While most ASIO Windows drivers offer sample buffer sizes in powers of 2 (64 samples, 128 samples, 256 samples, etc), the Focusrite drivers strangely offer dozens and dozens of selectable sample buffers. Luckily, the common power-of-2 values are among the offered values.

MOTU Track16

The Track16 is a small desktop interface that pushes most of its I/O connections out to a giant proprietary octopus of a breakout cable. On hot summer days, I choose to use this device with headphones rather than powering up my full production rig, which generates a ton of heat.
  • With ADAT, MIDI, and complex mixing/routing software, this device is a strong competitor with the Clarett 2Pre in terms of flexibility and feature set.

Novation Audio Hub 2x4

The Audio Hub 2x4 is a 3-port powered USB hub with Focusrite Scarlett audio interface technology inside. I frequently use this device for recording/sampling audio sound sources, and I use its USB ports to host eLicenser/iLok dongles.
  • The audio interface portion of this device is based on Focusrite's "1st Generation" USB interface technology, which gives us a chance to compare the improvements Focusrite has made with the 2nd generation drivers.
  • While the Audio Hub sports a pair of balanced main outputs, it only has unbalanced RCA-style inputs, so I had to use different audio cables when testing it from the ones I used with the other interfaces.
  • This device is also a little unusual in that it only offers a low/high-gain toggle switch on the inputs rather than adjustable gain pots. 

RME Fireface UFX (1st generation)

The Fireface UFX is a prosumer legend, offering tons of analog and digital I/O with top-tier performance, reliability, and flexibility in a single rack space form factor. This is my primary interface for writing and recording.
  • I own the first generation UFX, which has both USB 2 and Firewire support. I typically use it as a Firewire device, just to avoid possible USB contention in my studio.
  • In recent years, RME has released an updated model, called the Fireface UFX+, which includes USB 3 and Thunderbolt support, and the Fireface UFX II, which is USB 2 only.
  • RME's drivers are a little bit odd in that they don't appear to advertise more than one sample buffer size at a time like most others do. In order to perform the latency tests at different buffer sizes I had to use the RME control panel to choose the new buffer size then "reload" the driver in the test tool before performing each round of tests.
  • Since this is the only interface I currently own that supports two different data buses, I tested it both as a Firewire and as a USB device. 

Roland Duo-Capture EX

The Duo-Capture EX is one of my favorite entry-level interfaces, just because it packs a lot of features into a small, reliable and affordable package. I don't actively use my Duo-Capture these days, but I keep it around for testing and ad-hoc stuff.
  • The Roland drivers are kind of weird in that they do not use powers-of-two buffer sizes, and their configuration control panel also has a number of non-standard options and metrics. I had to choose the nearest approximate buffer size for the comparative tests. Also, for all control panel settings other than buffer size, I just left them at their factory defaults.
  • On Windows 10, the Duo-Capture driver installs automatically (no separate discs or downloads from Roland required), however I experienced a lot of driver instability on my first round of latency tests with this interface after the drivers installed (some tests straight-up failed while others took significantly longer or shorter to complete than expected). I rebooted my computer and re-tested the interface and didn't experience any instability at all, so that appeared to be a temporary issue.

Roland Tri-Capture

The Tri-Capture is an odd little device with an interesting combination of features at a low price. I have used this for recording/sampling from consumer audio devices.
  • Like with the Duo-Capture EX, the Tri-Capture's driver has some unusual options and non-standard sample buffer sizes. I note the differences in the test results.
  •  

Test Setup and Method

For this round of tests I used my current DAW PC on the most recent build of 64-bit Windows 10. The current specs of this system:
  • CPU: Intel Core i7-5930K @ 3.5GHz
  • Motherboard: ASUS X99-A/USB 3.1 ATX
  • RAM: 32GB of DDR4 2133 MT/s
In order to keep the playing field level, I used the same USB and audio cables for all tests, where possible. (Some devices had special I/O connectors that required different cables.) I ran multiple tests at each sample buffer size for each device and selected the best/lowest test results from each device/buffer size combination.

In cases where the test results seemed surprising or unexpected, I re-ran all tests on the device in question to ensure results were consistent and reproducible.

The purpose of these tests was only to determine the measurable RTL of each device at each sample buffer size; not to establish the most reliable low-latency sample rate/buffer size combo under heavy DSP loads. (See the official DAW Bench tests for that sort of thing.)

Test Results 

I ran latency tests at four of the most common buffer sizes, all at a sample rate of 44.1kHz. Some quick notes before we get to the raw data:
  • Not all interface drivers support powers-of-two sample buffer scaling. In those cases I've added a * by the name of the device in the tables and graphs. Here are the exceptions:
    • For the Duo-Capture EX I had to use 144, 288, and 576 samples in the 128, 256, and 512 samples tests, respectively. This put that interface at a slight disadvantage for each test.
    • For the Tri-Capture I had to use 288 and 576 samples in the 256 and 512 samples tests, respectively. This also put that interface at a slight disadvantage for each test.
    • For the U-Control UCA222, I had to pick 132 samples for the 128 samples test. (The other common sample rates were available.)
  • The drivers of some of the devices did not offer as many buffer size options as others. This is why the 64-samples and 128-samples tests don't include scores for all eight interfaces. The interfaces that didn't support all testable sample rates were the Duo Capture EX, the Tri-Capture, and the UCA222.
  • Each test lists two scores for the RME Fireface UFX: One as a USB device, and one as a Firewire device.
  • The U-Control UCA222 was the only interface that didn't have ASIO drivers, so I tested it as a Windows Audio device.
  • All test results are reported as round-trip time in milliseconds. Lower scores/shorter bars are better.

64 Samples @ 44.1kHz

128 Samples @ 44.1kHz


256 Samples @ 44.1kHz




512 Samples @ 44.1kHz



Observations/Summary

  • True to its reputation, the Fireface UFX performed the best in all tests, with its USB mode slightly scoring better than its FW mode every time. (The USB and FW scores were always within 1ms of each other.)
  • Even though its larger-than-average sample buffer sizes put it at a slight disadvantage in every test, the Roland Duo-Capture EX fared rather well, capturing the third-best score in every round of tests where it competed.
  • Despite having very similar-seeming driver and control panel to the Duo-Capture EX, the Tri-Capture lagged behind the other Roland interface in both tests where it competed.
  • Considering that the Audio Hub 2x4 is really a Focusrite Scarlett interface in a Novation-branded box, its scores clearly demonstrate the poor performance of that generation of Scarlett interfaces on Windows. It scored significantly worse than most other interfaces on all tests but the last- and that one was a surprise...
  • ...which was the odd performance results of the Clarett 2Pre USB. In the 64-samples and 128-samples tests the Clarett performs reasonably well, at least when compared to the Audio Hub. (In the 64-sample test the Clarett's round-trip latency is almost one third of the Audio Hub's score.) However as the sample buffer sizes increased, so did the Clarett's relative latency, gradually closing- AND THEN PASSING- the gap with the Audio Hub, making the Clarett the worst-performing interface in the 512-samples tests.
  • When I was reviewing the test results before I began writing this article, I was so surprised by the Clarett's scores that I re-connected it to my test system and ran all of the tests on the Clarett again- only to find that they were the same. In both rounds of tests on this device, its round-trip latency grew progressively worse (relative to other interfaces) as the sample buffer size increased. This may point to some inefficient code in the current driver that is exacerbated as sample buffers grow.

Sunday, October 15, 2017

Porting DAW Bench Projects to Ableton Live

Vin Curigliano's DAW Bench is the standard toolset for measuring digital audio workstation hardware and software performance. While the current DAW Bench suite includes projects for several different DAWs, Ableton Live isn't one of them- which was a problem for me, since I was planning to investigate the performance differences between Live and Cubase for an upcoming post. Rather than ditch DAW Bench and go to the effort of developing my own test suite from the ground up, I chose to port the DAW Bench Cubase sessions to Ableton Live. What follows is a detailed analysis of the original DAW Bench projects, and my account of the surprises and challenges I encountered when porting those projects to Live.



When I first began this effort I thought I was the first person to do such a thing, but I eventually discovered user on a music forum who did his own attempt at a Live port a few months back. His goal was different from mine, and his projects weren't direct ports (he only replicated DAW Bench's "DSP" tests, and he chose a different set of plugins for his projects from the ones used in the official tests), so I continued with my own effort.

It is October 2017 as I'm writing this, so my observations here are based on the 2017 edition of the DAW Bench DSP and VI projects.

Analyzing the DSP Projects

Each DAW Bench DSP project contains hundreds of instances of a given CPU-intensive effects plugin, simulating a very complex mix. The "score" of a DSP test is the number of plugin instances you can enable without encountering any glitches in the audible audio.
  • There are four audible tracks of pre-recorded audio, comprising the loop that plays while you enable FX plugins, listening for pops and drops. There are no effects plugins on these tracks.
  • There is another "monitor" track which is just a recording of a sine wave. I don't know its purpose, but its track volume is turned all the way down.
  • Next there are 40 more tracks of sine wave recordings, each with eight instances of the plugin under test filling the track's insert slots. Each instance is disabled when you first load the project. Track volume for each of these tracks is also turned down all the way.
  • All tracks are routed to the main stereo outs, but only the first four audio tracks are audible due to their volume settings.
  • There are three different DSP projects, one for each of these free-to-use effects plug-ins: SGA1566 (Shattered Glass Audio), MJUC jr. (Klanghelm), and a special version of ReaXcomp (Cockos) which is actually included with the DSP project download.
The DSP-1566 project on Cubase 9.

Analyzing the VI Projects

While the DSP projects are all about effects plugins, the VI tests only use instances of a virtual instrument plugin: Native Instruments Kontakt 5. The "score" of a VI test is the number of individual Kontakt "voices" you can have playing simultaneously before encountering audio glitches.
  • The project contains one "Multi1Orchband" instance of the Kontakt plugin that contains a 16-part multi (one part per MIDI channel) of different Kontakt instruments that comprise most of the audible content in the project.
  • Next there are ten "Multi2Poly" instances of Kontakt, each with 16 instances of the same instrument: A layered pad patch named "Light Breaks Through." Due to the patch's two sample layers, a single MIDI note played with this patch uses two Kontakt voices/oscillators. As with the sine tracks in the DSP projects, each instrument part in the Multi2Poly multis is turned down all the way. More on this later.
  • The project also contains 16 MIDI tracks, each routed to a different MIDI channel in the Multi1Orchband Kontakt instance.
  • Finally there are ten folders containing 16 MIDI tracks each, each one routed to its respective MIDI channel in one of the ten Multi2Poly instances. The MIDI tracks are simple clips that play a 16-bar sustained 10-note chord. So at play time, a single Multi2Poly instance can have up to 320 voices going at a time (16 parts of a 2-voice patch playing 10 notes each).
  • There are two flavors of VI projects, "CV" and "NCV." In the CV project. many of the instruments in the "Multi1Orchband" multi have convolution reverb enabled as an insert effect, while that effect is bypassed in the NCV project. Both projects use convolution reverb as a send effect on a few channels. Importantly, the Multi2Poly multi is identical between the CV and NCV versions of the project (no reverb effect actively enabled as an insert or a send), so the results between CV and NCV tests are generally not very significant.
  • The VI projects in DAW Bench were initially developed using Kontakt 4, whose factory library had a different layout from the library in Kontakt 5. While the current generation of the DAW Bench projects uses the Kontakt 5 plugin, the multis still use samples from the Kontakt 4 library. I am a licensed owner of Kontakt 4 but I do not currently have that version of the plugin installed anywhere, so I keep a folder containing all the necessary samples on my hard drive for when I need to do tests.
The VI CV project on Cubase 9.

Porting the DSP Projects

  • Session view vs Arrangement view: Cubase is a traditional "piano roll" DAW while Live offers both a piano-roll style Arrangement view and the clip/loop-based Session view. I chose to build my projects in Session view because of the looping nature of the tests (it's just a few measures of repeating audio, after all), and because this view offers better access to the plugin slots for every track (after some UI resizing, at least).
  • Audio tracks: The DAWBench DSP 2017 folder includes an "Audio" folder that contains the pre-recorded audio content (including sampled sine waves) used by these projects, which I easily imported into Live as stems. I grouped the tracks in the same order as the folders used in the Cubase projects.
  • Plugin presets: All instances of the effects plugins in these projects use identical settings, which is important, because some plugin presets can be more CPU-intensive than others. Since plugin settings are embedded in the DAW project, I didn't have preset files I could import into Live, so I noted all of the plugin values used in the Cubase projects and created presets matching those settings for each of the plugins in Live.
  • Plugin inserts: Just like with the Cubase projects, I inserted 8 instances of the desired effects plugin on each of the sine tracks. I enabled display of the insert slots in Session view and resized it so all 8 slots were visible at all times.
  • Levels and routing: All tracks are routed to Live's Master outs, but all of the "Sine" tracks are set at -inf volume.
The DSP-1566 project on Live 9.

Porting the VI Projects

  • MIDI tracks: For the MIDI parts that play into the "Multi1Orchband" instance of Kontakt, I dragged all of the MIDI clips out of Cubase and imported them into Live as MIDI clips. There were 16 MIDI clips for the Orchband multi, and another 16-bar MIDI clip for the polyphonic Sine tracks. This clip simply played a 10-note chord for a full 16 measures. Note: Due to an issue I discovered while performing benchmark testing, I discovered that the original 16-bar Poly tracks could not be used for reliable testing. (I will go into more detail on this in my next post.) Because of this I also created a 1-bar version of the Poly track that plays a sustained chord for the first seven 8ths of the measure. Session view allows me to add both the 1-bar clip and the 16-bar clip to every poly track and switch between them at will. Because of this I also edited separate copies of the original Cubase VI projects to use the same 1-bar loop.
  • Audio tracks: The VI projects include one audible pre-recorded audio track (a 2-bar drum loop to accompany the Orchband parts), and 16 sine wave tracks whose volume is turned to -inf/silent. I presume these additional tracks are to help simulate a real music project that is handling both virtual instruments and audio tracks.
  • EQ: In Cubase, each of the Sine audio tracks had channel EQ enabled, although it wasn't applying any boosts or cuts. This is not a default track setting, however, so I figured it was intentional. Since Live doesn't have built-in track effects, I instantiated an EQ Eight effect on each of the Sine tracks in Live, and only enabled the first four filter activators, setting them to the same frequencies as those in the Cubase projects, to simulate an equivalent processing load.
  • Levels and routing: All audio outputs are routed to the Master outs. The Cubase VI projects use the MIDI volume fader in the inspector to set the volume of each part in the Orchband multi. Live uses envelopes for this sort of thing, so I used the pink noise mixing method to set the MIDI levels of each of the audible MIDI parts. As mentioned before, all Sine audio tracks were set to -inf, and most importantly, all parts in the Multi2Poly instances of Kontakt were also turned down to -inf- which proved to be a problem during testing.
  • Kontakt multis:
    • I didn't want to have to go through the laborious process of browsing for the correct samples for each of the VI projects when loading them into Live, so in Cubase, I opened both the CV and NCV versions of the DAW Bench projects and saved the Kontakt multis as "monoliths," which include samples. I saved only one copy of the Multi2Poly multi (since it is identical in both flavors of the DAW Bench VI projects), and separate CV/NCV versions of the Multi1Orchband multi. To ensure that my choice of monolith vs. traditional NKI did not adversely affect my tests, I did do a project built that used the traditional sample loading method. I found that RAM consumption was identical whether you used a monolith or not (Kontakt appears to load all samples for all parts when it is able to do so).
    • Upon my first playback of my imported MIDI clips and multis, I discovered something very strange. One of the parts in the audible potion of the project was clearly out of tune, and not playing in the same way that it sounded when playing under Cubase. It took a while to debug, but I determined that the problem was ocurring in part 13 of the Orchband multi. It was a patch named "Mini Lead 2." This Kontakt patch is explicitly designed to be monophonic (to only play one note at a time). Strangely, the original Cubase DAW Bench projects play chords into this track and they are audible as chords. I don't know why this works under Cubase (because it's not supposed to). But basically, the patch's monophonic design was affecting how it interpreted the chords being sent to it under Live, resulting in strange and unexpected behavior. In order to make the project sound the same in Live as it does in Cubase, I had to bypass the Unisono behavior on Mini Lead 2.

      Here's a demonstration of the Unisono issue with the Mini Lead 2 part. It's a 16-bar track divided into four 4-bar sections: 1) The track as it's supposed to sound, 2) The Mini Lead 2 part correctly playing, soloed, 3) The track when the Mini Lead 2 part is not working correctly, and 4) The out-of-tune Mini Lead 2 part, soloed.

The VI CV project on Live 9.

The results

See my upcoming post about the DAW Bench results when I compared Live 9 to Cubase 9. I may also publish these project files at some point so others can do their own tests in Live.

Tuesday, September 5, 2017

Comparing DAW Performance of Recent Cubase Versions on Windows

I recently had to increase the buffer size setting on my audio interface to eliminate audio glitches in a music project on Cubase Pro 9. Since I almost never have to adjust my interface settings while producing a track, I wondered if perhaps Cubase 9 wasn't performing as well as previous Cubase versions I'd worked with. I searched around for some performance information, but I couldn't find any detailed, up-to-date comparisons of recent Cubase versions- so I decided to do my own.


Cubase versions under test

I decided to test the most recent available 64-bit versions of the last four major Cubase releases:
  • Cubase 6.5 - Originally released February 2012, the latest version is 6.5.5 from June 24, 2013.
  • Cubase 7.5 - Originally released December 2013, the latest version is 7.5.40 from Jan 19, 2015.
  • Cubase Pro 8.5 - Originally released December 2015, the latest version is 8.5.30 from Feb 22, 2017.
  • Cubase Pro 9.0 - Originally released December 2016, the latest version is 9.0.30 from July 20, 2017.
In addition to comparing basic performance of individual Cubase releases, I also wanted to examine the effects of ASIO-Guard, a feature Steinberg introduced with Cubase 7. By using smart management of CPU time and audio buffers, ASIO-Guard claims to increase the amount of plugins you can run without encountering audio glitches. Steinberg claims to have made improvements to ASIO-Guard over time, so I wanted to see how the feature had changed.

DAW Bench and Test Preparation

I installed each version side-by-side on my PC, patched them with the latest updates, and then downloaded the 2017 versions of the DAW Bench test projects. In case you're not familiar with DAW Bench, it's a collection of DAW projects assembled by audio professional Vin Curigliano to assess a digital audio workstation's ability to reliably produce audio while operating under heavy DSP workloads. When a computer's DSP resources are exhausted, audio suffers, with pops, drop-outs, and strange digital artifacts. Many factors contribute to a DAW system's ability to perform well: CPU, chipsets, drivers, operating system, DAW software, and audio interfaces all play a role.

The current iteration of DAW Bench includes five different Cubase test projects, broken into two categories.

  • The DSP projects contain some basic audio tracks with literally hundreds of instances of a specific effect loaded up on various tracks. These push your computer's computational digital signal processing capabilities to its limits. The "score" for a DSP test is the number of plugin instances that can be activated without glitching the audio.
  • The VI projects use instances of Native Instruments Kontakt to test your computer's virtual instrument oscillation/voice generation abilities by playing from hundreds to thousands of simultaneous notes of polyphony. The "score" for a VI test is the number of musical notes that can play simultaneously without glitching the audio.

Each of the DSP projects uses a different freely-available effects plugin:
  • DSP-1566 uses Shattered Glass Audio's SGA1566, which is a CPU-intensive emulation of a vintage tube amplifier.
  • DSP-MJUC uses Klanghelm's MJUC jr., a "variable-mu" compressor plugin.
  • DSP-REAX uses a specially-compiled version of Cockos ReaXcomp, a multi-band compressor. (Note: The correct version is included in the DAW Bench download, don't use the one from the Reaper site.)
There are also two flavors of the VI tests: The "VI-CV" tests use Kontakt's internal convolution reverb effect (using more DSP power), while the "VI-NCV" tests have no reverb enabled.

I performed the tests on my primary DAW PC. The full specs of the system are published elsewhere, but here's the pertinent information:
  • Processor: Intel i7 5930K @ 3.50GHz (6 physical cores)
  • RAM: 32GB
  • Video: NVIDIA GeForce GTX 960
  • Operating System: Windows 7 Professional SP-1, 64-bit
  • Audio Interface: RME FireFace UFX, in FireWire mode
  • Interface Settings: 44.1kHz, 256 samples.
  • Windows Optimization: The only Windows performance tweak I made was to select the High Performance power scheme in the Power Options control panel and to disable some unneeded startup processes and services. I have not adjusted any of the more arcane Windows settings such as the MMCSS options.
  • Cubase Optimization: In all of my tests I use the default Cubase performance settings, with the obvious exception of disabling/enabling ASIO-Guard for a specific round of tests. So this means I'm leaving Audio Priority to Normal, Activate Multi Processing is checked, Activate Steinberg Audio Power Scheme is unchecked (I'm using the built-in Windows High Performance scheme), and on versions of Cubase that offer various "ASIO-Guard Level" settings, I'm using the normal level.

DSP Test Results

The results of the DSP tests are below. For versions of Cubase with the ASIO-Guard features, separate scores are shown with the feature disabled ("no AG") or enabled ("AG"). Cubase 6.5 is the only tested version which lacks that feature.

DSP Test Raw Data (44.1kHz, 256 samples)
DSP Test Chart
The results weren't very dramatic, however they did show modest gains for the ASIO-Guard feature- particularly for versions 7.5 and 9.0. Cubase 8.5 with ASIO-Guard enabled scored the best for 2 out of 3 tests while Cubase 9.0 with ASIO-Guard disabled scored lowest in all three tests.

VI Test Results

The virtual instrument tests were a little more interesting. In the tests below, "VI-CV" are with Kontakt's convolution reverb effect enabled, while reverb is disabled in the "VI-NCV" tests.

VI Test Raw Data (44.1kHz, 256 samples)

VI Test chart

A few things stand out in these tests:
  • First, ASIO-Guard made dramatic improvements in both Cubase 8.5 and Cubase 9.0, while their ASIO-Guard gains weren't quite as impressive in the DSP tests. The feature shows a clear and demonstrable benefit, at least for some plugin duties.
  • The improvement ASIO-Guard made on Cubase 7.5 was much less impressive, and I am guessing it's because the Cubase 7.x implementation of ASIO-Guard did not fully support multi-timbral plugins such as Kontakt.
  • It was also interesting that while Cubase 9.0-with-ASIO-Guard gained the second-highest score in the test, without ASIO-Guard, Cubase 9.0 scored the lowest on these tests. Cubase 8.5 scored significantly higher, in both the ASIO-Guard enabled and disabled tests.

Final scores

I wanted to be able to rank individual Cubase versions in terms of performance, but I didn't want the VI tests to skew the numbers (since the VI scores reach up to the thousands while the DSP scores are all down in the low hundreds). In order to give each test equal weight, I divided the VI test scores by 10, and then I summed all 5 test scores for each DAW and ASIO-Guard setting.

Final Scores (raw)
Final Scores (chart)
With these adjusted performance scores, it appears that Cubase 8.5 with ASIO-Guard is the best-performing version of Cubase in recent years, although Cubase 9.0 still performs very well in second place so long as ASIO-Guard is enabled. However with ASIO-Guard disabled, Cubase 9.0 is the worst-performing version of Cubase of the versions tested.

Conclusions

  • Cubase 9 performs slightly worse than Cubase 8.5, given the same content and settings on the same system. Without ASIO-Guard, Cubase 9 performed about 4 percent worse than 8.5. With ASIO-Guard there was only around a 2 percent difference.
  • Core Cubase performance (without ASIO-Guard) has not changed significantly over time. With Cubase 7.5 and 8.5 performing around 1 percent better than Cubase 6.5 and Cubase 9.0 performing nearly 3% worse, Cubase has delivered more or less consistent performance across major releases.
  • ASIO-Guard can make a big difference, but it depends on the specific plugins and workload. Both Cubase 8.x and 9.x saw huge gains in the VI tests with ASIO-Guard enabled, but the gains were less impressive in the DSP tests.
  • Cubase makes very good use of multi-core processors and hyper-threading (*). Not all Windows DAWs handle modern CPUs the same, but Cubase has, for some time, been quite good at making use of both physical and logical CPU resources to deliver reliable audio under heavy DSP loads. Here's a screenshot of Windows Task Manager while Cubase is performing one of the DSP tests covered earlier. Every logical core of my i7 5930K is working at the maximum allowed by the Windows MMCSS settings (which reserve 20% of CPU power for background tasks). I plan to explore this stuff a little more in future posts.
(*) As of the time of this writing there is a known issue with Cubase on Windows 10 where Windows imposes a thread limit that can result in audio instability on CPUs with more than 14 logical cores (or more than 7 physical cores). For now, Steinberg recommends using Windows 8.1 or earlier for top Cubase performance on CPUs that exceed 14 logical cores, or using workarounds on Windows 10 which are documented at the above link to at least avoid the audio glitches resulting from this limitation.

Sunday, October 12, 2014

Native Instruments Acoustic Pianos Compared

Native Instruments has produced sample-based acoustic piano instruments in one form or another for quite some time. Because the company offers so many piano options, I frequently see people asking which ones are best, or what the differences are between the various piano products. I hope to answer most of those questions here. I'll also provide some audio demos of every current and recent Native Instruments piano- so you can hear the differences for yourself.



AKOUSTIK PIANO

In 2006, Native Instruments released Akoustik Piano, an instrument that included sampled versions of four different pianos. While it used the Kontakt engine under the hood, Akoustik Piano could run as a standalone application or a plugin. Native Instruments discontinued this product in 2009, however the sample content was reused in several ways in subsequent products.

You can learn more about Akoustik Piano and other legacy Native Instruments products at my Complete History of Komplete.

KONTAKT FACTORY LIBRARY

Kontakt ships with several basic pianos. They don't have fancy user interfaces, but they offer most of the standard tweakable parameters- and a number of not-so-standard parameters for pianos. Some of the piano samples in the Kontakt factory library come from Akoustik Piano, but the library also includes some piano sounds that don't appear anywhere else.

The Kontakt pianos are split across the Band and Orchestral categories, which have different feature sets. Despite their basic appearance these pianos do feature multiple velocity and microphone layers. Most of these pianos also have a very small memory footprint, partly because they are not one-sample-per-note.
  • Features (Band instruments): Mic and resonance control, compression, chorus, delay, convolution reverb, amp simulation, EQ, velocity curve, pitch bend range, transpose, tuning, parameter randomization.
  • Features (Orchestral instruments): EQ, tuning/scale, convolution reverb, velocity curve, pitch bend range, transpose, parameter randomization.
  • Presets/variations: None of the Kontakt acoustic pianos include presets or variations, although they are highly customizable.
  • Available in: Komplete 10, Komplete 10 Ultimate, Kontakt 5.

August Foerster Grand (Orchestral)


  • Original piano: Unknown model manufactured by August Foerster
  • Preset or NKI file used in sound demo: August Foerster Grand.nki
  • Sample data size: 0.38 GB
  • Sample rate/bit-depth: 44.1kHz/16-bit
  • Number of groups/zones: 16/605
  • Velocity layers: 10
  • Minimum Kontakt version required: 5.0.0
  • Notes: This is one of the oldest pianos in the Native Instruments catalog. Its default maximum voice setting was only 64, which was simply not enough voices for this sample clip, which includes a lot of sustan (making it a rather voice-hungry composition). I doubled the voice count to 128 for the demo recording.
  • Sound demo: Listen

Concert Grand (Orchestral)


  • Original piano: Steinway D
  • Preset or NKI file used in demo: Concert Grand.nki
  • Sample data size: 0.82 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 93/985
  • Velocity layers: 10
  • Minimum Kontakt version required: 5.0.0
  • Notes: This is a "lite" version of the New York Concert Grand.
  • Sound demo: Listen

Grand Piano (Band)


  • Original piano: Boesendorfer 290 Imperial
  • Preset or NKI file used in demo: Grand Piano.nki
  • Sample data size: 0.16 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 6/155
  • Velocity layers: 3
  • Minimum Kontakt version required: 5.0.0
  • Notes: This is a "lite" version of the Vienna Concert Grand.
  • Sound demo: Listen

Ragtime Piano (Band)

  • Original piano: Steingraeber 130
  • Preset or NKI file used in demo: Ragtime Piano.nki
  • Sample data size: 0.17 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 6/189
  • Velocity layers: 3
  • Minimum Kontakt version required: 5.0.0
  • Notes: This uses the same samples as Upright Piano.
  • Sound demo: Listen

Upright Piano (Band)


  • Original piano: Steingraeber 130
  • Preset or NKI file used in demo: Upright Piano.nki
  • Sample data size: 0.17 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 6/189
  • Velocity layers: 3
  • Minimum Kontakt version required: 5.0.0
  • Notes: This is a "lite" version of the Upright Piano from the Classic Piano Collection.
  • Sound demo: Listen


CLASSIC PIANO COLLECTION

From 2010 through 2014, these pianos were available for separate purchase or as a bundle, but they were dropped from Native Instruments' product lineup at the time of Komplete 10's release. These all take sample content from Akoustik Piano, like some of the instruments in the Kontakt library do- although the factory library versions are more compact and do not ship with any presets/variations. These versions utilize more individual samples than the Kontakt library versions, explaining their significantly larger memory footprint.
  • Features: Mic and resonance control, pedal style, velocity curve, tuning, convolution reverb, lid position.
  • Presets/variations: While the individual instruments do not use snapshots or have any built-in preset facility, each of these pianos includes several different NKI files, each representing different variations.
  • Available in: Komplete 9 and Komplete 9 Ultimate.

Berlin Concert Grand


  • Original piano: Bechstein D 280
  • Preset or NKI file used in demo: Berlin Concert Grand.nki
  • Sample data size: 2.56 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 93/984
  • Velocity layers: 10
  • Minimum Kontakt version required: 5.0.1
  • Notes: This is the only one of the Classic Piano Collection pianos that doesn't appear to have any representation in the Kontakt 5 factory library.
  • Sound demo: Listen

New York Concert Grand


  • Original piano: Steinway D
  • Preset or NKI file used in demo: New York Concert Grand.nki
  • Sample data size: 2.15 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 93/985
  • Velocity layers: 10
  • Minimum Kontakt version required: 4.8.0
  • Sound demo: Listen

Upright Piano


  • Original piano: Steingraeber 130
  • Preset or NKI file used in demo: Upright Piano.nki
  • Sample data size: 2.49 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 93/991
  • Velocity layers: 10
  • Minimum Kontakt version required: 5.0.1
  • Sound demo: Listen

Vienna Concert Grand


  • Original piano: Boesendorfer 290 Imperial
  • Preset or NKI file used in demo: Vienna Concert Grand.nki
  • Size in RAM: 141 MB
  • Sample data size: 2.96 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 102/1071
  • Velocity layers: 10
  • Minimum Kontakt version required: 5.0.1
  • Sound demo: Listen


DEFINITIVE PIANO COLLECTION

Introduced in 2014, these pianos are considered a higher-quality replacement for the "Classic" models that have since been dropped from the library, although they represent a different set of original pianos. These were produced by Galaxy Instruments, the creators of another Native Instruments piano, The Giant.
  • Features: EQ, transient shaping, compression, mic and resonance control, pedal style, velocity curve, tuning, convolution reverb, lid position.
  • Available in: Komplete 10, Komplete 10 Ultimate, Definitive Piano Collection.
  • Presets/variations: All of these pianos include different variations, but rather than using a built-in preset selector, these instruments use Kontakt's snapshot feature. To choose a preset you click the Snapshot button and then select the desired preset from the list:

The Gentleman


  • Original piano: Bechstein Model A Upright (1908)
  • Preset or NKI file used in demo: Basic Gentleman
  • Sample data size: 5.70 GB
  • Sample rate/bit-depth: 48kHz/24-bit
  • Number of groups/zones: 105/3148
  • Velocity layers: 16
  • Minimum Kontakt version required: 5.4.1
  • Sound demo: Listen

The Grandeur


  • Original piano: Hamburg Steinway D Concert Grand
  • Preset or NKI file used in demo: Basic Grandeur
  • Sample data size: 7.16 GB
  • Sample rate/bit-depth: 48/24-bit
  • Number of groups/zones: 105/3358
  • Velocity layers: 18
  • Minimum Kontakt version required: 5.4.1
  • Sound demo: Listen

The Maverick


  • Original piano: Bechstein Model A Grand (1905)
  • Preset or NKI file used in demo: Basic Maverick
  • Sample data size: 7.55 GB
  • Sample rate/bit-depth: 48kHz/24-bit
  • Number of groups/zones: 105/3282
  • Velocity layers: 18
  • Minimum Kontakt version required: 5.4.1
  • Sound demo: Listen


OTHER RELEASES

These items are all available as separate products.

Alicia's Keys


  • Original piano: Yamaha C3 Neo
  • Preset or NKI file used in demo: 01 Dry
  • Sample data size: 6.92 GB
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 46/6200
  • Velocity layers: ~12 (10-15 depending on the aspect of the piano)
  • Minimum Kontakt version required: 4.8.0
  • Developed by: Scarbee
  • Features: Mic and resonance control, pedal style, velocity curve, convolution reverb, attack/release adjust, repetition control.
  • Presets/variations: Preset selector in UI.
  • Available in: Komplete 10 Ultimate or separately.
  • Sound demo: Listen

Discovery Series: Cuba

This package includes two piano variations. At the time of writing, this is the only piano instrument in the Native Instruments lineup that has a groove/pattern playback system built-in. It is not very deeply sampled, but has a distinct and useful sound.

  • Features: EQ, compression, tape saturation, reverb, groove/pattern support.
  • Presets/variations: User interface has separate Mixer and Instrument preset selectors.
  • Available in: Komplete 10 Ultimate or separately.

GRAND PIANO

  • Original piano: unknown (possibly a Kawai KG-1C?)
  • Preset or NKI file used in demo: Grand Piano / Classic mixer
  • Sample data size: 6.92 GB (shared with upright piano)
  • Sample rate/bit-depth: 44.1kHz/24-bit
  • Number of groups/zones: 20/749 (shared with upright piano)
  • Velocity layers: 2
  • Minimum Kontakt version required: 5.4.1
  • Sound demo: Listen

UPRIGHT PIANO

  • Original piano: Steingraeber 130
  • Preset or NKI file used in demo: Upright Piano / Classic mixer
  • Sample data size: 0.57 GB (shared with grand piano)
  • Sample rate/bit-depth: 44.1kHz/16-bit
  • Number of groups/zones: 20/749 (shared with grand piano)
  • Velocity layers: 6
  • Minimum Kontakt version required: 5.4.1
  • Notes: This uses the same samples as the Upright Piano from the Kontakt Factory Library, although they have been converted to 16-bit.
  • Sound demo: Listen

The Giant


  • Original piano: Klavins Model 370
  • Preset or NKI file used in demo: Basic Giant
  • Sample data size: 3.49 GB
  • Sample rate/bit-depth: 48kHz/24-bit
  • Number of groups/zones: 98/2003
  • Velocity layers: 13
  • Minimum Kontakt version required: 5.4.1
  • Developed by: Galaxy Instruments
  • Features: EQ, transient shaping, compression, mic and resonance control, pedal style, velocity curve, tuning, convolution reverb.
  • Presets/variations: Preset selector in UI.
  • Available in: Komplete 10, Komplete 10 Ultimate, or separately.
  • Sound demo: Listen


ABOUT THE SOUND DEMOS

For each of the pianos described above, I recorded a minute-long excerpt of Felix Mendelssohn's Songs Without Words, Book 2, Op. 30: No. 1 Andante espressivo. This piece makes liberal use of pedal/sustain and is fairly dynamic, velocity-wise. The full playlist of clips follows, and I'll also provide links to individual clips in the descriptions below.

All clips were loudness normalized to -20 LUFS, and there is no processing or effects outside of what was already included with the instrument. For every example, I tried to use each instrument's "default" or "basic" setting. In the detailed instrument descriptions above, I list the presets used for each of the sample recordings. In cases where the instrument doesn't have a preset or snapshot facility, I simply list the NKI file I used, since Native Instruments sometimes includes separate NKI files as "presets."

SOUND DEMOS



CREDITS

Thanks to the following folks:
  • nielsdolieslager at the Native Instruments forum for filling me in on some details and recommending I check the voice count on the August Foerster piano.
  • kbaccki at the KVR forum for inspiring me to find a better metric to judge sample content size than the active RAM buffer.