Monday, April 2, 2018

Comparing USB Audio Interface Latency on Windows 10

I may never fully understand my love for audio interfaces. At the end of the day they're just devices that do a better job of recording and playing back music than the built-in audio of most computers, but I almost spend more time experimenting with and writing about them than I do using them to make music. I have done interface performance tests several times before, but that was all on Windows 7 (and mostly on a now-10-year-old DAW PC), so I was recently curious how well some of today's USB audio interfaces perform on a newer computer with the latest version of Windows.


About Audio Interface Latency

Audio interfaces are audio streaming devices, and on modern operating systems all streaming is "buffered" or "packeted." Rather than truly sending a constant binary stream of audio data, your computer bundles up tiny chunks of audio into separate data buffers that are reassembled at the destination end of the stream. This buffering introduces some amount of latency; that is, the fact that the audio data is buffered means that there is a small amount of built-in delay between when the audio data is first transmitted by one device (an audio interface) and received by another (your computer).

There is some amount of latency in both an interface's input and output audio path, and round-trip latency (RTL) is the combination of both of those times. RTL is the metric I tested for: What is the total amount of time an interface takes to send and receive audio given certain settings?

A latency measurement is only meaningful if you know two other values: Sample Rate and Buffer Size. The sample rate is the number of samples per second the audio stream is encoded at, and the buffer size is the number of individual samples included in each streaming buffer.

When you record at high sample rates, your computer processes more audio data, which usually requires larger sample buffers in order to handle audio as reliably as at lower sample rates. The buffer provides protection against glitches (pops and drop-outs in the audio stream), and the harder your computer is working, generally the bigger the buffer you need for glitch-free audio.

The trade-off (and the reason we're studying this at all) is that too big of a buffer at a given sample rate can result in such a great delay that it can become difficult or impossible for a musician to keep in time with the rest of the music while attempting to sing or record. When shopping for audio interfaces, it's good to know which devices offer you the lowest reliable round-trip latencies at given sample rates and buffer sizes.

Not all interfaces perform alike. There are many factors that contribute to interface performance, but the most important appears to be driver quality. A driver written for a specific device with efficiency and optimization in mind can significantly outperform a less optimized driver on similar hardware.

For my tests, I used a free tool called RTL Utility, by Oblique Audio. With this tool, you patch your audio interface's outputs to its own inputs, forming an audio loop, and measure the time it takes for a full output->input round trip at a given sample rate/buffer size.

The Devices Under Test

I have bought and sold many interfaces over the years. These are the ones that I still own and used for these tests.

Behringer U-Control UCA222

The U-Control UCA222 is a very low-cost interface with unbalanced RCA-style analog connectors that's designed more for consumer audio applications than for music production. I bought one so my wife could create digital recordings of her old cassette tapes.

  • Like most Behringer interfaces, the UCA222 does not require any special drivers or software; all modern versions of Windows recognize it as an audio recording/playback device. This also means, however, that there is no native ASIO driver for the interface, so I conducted my tests in "Windows Audio" mode for this device only.
  • The default Windows audio drivers also didn't export all of the traditional "powers of two" sample buffer sizes, so for some tests I had to pick the closest available buffer size for comparison.
  • Despite its low cost, this little interface includes a couple nice features like a physical direct monitor switch and an optical S/PDIF port for digital output.

Focusrite Clarett 2Pre USB

The Clarett 2Pre USB is a desktop interface with high-quality preamps and sophisticated routing/mixing technology. Mine serves as an external DAC for a computer I mostly use for multimedia purposes.

  • While the original Clarett 2Pre was a Thunderbolt-only device. The Clarett 2Pre USB is USB 2.0-only, although its connector is the small USB-C type, normally associated with newer standards. It includes the required USB cables, which is good, considering I did not already own any USB-C cables.
  • From what I've read, this device's USB implementation is the same as what's used in Focusrite's "2nd Generation" Scarlett line of interfaces.
  • While most ASIO Windows drivers offer sample buffer sizes in powers of 2 (64 samples, 128 samples, 256 samples, etc), the Focusrite drivers strangely offer dozens and dozens of selectable sample buffers. Luckily, the common power-of-2 values are among the offered values.

MOTU Track16

The Track16 is a small desktop interface that pushes most of its I/O connections out to a giant proprietary octopus of a breakout cable. On hot summer days, I choose to use this device with headphones rather than powering up my full production rig, which generates a ton of heat.
  • With ADAT, MIDI, and complex mixing/routing software, this device is a strong competitor with the Clarett 2Pre in terms of flexibility and feature set.

Novation Audio Hub 2x4

The Audio Hub 2x4 is a 3-port powered USB hub with Focusrite Scarlett audio interface technology inside. I frequently use this device for recording/sampling audio sound sources, and I use its USB ports to host eLicenser/iLok dongles.
  • The audio interface portion of this device is based on Focusrite's "1st Generation" USB interface technology, which gives us a chance to compare the improvements Focusrite has made with the 2nd generation drivers.
  • While the Audio Hub sports a pair of balanced main outputs, it only has unbalanced RCA-style inputs, so I had to use different audio cables when testing it from the ones I used with the other interfaces.
  • This device is also a little unusual in that it only offers a low/high-gain toggle switch on the inputs rather than adjustable gain pots. 

RME Fireface UFX (1st generation)

The Fireface UFX is a prosumer legend, offering tons of analog and digital I/O with top-tier performance, reliability, and flexibility in a single rack space form factor. This is my primary interface for writing and recording.
  • I own the first generation UFX, which has both USB 2 and Firewire support. I typically use it as a Firewire device, just to avoid possible USB contention in my studio.
  • In recent years, RME has released an updated model, called the Fireface UFX+, which includes USB 3 and Thunderbolt support, and the Fireface UFX II, which is USB 2 only.
  • RME's drivers are a little bit odd in that they don't appear to advertise more than one sample buffer size at a time like most others do. In order to perform the latency tests at different buffer sizes I had to use the RME control panel to choose the new buffer size then "reload" the driver in the test tool before performing each round of tests.
  • Since this is the only interface I currently own that supports two different data buses, I tested it both as a Firewire and as a USB device. 

Roland Duo-Capture EX

The Duo-Capture EX is one of my favorite entry-level interfaces, just because it packs a lot of features into a small, reliable and affordable package. I don't actively use my Duo-Capture these days, but I keep it around for testing and ad-hoc stuff.
  • The Roland drivers are kind of weird in that they do not use powers-of-two buffer sizes, and their configuration control panel also has a number of non-standard options and metrics. I had to choose the nearest approximate buffer size for the comparative tests. Also, for all control panel settings other than buffer size, I just left them at their factory defaults.
  • On Windows 10, the Duo-Capture driver installs automatically (no separate discs or downloads from Roland required), however I experienced a lot of driver instability on my first round of latency tests with this interface after the drivers installed (some tests straight-up failed while others took significantly longer or shorter to complete than expected). I rebooted my computer and re-tested the interface and didn't experience any instability at all, so that appeared to be a temporary issue.

Roland Tri-Capture

The Tri-Capture is an odd little device with an interesting combination of features at a low price. I have used this for recording/sampling from consumer audio devices.
  • Like with the Duo-Capture EX, the Tri-Capture's driver has some unusual options and non-standard sample buffer sizes. I note the differences in the test results.
  •  

Test Setup and Method

For this round of tests I used my current DAW PC on the most recent build of 64-bit Windows 10. The current specs of this system:
  • CPU: Intel Core i7-5930K @ 3.5GHz
  • Motherboard: ASUS X99-A/USB 3.1 ATX
  • RAM: 32GB of DDR4 2133 MT/s
In order to keep the playing field level, I used the same USB and audio cables for all tests, where possible. (Some devices had special I/O connectors that required different cables.) I ran multiple tests at each sample buffer size for each device and selected the best/lowest test results from each device/buffer size combination.

In cases where the test results seemed surprising or unexpected, I re-ran all tests on the device in question to ensure results were consistent and reproducible.

The purpose of these tests was only to determine the measurable RTL of each device at each sample buffer size; not to establish the most reliable low-latency sample rate/buffer size combo under heavy DSP loads. (See the official DAW Bench tests for that sort of thing.)

Test Results 

I ran latency tests at four of the most common buffer sizes, all at a sample rate of 44.1kHz. Some quick notes before we get to the raw data:
  • Not all interface drivers support powers-of-two sample buffer scaling. In those cases I've added a * by the name of the device in the tables and graphs. Here are the exceptions:
    • For the Duo-Capture EX I had to use 144, 288, and 576 samples in the 128, 256, and 512 samples tests, respectively. This put that interface at a slight disadvantage for each test.
    • For the Tri-Capture I had to use 288 and 576 samples in the 256 and 512 samples tests, respectively. This also put that interface at a slight disadvantage for each test.
    • For the U-Control UCA222, I had to pick 132 samples for the 128 samples test. (The other common sample rates were available.)
  • The drivers of some of the devices did not offer as many buffer size options as others. This is why the 64-samples and 128-samples tests don't include scores for all eight interfaces. The interfaces that didn't support all testable sample rates were the Duo Capture EX, the Tri-Capture, and the UCA222.
  • Each test lists two scores for the RME Fireface UFX: One as a USB device, and one as a Firewire device.
  • The U-Control UCA222 was the only interface that didn't have ASIO drivers, so I tested it as a Windows Audio device.
  • All test results are reported as round-trip time in milliseconds. Lower scores/shorter bars are better.

64 Samples @ 44.1kHz

128 Samples @ 44.1kHz


256 Samples @ 44.1kHz




512 Samples @ 44.1kHz



Observations/Summary

  • True to its reputation, the Fireface UFX performed the best in all tests, with its USB mode slightly scoring better than its FW mode every time. (The USB and FW scores were always within 1ms of each other.)
  • Even though its larger-than-average sample buffer sizes put it at a slight disadvantage in every test, the Roland Duo-Capture EX fared rather well, capturing the third-best score in every round of tests where it competed.
  • Despite having very similar-seeming driver and control panel to the Duo-Capture EX, the Tri-Capture lagged behind the other Roland interface in both tests where it competed.
  • Considering that the Audio Hub 2x4 is really a Focusrite Scarlett interface in a Novation-branded box, its scores clearly demonstrate the poor performance of that generation of Scarlett interfaces on Windows. It scored significantly worse than most other interfaces on all tests but the last- and that one was a surprise...
  • ...which was the odd performance results of the Clarett 2Pre USB. In the 64-samples and 128-samples tests the Clarett performs reasonably well, at least when compared to the Audio Hub. (In the 64-sample test the Clarett's round-trip latency is almost one third of the Audio Hub's score.) However as the sample buffer sizes increased, so did the Clarett's relative latency, gradually closing- AND THEN PASSING- the gap with the Audio Hub, making the Clarett the worst-performing interface in the 512-samples tests.
  • When I was reviewing the test results before I began writing this article, I was so surprised by the Clarett's scores that I re-connected it to my test system and ran all of the tests on the Clarett again- only to find that they were the same. In both rounds of tests on this device, its round-trip latency grew progressively worse (relative to other interfaces) as the sample buffer size increased. This may point to some inefficient code in the current driver that is exacerbated as sample buffers grow.

No comments: