Lossless audio codec comparison revision 5 - CDDA part
Introduction
This document compares the performance of lossless audio codecs on CDDA material, which are PCM audio files with a bit depth of 16 bit per sample and a samplerate of 44.1kHz.
The test corpus consists of 64 sources of very different sound. An attempt was made to create a balanced though extensive set covering many possible sounds. While most sources contain music, not all do.
Method
To compare the performance of each codec, the following steps are followed for each combination of corpus file, codec and codec setting:
- A WAV file is placed on a large enough ramdisk
- An MD5sum is calculated for the WAV file excluding its header
- The WAV file is encoded by the chosen codec provided with the required settings. The amount of CPU time required to do this conversion is measured and the resulting filesize is recorded
- The encoded file is decoded by the chosen codec. The amount of CPU time required to do this conversion is measured
- An MD5sum is calculated for the resulting decoded file excluding its header
- The MD5sum of the provided WAV file and the decoded file are compared
The following codecs and settings are used:
Codec | Settings used |
---|---|
FLAC | -0, -1, -2, -3, -4, -5, -6, -7, -8 |
ALAC | --fast, [-] |
WavPack | -f, [-], -h, -hh, -x4f, -x4, -x4h, -x4hh |
TAK | -p0, -p0e, -p0m, -p1, -p1e, -p1m, -p2, -p2e, -p2m, -p3, -p3e, -p3m, -p4, -p4e, -p4m |
OptimFROG | presets 0 through 10 and max |
Monkeys Audio | -c1000, -c2000, -c3000, -c4000, -c5000 |
La | [-], -high |
MP4ALS | -a -o5, -a, -a -o20, -a -o40, -a -b, -a -b -o20, -a -b -o40, -a -b -o1023, -7 |
WMA | [-] |
Shorten | [-] |
TTA | [-] |
For each codec, the latest Windows binary provided by the author of the codec is used, no specially tuned compiles are used. In case of ALAC, encoding is done with refalac64 as provided by QAAC. In case of WMA, encoding is done with WMAEncode.exe (which uses the encoder provided by Windows 10) and decoding with FFmpeg 5.0.
Measurements are made on a Windows 10 machine with a AMD A4-5000 CPU with 4GB of RAM. This CPU has all x86 instruction set extensions up to and including AVX (i.e. lacks AVX2). Measuring the CPU-time used is done with timer64.exe, part of the 7-max/7-benchmark suite.
Timing is done per track. This measured time is divided by the content length, i.e. the execution time of the encoding or decoding process in seconds divided by the playback length of the track or chapter in seconds. The result of this division is called CPU-usage. The filesize of the encoded file is divided by the filesize of the original WAV file to calculate a compression.
Results per source are obtained by averaging the compression and CPU-usage so each track or chapter contributes the same amount to the average, i.e. length of the track or chapter is not incorporated.
The average of all sources is obtained by averaging the results per source, again without any weighing. The total results are therefore not influenced by the length of the corpus content, each source contributes an equal amount to the average.
Results (PDF)
- Average of all CDDA sources
- Results per source, PDF, CSV
The best results will be obtained in the bottom left corner of the graphs: this represents the best compression (smallest file size) and the lowest CPU-usage (fastest compression and/or decompression).
In the graph each codec is represented by a group of markers connected by a line. Each combination of settings as mentioned in the previous table corresponds to one marker, in the order listed in the table. The first combination of settings is (usually) the fastest and the last the slowest. Therefore, the marker closest to the upper left corner of the graph corresponds to the first listed combination of settings, and the marker closest to the lower right corner corresponds to the last listed combination of settings.
Discussion
Looking at the average of all sources, there is not a single clear winner. In case maximum compression is desired, OptimFROG does best, with the highest compressing settings having quite a clear margin over the next best competitor. When a balance between compression and speed is desired, TAK seems to hold a good spot, with its decoding speed only second to FLAC while achieving much higher compression.
When looking at the results per source, the results shift a bit, but large shifts are only found for peculiar sources. For example, OptimFROG does exceptionally well on the diffuse sound sources. Other big difference are visible on mono audio encoded as stereo (with the two channels carrying bit-for-bit exact same content) where it can be clearly seen that Shorten and some presets of refalac and FLAC do not perform stereo decorrelation.
Other peculiar results can be spotted in the graph for Ryodi Ikeda - Dataplex, where Monkey Audio's insane preset (which is the slowest and usually best compressing) is outperformed by its other presets. Something similar happens with La and OptimFROG. Also Junichi Masuda Go Ichinose - Pokemon Gold stands out as being different.
Sources used
Solo instruments | |||
---|---|---|---|
Artist - Album | Main instrumentation | Comment | Year |
Jeroen van Veen - Einaudi: Waves The Piano Collection | Piano | 2013 | |
Jean Guillou - The Great Organ of St. Eustache, Paris | Church organ | 2006 | |
Bobby McFerrin - The Voice | Voice | 1984 | |
Dan Brown - Angels & Demons | Voice | Audiobook | 2009 |
Carmen Cavallaro - Songs of Our Times, Song Hits of 1921 | Piano | [5][1] | 1942 |
Konrad Junghanel - Europaische Lautenmusik Volume 1 | 11-string baroque lute | 1989 | |
Nicolò Paganini - 24 Capricci (Salvatore Accardo) | Violin | 1978 | |
Acoustic ensembles and bands | |||
Loosely sorted by size of ensemble | |||
Artist - Album | Main instrumentation | Comment | Year |
Alela Diana - The Pirate's Gospel (tracks 1, 2, 6, 7, 10 and 11) | Voice and guitar with occasional vocal stacking | 2004 | |
Fredrik Fors and Sveinung Bjelland - Les Nouveaux Musiciens | Clarinet and piano | 2004 | |
Rosenberg Trio - Djangolists | 2 guitars and double bass | 2010 | |
Szakcsi - Viragom, Viragom | 2 Voices, piano, flute | 1998 | |
Berlage Saxophone Quartet - SaxoFOLK | 4 saxophones | 2014 | |
Apocalyptica - Inquisition Symphony | 4 cello's | 1998 | |
Flanders Recorder Quartet - Armonia di Flauti | 4 recorders | 1998 | |
Dave Brubeck Quartet - Time Out | Saxophone, piano, double bass, drums | 1959 | |
Equilibrium - Rekreatur (Acoustic bonus CD) | Flute, accordion, guitar, bass guitar, drums | Very fast paced | 2010 |
Dvorak - String Quartet No. 3 | 2 violins, alto violin, cello | 1993 | |
Alison Krauss & Union Station - New Favourite | Voice, guitar, steel guitar, fiddle, upright bass | 2001 | |
RiverBend Bluegrass Live at Jacoby Arts Center on 2021-07-29 | Voice, mandolin, banjo, violin, steel guitar, double bass | Live, [1] | 2021 |
Piet Jeegers Clarinet Choir - Volume 4 | 15 clarinets (1x E♭, 8x B♭, 2x Alto, 2x Bass, 1x Contra Alto, 1x Contra Bass) | 2002 | |
Amplified ensembles and bands | |||
Loosely sorted by amount of 'overdrive' | |||
Artist - Album | Main instrumentation | Comment | Year |
Coldplay - Parachutes | Voice, electric guitar, bass guitar, drums | 2000 | |
Michael Bublé - Michael Bublé Meets Madison Square Garden | Voice, electric guitar, piano, drums, brass, crowd | 2009 | |
Albert King - Years Gone By | Voice, 2 electric guitars, keyboard, bass guitar, drums | Vinyl rip | 1969 |
Status Quo - Whatever You Want | Voice, 2 electric guitars, bass guitar, drums | Vinyl rip | 1979 |
Muse - HAARP | Voice, overdriven guitar, bass guitar, drums | Live | 2008 |
Iron Maiden - Powerslave | Voice, 2 overdriven guitars, bass guitar, drums | 1984 | |
Animals as Leaders - Weightless | 2 overdriven 8-string guitars, drums | 2011 | |
In Flames - Soundtrack to Your Escape | Growling vocals, 2 overdriven guitars, bass guitar, drums | 2004 | |
Nightwish - Wishmaster | Voice, overdriven guitar, bass guitar, keyboard, drums | 2000 | |
Epica - The Holographic Principle | Clean and growling vocals, guitar, synth, bass guitar, drums, orchestra and choir | 2016 | |
Big bands | |||
Artist - Album | Main instrumentation | Comment | Year |
Bert Kaempfert - A Swingin' Safari | Big band with string section and amplified vocals | 1962 | |
Dutch Swing College Band - Digital Dutch | Dixieland band | [3] | 1982 |
Fanfare Ciocarlia - Baro Biao; World Wide Wedding | Fanfare | 1999 | |
Glenn Miller - The Best Of The Lost Recordings And Secret Broadcasts | Big band | [4] | 1943 |
John Allmark Jazz Orchestra Live at The Met on 2019-05-20 | Big band | Live, [1] | 2019 |
Ida Gotkovsky - Works for Symphonic Band | Symphonic Band (also known as concert band or wind orchestra) | 1989 | |
Orchestral and choral | |||
Loosely sorted by size of ensemble | |||
Artist - Album | Main instrumentation | Comment | Year |
Confido Domino Minsk - Sacred Choral Music From White Russia | Choir | 1994 | |
Vivaldi - Four Seasons (Amsterdam Baroque Orchestra, cond. Ton Koopman) | Baroque orchestra (small) | 1996 | |
Dario Marianelli - Pride & Prejudice | Piano and chamber orchestra | Film score | 2005 |
J.S. Bach - Magnificat | Baroque orchestra (small), vocal soloists and choir | Live | 2012 |
Michael Giacchino - The Incredibles | 'Processed' orchestra, very upfront brass [7] | Film score | 2004 |
Nicholas Hooper - Harry Potter and the Order of the Phoenix | 'Processed' orchestra and choir [7] | Film score | 2004 |
Tchaikovsky - The Nutcracker (Kirov Orchestra, cond. Valery Gergiev) | Ballet orchestra (intermediately sized) | 1998 | |
Stravinsky - Le Sacre du Printemps / Petrouchka (Isreal Philharmonic Orchestra cond. Leonard Bernstein) | Large symphonic orchestra | 1983 | |
Verdi - Requiem (Berliner Philharmoniker cond. Claudio Abbado) | Large symphonic orchestra, vocal soloists and very large choir | 2001 | |
Mahler - Symphony no.8 (Concertgebouw orchestra cond. Bernard Haitink) | Very large symphonic orchestra, vocal soloists and exceptionally large choir [6] | 1971 | |
Electronic music | |||
Loosely sorted by how 'irregular' music is | |||
Artist - Album | Description | Comment | Year |
The Ambient Visitor - Across the Milky Way | Very slow ambient music, drone | 2021 | |
Kraftwerk - Autobahn | Analog synthesizers, vocoder | 1974 | |
Fatboy Slim - You've Come a Long Way, Baby | Samplers (vocal and drum), synthesizers | 1998 | |
Owl City - Ocean Eyes | Audibly pitch corrected voice, synthesizers, drum computers | 2009 | |
Taylor Swift - 1989 | Stacked voice, synthesizers, drum computers | 2014 | |
Lil Wayne - Tha Carter III | Voice, samplers, keyboard, drum computer | 2008 | |
Xzibit - At the Speed of Life | Voice, samplers, drum computer | 1996 | |
Daft Punk - Alive 2007 | Sampling, vocoded voice | Live | 2007 |
Junichi Masuda, Go Ichinose - Pokemon Gold | Game Boy emulator (2x square wave, noise generator, wavetable) | 1999 | |
The Prodigy - Invaders Must Die | Voice, samplers | 2009 | |
Ubiktune - SOUNDSHOCK: FM FUNK MADDNESS!! | FM Synthesizers | 2011 | |
Skrillex - Bangarang | Highly 'overdriven' synthesizers, vocal samples, drum synth | 2011 | |
Merzbow - Cloud Cock OO Grand | Very noisy music | 1989 | |
Other | |||
Artist - Album | Description | Comment | Year |
Ryoji Ikeda - dataplex | Dominated by transients | 2005 | |
Horacio Vaggione - Fluides | Very short sounds (microsound) | 2018 | |
N/A | Recordings of diffuse soundfields produced by choirs, ensembles and orchestras [2] | Own recordings | 2015 |
N/A | Rarewares test samples (https://www.rarewares.org/test_samples/) | 2005 | |
N/A | Various recordings of ambient sounds taken from Freesound.org [8] |
[1] Available on archive.org
[2] Consists of: Recording of a choir in a small church; Recording of a (different) choir (part of recording is a capella, part of recording accompanied by a small ensemble) in a small theater; Recording of music directors exam, featuring first a small, later an intermediately sized and finally a very large choir in a concert hall; Recording of a choir, orchestra and solists in a intermediately sized church; All recordings with boundary microphones spaced 3 meters apart on a balcony, as far from the performers as possible
[3] 'Launch title' for first CD players
[4] Professional vinyl rip and restoration from a radio transcription disc
[5] Professional vinyl rip from a 78rpm commercial release
[6] Mahler's symphony no.8 is nicknamed 'Symphony of a Thousand' for its exceptionally large orchestra and choir
[7] Orchestra has been recorded in seperate sections (with sound barriers) so mix can be varied and some instruments are (at times) mixed in much louder or quieter than the really are. Also, for Harry Potter and the Order of the Phoenix, choir had been dubbed over orchestra. See https://youtu.be/0sASbCxnjAg for Harry Potter and the Order of the Phoenix and https://www.youtube.com/watch?v=zA1XcaQI5eU for Incredibles
[8] Sound include birdsong, traffic and rain