Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!


Click here to return to the 'Rip losslessly to flac with abcde' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Rip losslessly to flac with abcde
Authored by: Gerk on Oct 21, '05 07:47:01AM

lossless is subjective term. Sadly, flac is NOT a lossless format in the true sense of the term although it claims to be. We tested quite a few different "lossless" formats and a couple of the big players (namely flac and apple's lossless) aren't really lossless after all. We put a bunch of these codes through some grueling tests to see just how lossless they were and flac showed loss on compression/decompression. In fact it showed significant audio artifacts. Apple's lossless was worse however!

Just wanted to chime in here with that information.

It's a good format, but for people that are trying to choose a good format for long term archival that is truly lossless I would recommend using either wav/aiff (which are truly lossless as there is no conversion!) or monkey audio's .ape.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: hamarkus on Oct 21, '05 08:01:11AM

Wouldn't zip or sitx be truely lossless archival audio formats. Sure they cannot be decoded as they are played, but decoding would only take a few seconds anyway.

Just compressed an aiff file (3.6 MB) with Apple's zip: 2.1 MB, using sitx they size comes down to 1.2 MB.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: shoelzer on Oct 21, '05 08:13:25AM

Gerk, can you provide more info about the "grueling tests" and what kinds of "audio artifacts" you found. It seems that a lossless codec test wouldn't need to look for audio artifacts; just do a binary compare of the input and output wav/aiff files.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: Gerk on Oct 21, '05 09:13:27AM

We used a couple of different methods to test the codecs with some specially crafted audio samples.

We did both binary bit comparison (cmp command) as well as a testing the original WAV/AIFF against a compressed/decomrpessed one by putting the original sound in phase (0 degrees) and the comparing sound out of phase (180 degrees) In a perfect world they would cancel each other out perfectly ... which they didn't -- there were artifacts. The same files that had artifacts failed the binary comparison tests.

For some simpler audio files it was lossless, for the specially crafted ones it was not. The specially crafted files contained things like white noise, generated audio tones (sawtooth, square wav, etc), spectrum sweeps and other complex audio samples.

We're still gathering some of the last findings and will be posting all of our results. The tests were done by several audio professionals running different operating systems (OSX/Windows/Linux/BSD) to rule out software type/versions causing inaccurate results. Once it's posted I'll try and remember to post the link in this thread -- it may take a while still.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: fracai on Oct 21, '05 09:38:58AM

Then I think your tests WERE flawed.

Without seeing the full results it's hard to comment, but it sounds like the music tests were fine. I don't know many people who sit around listening to white noise and sine waves as a means of entertainment.

It seems to me that flac is lossless for the sound files it was intended to process.

I'll be interested to see your full results and testing measures of course. Where will these be posted when done?

---
i am jack's amusing sig file



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: shoelzer on Oct 21, '05 09:58:12AM

It doesn't matter if the source file is music or noise. Lossless should be lossless. If there are any errors with any type of input, the audio codec isn't truly lossless.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: kd4ttc on Oct 21, '05 10:18:27AM

No, that is not true. Giving input which does not meet the specification is an invalid test. These codecs expect audio. If you throw at it non-audio you have an invalid test. If you have a system that will do arithmetic on dollars and cents and you give a test suite including data with 10ths of cents you have an unreasonable test.

Even the 180 out of phase test is a mistake if you do not adjust for different latencies. And if the latencies vary over time it should also not matter.

---
Stephen Holland, M.D.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: shoelzer on Oct 21, '05 10:51:55AM

It is true. These codecs expect audio in the sense that the input must be a valid audio format supported by the codec but it does not matter what type of signal was encoded in that audio format. It could be music, a square wave, noise, or a list of winning lottery numbers -- it makes no difference to the encoder.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: BadgerUMD on Oct 21, '05 11:05:30AM

Not really true -- a true square wave has infinite frequencies (or at least higher than the sampling rate and your range of hearing), which any encoder (even "lossless" WAV) will alias to lower frequencies that you "can" hear. So if one encoder suppresses these alias frequencies better it doesn't make it more "lossless" -- you've simply passed data to the encoder that the writes didn't expect -- non-audio data.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: shoelzer on Oct 21, '05 11:15:27AM

Yes, it really is true. I'm talking about the codec being lossless. If the audio samples coming out match the audio samples that went in, it's lossless. You're talking about the sampling process being lossy, which of course is true, but it's a totally different issue.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: kd4ttc on Oct 21, '05 03:11:31PM

The thing the codec is trying to be smart about is to work with audio that really will come along. If the inventors of a codec realize through analysis of real audio that certain frequencies never come up in real life, then develop a codec that takes advantage of that truth of nature they may have an insight that allows a significant advance in technology. So if it is lossless with real audio but loses certain frequency components in fantasy signals I would still call it lossless.

Just because an arbitrary waveform has a certain frequency in the range of 50 Hz to 20 kHz does not make it a valid audio input. True square waves of pressure in air cannot be generated by any known phyisical method. No moving body can instantaneously move. Even electronic circuits that generate square waves are limited by rise times and may exhibit slight ringing.

As far as square waves and audio circuits, a crude test of audio gear is to run a square wave through and see what comes out. However, the purpose of this is not to look for whether the process drops a bit. One scopes the output looking for rolloff, ringing, and high and low frequency losses.

For audio testing that would be relevant consider something like the test disk at http://www.interstudio.co.uk/sndchk.htm That will privide not only audio that is well selected for testing sound reproduction equipment, but would be a suite that others could use in independent verification of the system.

Running a square waveform into a codec may be an easy test, but the ease with which it is done misses the point of why the testing is done in the first place.

---
Stephen Holland, M.D.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: DylanMuir on Oct 25, '05 01:49:03AM
"Lossless compression" means no information is thrown away.

MP3 and FLAC are analogous to JPEG and GIF, or ZIP. JPEG and MP3 throw away information that is considered unimportant, or below our threshold of detection. FLAC, GIF and ZIP should take some binary stream and compress it in a way that that stream can be recovered PERFECTLY.

Arguing about square waves misses the point -- FLAC can encode any valid PCM stream and reproduce it bit for bit on decompression. The sampling process that makes a true square wave impossible is not the issue here.

D.

[ Reply to This | # ]

Rip losslessly to flac with abcde
Authored by: n8gray on Oct 21, '05 12:04:13PM
No, that is not true. Giving input which does not meet the specification is an invalid test. These codecs expect audio. If you throw at it non-audio you have an invalid test. If you have a system that will do arithmetic on dollars and cents and you give a test suite including data with 10ths of cents you have an unreasonable test.

Sorry, you're flat-out wrong. FLAC is designed to compress any PCM stream so that it can be decompressed without loss. It is designed to be very efficient for music, but it should never lose information for any PCM stream including white noise, square waves, etc. If somebody was complaining that FLAC didn't achieve a high compression ratio on white noise, then your criticism would be justified. Indeed, it's likely that FLAC "compression" would make a white noise file larger.

Even the 180 out of phase test is a mistake if you do not adjust for different latencies. And if the latencies vary over time it should also not matter.

The out-of-phase test would be trivial to get right, but why bother? The only thing that matters is the binary difference test. Actually, I guess it might be a useful way to find out if the errors were spread out over the entire file or just in one chunk somewhere.

[ Reply to This | # ]

Problems with your test methods ...
Authored by: BadgerUMD on Oct 21, '05 10:02:41AM

In reading your test methods, I have to say it seems as if each one is significantly flawed in some way (although you may have omitted details for brevity's sake). Warning -- this post is overly long! I apologize, but I thought that Gerk might want some constructive criticism.

>> "We did both binary bit comparison (cmp command) ..."

I can't see this test being meaningful for any comparison of the files involved -- since the files are of different formats, insignificant format differences would result in significant changes in the files. If you are comparing the output to the sound card, however, that *should* work (more about why it may not later).

>> "... the original WAV/AIFF against a compressed/decompressed one by putting the original sound in phase (0 degrees) and the comparing sound out of phase (180 degrees) ..."

I can see a LOT of ways this test could fail, and it wouldn't make Flac (or others) "un-lossless". Mostly they have to do with sampling and the way you took the sound 180 degrees out of phase (more on sampling later). Mostly though is because we don't live in a perfect world, from a signal processing point of view, and except for very simple waveforms, I don't think you could cancel out sounds this way anyway.

>> "For some simpler audio files it was lossless, for the specially crafted ones it was not. The specially crafted files contained things like white noise, generated audio tones (sawtooth, square wav, etc), spectrum sweeps and other complex audio samples."

I had the most trouble with this. There is NO digital format which can faithfully reproduce the sounds you hear. This is because all of the digital formats work by sampling the sound signal (usually at about 44000 Hz) and basically recording the signal's amplitude every 1/44000 seconds. Just doing this loses information -- there is absolutely no way (especially with complex sounds) you could *NOT* lose information doing this. This means that any "lossless" format needs to be lossy to be represented digitally. (although usually this can be decreased to where you can't hear it).

Then, the even bigger problem is that the true white noise, sawtooth, and square wave samples all have infinite frequency components (or at least WAY above 22000Hz, the highest frequency you can get with 44000 sampling). That means, when the audio sample is sampled, those higher frequency artifacts (which you can't hear anyway) are aliased to lower frequency components that you CAN hear. These artifacts may be absent from WAV if the WAV encoder you used pre-filters the incoming filter before sampling (a common practice) and present in other formats because they assume that no one would be encoding a sound that no one could hear anyway (which is what a CD's input to these encoders would be). So a more fair comparison would be to input only those sounds into the encoders which a human can hear -- white noise et. al. don't represent those sounds.

I hope this helps you revise your tests, or at least describe them in such a way that your article is well received.



[ Reply to This | # ]
Problems with your test methods ...
Authored by: c15zyx on Oct 21, '05 10:44:24AM

Some good points here. Having sounds compared by using phasing is kinda of pointless. In general when generating any kind of extra processing it's hard to keep track of possible added variabilities along the way.

All you need to do is take pcm data straight from the input wav and encode. Then decode to wav and extract and compare the pcm data. Since pcm data contains only raw audio samples, you can just bit-compare the results. Nice and simple. A difference between two values in this case can mean one of 3 things- the encoder is buggy, the decoder is buggy, or the underlying mathematical algorithms are spotty.



[ Reply to This | # ]
Be more careful with presentation
Authored by: n8gray on Oct 21, '05 12:22:31PM
It appears that your methodology is sound, but you should be more careful about how you present these results. It's quite likely you're revealing bugs with one implementation of the FLAC algorithm, not fundamental problems in the design of FLAC. Unless you show your test files to the FLAC team and they say "yes, this is a design flaw and no, we aren't going to fix it" then claiming that "FLAC is not a lossless format" is inaccurate, misleading, and defamatory, and you're opening yourself up to a boatload of justified flameage.

[ Reply to This | # ]
Be more careful with presentation
Authored by: Githon on Oct 23, '05 09:56:26AM

Because saying "This codec loses something, therefore is not lossless" is entirely inflammatory and unfair.

Just like saying "The phone company isn't giving you a free cell phone" when part of your bill says "Free phone fee: $270".



[ Reply to This | # ]
Be more careful with presentation
Authored by: n8gray on Oct 25, '05 12:08:51AM
Because saying "This codec loses something, therefore is not lossless" is entirely inflammatory and unfair.

No, because saying "FLAC is not a lossless codec" is different from saying "one particular implementation of FLAC has a bug that can cause loss." Claiming that the FLAC algorithm is not lossless is an accusation that its authors are intentionally misleading people by saying otherwise. I would say that without some serious evidence, yes, such a claim is inflammatory and unfair. It's FUD, plain and simple, because it leads people to the incorrect conclusion.

Just like saying "The phone company isn't giving you a free cell phone" when part of your bill says "Free phone fee: $270".

No, not really. It's more like claiming "The phone company is lying and isn't giving anybody a free cell phone" when they actually just accidentally billed you for it but were completely willing to correct their mistake. The difference is what you're saying about the intent of the other party.

[ Reply to This | # ]

Be more careful with presentation
Authored by: Githon on Oct 30, '05 11:54:34AM

And you're certainly not assigning malicious intent to his statements, no sir.



[ Reply to This | # ]
Be more careful with presentation
Authored by: n8gray on Nov 01, '05 04:42:07PM
And you're certainly not assigning malicious intent to his statements, no sir.

Actually, if you read my post you might notice I said nothing about his intent, only the nature of his statements. I can understand the mistake -- it's a subtle distinction.

[ Reply to This | # ]

Rip losslessly to flac with abcde
Authored by: genericuser on Oct 21, '05 08:43:55AM

Tests over at Hydrogen Audio tend to disagree with you. Could you give specifics as the few tests I've seen of WAV versus WAV>ALAC>WAV were bit identical.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: c15zyx on Oct 21, '05 09:11:45AM

Then your tests were possibly flawed or you used miscompiled binaries (correct algorithms and implementation are good, but you still must trust the compiler to generate correct code).

Comparing raw pcm data before and after encoding to FLAC shows bit-identical results in all samples I've tested. It's possible (but unlikely) there might be a bug, but if that's the case you should report it to the FLAC developers.

Apple Lossless had a bug in earlier versions available that caused truncation for some samples, but I think the problem might have been fixed, either way I'd prefer FLAC or wavpack.



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: Gerk on Oct 21, '05 09:18:16AM

See my reply above for more information. Of note the tests that failed were not with music, but with sine wav sweeps, white noise, and generated audio tones. We also worked with various operating systems using both pre-compiled binaries and in some cases we compiled from the latest sources.

If we find reproducible problems we'll be sure to submit bugs to the developers along with the samples that triggered the problems, that was one of the points of the exercise! :)



[ Reply to This | # ]
Rip losslessly to flac with abcde
Authored by: TekkNick on Oct 21, '05 10:49:30AM

FLAC is indeed lossless. I just finished re-RIPing my entire collection so that as new/better formats come out I can re-encode without that hassle. I used FLAC to minimize the storage requirement. Paranoia got the best of me. I was not going to go through all that hassle (some 500 discs) only to find out that the decoded FLAC files weren't bit-for-bit copies of the originals. I wrote a script that encoded, decoded and diffed every track before permanently deleting the WAV. I can't speak for previous versions, but flac 1.1.2 is as reliable as zip with a lot higher compression ratio.



[ Reply to This | # ]