gapless playback of *some* MP3 files destroyed in the end after -c copy

[Christoph Anton Mitterer]

Summary of the bug:

Originally from ​https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797965
which I just thought I could close.

What I did was applying R128 gain info to audio files with bs1770gain (which uses ffmpeg internally).

Long time ago, I found that this corrupts gapless playback by adding small silence or other distortions at the end of the "first" track.

I've just retried with current toolset (current in the sense of Debian unstable, that is: ffmpeg 4.4.2).

At first I thought it would work for both now, Opus and MP3, because some new test files I had used, were processed correctly.
But then I dug out my test files from back then, where ffmpeg failed again with MP3 (but not with Opus).

To simplify things, let's keep bs1770gain out of the game and just use ffmpeg.

1) I have two WAVs where the 1st goes right over in the 2nd and which should play back gaplessly.
2) I encode them to opus and MP3 with:

opusenc --vbr --bitrate 128 --discard-comments --discard-pictures
lame --verbose -q 0 -v -V 4 --noreplaygain --id3v2-utf16 --add-id3v2 --id3v1-only

3) I decode these again with:

opusdec
lame --decode

4) Now I copy the Opus/MP3 files with ffmpeg like:

ffmpeg -i a.opus -acodec copy -y copy_a.opus
ffmpeg -i b.opus -acodec copy -y copy_b.opus
ffmpeg -i a.mp3 -acodec copy -y copy_a.mp3
ffmpeg -i b.mp3 -acodec copy -y copy_b.mp3
My understanding is, that the generated files should be audio-wise identical to their sources.

5) I dcode copy_*.* as in (3).
6) diff each WAV pair

For Opus, both pairs are bitwise identical
For MP3, only the B file is, but the A file differs, which one can also clearly see at the end of the waveform, where there's a bit more silence (and the audio ends with a strange spike).

So something gets lost in the copying process of ffmpeg, and apparently this also depends on the file (as said, I checked another pair of files, where this worked even for MP3.

If any ffmpeg developer is interested in looking into this, I'd rather privately share the sample files if that's possible (just tell me your email and I can send you a link), the material is copyrighted.

Thanks,
Chris

[Balling]

Is this still broken, Chris?

[Christoph Anton Mitterer]

Hey Balling.

First, please have a look at the messages starting with:
​https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=797965#197

A while ago (April last year) I've had that re-checked and it seemed at first that now it works.
However, I then checked it again with the very same (gapless) WAV files that I've had used in the beginning... and with them I still heard some distortion (a tiny but audible pop).

So conclusion from that is, that it seems to depend on the source file, whether it works or not.

I've just repeated the test with versions as of current Debian unstable, that is:
lame 3.100
mpv 0.35.1
ffmpeg 5.1.2

First I take my original WAVs, encode them with lame via:
lame --verbose -q 0 -v -V 4 --noreplaygain --id3v2-utf16 --add-id3v2 --id3v1-only 0.wav
lame --verbose -q 0 -v -V 4 --noreplaygain --id3v2-utf16 --add-id3v2 --id3v1-only 1.wav

Then I check whether the resulting MP3s play back gaplessly (with no distortion).
mpv 0.mp3 1.mp3
=> that works (or at least I cannot hear anything).

Then I do:
ffmpeg -i 0.mp3 -acodec copy -y a.mp3
ffmpeg -i 1.mp3 -acodec copy -y b.mp3

and checking again with:
mp4 a.mp3 b.mp3
=> there is no real gap, but as said above, there's still an audible click sound.

If you need my source file I could share them in private with you (not sure whether the material is copyrighted).

Cheers,
Chris.

[Balling]

that works (or at least I cannot hear anything).

Why are you not comparing sample accuracy in Audition (start gap, end gap, sync of samples)??

[Christoph Anton Mitterer]

(btw: messed up the filenames in the post above, corrected that now)

Well I have no Audition, but verified now with Audacity (though this requires me to lame --decode the MP3s first and use the resulting WAVs, since Audacity does seem to generally not support gapless information).

When doing so, the WAVs from 0.mp3 / 1.mp3 align perfectly together...

But the ones fed through lame (a.mp3 / b.mp3) show a clearly visible distortion (see attached PNG).

[Christoph Anton Mitterer]

distortion mentioned in comment 4

[Balling]

show a clearly visible distortion

In your picture you put two audios one after the other. Obviously to compare start gap, end gap, sync of samples you should put them one on top of the other.

As for distortion that would be not a gap, but a bug in mp3 decoder itself.

[Christoph Anton Mitterer]

No (at least if you mean lame --decode), because that distortion *only* occurs with the files that I fed through:

ffmpeg -i 0.mp3 -acodec copy -y a.mp3

So it does not show up with the (decoded) 0.mp3/1.mp3 ... but it does show up with te (decoded) a.mp3/b.mp3.

Well maybe it's not a gap (anymore - there used to be one).. but it's at least some artefact ffmpeg seems to add.

[Balling]

Okay, share the files with my email, ​val.zapodvz@gmail.com

I cannot reproduce this with out native decoder (did not try lame --decode). The wav files before and after -c copy are decoding the same.

[Christoph Anton Mitterer]

You should have received a Google Drive link. (Please tell me once you no longer need the download.)

What I do is basically:
lame --verbose -q 0 -v -V 4 --noreplaygain --id3v2-utf16 --add-id3v2 --id3v1-only 16.wav
lame --verbose -q 0 -v -V 4 --noreplaygain --id3v2-utf16 --add-id3v2 --id3v1-only 17.wav
ffmpeg -i 16.mp3 a.wav
ffmpeg -i 17.mp3 b.wav

diff 16.mp3 a.mp3
diff 17.mp3 b.mp3
=> both differ (but could of course be metadata)

thus checking the decoded WAV:
ffmpeg -i 16.mp3 16.mp3.wav
ffmpeg -i 17.mp3 17.mp3.wav
ffmpeg -i a.mp3 a.wav
ffmpeg -i b.mp3 b.wav

diff 16.mp3.wav a.wav
=> differ
diff 17.mp3.wav b.wav
=> equal

[Balling]

Downloaded. You can just use -bitexact option so that wav files metadata is not inserted or use Beyond Compare...

[Balling]

ffmpeg -i 16.mp3 a.wav
ffmpeg -i 17.mp3 b.wav

Where is -c copy? Why was there -c copy before?

[Christoph Anton Mitterer]

Ah, I had used -acodec copy (which I assume is the same here as -c copy?) above in comment #2, but forgot it before.
Still, the same results with that when I repeat it now:
$ lame --verbose -q 0 -v -V 4 --noreplaygain --id3v2-utf16 --add-id3v2 --id3v1-only 16.wav
$ lame --verbose -q 0 -v -V 4 --noreplaygain --id3v2-utf16 --add-id3v2 --id3v1-only 17.wav
$ ffmpeg -i 16.mp3 -acodec copy -y a.mp3
$ ffmpeg -i 17.mp3 -acodec copy -y b.mp3
$ diff 16.mp3 a.mp3
Binary files 16.mp3 and a.mp3 differ
$ diff 17.mp3 b.mp3
Binary files 17.mp3 and b.mp3 differ
$ ffmpeg -i 16.mp3 16.mp3.wav
$ ffmpeg -i 17.mp3 17.mp3.wav
$ ffmpeg -i a.mp3 a.wav
$ ffmpeg -i b.mp3 b.wav
$ diff 16.mp3.wav a.wav
Binary files 16.mp3.wav and a.wav differ
$ diff 17.mp3.wav b.wav
$

[Balling]

Yep, there is in fact a remainder problem.

In fact this may be a bug in LAME: ffmpeg -i 16.wav cmancasw1.mp3

prints

[libmp3lame @ 000001684bdb0180] Trying to remove 1152 samples, but the queue is empty

​https://superuser.com/questions/796977/trying-to-remove-1152-samples-but-the-queue-is-empty

And many other results through 10 years.

​https://www.google.com/search?q=Trying+to+remove+1152+samples

[Balling]

You can reproduce it from sample from #6600 (nope, sample is only part of full song). And also it was in the #3200, 9 years ago. LULZ.

[Christoph Anton Mitterer]

I see. Well not sure how much LAME is still mainained... last commit to their SVN seems to be from 2021.

If you think it's not in ffmpeg, then I can forward the issue there, unless you'd like to do so yourself.

MP3 is in fact no longer that important for me (neither AAC), since Opus works now gaplessly on all the devices I use ;-)

[Balling]

Nope, our bug. lame --decode decodes lame created file fine, but after -c copy file has extra samples in the end.

Indeed, bad file has in lame --decode:

skipping initial 1105 samples (encoder+decoder delay)
skipping final 1152 samples (encoder padding-decoder delay)

While good file has

skipping initial 1105 samples (encoder+decoder delay)
skipping final 1187 samples (encoder padding-decoder delay)

and yes, -c copy does corrupt this metadata, and yes the difference in Audacity is 35 samples. Wow.

Metadata

MusicLength: 1825070

vs

MusicLength: 1825175

[Balling]

Thankfully our decoder is not affected, and not all mp3 files are affected, sample with skipping final 311 samples (encoder padding-decoder delay) does not change after -c copy.

[Balling]

I understand that according to the LAME FAQ file, resultant MP3 files are zero padded in the front and back to make sure the inverse MDCT is performed properly, but also because the windows overlap.

Yet our -c copy removes it all. Wow.

​https://stackoverflow.com/questions/48411053/how-to-compute-the-number-of-extra-samples-added-by-lame-or-ffmpeg

[Balling]

You can reproduce the same with just silence wav:

ffmpeg -f lavfi -i anullsrc -ar 44100 -ac 2 -t 00:01:31.53334 -c:a pcm_s16le empty.wav

that created file with 4036620 samples per channel.