Compare two audio files
Question:
Basically, I have a lot of audio files representing the same song. However, some of them are worse quality than the original, and some are edited to where they do not match the original song anymore. What I’d like to do is programmatically compare these audio files to the original and see which ones match up with that song, regardless of quality. A direct comparison would obviously not work because the quality of the files varies.
I believe this could be done by analyzing the structure of the songs and comparing to the original, but I know nothing about audio engineering so that doesn’t help me much. All the songs are of the same format (MP3). Also, I’m using Python, so if there are bindings for it, that would be fantastic; if not, something for the JVM or even a native library would be fine as well, as long as it runs on Linux and I can figure out how to use it.
Answers:
First, you will have to change your domain of comparison. Analyzing raw samples from the uncompressed files will get you nowhere. Your distance measure will be based on one or more features that you extract from the audio samples. Wikipedia lists the following features as commonly used for Acoustic Fingerprinting:
Perceptual characteristics often exploited by audio fingerprints include average zero crossing rate, estimated tempo, average spectrum, spectral flatness, prominent tones across a set of bands, and bandwidth.
I don’t have programmatic solutions for you but here’s an interesting attempt at reverse engineering the YouTube Audio ID system. It is used for copyright infringement detection, a similar problem.
This is actually not a trivial task. I do not think any off-the-shelf library can do it. Here is a possible approach:
- Decode mp3 to PCM.
- Ensure that PCM data has specific sample rate, which you choose beforehand (e.g. 16KHz). You’ll need to resample songs that have different sample rate. High sample rate is not required since you need a fuzzy comparison anyway, but too low sample rate will lose too much details.
- Normalize PCM data (i.e. find maximum sample value and rescale all samples so that sample with largest amplitude uses entire dynamic range of data format, e.g. if sample format is signed 16 bit, then after normalization max. amplitude sample should have value 32767 or -32767).
- Split audio data into frames of fixed number of samples (e.g.: 1000 samples per frame).
- Convert each frame to spectrum domain (FFT).
- Calculate correlation between sequences of frames representing two songs. If correllation is greater than a certain threshold, assume the songs are the same.
Python libraries:
- PyMedia (for step 1)
- NumPy (for data processing) — also see this article for some introductory info
An additional complication. Your songs may have a different length of silence at the beginning. So to avoid false negatives, you may need an additional step:
3.1. Scan PCM data from the beginning, until sound energy exceeds predefined threshold. (E.g. calculate RMS with a sliding window of 10 samples and stop when it exceeds 1% of dynamic range). Then discard all data until this point.
Copying from that answer:
The exact same question that people at the old AudioScrobbler and currently at MusicBrainz have worked on since long ago. For the time being, the Python project that can aid in your quest, is Picard, which will tag audio files (not only MPEG 1 Layer 3 files) with a GUID (actually, several of them), and from then on, matching the tags is quite simple.
If you prefer to do it as a project of your own, libofa might be of help. The documentation for the Python wrapper perhaps will help you the most.
Basically, I have a lot of audio files representing the same song. However, some of them are worse quality than the original, and some are edited to where they do not match the original song anymore. What I’d like to do is programmatically compare these audio files to the original and see which ones match up with that song, regardless of quality. A direct comparison would obviously not work because the quality of the files varies.
I believe this could be done by analyzing the structure of the songs and comparing to the original, but I know nothing about audio engineering so that doesn’t help me much. All the songs are of the same format (MP3). Also, I’m using Python, so if there are bindings for it, that would be fantastic; if not, something for the JVM or even a native library would be fine as well, as long as it runs on Linux and I can figure out how to use it.
First, you will have to change your domain of comparison. Analyzing raw samples from the uncompressed files will get you nowhere. Your distance measure will be based on one or more features that you extract from the audio samples. Wikipedia lists the following features as commonly used for Acoustic Fingerprinting:
Perceptual characteristics often exploited by audio fingerprints include average zero crossing rate, estimated tempo, average spectrum, spectral flatness, prominent tones across a set of bands, and bandwidth.
I don’t have programmatic solutions for you but here’s an interesting attempt at reverse engineering the YouTube Audio ID system. It is used for copyright infringement detection, a similar problem.
This is actually not a trivial task. I do not think any off-the-shelf library can do it. Here is a possible approach:
- Decode mp3 to PCM.
- Ensure that PCM data has specific sample rate, which you choose beforehand (e.g. 16KHz). You’ll need to resample songs that have different sample rate. High sample rate is not required since you need a fuzzy comparison anyway, but too low sample rate will lose too much details.
- Normalize PCM data (i.e. find maximum sample value and rescale all samples so that sample with largest amplitude uses entire dynamic range of data format, e.g. if sample format is signed 16 bit, then after normalization max. amplitude sample should have value 32767 or -32767).
- Split audio data into frames of fixed number of samples (e.g.: 1000 samples per frame).
- Convert each frame to spectrum domain (FFT).
- Calculate correlation between sequences of frames representing two songs. If correllation is greater than a certain threshold, assume the songs are the same.
Python libraries:
- PyMedia (for step 1)
- NumPy (for data processing) — also see this article for some introductory info
An additional complication. Your songs may have a different length of silence at the beginning. So to avoid false negatives, you may need an additional step:
3.1. Scan PCM data from the beginning, until sound energy exceeds predefined threshold. (E.g. calculate RMS with a sliding window of 10 samples and stop when it exceeds 1% of dynamic range). Then discard all data until this point.
Copying from that answer:
The exact same question that people at the old AudioScrobbler and currently at MusicBrainz have worked on since long ago. For the time being, the Python project that can aid in your quest, is Picard, which will tag audio files (not only MPEG 1 Layer 3 files) with a GUID (actually, several of them), and from then on, matching the tags is quite simple.
If you prefer to do it as a project of your own, libofa might be of help. The documentation for the Python wrapper perhaps will help you the most.