How MP3 audio compression works

The problem MP3 solves

A CD-quality audio track—44,100 samples per second, 16 bits per sample, two channels—consumes about 1.4 megabytes per second, or roughly the size of a high-resolution photo. A three-minute song is around 30 megabytes. MP3, standardized in 1993 as part of the MPEG-1 Audio codec in the ISO/IEC 11172-3 specification, fits the same song into a few megabytes: typically around 1–2 megabytes at 128 kilobits per second, roughly a tenth of the original data. It does not achieve this by recording sound more cheaply. It achieves it by storing only what a human listener would actually hear.

The core idea: throw away what you cannot hear

Most audio formats use lossless compression: they store every sample, just encoded more efficiently. MP3 is lossy, meaning it permanently discards information. The trick is choosing what to discard intelligently, guided by a field called psychoacoustics—the study of how the human ear and brain perceive sound. Your hearing is far less sensitive than your eye: it has a limited frequency range, varying sensitivity across that range, and, most usefully for compression, an attentional weakness where one sound can “mask” another. A codec that understands these weaknesses can remove masked components from a recording and few listeners will ever notice.

Two kinds of masking

The ear exhibits masking in two principal forms. Temporal masking is a time effect: a loud sound briefly before or after a quieter one reduces the quieter one’s audibility—which is why a clap can erase the tick of a doorknob behind it. Simultaneous, or frequency, masking is a pitch effect: a loud tone at one frequency makes nearby frequencies harder to hear. The louder the masker, the wider the band it covers and the less detail the ear resolves inside it; in quiet passages, or in frequency range the ear is least sensitive to, far more detail is discernible. MP3 leans hard on frequency masking, because it works at a scale that maps onto how audio is decomposed—into bands of frequency.

Turning sound into frequency bands

To exploit frequency masking, the decoder needs the signal described in the frequency domain rather than the time domain. The MP3 pipeline accomplishes this with a hybrid of the fast Fourier transform (FFT), which converts a block of audio samples into its frequency components and is the same mathematics that lets apps show equalizers, and a modified discrete cosine transform (MDCT), a variation of the discrete cosine transform. Together, a 1,152-long block of samples—about 26 milliseconds of audio—is split into 576 short-transform coefficients (32 per sub-band across 18 sub-bands) and 1,080 long-transform coefficients (576 per channel), yielding a granular frequency representation of each short block. The coefficients are not the final bitstream, but they are the currency the encoder can spend, keep, or discard to hit a bitrate target.

Quantization: spending bits where they matter

The next step is quantization: assigning each frequency coefficient a small number of bits, or none at all. The encoder consults a psychoacoustic model—its estimate of a masking threshold at each frequency at each moment—and budgets bits accordingly. Where a loud noise floor exists, nearby coefficients can be rounded coarsely or zeroed without audible cost. Where the signal is quiet and the ear is sensitive, coefficients are preserved at fine resolution. The choice of how aggressively to quantize is the essence of the bitrate setting: more bits per second means finer quantization everywhere, which is why 320 kilobits per second sounds closer to the source than 96 kilobits per second.

Bit reservoir, stereo coupling, and the final layer

A few engineering refinements complete the codec. The bit reservoir allows the encoder to borrow bits from short, low-information blocks to spend on hard, high-information ones, smoothing quantization errors over time instead of letting them pile up in any single instant. Stereo processing exploits the fact that the left and right channels overlap heavily by a factor of three to twelve, encoding the common parts once and only the differences separately. And an optional Layer III sub-band bit allocation, together with spectral and temporal band splitting, lets the encoder decide at several granularities whether to spend bits in a single band—protecting a transient attack—or to merge bands where detail does not matter. The result is a compact bitstream the decoder reconstructs with the inverse MDCT, an overlap-add step to smooth block edges, a filter bank that reassembles the 32 sub-bands into a full-spectrum audio signal, and, in many decoders, a limiters stage to keep the result well-behaved on downstream hardware.

Where the trade-offs hide

The cost of aggressive quantization is a specific kind of artifact: instead of a pure tone, a listener may hear a faint, metallic “swish” or “underwater” muddiness, most obvious on quiet passages, cymbal tails, and reverb. These are pre-echo effects (when the masking estimate underestimates what is audible just before a loud transient) and ringing artifacts from coarse frequency resolution. At high bitrates they fade; at low bitrates they become apparent, especially on cheap headphones, which is why “low bitrate MP3” has a distinct sound. VBR encoders address the unevenness by spending bits where they matter and saving them elsewhere—a tradeoff the codec was explicitly designed for.

What this means for you

Why it still matters

MP3 is now over three decades old and has been largely succeeded in the streaming era by AAC and, for voice, Opus and the Opus-family codecs in VoIP. Yet the underlying idea—psychoacoustic lossy compression, quantizing against a masking threshold, and trading bitrate against artifact risk—is not historical. It is the working method behind essentially every lossy audio and video format you meet: AAC and HE-AAC, which reuse a cousin of the MDCT plus a spectral envelope and a separate psychoacoustic toolset; Opus, blending a low-delay CELT-style toolchain with an SILK speech model; and the video codecs on streaming platforms, which exploit analogous perceptual redundancy. Understanding MP3 is, in effect, understanding the template.

Sources: ISO/IEC 11172-3 (MPEG-1 Audio); Burr-Brown Application Note 135; Wikipedia: MP3.

Corrections: Report an error at corrections@wireshaman.com.