What Is Quantization?
Quantization stores a network's weights, and sometimes its activations, at lower numeric precision than the format they were trained in. The architecture and the parameter count are unchanged. What changes is how many bits each number occupies, and therefore how much memory and bandwidth the whole thing requires.
Precision, and what is traded for what
Training is usually done in a 16- or 32-bit floating-point format. Quantization maps those values onto a smaller format — 8-bit integers, 4-bit integers, or one of the narrow floating-point formats designed for the purpose — using a scale and, in the asymmetric schemes, an offset called the zero-point, recorded per tensor, per channel, or per small block. The finer the grouping, the better the mapping fits the values it is approximating, and the more bookkeeping it costs to store.
The gain is memory and bandwidth, and both matter more than they sound: a configuration that no longer fits in a device's memory is not slow, it is unavailable. The cost is fidelity. Every quantized value approximates the one it replaced, and the effect is not uniform across a network — some layers tolerate the approximation and some do not, which is why mixed schemes that keep sensitive tensors wider are common.
Weights and activations are quantized under different constraints, and most deployments quantize only the first. A weight is a fixed value, known before anything runs, so the interval it has to fit into can be measured once and recorded. An activation is produced during the run and its range depends on the input, and a small number of unusually large values will stretch the interval every other value in the group has to share. That asymmetry is why activation quantization is where schemes become elaborate — isolating outliers, quantizing in finer groups, holding particular channels wider — and why weight-only arrangements are the ordinary starting point.
| Format | Where it is used |
|---|---|
| 32-bit floating point | The reference format for training, and the baseline others are compared against. |
| 16-bit floating point | The usual training and serving format for large networks. |
| 8-bit integer | A widely supported quantized format for serving. |
| 4-bit integer | A more aggressive format used where memory is the binding constraint. |
| Mixed precision | Different formats for different tensors, keeping the sensitive ones wider. |
Support is a hardware and library question as much as a numerical one. A format the target device does not implement natively can end up slower than the wider format it replaced, having saved memory and spent the saving on converting values back before each multiplication. Where narrower storage does buy speed, the reason is usually bandwidth rather than arithmetic: generating one token at a time reads every weight once, so a step spent waiting on memory rather than on the arithmetic units finishes sooner when there are fewer bytes to fetch.
Post-training quantization and quantization-aware training
A network can be quantized after training, by running a calibration set of representative inputs through it, recording the ranges its activations take, and choosing scales that fit them — the weights need no calibration for this, their ranges being already known. Or the eventual quantization can be simulated during training, so that the values learned are values that survive it. The first is cheap and needs no training run. The second costs a training run and generally tolerates lower precision for the same behaviour.
The calibration set is the part most easily under-specified. Scales chosen to fit the values a network produced on one collection of inputs are scales fitted to that collection, and a distribution the calibration data did not contain is approximated by an interval chosen without reference to it. A quantized copy can therefore be checked and found acceptable on inputs resembling the calibration data while the approximation is at its worst on inputs that do not resemble it, and nothing in the artefact records which is which.
In both cases the effect on output is a measurement rather than a specification. Two quantized copies of the same network at the same bit width, produced by different methods or different calibration data, are different artefacts and have to be evaluated as such.
How the Registry classifies a quantization run
Quantizing a trained network and measuring the memory reduction on target hardware is filed as data processing here: an existing artefact is transformed, and the quantity reported is derived from the result rather than looked up.
The Registry's canonical brief for this entry is filed as:
Quantize a trained network to a lower numeric precision and measure the memory reduction on the target hardware.
Submitted for assessment it is
classified as Data processing, and its wording is hashed once — to
0ee8dedd8e4397fd…, the first sixteen of sixty-four hexadecimal
characters — with the wording itself never stored. The hash is what the derivation
reads.
That class's own page is /tasks/analysis.
Classification is one of three inputs. The other two are the configuration submitted with the task, and the permanent chart derived from that configuration — fixed by the model name, the training cutoff and the temperature alone, and never reading the task at all. The same ascendant, ruling planet and harmony therefore appear on every assessment a given configuration receives, whatever it was asked to do. The derivation is published in full at /method.
What this page does not claim about quantization
How far a particular network's behaviour moves under a particular quantization scheme is a measurement, and this registry has taken none. The canonical brief above names the operation, not its result.
The Registry does not run this task, does not inspect any system's output for it, and validates no assessment it issues against what afterwards happens. What it does is compute — from a published method, for one submitted task and one submitted configuration — a verdict and a recommended execution window. It computes neither on this page.
Questions about quantization
- What is quantization?
- Quantization stores a network’s weights, and sometimes its activations, at lower numeric precision than the format they were trained in, using a recorded scale to map between the two. The architecture is unchanged; the memory and bandwidth required are reduced.
- What does 4-bit quantization mean?
- It means each stored value occupies four bits rather than the sixteen or thirty-two of the training format, with a scale recorded per tensor, channel or block to map the small integers back onto the original range. It is the more aggressive end of common practice, used where memory is the binding constraint.
- Does quantization change how a configuration behaves?
- It changes the numbers the computation runs on, so the output can differ. How much it differs is a measurement taken on the cases that matter, and it varies by network, by scheme and by which tensors were left at wider precision.
- Are weights and activations quantized the same way?
- Usually not, and most deployments quantize weights alone. A weight is fixed before the run, so its range can be measured once and recorded. An activation is produced during the run, its range depends on the input, and a few unusually large values stretch the interval the rest of the group must share — which makes activation quantization the harder of the two and the place elaborate schemes are needed.
- Is quantization the same as distillation?
- No. Quantization stores the same network’s weights more cheaply. Distillation trains a different, smaller network on the outputs of a larger one. The first produces a compressed representation; the second produces a new artefact that has to be evaluated from scratch.
Related entries
Order an assessment for data processing tasks
The Registry issues a permanent, numbered task risk assessment for one submitted task and one submitted configuration. EUR 1.90 Standard, EUR 4.90 Extended, EUR 14.90 Full Chart, which adds the permanent chart. Assessments from EUR 1.90; machine-readable at /pricing.json.