Precision

How many bits to keep, on which axis, and which backend can honor it

Overview

Compression asks four questions. What block of parameters to remove is granularity, which ones matter is criteria, when to act is schedules — and how few bits to keep the survivors in is precision.

A precision request has six parts:

Part Argument Values
Weight width weight_bits 4, 8, 16, or a {layer_name: width} dict
Activation width act_bits 8, 16
Weight axis qscheme 'per_tensor', 'per_channel', 'per_group'
Symmetry symmetric True (every zero-point is 0), False (affine)
Q/DQ placement qdq_placement 'per_op', 'skip_conv_add' (the pt2e cell only)
Activations left in floating point skip_activations a list of module names (the pt2e cell only)

Two conventions run through the grammar:

  • A width of 16 means not quantized: the tensor keeps the model’s floating-point dtype. One label then covers fp16 and fp32 models, so INT8 weight-only quantization is written W8A16 whatever the model’s float type. weight_bits={'head': 16} is how you keep one layer out of the quantization.
  • The finer the axis, the more scales: per_tensor keeps one scale for the whole tensor, per_channel one per output channel, per_group one per fixed-size block of weights.
  • A placement is about where the pairs sit, not how many bits: per_op quantizes the result of every operator the flow annotates; skip_conv_add leaves one edge — the residual branch’s convolution feeding its add — without a Q/DQ pair, so that result reaches the add at accumulator precision. It changes the arithmetic, so it is opt-in and recorded on the spec.

Not every combination exists. A backend can only apply the cells its observers and kernels implement, and only some of those survive an ONNX export. PRECISION_SUPPORT is that matrix, and Quantizer validates every request against it at construction: a cell a backend cannot honor raises immediately, naming a backend that can, instead of being silently rounded to something the backend can do.

Precision cells

A PrecisionCell is one row of the matrix: a backend, a precision, and what that pair can do.

source

PrecisionCell

def PrecisionCell(
    backend:str, weight_bits:int, act_bits:int, qschemes:tuple[str, ...], symmetries:tuple[bool, ...],
    per_layer:bool, exports:bool, note:str, default_group_size:int | None=None, qdq_placements:tuple[str, ...]=(),
    skips_activations:bool=False
)->None:

What one (backend, weight width, activation width) combination can do

The support matrix

The table below is rendered from PRECISION_SUPPORT rather than written by hand, so it cannot drift away from what the code enforces.

Markdown(precision_table())

source

precision_table

def precision_table()->str:

Markdown view of PRECISION_SUPPORT — the matrix is the code, this is only its rendering

The resolved spec

_resolve_spec(...) turns a request into exactly one QuantSpec, or raises. Quantizer attaches the result to the model it quantizes, where quant_spec(model) reads it back — that is what export_qdq consults to refuse a precision it cannot write.

source

QuantSpec

def QuantSpec(
    backend:str, method:str, weight_bits:int, act_bits:int, qscheme:str, symmetric:bool, group_size:int | None=None,
    layer_bits:dict | None=None, qdq_placement:str | None=None, skip_activations:tuple[str, ...] | None=None,
    engine:str | None=None
)->None:

The single precision cell a Quantizer resolved to — attached to the model it quantizes

source

quant_spec

def quant_spec(
    model, # Any model, quantized or not
)->QuantSpec | None:

The precision Quantizer applied to model, or None when fasterai did not quantize it

FakeQuantSpec is the same idea for a precision nothing deploys: FakeQuantizer rounds a model onto a width pair in floating point and attaches one of these, which fake_quant_spec(model) reads back. It sits here so that “what precision is this model at?” needs no quantizer import.

Three divergences from QuantSpec are deliberate, not slips:

QuantSpec FakeQuantSpec
a tensor left in floating point 16, since no backend runs a 16-bit grid None; 16 is a real 16-bit grid
label renders that as W8A16 W8AF
per-layer activations skip_activations, a tuple of names layer_act_bits, a width per name

Three fields have no counterpart on the deployed side, and all three come from FakeQuantizeCallback: trained is True when the model was fitted through the rounding, and False on every post-training path; weight_widths and act_widths hold the ladder of widths a schedule was asked to step down, when one was, so a model that arrived at its final width through a ladder is not read as one trained at that width throughout. They record the request, not the rungs a given fit had time to visit. weight_bits and act_bits are the widths the model ended at either way.

source

FakeQuantSpec

def FakeQuantSpec(
    weight_bits:int | None, act_bits:int | None, qscheme:str, symmetric:bool, observer:str,
    group_size:int | None=None, layer_bits:dict | None=None, layer_act_bits:dict | None=None, trained:bool=False,
    weight_widths:tuple[int, ...] | None=None, act_widths:tuple[int, ...] | None=None
)->None:

The widths a model was rounded to, and whether it trained through them — attached to that model

source

fake_quant_spec

def fake_quant_spec(
    model, # Any model, rounded or not
)->FakeQuantSpec | None:

The widths FakeQuantizer rounded model to, or None when fasterai did not round it

Resolution

One resolver, one place where a request is accepted or refused. Every refusal names the argument at fault and, when there is one, a backend that can honor it.

Usage Examples

The grammar is reached through Quantizer, which forwards its precision arguments to _resolve_spec(...):

from fasterai.quantize.quantizer import Quantizer
from fasterai.core.all import quant_spec

# The deployable cell: symmetric INT8, per-channel weights, exportable as QDQ ONNX
quantizer = Quantizer(backend='pt2e', weight_bits=8, act_bits=8, qscheme='per_channel', symmetric=True)
model_q = quantizer.quantize(model, calibration_dl=dls.valid)
print(quant_spec(model_q).as_dict())
# {'backend': 'pt2e', 'method': 'static', 'weight_bits': 8, 'act_bits': 8, 'qscheme': 'per_channel',
#  'symmetric': True, 'group_size': None, 'layer_bits': None, 'qdq_placement': 'per_op',
#  'skip_activations': None, 'engine': None}

# INT8 weight-only, one scale per group of 64 weights
Quantizer(backend='torchao', weight_bits=8, act_bits=16, qscheme='per_group', group_size=64)

# Keep the classifier out of the quantization (16 = left in floating point)
Quantizer(backend='x86', weight_bits={'fc': 16})

# The same dict on torchao, over the Linear layers it rewrites: INT8 everywhere except one layer
Quantizer(backend='torchao', weight_bits={'layers.0.linear1': 16}, act_bits=16)

A {layer_name: width} dict says which layers are quantized; the width they are quantized at is the uniform one, so the two spellings below resolve to the same cell. Layers the dict does not name keep that uniform width — the dict is an override list, not a whitelist.

_resolve_spec('torchao', weight_bits={'fc': 16}, act_bits=16).weight_bits          # 8
_resolve_spec('torchao', weight_bits={'fc': 16, 'head': 8}, act_bits=16).weight_bits  # 8

qdq_placement names where the Q/DQ pairs sit rather than how many bits they keep. Only the pt2e cell has that axis, so it is the only one whose spec carries a value; everywhere else the field stays None and naming a placement raises:

Quantizer(backend='pt2e', qdq_placement='skip_conv_add')  # the residual conv->add edge stays unquantized
_resolve_spec('pt2e').qdq_placement                       # 'per_op' — asking for it explicitly is the same request
_resolve_spec('x86').qdq_placement                        # None — the FX flow has no such axis

skip_activations is the same kind of axis: it names modules whose activations keep the model’s float dtype, while their weights and their inputs stay INT8. Only the pt2e cell can do it, because it is the only flow fasterai annotates itself:

Quantizer(backend='pt2e', skip_activations=['features.0'])  # this block's activations stay in floating point
_resolve_spec('pt2e', skip_activations=['features.0']).skip_activations   # ('features.0',)
_resolve_spec('x86', skip_activations=['features.0'])
# ValueError: backend='x86' quantizes its activations for the whole graph at once: it cannot leave one
# module's activations in floating point. The backend(s) that can: ['pt2e'].

Asking for a cell a backend cannot honor raises, and says who can:

Quantizer(backend='x86', symmetric=True)
# ValueError: backend='x86' W8A8 cannot honor symmetric=True — its observers are affine by
# construction: activations carry a non-zero zero-point. The backend(s) that can: ['pt2e', 'torchao'].

Quantizer(backend='torchao', weight_bits={'fc': 8})   # W8A8: torchao's dynamic-activation recipe
# ValueError: backend='torchao' cannot honor a per-layer `weight_bits` dict at W8A8, only at W8A16:
# add act_bits=16.

See Also

Tests live in nbs/tests/test_precision.ipynb.