Quantization Methods Compared

Size and latency of seven precision configurations, three of them fasterai Quantizer calls, on a CPU and on a GPU

Overview

This notebook times seven precision configurations on a CPU model and six on a GPU model, and reports their size and latency. Three come from fasterai’s Quantizer (x86/static, torchao/int8_weight_only, torchao/int8_dynamic); the rest use .half() and torchao’s quantize_ directly. Quantizer backends and methods not run here (pt2e, qnnpack, fbgemm, onednn, method='dynamic', method='qat') are out of scope. No accuracy is measured for the quantized models; the training table below reports the baseline’s validation accuracy only.

• CPU section: ResNet-18 fine-tuned for 3 epochs on Oxford Pets at 64x64, timed on one training batch (dls.one_batch(), at ImageDataLoaders.from_name_func’s default bs=64; the page does not print the batch shape).
• GPU section: torchvision ResNet-50 with random weights (weights=None) on a random batch of 32 images at 224x224, on the GPU that the page names below.

Protocol for both sections: each configuration is timed in one block of 50 iterations after 10 warm-up iterations (3 for the torch.compile row), wall-clock time.time(), with torch.cuda.synchronize() around the GPU blocks; the blocks run once each, in the order shown, baseline first. No dispersion, no interleaved comparison and no output-parity check against the FP32 model is reported. Sizes are the state_dict files on disk, in MB of 10^6 bytes; the compiled model’s size is not measured.

import torch, torch.nn as nn, time
from torchvision.models import resnet18
from copy import deepcopy
from fastai.vision.all import *
from fasterai.quantize.quantizer import Quantizer

# Setup
path = untar_data(URLs.PETS)
files = get_image_files(path/"images")
def label_func(f): return f[0].isupper()
dls = ImageDataLoaders.from_name_func(path, files, label_func, item_tfms=Resize(64))

learn = vision_learner(dls, resnet18, metrics=accuracy)
learn.unfreeze()
learn.fit(3)

model = learn.model.cpu().eval()
sample = dls.one_batch()[0].cpu()
print(f"Baseline model: {sum(p.numel() for p in model.parameters()):,} params")

epoch	train_loss	valid_loss	accuracy	time
0	0.517184	0.375837	0.846414	00:02
1	0.331877	0.268282	0.882950	00:02
2	0.248571	0.243887	0.896482	00:02

Baseline model: 11,704,896 params

# Helper: measure size, latency, accuracy
import tempfile, os

def measure(m, name, sample=sample, n_runs=50):
    m.eval()
    # Size
    tmp = tempfile.mktemp(suffix='.pt')
    torch.save(m.state_dict(), tmp)
    size_mb = os.path.getsize(tmp) / 1e6
    os.remove(tmp)
    
    # Latency
    x = sample.to(dtype=next((p.dtype for p in m.parameters()), torch.float32))
    with torch.no_grad():
        for _ in range(10): m(x)  # warmup
        t0 = time.time()
        for _ in range(n_runs): m(x)
        latency = (time.time() - t0) / n_runs * 1000
    
    print(f"{name:30s}  size={size_mb:6.1f} MB  latency={latency:6.2f} ms")
    return {'name': name, 'size_mb': size_mb, 'latency_ms': latency}

1. Baseline (FP32)

No quantization: the fine-tuned model as trained.

results = []
results.append(measure(model, "FP32 (baseline)"))

FP32 (baseline)                 size=  46.9 MB  latency= 25.89 ms

2. FP16 (half precision)

.half() on the whole model. The file halves. On this CPU the latency is 633 ms against 26 ms for FP32, so this row is a size result, not a speed one.

model_fp16 = deepcopy(model).half()
results.append(measure(model_fp16, "FP16 (half)", sample=sample.half()))

FP16 (half)                     size=  23.5 MB  latency=633.13 ms

3. W8A8 static (x86 backend)

Weights and activations in INT8, with a calibration pass over dls.valid. At 16.8 ms against 25.9 ms it is the largest CPU latency difference on this page, and the only one above 10%, at a quarter of the size. With one timing block per configuration and no dispersion reported, the page cannot say how much of any difference is run-to-run variation.

model_w8a8 = Quantizer(backend='x86', method='static').quantize(deepcopy(model), dls.valid)
results.append(measure(model_w8a8, "W8A8 static (x86)"))

W8A8 static (x86)               size=  11.9 MB  latency= 16.75 ms

4. W8A32 weight-only (torchao)

Only the weights are INT8; activations stay FP32; no calibration. The torchao backend rewrites Linear layers only, not convolutions (see the Quantizer reference), which is why the file barely moves on a ResNet, 45.3 MB against 46.9 MB. The latency comes out 7% below FP32, 24.0 ms against 25.9 ms; with a single timing block per configuration this page cannot separate that from run-to-run variation.

model_w8a32 = Quantizer(backend='torchao', method='int8_weight_only').quantize(deepcopy(model))
results.append(measure(model_w8a32, "W8A32 weight-only (torchao)"))

W8A32 weight-only (torchao)     size=  45.3 MB  latency= 24.01 ms

5. W8A8 dynamic (torchao)

INT8 weights, activations quantized on the fly at run time; no calibration. Linear layers only, so the same 45.3 MB as W8A32; here 25.5 ms against 25.9 ms for FP32, a difference one timing block cannot resolve.

model_w8a8d = Quantizer(backend='torchao', method='int8_dynamic').quantize(deepcopy(model))
results.append(measure(model_w8a8d, "W8A8 dynamic (torchao)"))

W8A8 dynamic (torchao)          size=  45.3 MB  latency= 25.48 ms

6. W4A32 (INT4 weight-only)

Weights in 4 bits through torchao’s IntxWeightOnlyConfig, applied to Conv2d and Linear layers. The file is about four times smaller; on this CPU the latency is higher than FP32, 30.2 ms against 25.9 ms.

from torchao.quantization import quantize_, IntxWeightOnlyConfig

model_w4 = deepcopy(model)
quantize_(model_w4, IntxWeightOnlyConfig(weight_dtype=torch.int4),
          filter_fn=lambda mod, fqn: isinstance(mod, (nn.Conv2d, nn.Linear)))
results.append(measure(model_w4, "W4A32 (INT4 weight-only)"))

W4A32 (INT4 weight-only)        size=  12.0 MB  latency= 30.18 ms

7. W4A16 (INT4 weights, FP16 activations)

The INT4 model above with activations in half precision. Nearly the same size as W4A32, 11.9 MB against 12.0 MB; on CPU its latency is in the same range as the half-precision row of section 2, 658 ms against 633 ms.

model_w4a16 = deepcopy(model).half()
quantize_(model_w4a16, IntxWeightOnlyConfig(weight_dtype=torch.int4),
          filter_fn=lambda mod, fqn: isinstance(mod, (nn.Conv2d, nn.Linear)))
results.append(measure(model_w4a16, "W4A16 (INT4 + half)", sample=sample.half()))

W4A16 (INT4 + half)             size=  11.9 MB  latency=658.23 ms

GPU benchmarks

The same methods on ResNet-50 with random weights (weights=None) and a random batch of 32 images at 224x224, plus torch.compile on the FP16 model. Latency only: with random weights there is no accuracy to compare and no parity check against a trained reference. The size printed for the compiled model is typed into the cell, not measured.

if torch.cuda.is_available():
    device = torch.device('cuda')
    print(f"GPU: {torch.cuda.get_device_name(0)}")
    
    # ResNet-50, batch=32 — realistic workload that saturates the GPU
    from torchvision.models import resnet50 as tv_resnet50
    gpu_model = tv_resnet50(weights=None).eval()
    gpu_sample = torch.randn(32, 3, 224, 224)
    
    def measure_gpu(m, name, sample_in=None, n_runs=50):
        if sample_in is None: sample_in = gpu_sample
        m = m.to(device).eval()
        x = sample_in.to(device, dtype=next((p.dtype for p in m.parameters()), torch.float32))
        
        import tempfile, os
        tmp = tempfile.mktemp(suffix='.pt')
        torch.save(m.cpu().state_dict(), tmp)
        size_mb = os.path.getsize(tmp) / 1e6
        os.remove(tmp)
        m = m.to(device)
        
        with torch.no_grad():
            for _ in range(10): m(x)
            torch.cuda.synchronize()
            t0 = time.time()
            for _ in range(n_runs): m(x)
            torch.cuda.synchronize()
            latency = (time.time() - t0) / n_runs * 1000
        
        print(f"{name:35s}  size={size_mb:6.1f} MB  latency={latency:6.2f} ms")
        return {'name': name, 'size_mb': size_mb, 'latency_ms': latency}
    
    gpu_results = []
    
    # FP32
    gpu_results.append(measure_gpu(deepcopy(gpu_model), "FP32 (baseline)"))
    
    # FP16
    gpu_results.append(measure_gpu(deepcopy(gpu_model).half(), "FP16 (half)",
                                   sample_in=gpu_sample.half()))
    
    # W8A32 torchao
    m_w8 = Quantizer(backend='torchao', method='int8_weight_only').quantize(deepcopy(gpu_model))
    gpu_results.append(measure_gpu(m_w8, "W8A32 weight-only"))
    
    # W4A32
    from torchao.quantization import quantize_, IntxWeightOnlyConfig
    m_w4 = deepcopy(gpu_model)
    quantize_(m_w4, IntxWeightOnlyConfig(weight_dtype=torch.int4),
              filter_fn=lambda mod, fqn: isinstance(mod, (nn.Conv2d, nn.Linear)))
    gpu_results.append(measure_gpu(m_w4, "W4A32 (INT4 weight-only)"))
    
    # W4A16
    m_w4fp16 = deepcopy(gpu_model).half()
    quantize_(m_w4fp16, IntxWeightOnlyConfig(weight_dtype=torch.int4),
              filter_fn=lambda mod, fqn: isinstance(mod, (nn.Conv2d, nn.Linear)))
    gpu_results.append(measure_gpu(m_w4fp16, "W4A16 (INT4 + half)",
                                   sample_in=gpu_sample.half()))
    
    # FP16 + torch.compile
    import logging
    logging.getLogger("torch._inductor").setLevel(logging.ERROR)
    m_comp = torch.compile(deepcopy(gpu_model).half().to(device), mode='max-autotune')
    x_comp = gpu_sample.half().to(device)
    with torch.no_grad():
        for _ in range(3): m_comp(x_comp)  # compile warmup
        torch.cuda.synchronize()
        t0 = time.time()
        for _ in range(50): m_comp(x_comp)
        torch.cuda.synchronize()
        comp_lat = (time.time() - t0) / 50 * 1000
    gpu_results.append({'name': 'FP16 + torch.compile', 'size_mb': 49.7, 'latency_ms': comp_lat})
    print(f"{'FP16 + torch.compile':35s}  size=  49.7 MB  latency={comp_lat:6.2f} ms")
    
    import pandas as pd
    df_gpu = pd.DataFrame(gpu_results)
    df_gpu['speedup'] = df_gpu['latency_ms'].iloc[0] / df_gpu['latency_ms']
    print()
    print(df_gpu[['name', 'size_mb', 'latency_ms', 'speedup']].to_string(index=False))
else:
    print("No GPU available — skip GPU benchmarks")

GPU: NVIDIA GeForce RTX 5090
FP32 (baseline)                      size= 102.5 MB  latency=  7.66 ms
FP16 (half)                          size=  51.3 MB  latency=  4.24 ms
W8A32 weight-only                    size=  96.4 MB  latency=  7.66 ms
W4A32 (INT4 weight-only)             size=  26.4 MB  latency=  8.38 ms
W4A16 (INT4 + half)                  size=  26.0 MB  latency=  5.09 ms
FP16 + torch.compile                 size=  49.7 MB  latency=  2.25 ms

                    name    size_mb  latency_ms  speedup
         FP32 (baseline) 102.542935    7.661738 1.000000
             FP16 (half)  51.322647    4.239492 1.807230
       W8A32 weight-only  96.412315    7.661681 1.000007
W4A32 (INT4 weight-only)  26.376239    8.378868 0.914412
     W4A16 (INT4 + half)  26.040111    5.086675 1.506237
    FP16 + torch.compile  49.700000    2.250066 3.405117

Comparison

CPU section, from the results list above.

import pandas as pd

df = pd.DataFrame(results)
df['size_reduction'] = df['size_mb'].iloc[0] / df['size_mb']
df['speedup'] = df['latency_ms'].iloc[0] / df['latency_ms']

print(df[['name', 'size_mb', 'latency_ms', 'size_reduction', 'speedup']].to_string(index=False))

                       name   size_mb  latency_ms  size_reduction  speedup
            FP32 (baseline) 46.912607   25.891147        1.000000 1.000000
                FP16 (half) 23.477471  633.130608        1.998197 0.040894
          W8A8 static (x86) 11.870623   16.752791        3.951992 1.545483
W8A32 weight-only (torchao) 45.344999   24.007483        1.034571 1.078462
     W8A8 dynamic (torchao) 45.340899   25.482626        1.034664 1.016031
   W4A32 (INT4 weight-only) 12.047031   30.175323        3.894122 0.858024
        W4A16 (INT4 + half) 11.918135  658.228807        3.936237 0.039335

Summary

The figures are the ones printed above, rounded to the precision shown; this table restates them.

Configuration	CPU: ResNet-18, one training batch at 64x64	GPU: ResNet-50, batch of 32 at 224x224
FP32	46.9 MB, 25.9 ms	102.5 MB, 7.66 ms
FP16 (.half())	23.5 MB, 633 ms	51.3 MB, 4.24 ms
W8A8 static (x86)	11.9 MB, 16.8 ms	not run
W8A32 weight-only (torchao)	45.3 MB, 24.0 ms	96.4 MB, 7.66 ms
W8A8 dynamic (torchao)	45.3 MB, 25.5 ms	not run
W4A32 (INT4 weight-only)	12.0 MB, 30.2 ms	26.4 MB, 8.38 ms
W4A16 (INT4 + half)	11.9 MB, 658 ms	26.0 MB, 5.09 ms
FP16 + torch.compile	not run	size not measured, 2.25 ms

Measured on: CPU section, an x86 CPU that the page does not identify; ResNet-18 fine-tuned 3 epochs on Oxford Pets, one training batch at 64x64 (default bs=64, shape not printed). GPU section, NVIDIA GeForce RTX 5090; ResNet-50 with random weights, random 32x3x224x224 input. Each configuration: one block of 50 iterations after 10 warm-ups (3 for the compiled row), wall-clock time, run once, no dispersion, no interleaving, no parity check. Nothing here was measured on other hardware, batch sizes or models.

Tool	Used for
Quantizer(backend='x86', method='static')	W8A8 static with calibration
Quantizer(backend='torchao', method='int8_weight_only')	W8A32 weight-only
Quantizer(backend='torchao', method='int8_dynamic')	W8A8 dynamic
torchao quantize_ with IntxWeightOnlyConfig(weight_dtype=torch.int4)	W4A32 and W4A16

---

See Also

• Quantizer API - backends, methods and the precision grammar
• Deployable Export - exporting a quantized model and checking what was produced
• Walkthrough - end-to-end optimization pipeline