Sensitivity Analysis

Per-layer sensitivity analysis for compression methods

Overview

Not all layers in a neural network are equally important. Some are fragile — compressing them even slightly degrades performance — while others are robust and can be heavily compressed with minimal impact. Sensitivity analysis measures this per-layer fragility, enabling smarter compression strategies.

The SensitivityAnalyzer works by compressing one layer at a time, measuring the impact on a user-provided evaluation metric, and ranking layers by their sensitivity (delta from baseline).

Key Features:

Supports three compression types: sparsity (weight zeroing), pruning (structural filter removal), and quantization (precision reduction)
Generates non-uniform per-layer targets via to_layer_targets() — fragile layers get less compression, robust layers get more
Uses fasterai’s Sparsifier and Pruner internally for consistent results
Visualization with plot() and export to pandas with to_dataframe()

When to Use

Compression Type	`level` parameter	What it tests	Best for
`"sparsity"`	% of weights zeroed (e.g., 50)	Unstructured weight removal	Generating per-layer sparsity targets
`"pruning"`	% of filters removed (e.g., 30)	Structural filter pruning	Identifying layers to protect via `ignored_layers`
`"quantization"`	Bit width (e.g., 8)	Precision reduction	Finding layers that need higher precision

Data Classes

Sensitivity results are returned as structured dataclasses for easy inspection, sorting, and export.

Skipping import of cpp extensions due to incompatible torch version 2.9.1+cu128 for torchao version 0.16.0             Please see https://github.com/pytorch/ao/issues/2919 for more info

source

LayerSensitivity


def LayerSensitivity(
    name:str, layer_type:str, params:int, baseline_metric:float, compressed_metric:float, delta:float,
    group_id:int | None=None, group_members:list[str]=<factory>, prunable:bool=True
)->None:

Sensitivity result for a single layer.

LayerSensitivity holds the result for a single layer: the metric before and after compression, the delta (positive = degradation), and layer metadata. Use as_dict() to serialize.

source

SensitivityResult


def SensitivityResult(
    compression_type:str, compression_level:float, baseline_metric:float, layers:list[LayerSensitivity],
    metric_name:str='accuracy', higher_is_better:bool=True
)->None:

Structured result from sensitivity analysis.

SensitivityResult aggregates all per-layer results. It provides methods to inspect, rank, visualize, and convert sensitivity data into actionable compression targets.

Key Methods

source

SensitivityResult.top


def top(
    n:int=5, # number of layers to return
    most_sensitive:bool=True, # True=highest delta (fragile), False=lowest (robust)
)->list[LayerSensitivity]:

Return top N most or least sensitive layers.

Layers that could not be pruned (prunable=False) are excluded — a no-op prune is NOT evidence of robustness, so it must never rank as compressible.

source

SensitivityResult.to_layer_targets


def to_layer_targets(
    model:nn.Module, # model (used for parameter counts)
    target_pct:float=50, # target mean compression percentage
    min_pct:float=0, # minimum compression for any layer
    max_pct:float=90, # maximum compression for any layer
    gamma:float=1.0, # exponent for sensitivity scaling (higher = more differentiation)
)->dict[str, float]:

Convert sensitivity to non-uniform per-layer compression targets.

High sensitivity layers get lower compression, robust layers get higher. Uses parameter-weighted optimization to hit target_pct exactly.

Coupled layers (same group_id) are optimized as a SINGLE knob — they physically share one pruning ratio — then the group’s target is expanded back to every member, so the returned dict stays per-layer. Layers that are not prunable receive min_pct.

The to_layer_targets() method converts sensitivity scores into a dict[str, float] mapping layer names to compression percentages. This dict can be passed directly to Sparsifier.sparsify_model() or SparsifyCallback(sparsity=...) for non-uniform compression.

The gamma parameter controls differentiation: higher values protect fragile layers more aggressively.

targets = result.to_layer_targets(model, target_pct=50, min_pct=10, max_pct=80, gamma=1.5)
# {'conv1': 62.5, 'layer1.0.conv1': 65.3, 'layer1.1.conv2': 10.0, ...}

source

SensitivityResult.summary


def summary(
    top:int=5, # number of layers to show per category
)->None:

Print a formatted summary of sensitivity analysis.

source

SensitivityResult.plot


def plot(
    figsize:tuple=(12, 5), # figure size (width, height)
)->None:

Plot sensitivity as a bar chart.

source

SensitivityResult.to_dataframe


def to_dataframe(
    
):

Convert to pandas DataFrame.

SensitivityAnalyzer

The SensitivityAnalyzer class provides full control over the analysis process. For quick one-off analysis, see analyze_sensitivity() below.

source

SensitivityAnalyzer


def SensitivityAnalyzer(
    model:nn.Module, # model to analyze
    sample:torch.Tensor, # example input (for Pruner dependency analysis)
    eval_fn:Callable[[nn.Module], float], # evaluation function returning metric
    criteria:Criteria=<fasterai.core.criteria.Criteria object>, # fasterai criteria for importance scoring
    higher_is_better:bool=True, # whether higher metric values are better
    metric_name:str='accuracy', # name of the metric for display
    device:str | torch.device | None=None, # device for computation
    calibration_data:torch.Tensor | None=None, # for observer-based quantization
):

Analyze per-layer sensitivity to compression methods.

Uses fasterai’s Sparsifier for sparsity analysis and Pruner for structural pruning. Supports sparsity (weight zeroing), pruning (structural), and quantization.

The eval_fn should take a nn.Module and return a scalar metric (e.g., accuracy, loss). The analyzer will call it once per layer, so it should be reasonably fast — a forward pass on a small validation batch is typical.

Key Methods

source

SensitivityAnalyzer.analyze


def analyze(
    compression:Literal['sparsity', 'pruning', 'quantization']='sparsity', # compression type
    level:float=50, # compression level (% for sparsity/pruning, bits for quant)
    granularity:str='weight', # granularity for sparsity (fasterai granularities)
    layers:list[str] | None=None, # specific layer names to analyze (None = all)
    layer_types:type | tuple[type, ...] | None=None, # restrict to a module type or tuple of types, e.g. nn.Conv2d or (nn.Conv2d, nn.Linear) (None = all compressible)
    quant_per_channel:bool=True, # use per-channel quantization
    quant_activations:bool=False, # also quantize activations
    verbose:bool=True, # print progress
)->SensitivityResult:

Analyze per-layer sensitivity to compression.

For pruning, each layer is pruned with the per-layer target {name: level} — the exact operation Pruner/PruneCallback perform — so the reported Δ faithfully predicts the degradation of really pruning that layer at level. Residual/skip-coupled layers prune together and therefore share a Δ (tagged with a common group_id); layers that cannot be pruned independently (output Linear, attention) are marked prunable=False.

Pass layer_types to restrict the analysis to specific module types — a single type or a tuple, e.g. layer_types=nn.Conv2d analyses only convolutions and skips the classifier Linear.

Pruning is group-aware and faithful. In pruning mode each layer is pruned with the exact per-layer target {name: level} that Pruner/PruneCallback use, so the reported delta is the degradation you get from really pruning that layer at level — not an approximation. Residual/skip-connected layers (e.g. a ResNet stem conv, block conv2s, downsamples) share output channels and can only be pruned together, so they are pruned as a group and report the same delta, tagged with a shared group_id. Layers that cannot be pruned independently (the output Linear, attention qkv) are marked prunable=False and surfaced separately rather than ranked as “robust”. to_layer_targets() collapses each group_id to a single knob so coupled layers aren’t double-counted.

⚠️ The delta is level-specific — an analysis at level=50 predicts a 50% prune. To plan a 10% prune, analyze at level=10.

For quantization analysis, additional parameters control the behavior:

quant_per_channel=True — per-channel quantization (more accurate, standard for weights)
quant_activations=False — set to True to also quantize activations (slower but more realistic)
level is interpreted as bit width (e.g., 8 for INT8) instead of percentage

source

SensitivityAnalyzer.sweep


def sweep(
    compression:Literal['sparsity', 'pruning', 'quantization']='sparsity', # compression type
    levels:list[float] | None=None, # compression levels to test (default: [25, 50, 75])
    kwargs:VAR_KEYWORD
)->list[SensitivityResult]:

Run sensitivity analysis at multiple compression levels.

Convenience Function

For quick one-off analysis without creating an analyzer instance:

source

analyze_sensitivity


def analyze_sensitivity(
    model:nn.Module, # model to analyze
    sample:torch.Tensor, # example input tensor
    eval_fn:Callable[[nn.Module], float], # evaluation function returning metric
    compression:Literal['sparsity', 'pruning', 'quantization']='sparsity', # compression type
    level:float=50, # compression level (% for sparsity/pruning, bits for quant)
    criteria:Criteria=<fasterai.core.criteria.Criteria object>, # fasterai criteria for importance scoring
    higher_is_better:bool=True, # whether higher metric values are better
    metric_name:str='accuracy', # name of the metric for display
    granularity:str='weight', # granularity for sparsity
    verbose:bool=True, # print progress
    kwargs:VAR_KEYWORD
)->SensitivityResult:

One-line sensitivity analysis using fasterai compression methods.

Usage Example

from fasterai.analyze.sensitivity import analyze_sensitivity

result = analyze_sensitivity(model, sample, eval_fn, compression="sparsity", level=50)

# Inspect results
result.summary()                          # formatted console output
fragile = result.top(5, most_sensitive=True)  # most sensitive layers

# Generate per-layer targets for non-uniform compression
targets = result.to_layer_targets(model, target_pct=50, min_pct=10, max_pct=80)
# Pass directly to Sparsifier: sparsifier.sparsify_model(sparsity=targets)

Overview

When to Use

Data Classes

LayerSensitivity

SensitivityResult

Key Methods

SensitivityResult.top

SensitivityResult.to_layer_targets

SensitivityResult.summary

SensitivityResult.plot

SensitivityResult.to_dataframe

SensitivityAnalyzer

SensitivityAnalyzer

Key Methods

SensitivityAnalyzer.analyze

SensitivityAnalyzer.sweep

Convenience Function

analyze_sensitivity

Usage Example

See Also