Sparsifier

Make your neural network sparse

Overview

A sparse vector, as opposed to a dense one, is a vector which contains a lot of zeroes. When we speak about making a neural network sparse, we thus mean that the network’s weights are mostly zeroes.

With fasterai, you can do that thanks to the Sparsifier class.

Found permutation search CUDA kernels [ASP][Info] permutation_search_kernels can be imported. —

source

Sparsifier


def Sparsifier(
    model:nn.Module, # The model to sparsify
    granularity:str, # Granularity of sparsification (e.g., 'weight', 'filter')
    context:str, # Context for sparsification ('global' or 'local')
    criteria:Criteria, # Criteria to determine which weights to keep
    nm:bool=False, # Whether to use N:M sparsity pattern (forces 2:4 sparsity)
    layer_type:Type[nn.Module]=<class 'torch.nn.modules.conv.Conv2d'>, # Type of layers to apply sparsification to
):

Class providing sparsifying capabilities

The Sparsifier class allows us to remove some weights, that are considered to be less useful than others. This can be done by first creating an instance of the class, specifying:

  • The granularity, i.e. the part of filters that you want to remove. Typically, we usually remove weights, vectors, kernels or even complete filters.
  • The context, i.e. if you want to consider each layer independently (local), or compare the parameters to remove across the whole network (global).
  • The criteria, i.e. the way to assess the usefulness of a parameter. Common methods compare parameters using their magnitude, the lowest magnitude ones considered to be less useful.

Key Methods

User can pass a single layer to sparsify by using the Sparsifier.sparsify_layer method.

source

Sparsifier.sparsify_layer


def sparsify_layer(
    m:nn.Module, # The layer to sparsify
    sparsity:float, # Target sparsity level (percentage)
    round_to:int | None=None, # Round to a multiple of this value
)->None:

Apply sparsification to a single layer

Most of the time, we may want to sparsify the whole model at once, using the Sparsifier.sparsify_model method, indicating the percentage of sparsity you want to apply.

source

Sparsifier.sparsify_model


def sparsify_model(
    sparsity:float | dict, # Target sparsity level or per-layer dict
    round_to:int | None=None, # Round to a multiple of this value
)->None:

Apply sparsification to all matching layers in the model

Advanced Options

In some case, you may want to impose the remaining amount of parameters to be a multiple of a given number (e.g. 8), this can be done by passing the round_to parameter.

Instead of passing a single value of sparsity, a dictionary of per-layer sparsities can be provided. This allows fine-grained control over which layers get sparsified and by how much.

Example: Apply different sparsity levels to specific layers:

sparsity_levels = {
    'conv1': 30,           # 30% sparsity on first conv
    'layer1.0.conv1': 50,  # 50% sparsity
    'layer2.0.conv1': 70,  # 70% sparsity (more aggressive)
}
sparsifier.sparsify_model(sparsity=sparsity_levels)

This works seamlessly with SensitivityResult.to_layer_targets() to apply sensitivity-aware compression.

See Also

  • SparsifyCallback - Apply sparsification during fastai training
  • Criteria - Different importance measures for selecting what to sparsify
  • Granularity - Control what gets sparsified (weights, filters, etc.)
  • Schedules - Control sparsification progression during training
  • Pruner - Structured pruning that removes filters entirely