Overview

Methods • Features • Installation • Tutorials • Citing • License

fasterai is a library created to make neural network smaller and faster. It essentially relies on common compression techniques for networks such as pruning, knowledge distillation, Lottery Ticket Hypothesis, …

The core feature of fasterai is its Sparsifying capabilities, constructed around 4 main modules: granularity, context, criteria, schedule. Each of these modules is highly customizable, allowing you to change them according to your needs or even to come up with your own!

Project Documentation

Visit Read The Docs Project Page or read following README to know more about using fasterai.

Features

1. Sparsifying

Make your model sparse according to a:
- Sparsity: the percentage of weights that will be replaced by 0
- Granularity: the granularity at which you operate the pruning (removing weights, vectors, kernels, filters)
- Context: prune either each layer independantly (local pruning) or the whole model (global pruning)
- Criteria: the criteria used to select the weights to remove (magnitude, movement, …)
- Schedule: which schedule you want to use for pruning (one shot, iterative, gradual, …)

This can be achieved by using the SparsifyCallback(sparsity, granularity, context, criteria, schedule)

2. Pruning

Once your model has useless nodes due to zero-weights, they can be removed to not be a part of the network anymore.

This can be achieved by using the PruneCallback(sparsity, context, criteria, schedule)

3. Regularization

Instead of explicitely make your network sparse, let it train towards sparse connections by pushing the weights to be as small as possible.

Regularization can be applied to groups of weights, following the same granularities as for sparsifying, i.e.: - Granularity: the granularity at which you operate the regularization (weights, vectors, kernels, filters, …)

This can be achieved by using the RegularizationCallback(granularity)

4. Knowledge Distillation

alt text

Distill the knowledge acquired by a big model into a smaller one, by using the KnowledgeDistillation callback.

5. Lottery Ticket Hypothesis

Find the winning ticket in you network, i.e. the initial subnetwork able to attain at least similar performances than the network as a whole.

The FasterAI Ecosystem

fasterai is part of a family of libraries designed to make neural network optimization accessible:

Package Purpose When to Use
fasterai Compression techniques Pruning, sparsification, distillation, quantization during training
fasterbench Benchmarking Measuring model size, speed, memory, compute, and energy
fasterlatency Latency prediction Hardware-aware neural architecture search
fasterrecipes High-level workflows Quick compression with sensible defaults

Typical Workflow

  1. Benchmark your model with fasterbench to identify bottlenecks
  2. Compress using fasterai techniques (pruning, distillation, quantization)
  3. Validate compression impact with fasterbench
  4. Deploy the optimized model

Or use fasterrecipes for one-line compression with sensible defaults!

Quick Start

0. Import fasterai

from fasterai.sparse.all import *

1. Create your model with fastai

learn = cnn_learner(dls, model)

2. Get you Fasterai Callback

sp_cb=SparsifyCallback(sparsity, granularity, context, criteria, schedule)

3. Train you model to make it sparse !

learn.fit_one_cycle(n_epochs, cbs=sp_cb)

Installation

pip install git+https://github.com/nathanhubens/fasterai.git

or

pip install fasterai

Tutorials

Citing

@software{Hubens,
  author       = {Nathan Hubens},
  title        = {fasterai},
  year         = 2022,
  publisher    = {Zenodo},
  version      = {v0.1.6},
  doi          = {10.5281/zenodo.6469868},
  url          = {https://doi.org/10.5281/zenodo.6469868}
}

License

Apache-2.0 License.