Get more out of your backward pass

BackPACK is a library built on top of PyTorch to make it easy to extract more information from a backward pass. Some of the things you can compute:

Compute the gradient with Pytorch

from torch.nn import CrossEntropyLoss, Linear
from utils import load_mnist_data

X, y = load_mnist_data()
model = Linear(784, 10)
lossfunc = CrossEntropyLoss()
loss = lossfunc(model(X), y)


for param in model.parameters():

Install with

pip install backpack-for-pytorch 

If you use BackPACK in your research, please cite download bibtex

    title = {BackPACK: Packing more into Backprop},
    author = {Felix Dangel and Frederik Kunstner and Philipp Hennig},
    booktitle = {International Conference on Learning Representations},
    year = {2020},
    url = {}