Abstract: 

Quantization refers to the practice of taking a neural network's painstakingly-tuned FP32 parameters and rounding that to an integer. Why would anyone possibly want to do that?! For one, it makes the model lighter, reduces how much power it consumes, and speeds up inferences - all without obliterating model accuracy! Sounds unreal, except it isn’t.

In this session, we'll learn more about quantization theory from first principles, and see how you can implement it yourself in PyTorch. We'll break down all of the available approaches to quantize your model, their benefits and pitfalls, and most importantly how you can make an informed decision for your use case. Finally, we put our learnings to the test on a large non-academic model to see how this works in the real world.

Bio: 

Suraj is an ML engineer and developer advocate at Meta AI. In a previous life, he was a data scientist in personal finance. After being bitten by the deep learning bug, he worked in healthcare research (predicting patient risk factors) and behavioral finance (preventing overly-risky trading). Outside of work, you can find him hiking barefoot in the Catskills or being tossed on the Aikido mat.