All projects

CompleteUpdated Added

AI Quantization Laboratory

An offline-capable browser laboratory for learning how number formats and quantization work, from individual floating-point bits to quantized matrix math and the predictions of small neural networks.

About this project

At around this time, I was starting to explore tools that enabled me to run LLMs locally on my own hardware as opposed to running them in the cloud. As part of this endeavor, I was often faced with choosing between models at different quantization levels. There was a balancing act: the more heavily a model was quantized, the smaller it became (which would allow me to run a larger model on more moderate hardware with a smaller memory footprint), but the less accurate it became.

I had utilized quantization before, primarily as a means for compressing large data files. While I had an understanding of the basics, I really desired to learn more about the underlying mathematics that described the type of quantization that existed within LLMs. With that goal, the AI Quantization Laboratory was created as a means to learn this subject. Not only was this project effective at increasing my own understanding of quantization for LLMs, but it also deepened my appreciation of software-composing AI agents as tools for fun, tailored, and interactive learning.

Screenshots

Histograms and charts comparing a normally distributed tensor with its INT8, INT4 and FP8 quantized reconstructions (view full size)
The Quantization Workbench comparing INT8, INT4 and FP8 versions of the same tensor.
The Matrix Laboratory multiplying two INT8-stored matrices with FP32 products and accumulation, and a table splitting each result's error into storage and arithmetic error (view full size)
Tracing how storage and arithmetic precision each contribute to the error of a matrix product.
Accuracy and agreement for three quantized versions of a two-moons classifier, with decision-region maps for the FP64 baseline and the W8A8 INT8 network (view full size)
Comparing a small network's predictions and decision regions before and after quantization.

Resources