*Equal taxpayers
To deploy machine learning models on the device, professionals use compression algorithms to shrink and speed up the models while maintaining their high-quality output. A critical aspect of compression in practice is model comparison, including tracking many compression experiments, identifying subtle changes in model behavior, and negotiating complex trade-offs between accuracy and efficiency. However, existing compression tools do not support comparison well, leading to tedious and sometimes incomplete analyzes spread across disjoint tools. To support real-world benchmarking workflows, we developed an interactive visual system called Compress & Compare. Within a single interface, Compress and Compare demonstrates promising compression strategies by visualizing provenance relationships between compressed models and reveals compression-induced behavioral changes by comparing model predictions, weights, and activations. We demonstrate how Compress and Compare supports common compression analysis tasks through two case studies, debugging failed compression in generative language models and identifying compression artifacts in image classification models. We further evaluated Compress and Compare in a user study with eight compression experts, illustrating its potential to provide structure to compression workflows, help professionals develop intuition about compression, and encourage deep analysis. of the effect of compression on the behavior of the model. Through these evaluations, we identified specific compression challenges that future visual analytics tools should consider and compression and comparison visualizations that can be generalized to broader model comparison tasks.
