Inkling-2-Mini in 5.4 GB Memory
Inkling-2-Mini is a compressed language model that achieves impressive performance with a reduced memory footprint. Quantization compiler LookAtMyQuant plays a key role in determining how much information loss occurs during compression, optimizing the model to fit within size constraints while minimizing damage to its output. The model demonstrates strong capabilities on various mathematical problem sets, scoring well on tests like GSM8K and HumanEval while using significantly less memory compared to the original model.
Rewritten by urgent.news; may contain errors — check the original.