fredericowieser

TinyLlama_v1.1_mix_wikitext_alpaca_1bit_BitDistiller_baseline

fredericowieser/TinyLlama_v1.1_mix_wikitext_alpaca_1bit_BitDistiller_baseline

TinyLlama_v1.1_mix_wikitext_alpaca_1bit_BitDistiller_baseline is a machine learning model from fredericowieser. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

This is a 1-bit quantized version of TinyLlama v1.1, trained using BitDistiller with asymmetric quantization and self-distillation (CAKLD) to optimize accuracy retention under extreme compression. The model is fine-tu…