jusjinuk

Llama-3.1-8B-Instruct-4bit-GuidedQuant-QTIP

jusjinuk/Llama-3.1-8B-Instruct-4bit-GuidedQuant-QTIP

Llama-3.1-8B-Instruct-4bit-GuidedQuant-QTIP is a machine learning model from jusjinuk. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.1.

- Base model: meta-llama/Llama-3.1-8B-Instruct - Quantization method: BlockLDLQ with GuidedQuant Hessian - Target bit-width: 4 - Backend kernel: QTIP kernel (HYB variant) - Calibration data: RedPajama (1024 sentences…