Meta-Llama-3.1-8B-Instruct-gptq-w4a16-c2

Meta-Llama-3.1-8B-Instruct-gptq-w4a16-c2

Gptq 4-bit weight quantization (w4a16, group size 128) of meta-llama/Meta-Llama-3.1-8B-Instruct, calibrated on C2 (C4 web text (generic text)). One of the 30 compressed configurations released with the ICAIF '26 paper Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMs, which studies when post-training compression of financial LLMs needs domain-matched calibration data.

Code, calibration corpora, evaluation suite and all other configurations: github.com/JunyiYe/finance-ptc-calibration - full collection: Financial LLM Compression & Calibration (ICAIF 2026).

Usage

The checkpoint is in compressed-tensors format and loads directly in vLLM, or in transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("jy395/Meta-Llama-3.1-8B-Instruct-gptq-w4a16-c2", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("jy395/Meta-Llama-3.1-8B-Instruct-gptq-w4a16-c2")

To reproduce the paper's evaluation (ten financial tasks), see the evaluation suite in the code release.

License

Built with Llama. This model is a derivative of Meta-Llama-3.1-8B-Instruct and is distributed under the Llama 3.1 Community License.

Citation

@inproceedings{ye2026damage,
  title     = {Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMs},
  author    = {Ye, Junyi and Yu, Mengjia and Hazra, Debapriya and Wang, Guiling},
  booktitle = {Proceedings of the 7th ACM International Conference on AI in Finance (ICAIF '26)},
  year      = {2026},
  address   = {Milan, Italy},
  publisher = {ACM},
  doi       = {10.1145/3858063.3858724}
}