ProLong-512k-8B-WritingPrompts

ProLong-512k-8B-WritingPrompts

ProLong-512k-8B-CLIPPER is a fine-tuned version of princeton-nlp/Llama-3-8B-ProLong-512k-Instruct using supervised finetuning over chtmp223/CLIPPER dataset. Please check our paper for more details on the method.

📒 Model Details

Model Description

• Language(s) (NLP): English
• License: Apache-2.0
• Finetuned from model: princeton-nlp/Llama-3-8B-ProLong-512k-Instruct](https://huggingface.co/princeton-nlp/Llama-3-8B-ProLong-512k-Instruct)

Model Sources

• Repository: Github repository.
• Paper: https://arxiv.org/abs/2502.14854

💻 Training Details

Training Data

chtmp223/CLIPPER-WritingPrompts

Training Procedure

Software

Training code is adapted from https://github.com/princeton-nlp/ProLong.

🤗 Inference

Inference is done with vLLM on 1 A100-80GB.

📜 Citation

@misc{pham2025clippercompressionenableslongcontext,
      title={CLIPPER: Compression enables long-context synthetic data generation}, 
      author={Chau Minh Pham and Yapei Chang and Mohit Iyyer},
      year={2025},
      eprint={2502.14854},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.14854}, 
}