epfml

landmark-attention-llama7b-wdiff

epfml/landmark-attention-llama7b-wdiff

landmark-attention-llama7b-wdiff is a text generation model from epfml. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-sa-4.0.

This repo hosts the weight diff between LLaMA 7B trained with landmark attention for 15000 steps on RedPajama and the original model. Please visit the Github repository for further instructions on how to recover the f…