mattshumer

Llama-3-8B-16K

mattshumer/Llama-3-8B-16K

Llama-3-8B-16K is a text generation model from mattshumer. Use it when you need the model to write or continue text. It is set up for transformers.

This is an extended (16K) context version of LLaMA 3 8B (base, not instruct). Trained for five hours on 8x A6000 GPUs, using the Yukang/LongAlpaca-16k-length dataset.