Downloads · 30 days
0
Llama2-13B-Lora-Tiny-Opendomain-Example
Blackroot/Llama2-13B-Lora-Tiny-Opendomain-Example
Llama2-13B-Lora-Tiny-Opendomain-Example is a machine learning model from Blackroot. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as unlicense.
Access
Public
Updated Jul 24, 2023
Repo size
26.3 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin26.3 MB · 100%
At a glance
- License
- unlicense
- Access
- Public
- Created
- Jul 24, 2023
- Updated
- Jul 24, 2023
- SHA
- f164cd55
Model card
From the Hugging Face model README
Base model
Thanks to Meta Ai for the base model
https://huggingface.co/meta-llama/Llama-2-13b-hf
Demo LORA
This LORA is meant to serve as a simple, repeatable demo, on about one-hour of training for an LLM. The results are interesting, and there's a clear change in behaviour for the model when the lora is applied. It does produce generally longer output, though it's also somewhat off the rails because this training is very haphazard. This model is meant to serve as a demo piece on small-set training and how short run training can impact an LLMs abilities.
Training procedure
The training procedure for this model is outlined step-by-step here:
- Setup https://github.com/CoffeeVampir3/ez-trainer/wiki/Train-the-Stuff-and-Stuff
- Actual training configs: https://github.com/CoffeeVampir3/ez-trainer/wiki/Actual-Training-Example
Datasets:
As outlined in the training procedure step-by-step: https://huggingface.co/datasets/Blackroot/Tiny-Open-Domain-Books
- License
- unlicense
- Created
- Jul 24, 2023
- Updated
- Jul 24, 2023