Downloads · 30 days
8
11% of all-time downloads
WizardOrca-7bv2-lora
Vezora/WizardOrca-7bv2-lora
WizardOrca-7bv2-lora is a machine learning model from Vezora. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
license: apache-2.0 --- This is a qlora trained llama 7b v2 Adapter trained with 1,250 high quality examples from uncencored WizardOrca dataset+Custom gpt4dataset (examples selected by length).
All-time downloads
73
Public
Repo size
268 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin268 MB · 100%
At a glance
- Library
- peft
- Access
- Public
- Created
- Jul 20, 2023
- Updated
- Jul 20, 2023
- SHA
- eabcfbed
Model card
From the Hugging Face model README
license: apache-2.0
This is a qlora trained llama 7b v2 Adapter trained with 1,250 high quality examples from uncencored WizardOrca dataset+Custom gpt4dataset (examples selected by length).
|4 epochs| 2e-5 learning rate| 1 microbatch size| 128 batch size| adam8it|2048 tokens|
The model can use the standard llama v2 prompting or use the alpaca chat prompting as this dataset was converted to alpaca format.
Footnotes
The model has not lost its ability to interpret 4096 tokens, regardless of this adapter being trained on 2048 tokens. The model preforms execptionally well based of my preliminary human evaluation. Benchmarks coming soon.
(trained with oobabooga webui) https://github.com/oobabooga/text-generation-webui Main Orginal dataset creator: Psmathur https://huggingface.co/psmathur
- Library
- peft
- Created
- Jul 20, 2023
- Updated
- Jul 20, 2023