Downloads · 30 days
40
5% of all-time downloads
dolly-v2-7b-sharded
ethzanalytics/dolly-v2-7b-sharded
dolly-v2-7b-sharded is a text generation model from ethzanalytics. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
<a href="https://colab.research.google.com/gist/pszemraj/6eb7ccce28ea6aa07b8ec86388ac010e/sharded-instruction-model-playground.ipynb" <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Co…
All-time downloads
754
Public
Repo size
55.1 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin27.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- gpt_neox
- Access
- Public
- Created
- Apr 22, 2023
- Updated
- Apr 25, 2023
- SHA
- 3a569e34
Try a prompt
Datasets
Model card
From the Hugging Face model README
dolly-v2-7b: sharded checkpoint
<a href="https://colab.research.google.com/gist/pszemraj/6eb7ccce28ea6aa07b8ec86388ac010e/sharded-instruction-model-playground.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>This is a sharded checkpoint (with ~4GB shards) of the databricks/dolly-v2-7b model. Refer to the original model for all details.
- this enables low-RAM loading, i.e. Colab :)
Basic Usage
install transformers, accelerate, and bitsandbytes.
pip install -U -q transformers bitsandbytes accelerate
Load the model in 8bit, then run inference:
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "ethzanalytics/dolly-v2-7b-sharded"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name, load_in_8bit=True, device_map="auto",
)
- Task
- Text Generation
- Library
- transformers
- Type
- gpt_neox
- License
- mit
- Languages
- en
- Created
- Apr 22, 2023
- Updated
- Apr 25, 2023