Downloads · 30 days
0
mlc_llm-dolly-v2-int4
wshhyh/mlc_llm-dolly-v2-int4
mlc_llm-dolly-v2-int4 is a machine learning model from wshhyh. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
https://huggingface.co/databricks/dolly-v2-12b
Access
Public
Updated May 4, 2023
Repo size
7.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin7.4 GB · 99%
At a glance
- Access
- Public
- Created
- May 4, 2023
- Updated
- May 4, 2023
- SHA
- a256e10b
Model card
From the Hugging Face model README
https://huggingface.co/databricks/dolly-v2-12b
converted to mlc_llm format using https://github.com/mlc-ai/mlc-llm
command
python build.py --model dolly-v2-12b --target vulkan --quantization-mode int4 --quantization-sym --max-seq-len 2048
not fully tested because i got "CUDA: out of memory"
- Created
- May 4, 2023
- Updated
- May 4, 2023