Downloads · 30 days
9
11% of all-time downloads
opt-125M-autoround-lm_head-true-symTrue
LnL-AI/opt-125M-autoround-lm_head-true-symTrue
opt-125M-autoround-lm_head-true-symTrue is a text generation model from LnL-AI. Use it when you need the model to write or continue text. It is set up for transformers.
All-time downloads
79
Public
Parameters
164M
145 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors145 MB · 98%
Parameter types
How the weights are stored.
I32124M · 75%
Try a prompt
Model card
From the Hugging Face model README
env CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0 python3 main.py \
--model_name facebook/opt-125M \
--device 0 \
--group_size 128 \
--bits 4 \
--seqlen 2048 \
--iters 1000 \
--use_quant_input \
--quant_lm_head \
--disable_eval \
--n_blocks 22 \
--sym \
--deployment_device 'gpu' \
--disable_low_gpu_mem_usage \
--output_dir "/monster/data/zx/opt-125M-quant_lm_head_true"
quant model path:
/monster/data/zx/opt-125M-quant_lm_head_true/opt-125m-autoround-w4g128-gpu
- Task
- Text Generation
- Library
- transformers
- Type
- opt
- Created
- Apr 30, 2024
- Updated
- Apr 30, 2024