Downloads · 30 days
3
4% of all-time downloads
Lumimaid-v0.2-70B-awq
TeeZee/Lumimaid-v0.2-70B-awq
Lumimaid-v0.2-70B-awq is a machine learning model from TeeZee. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
To run on 2x3090 with 48 GB VRAM use vLLM:
All-time downloads
84
Public
Parameters
70.6B
39.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors39.8 GB · 100%
Parameter types
How the weights are stored.
I3268.5B · 97%
Base models
Model card
From the Hugging Face model README
To run on 2x3090 with 48 GB VRAM use vLLM:
docker run --runtime nvidia --gpus all -v vllm-models:/root/.cache/huggingface --env "HUGGING_FACE_HUB_TOKEN=<your token>" -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model TeeZee/Lumimaid-v0.2-70B-awq --tensor-parallel-size 2 --quantization awq --max_model_len 8192 --dtype half --gpu_memory_utilization 0.97
- Type
- llama
- Created
- Mar 21, 2025
- Updated
- Mar 21, 2025