Downloads · 30 days
5
1% of all-time downloads
Meta-Llama-3.2-1B-Instruct-nc2-bs1-token1024-neuron-220
cszhzleo/Meta-Llama-3.2-1B-Instruct-nc2-bs1-token1024-neuron-220
Meta-Llama-3.2-1B-Instruct-nc2-bs1-token1024-neuron-220 is a machine learning model from cszhzleo. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
All-time downloads
845
Public
Repo size
2.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.5 GB · 99%
At a glance
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Oct 24, 2024
- Updated
- Oct 24, 2024
- SHA
- 87547700
Model card
From the Hugging Face model README
environment
optimum-neuron 0.0.25
neuron 2.20.0
transformers-neuronx 0.12.313
transformers 4.45.2
export
optimum-cli export neuron --model meta-llama/Llama-3.2-1B-Instruct --batch_size 1 --sequence_length 1024 --num_cores 2 --auto_cast_type fp16 ./models-hf/meta-llama/Llama-3.2-1B-Instruct
run
docker run -it --name llama-31 --rm \
-p 8080:80 \
-v /home/ec2-user/models-hf/:/models \
-e HF_MODEL_ID=/models/meta-llama/Llama-3.2-1B-Instruct \
-e MAX_INPUT_TOKENS=256 \
-e MAX_TOTAL_TOKENS=4096 \
-e MAX_BATCH_SIZE=1 \
-e LOG_LEVEL="info,text_generation_router=debug,text_generation_launcher=debug" \
--device=/dev/neuron0 \
neuronx-tgi:latest \
--model-id /models/meta-llama/Llama-3.2-1B-Instruct \
--max-batch-size 1 \
--max-input-tokens 256 \
--max-total-tokens 1024
test
curl 127.0.0.1:8080/generate -X POST -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' -H 'Content-Type: application/json'
- Type
- llama
- License
- mit
- Created
- Oct 24, 2024
- Updated
- Oct 24, 2024