Downloads · 30 days
13
28% of all-time downloads
Gonsoo/AWS-NeuronCC-2-14-llama-3-Korean-Bllossom-8B
AWS-NeuronCC-2-14-llama-3-Korean-Bllossom-8B is a machine learning model from Gonsoo. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This model is an AWS Neuron compiled version, neuron-cc 2.14, of the Korean fine-tuned model MLP-KTLim/llama-3-Korean-Bllossom-8B, available at https://huggingface.co/MLP-KTLim/llama-3-Korean-Bllossom-8B. It is intend…
Downloads · 30 days
13
28% of all-time downloads
All-time downloads
47
Public
Repo size
21.1 MB
Likes
0
Public
Click a slice to open those files.
.neff21.1 MB · 70%
From the Hugging Face model README
This model is an AWS Neuron compiled version, neuron-cc 2.14, of the Korean fine-tuned model MLP-KTLim/llama-3-Korean-Bllossom-8B, available at https://huggingface.co/MLP-KTLim/llama-3-Korean-Bllossom-8B. It is intended for deployment on Amazon EC2 Inferentia2 and Amazon SageMaker. For detailed information about the model and its license, please refer to the original MLP-KTLim/llama-3-Korean-Bllossom-8B model page
This model is compiled with neuronx-cc version, 2.14 It can be deployed with [v1.0-hf-tgi-0.0.24-pt-2.1.2-inf-neuronx-py310](https://github.com/aws/deep-learning-containers/releases?q=tgi+AND+neuronx&expanded=true on SageMaker Endpoint because this inference docker image is only used on SageMaker
This model can be deployed to Amazon SageMaker Endtpoint with this guide, S3 에 저장된 모델을 SageMaker INF2 에 배포하기
In order to do neuron-compilation and depoly in detail , you can refer to Amazon ECR 의 도커 이미지 기반하에 Amazon EC2 Inferentia2 서빙하기
At a minimum hardware, you can use Amazon EC2 inf2.xlarge and more powerful family such as inf2.8xlarge, inf2.24xlarge and inf2.48xlarge. The detailed information is Amazon EC2 Inf2 Instances
Gonsoo Moon, [email protected]