Downloads · 30 days
0
Q2
Vicious-Vineyards/Q2
Q2 is a machine learning model from Vicious-Vineyards. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
import json import sagemaker import boto3 from sagemaker.huggingface import HuggingFaceModel, gethuggingfacellmimageuri
Access
Public
Updated Feb 9, 2024
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
Other1.5 KB · 63%
At a glance
- Access
- Public
- Created
- Feb 9, 2024
- Updated
- Feb 9, 2024
- SHA
- bbc2d8bb
Model card
From the Hugging Face model README
import json import sagemaker import boto3 from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uri
try: role = sagemaker.get_execution_role() except ValueError: iam = boto3.client('iam') role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']
Hub Model configuration. https://huggingface.co/models
hub = { 'HF_MODEL_ID':'mistralai/Mixtral-8x7B-Instruct-v0.1', 'SM_NUM_GPUS': json.dumps(8) }
create Hugging Face Model Class
huggingface_model = HuggingFaceModel( image_uri=get_huggingface_llm_image_uri("huggingface",version="1.1.0"), env=hub, role=role, )
deploy model to SageMaker Inference
predictor = huggingface_model.deploy( initial_instance_count=1, instance_type="ml.g5.48xlarge", container_startup_health_check_timeout=300, )
send request
predictor.predict({ "inputs": "Digital Deviance Detective", })
- Created
- Feb 9, 2024
- Updated
- Feb 9, 2024