Downloads · 30 days
9
9% of all-time downloads
helenai/jais-family-13b-chat-ov-int4-sym
jais-family-13b-chat-ov-int4-sym is a machine learning model from helenai. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is the inceptionai/jais-family-13b-chat model converted to OpenVINO with INT4 symmetric channel-wise weight compression.
Downloads · 30 days
9
9% of all-time downloads
All-time downloads
97
Public
Repo size
7.2 GB
Likes
0
Public
Click a slice to open those files.
.bin7.2 GB · 100%
From the Hugging Face model README
This is the inceptionai/jais-family-13b-chat model converted to OpenVINO with INT4 symmetric channel-wise weight compression.
pip install huggingface-hub[cli]
huggingface-cli download helenai/jais-family-13b-chat-ov-int4-sym --local-dir jais-family-13b-chat-ov-int4-sym
The recommend way to run inference with this model is with OpenVINO GenAI. It is the only package needed for inference - no need to install Transformers or PyTorch.
pip install --pre --upgrade openvino-genai openvino openvino-tokenizers --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/pre-release
curl -O https://raw.githubusercontent.com/helena-intel/snippets/refs/heads/main/llm_chat/python/llm_chat.py
python llm_chat.py jais-family-13b-chat-ov-int4-sym GPU
Check out OpenVINO GenAI documentation and OpenVINO GenAI samples.
openvino_version : 2025.2.0-18660-3ceeeb52d64
advanced_parameters : {'statistics_path': None, 'awq_params': {'subset_size': 32, 'percent_to_apply': 0.002, 'alpha_min': 0.0, 'alpha_max': 1.0, 'steps': 100}, 'scale_estimation_params': {'subset_size': 64, 'initial_steps': 5, 'scale_steps': 5, 'weight_penalty': -1.0}, 'gptq_params': {'damp_percent': 0.1, 'block_size': 128, 'subset_size': 128}, 'lora_correction_params': {'adapter_rank': 8, 'num_iterations': 3, 'apply_regularization': True, 'subset_size': 128, 'use_int8_adapters': True}}
all_layers : False
awq : False
backup_mode : int8_asym
gptq : False
group_size : -1
ignored_scope : []
lora_correction : False
mode : int4_sym
ratio : 1.0
scale_estimation : False
sensitivity_metric : weight_quantization_error
optimum_intel_version : 1.22.0
optimum_version : 1.24.0
pytorch_version : 2.5.1+cpu
transformers_version : 4.48.3