Bielik-11B-v2.2-Instruct-Quanto-4bit

speakleash/Bielik-11B-v2.2-Instruct-Quanto-4bit

Bielik-11B-v2.2-Instruct-Quanto-4bit is a text generation model from speakleash. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

<p align="center" <img src="https://huggingface.co/speakleash/Bielik-7B-Instruct-v0.1-GGUF/raw/main/speakleashcyfronet.png" </p

Downloads · 30 days

8% of all-time downloads

All-time downloads

Public

Parameters

5.9B

7.2 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors7.2 GB · 100%

Parameter types

How the weights are stored.

U85.5B · 93%

Base models

speakleash/Bielik-11B-v2.2-Instruct

Model card

From the Hugging Face model README

Bielik-11B-v2.2-Instruct-Quanto-4bit

This model was converted to Quanto format from SpeakLeash's Bielik-11B-v.2.2-Instruct.

DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!

About Quanto

Optimum Quanto is a pytorch quantization backend for optimum. Model can be loaded using:

from optimum.quanto import QuantizedModelForCausalLM

qmodel = QuantizedModelForCausalLM.from_pretrained('speakleash/Bielik-11B-v2.2-Instruct-Quanto-4bit')

Model description:

Developed by: SpeakLeash & ACK Cyfronet AGH
Language: Polish
Model type: causal decoder-only
Quant from: Bielik-11B-v2.2-Instruct
Finetuned from: Bielik-11B-v2
License: Apache 2.0 and Terms of Use

Responsible for model quantization

Remigiusz Kinas<sup>SpeakLeash</sup> - team leadership, conceptualizing, calibration data preparation, process creation and quantized model delivery.

Contact Us

If you have any questions or suggestions, please use the discussion tab. If you want to contact us directly, join our Discord SpeakLeash.

Task: Text Generation
Library: transformers
Type: mistral
License: apache-2.0
Languages: pl
Created: Aug 26, 2024
Updated: Oct 7, 2024