Qwen2.5-Coder-7B-Instruct-4bit

Converted version of Qwen2.5-Coder-7B-Instruct to 4-bit using bitsandbytes. For more information about the model, refer to the model's page.

Impact on performance

Impact of quantization on a set of models.

We evaluated the models using the PoLL (Pool of LLM) technique a panel of giga-models (GPT-4o, Gemini Pro 1.5, and Claude-Sonnet 3.5). The scoring ranged from 0, indicating a model unsuitable for the task, to 5, representing a model that fully met expectations. The evaluation was based on 67 instructions across four programming languages: Python, Java, JavaScript, and Pseudo-code. All tests were conducted in a French-language context, and models were heavily penalized if they responded in another language, even if the response was technically correct.

Performance Scores (on a scale of 5):

The impact of quantization is negligible.

Prompt Pattern

Here is a reminder of the command pattern to interact with the model:

<|im_start|>user\n{user_prompt_1}<|im_end|><|im_start|>assistant\n{model_answer_1}<|im_end|>...