Lite-Oute-1-300M-Instruct-openvino

This model was converted to OpenVINO from OuteAI/Lite-Oute-1-300M-Instruct using optimum-intel via the export space.

Lite-Oute-1-300M-Instruct

Lite-Oute-1-300M-Instruct is a Lite series model based on the Mistral architecture, comprising approximately 300 million parameters. <br> This model aims to improve upon our previous 150M version by increasing size and training on a more refined dataset. The primary goal of this 300 million parameter model is to offer enhanced performance while still maintaining efficiency for deployment on a variety of devices. <br> With its larger size, it should provide improved context retention and coherence, however users should note that as a compact model, it still have limitations compared to larger language models. <br> The model was trained on 30 billion tokens with a context length of 4096.

Available versions:

<a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct">Lite-Oute-1-300M-Instruct</a> <br> <a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct-GGUF">Lite-Oute-1-300M-Instruct-GGUF</a> <br> <a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M">Lite-Oute-1-300M</a> <br> <a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M-GGUF">Lite-Oute-1-300M-GGUF</a> <br>

Chat format

[!IMPORTANT] This model uses ChatML template. Ensure you use the correct template:

<|im_start|>system
[System message]<|im_end|>
<|im_start|>user
[Your question or message]<|im_end|>
<|im_start|>assistant
[The model's response]<|im_end|>

Benchmarks:

<table style="text-align: left;"> <tr> <th>Benchmark</th> <th>5-shot</th> <th>0-shot</th> </tr> <tr> <td>ARC Challenge</td> <td>26.37</td> <td>26.02</td> </tr> <tr> <td>ARC Easy</td> <td>51.43</td> <td>49.79</td> </tr> <tr> <td>CommonsenseQA</td> <td>20.72</td> <td>20.31</td> </tr> <tr> <td>HellaSWAG</td> <td>34.93</td> <td>34.50</td> </tr> <tr> <td>MMLU</td> <td>25.87</td> <td>24.00</td> </tr> <tr> <td>OpenBookQA</td> <td>31.40</td> <td>32.20</td> </tr> <tr> <td>PIQA</td> <td>65.07</td> <td>65.40</td> </tr> <tr> <td>Winogrande</td> <td>52.01</td> <td>53.75</td> </tr> </table>

First make sure you have optimum-intel installed:

pip install optimum[openvino]

To load your model you can do as follows:

from optimum.intel import OVModelForCausalLM

model_id = "FM-1976/Lite-Oute-1-300M-Instruct-openvino"
model = OVModelForCausalLM.from_pretrained(model_id)