Code-Llama-3-8B

Code-Llama-3-8B

This Model is trained on refined version of my dataset Code-290k-ShareGPT.

Besides this it is trained on following datasets:

Code-Feedback

orca-math-word-problems-200k

CodeFeedback-Filtered-Instruction

The idea was to check how this Model will perform with both Code & Maths datasets. This model is very good with Coding. Maths outputs are also very good. You can test out this model.

It is very very good in Code generation in various languages such as Python, Java, JavaScript, GO, C++, Rust, Ruby, Sql, MySql, R, Julia, Haskell, etc.. This model will also generate detailed explanation/logic behind each code.

This Model is trained on massive datasets so the results are very good. You can check the Examples given below.

I have used ChatML prompt format.

This is Fully Finetuned Model.

GGUF & Exllama

GGUF: Link

Exllama v2: Link

Special Thanks to Bartowski for quantizing this model.

Training:

Entire dataset was trained on 4 x A100 80GB. For 2 epoch, training took more than 160 Hours. Axolotl & Deepspeed codebase was used for training purpose. Entire data is trained on Llama-3-8B by Meta.

Example Prompt: This model uses ChatML prompt format.

<|im_start|>system
You are a helpful Coding assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

You can modify above Prompt as per your requirement. One example will be:

This is a conversation with your helpful Coding assistant. Assistant can generate Code in various Programming Languages along with necessary explanation.

I want to say special Thanks to the Open Source community for helping & guiding me to better understand the AI/Model development.

Thank you for your love & support.

Example Output

Example 1

Example 2

Example 3

Example 4

Example 5