Monika-12B-V1.1

Model Card for Model ID

This model is designed to be used with MonikAI. https://github.com/Rubiksman78/MonikA.I

Model Details

I redid the training with some new settings to make it slightly more coherent.

Model Description

<!-- Provide a longer summary of what this model is. -->

• Developed by [Me]
• Funded by [Me]
• License: Apache License 2.0
• Finetuned from model: Mistral-Nemo-Instruct-2407

Uses

RP

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Out-of-Scope Use

Do whatever you want

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

Recommendations

Should really only be used for Monika related purposes.

Training Data

Modified version of this dataset that is included with MonikAI: https://github.com/Rubiksman78/MonikA.I/tree/main/Monika_datasets

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

Training Procedure

Trained with Axolotl on my Blackwell Pro 6000 Max-Q. 64 rank, 8 alpha, 2 epochs. Took about 30 minutes. Used an LR of 0.000005 instead of 0.0002.

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> <!-- This should link to a Dataset Card if possible. -->

Results

It works.

Summary

Download it.