Mistral_Trained_V3

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This model generates answers to questions in a more honest/uncertain way. When the model believes it does not know the answer to a question, it will explain it doesnt know and why.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

• Developed by: Jens Groot
• Model type: Text Generation
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model: mistralai/Mistral-7B-Instruct-v0.3

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: github.com/Airslammer/Thesis

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

It should be used in Q&A environments

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

It should not be used outside of Q&A environments or to generate offensive and/or malicious answers

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

The data it is trained on is relatively small, so its knowledge may be limited.

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

(https://github.com/sylinrl/TruthfulQA). This dataset has been slightly altered to train the model. The data was altered before the training in two ways.

The first way was to edit the "i have no comment" responses into actual responses where the model explains why they cannot answer

The second way that a 85/15 split was done using:

train_data, test_data = train_test_split( df, test_size=0.15, # 85 - 15 split random_state=16 # seed 16 to make the research redoable )

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Training Hyperparameters

train_args = UnslothTrainingArguments( output_dir="./kaggle/working/fourth-llama-qlora", # Kaggle directory, change when doing it on own pc num_train_epochs=3, # number of times data gets taught, higher and overfitting would be an issue. Lower and it may not learn enough per_device_train_batch_size=2, gradient_accumulation_steps=4, learning_rate=2e-4, # Standard learning rate for LoRa warmup_steps=30, # Steps that stabilize the earlier part of training logging_steps=50, weight_decay=0.05, lr_scheduler_type="cosine", # Uses cosine curve to to decay learning rate save_steps=200, # checkpoint every 200 steps for training safety reasons optim="adamw_8bit", fp16=True, # Depends on the computer and if you use cpu bf16=False, # or if you use gpu save_total_limit=3, report_to="none", seed=16, # seed for reproducibility resume_from_checkpoint=False, # Turn on when training runs out of time )

trainer = UnslothTrainer( model=model, args=train_args, train_dataset=tokenized_data, tokenizer=tokenizer )

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

https://github.com/sylinrl/TruthfulQA again, but now the 15% part.

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Bertscore, Manual Annotation: Semantic Equivalence and Uncertainty

Results

In the comparison of this model and Airslammer/LLama_Trained_V3, prompt configuration mattered less then the model type in how correct it was and when it correctly used uncertainty. Mistral performed better then Llama.

Framework versions

• PEFT 0.18.1