Vanity-SBU-Mistral-7B-Instruct-v0.2

Model Details

This model was fine-tuned on a curated dataset of 10,000 license plate examples, including vanity and standard plates, to classify plates as vanity or non-vanity and assess their acceptance or rejection. It leverages Chain of Thought (CoT) reasoning for better interpretability of complex and nuanced semantic patterns in vanity plates. The training utilized the PEFT library with Low-Rank Adaptation (LoRA) for efficient memory and computation.

Model Description

• Developed by: Anonymous Author(s)
• Model type: Text classification and interpretation
• Language(s) (NLP): English
• Finetuned from model [optional]: mistralai/Mistral-7B-Instruct-v0.2

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: https://drive.google.com/drive/folders/1_12PC4b5PdDW4Dv0rF1mBDx7U2Q-QYIW
• Paper [optional]: https://drive.google.com/drive/folders/1Pm3dU1B_Ya6qRCXQNTE4ZcF0SGOYXZ-o?usp=sharing

Uses

The model is intended for research and development, focusing on the semantic and contextual interpretation of vanity license plates. It can classify plates, determine their vanity status, and generate explanations for approval or rejection decisions.

Training Details

Training Data

The training dataset includes 10,000 license plates, categorized into accepted and rejected vanity plates, along with standard plates. Semantic features like Named Entity Recognition (NER) and lexical features were integrated for contextual insights. Dataset link - https://drive.google.com/drive/folders/1_12PC4b5PdDW4Dv0rF1mBDx7U2Q-QYIW

Training Procedure

The model was fine-tuned using Low-Rank Adaptation (LoRA) with NF4 quantization for memory efficiency and gradient checkpointing to reduce resource usage. The training employed the Paged AdamW optimizer with a learning rate of 2e-4, batch size of 4, and ran for 5 epochs on NVIDIA GPUs in Google Colab. The process focused on minimizing the causal language modeling loss, enabling effective handling of nuanced semantic tasks.

<!-- #### Preprocessing [optional] [More Information Needed] -->

Training Hyperparameters

• Training regime: Mixed precision (fp16)
• Quantization: NF4 with gradient checkpointing
• Optimizer: Paged AdamW
• Learning rate: 2e-4
• Batch size: 4
• Epochs: 5

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

20% of the annotated dataset was used as the test set, alongside evaluations on external datasets like the NY License Plate dataset.

<!-- #### Factors --> <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. --> <!-- [More Information Needed] -->

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Metrics included accuracy, precision, recall, and F1 score for classification. BLEU and ROUGE scores were used to evaluate interpretative explanations.

Results

• Vanity classification accuracy: 78.32%
• Approval classification accuracy: 75.18%
• BLEU for explanations: 60.25

<!-- #### Summary ## Model Examination [optional] --> <!-- Relevant interpretability work for the model goes here --> <!-- [More Information Needed] --> <!-- ## Environmental Impact --> <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> <!-- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). - *Hardware Type:* [More Information Needed] - *Hours used:* [More Information Needed] - *Cloud Provider:* [More Information Needed] - *Compute Region:* [More Information Needed] - *Carbon Emitted:* [More Information Needed] --> <!-- ## Technical Specifications [optional] ### Model Architecture and Objective [More Information Needed] ### Compute Infrastructure [More Information Needed] --> <!-- #### Hardware [More Information Needed] --> <!-- #### Software [More Information Needed] --> <!-- ## Citation [optional] <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> <!-- *BibTeX:* --> <!-- [More Information Needed] --> <!-- *APA:* --> <!-- [More Information Needed] --> <!-- ## Glossary [optional] --> <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. --> <!-- [More Information Needed] --> <!-- ## More Information [optional] --> <!-- [More Information Needed] --> <!-- ## Model Card Authors [optional] --> <!-- [More Information Needed] --> <!-- ## Model Card Contact --> <!-- [More Information Needed] -->

Framework versions

• PEFT 0.13.2