Downloads · 30 days
0
llama_3.2_rag_categorisation_fine_tuning
Konfi7000/llama_3.2_rag_categorisation_fine_tuning
llama_3.2_rag_categorisation_fine_tuning is a machine learning model from Konfi7000. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: Konfi7000 - License: apache-2.0 - Finetuned from model : unsloth/llama-3.2-1b-bnb-4bit
Access
Public
Updated Oct 3, 2024
Repo size
226 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors45.1 MB · 83%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Oct 2, 2024
- Updated
- Oct 3, 2024
- SHA
- fd6eacc6
Base models
Model card
From the Hugging Face model README
Uploaded model
- Developed by: Konfi7000
- License: apache-2.0
- Finetuned from model : unsloth/llama-3.2-1b-bnb-4bit
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
Description
This model has been fine tuned to categorise user prompts into one of 47 Concepts or HCCH Principle Themes. The model should also be able to return “No Category” if no Concept or Theme is applicable.
The fine tuning data consists of 20 generated queries per Concept and Theme. Queries were generated using Meta Llama 3.2 (1B). Here’s one example:
instruction
Classify the following user query into one of the predefined categories and do not create new categories. Select from the following list: 'Revision/reform (concept)', 'Comparative method (concept)’, …, 'No Category'
user_query
What are potential solutions to address concerns about jurisdiction and transparency in revision or reform processes for existing international treaties?
category
Revision/reform (concept)
20 examples for “No Category” were generated using OpenAI ChatGPT-4o:
user_query
What was the main cause of the American Civil War?
category
No Category
In a further attempt to classify irrelevant queries, a further 20 “No Category” queries were generated ChatGPT-4o, which ask a legal question but have nothing to do with private international law:
user_query
What are the rights of tenants when facing eviction?
category
No Category
The fine tuning data contains a total of 980 query examples.
Evaluation
980 new, i.e., unseen, queries were generated (again using the same Llama and ChatGPT models in the same manner as above).
Overall Accuracy: 83.98%
Weighted Average F1-Score: 89.87%
F1-Score per Category
Sorted by descending F1-Score
| Category | F1-Score |
|---|---|
| 1 Arbitral awards vs. court decisions (concept) | 1 |
| 2 Comparative method (concept) | 1 |
| 3 Connection criteria/factors (concept) | 1 |
| 4 Exclusion of renvoi (principle) | 1 |
| 5 Freedom of choice (principle) | 1 |
| 6 Neutral law / law of a 3rd country (concept) | 1 |
| 7 PIL codification (concept) | 1 |
| 8 Partial choice (principle) | 1 |
| 9 Persuasive authority (concept) | 1 |
| 10 Preamble (principle) | 1 |
| 11 Principles/soft law (concept) | 1 |
| 12 Refer to an instrument (concept) | 1 |
| 13 Revision/reform (concept) | 1 |
| 14 Scope of the chosen law (principle) | 1 |
| 15 Severability (principle) | 1 |
| 16 Tacit choice (concept) | 1 |
| 17 Weaker/vulnerable parties (concept) | 1 |
| 18 Arbitrators vs. State judges (concept) | 0.974 |
| 19 Assignment (principle) | 0.974 |
| 20 Express and tacit choice (principle) | 0.974 |
| 21 Incorporate rules by way of reference (concept) | 0.974 |
| 22 Modification of a choice of law clause (concept) | 0.974 |
| 23 Respect choice of law clause (concept) | 0.974 |
| 24 Rules of law (principle) | 0.974 |
| 25 non-State law / rules of law (concept) | 0.974 |
| 26 Absence of choice (concept) | 0.947 |
| 27 Conventions/hard law (concept) | 0.947 |
| 28 Express choice (concept) | 0.947 |
| 29 Connection (concept) | 0.919 |
| 30 Formal validity of the choice of law (principle) | 0.919 |
| 31 Institutional rules (arbitration) (concept) | 0.919 |
| 32 No Category | 0.919 |
| 33 Agreement on the choice of law and battle of forms (principle) | 0.889 |
| 34 Establishment (principle) | 0.889 |
| 35 Legislation (concept) | 0.889 |
| 36 International contracts (commercial) (concept) | 0.824 |
| 37 Public policy (principle) | 0.824 |
| 38 Mandatory rules (principle) | 0.788 |
| 39 Public policy (concept) | 0.788 |
| 40 Scope of the Principles (principle) | 0.788 |
| 41 Absence of choice (principle) | 0.75 |
| 42 Arbitration (principle) | 0.75 |
| 43 Party autonomy (concept) | 0.75 |
| 44 Dépeçage (principle) | 0.710 |
| 45 Arbitration (concept) | 0.667 |
| 46 Dépeçage (concept) | 0.519 |
| 47 Party autonomy (principle) | 0.519 |
| 48 Mandatory rules (concept) | 0.462 |
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- Oct 2, 2024
- Updated
- Oct 3, 2024