Downloads · 30 days
6
12% of all-time downloads
Netta1994/setfit_baai_newrelic_gpt-4o_improved-cot_chat_few_shot_only_reasoning_1726751494.1082
setfit_baai_newrelic_gpt-4o_improved-cot_chat_few_shot_only_reasoning_1726751494.1082 is a text classification model from Netta1994. Use it when you need a label for a piece of text. It is set up for setfit.
This is a SetFit model that can be used for Text Classification. This SetFit model uses BAAI/bge-base-en-v1.5 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
Downloads · 30 days
6
12% of all-time downloads
All-time downloads
49
Public
Parameters
109M
438 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors438 MB · 100%
From the Hugging Face model README
This is a SetFit model that can be used for Text Classification. This SetFit model uses BAAI/bge-base-en-v1.5 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
| Label | Examples |
|---|---|
| 0 | <ul><li>"Reasoning:\nThe answer provided does not align with the content of the documents. It offers general advice on saving money rather than specific insights from the provided documents that relate to ORGANIZATION's specific guidelines or context about financial prudence or savings.\n\nEvaluation: Bad"</li><li>'Reasoning:\nThe answer is correct; it properly identifies several specific pet peeves mentioned in the document, such as sabotaging work, unwanted advances, and derogatory comments. However, the answer contains numerous repetitions of names and accidental insertions of text fragments which make it difficult to read. This detracts from the clarity and quality, despite being factually correct.\n\nEvaluation: Bad'</li><li>"Reasoning:\nThe answer given does not provide any information or instructions about accessing the company's training resources. Instead, it lists various unrelated methods such as accessing personal documents, managing passwords, and requesting learning budgets, based on the provided documents. The answer does not directly address the question.\n\nEvaluation: Bad"</li></ul> |
| 1 | <ul><li>'Reasoning:\nThe answer accurately captures the key points from the document regarding how feedback should be given. It mentions giving feedback at the time of the event, focusing on the situation rather than the person, aiming to help rather than shame, being clear and direct, and showing appreciation. It also covers tips for receiving feedback. The answer presents these points clearly and is aligned with the provided document.\n\nEvaluation: Good'</li><li>'Reasoning:\nThe answer effectively captures the reasons for proactively sharing information from high-level meetings, such as providing transparency, ensuring that team members have the necessary context, aligning the team, and fostering a sense of purpose. These points are supported by the provided documents, particularly Document 4.\n\nEvaluation: Good'</li><li>'Reasoning:\nThe provided answer accurately describes the procedure for reporting car travel expenses for reimbursement, including precise details such as tracking kilometers and sending details to specific email addresses. This information directly corresponds to the content provided in Document 1.\n\nFinal result: Good'</li></ul> |
| Label | Accuracy |
|---|---|
| all | 0.5522 |
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Netta1994/setfit_baai_newrelic_gpt-4o_improved-cot_chat_few_shot_only_reasoning_1726751494.1082")
# Run inference
preds = model("Reasoning:
The answer given in the response correctly reflects the information in Document 1, which states that questions regarding travel reimbursement should be directed to finance@ORGANIZATION_2.<89312988>. The required email address is present in the document and clearly mentions who to contact.
Evaluation: Good")
<!--
### Downstream Use
*List how someone could finetune this model on their own dataset.*
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
| Training set | Min | Median | Max |
|---|---|---|---|
| Word count | 21 | 47.4462 | 85 |
| Label | Training Sample Count |
|---|---|
| 0 | 32 |
| 1 | 33 |
| Epoch | Step | Training Loss | Validation Loss |
|---|---|---|---|
| 0.0061 | 1 | 0.2243 | - |
| 0.3067 | 50 | 0.2608 | - |
| 0.6135 | 100 | 0.2456 | - |
| 0.9202 | 150 | 0.1701 | - |
| 1.2270 | 200 | 0.0069 | - |
| 1.5337 | 250 | 0.0026 | - |
| 1.8405 | 300 | 0.0021 | - |
| 2.1472 | 350 | 0.002 | - |
| 2.4540 | 400 | 0.0018 | - |
| 2.7607 | 450 | 0.0016 | - |
| 3.0675 | 500 | 0.0015 | - |
| 3.3742 | 550 | 0.0015 | - |
| 3.6810 | 600 | 0.0014 | - |
| 3.9877 | 650 | 0.0014 | - |
| 4.2945 | 700 | 0.0014 | - |
| 4.6012 | 750 | 0.0013 | - |
| 4.9080 | 800 | 0.0013 | - |
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->