EmergentMethods_-_Phi-3-mini-4k-instruct-graph-4bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Phi-3-mini-4k-instruct-graph - bnb 4bits

• Model creator: https://huggingface.co/EmergentMethods/
• Original model: https://huggingface.co/EmergentMethods/Phi-3-mini-4k-instruct-graph/

Original model description:

license: cc-by-nc-sa-4.0 datasets:

• EmergentMethods/AskNews-GraphER-v0 tags:
• entity-relationship-extraction
• news-analysis language:
• en pipeline_tag: text-generation

---

Model Card for Phi-3-mini-4k-instruct-graph

This model is a fine-tuned version of Microsoft's Phi-3-mini-4k-instruct, specifically tailored for entity relationship extraction from general text data. It aims to match the quality and accuracy of GPT-4 in generating entity relationship graphs while offering improved efficiency for large-scale processing.

Model Details

• Developed by: Emergent Methods
• Funded by: Emergent Methods
• Shared by: Emergent Methods
• Model type: microsoft/phi-3-mini-4k-instruct (fine-tuned)
• Language(s): English
• License: Creative Commons Attribution Non Commercial Share Alike 4.0
• Finetuned from model: microsoft/phi-3-mini-4k-instruct

For more information, see our blog post.:

📰 Blog <br>

Uses

This model is designed for generating structured JSON data representing entity relationships in general text data. It can be used for:

• Enhanced information retrieval in various text databases
• Exploration of temporal relationships and evolving narratives across different types of documents
• Advanced predictive modeling for trend analysis in diverse textual sources

The model is particularly useful for applications requiring high-throughput processing of large volumes of text data, such as content aggregation platforms, research databases, and comprehensive text analysis systems.

Bias, Risks, and Limitations

Although the goal of the dataset is to reduce bias, and improve diversity, it is still biased to western languages and countries. This limitation originates from the abilities of Llama2 for the translation and summary generations. Further, any bias originating in Llama2 training data will also be present in this dataset, since Llama2 was used to summarize the open-web articles. Further, any biases present in Microsoft Phi-3 will be present in the present dataset.

Training Details

• Training Data: Over 7,000 stories and updates from AskNews, curated to avoid topic overlap
• Training Procedure: Fine-tuned using the Transformers library, SFTTrainer, PEFT, and QLoRA

Evaluation Results

Compared to GPT-4o (ground truth), Claude Sonnet 3.5, and the base Phi-3 model:

Environmental Impact

• Hardware Type: 1x A100 SXM
• Hours used: 3 hours
• Carbon Emitted: 0.44 kg (According to Machine Learning Impact calculator)

How to Get Started with the Model (Sample inference code)

This code snippets show how to get quickly started with running the model on a GPU:

import torch 
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline 

torch.random.manual_seed(0) 
model = AutoModelForCausalLM.from_pretrained( 
    "EmergentMethods/Phi-3-mini-4k-instruct-graph",  
    device_map="cuda",  
    torch_dtype="auto",  
    trust_remote_code=True,  
) 

tokenizer = AutoTokenizer.from_pretrained("EmergentMethods/Phi-3-mini-4k-instruct-graph") 

messages = [ 
    {"role": "system", "content": """
A chat between a curious user and an artificial intelligence Assistant. The Assistant is an expert at identifying entities and relationships in text. The Assistant responds in JSON output only.

The User provides text in the format:

-------Text begin-------
<User provided text>
-------Text end-------

The Assistant follows the following steps before replying to the User:

1. **identify the most important entities** The Assistant identifies the most important entities in the text. These entities are listed in the JSON output under the key "nodes", they follow the structure of a list of dictionaries where each dict is:

"nodes":[{"id": <entity N>, "type": <type>, "detailed_type": <detailed type>}, ...]

where "type": <type> is a broad categorization of the entity. "detailed type": <detailed_type>  is a very descriptive categorization of the entity.

2. **determine relationships** The Assistant uses the text between -------Text begin------- and -------Text end------- to determine the relationships between the entities identified in the "nodes" list defined above. These relationships are called "edges" and they follow the structure of:

"edges":[{"from": <entity 1>, "to": <entity 2>, "label": <relationship>}, ...]

The <entity N> must correspond to the "id" of an entity in the "nodes" list.

The Assistant never repeats the same node twice. The Assistant never repeats the same edge twice.
The Assistant responds to the User in JSON only, according to the following JSON schema:

{"type":"object","properties":{"nodes":{"type":"array","items":{"type":"object","properties":{"id":{"type":"string"},"type":{"type":"string"},"detailed_type":{"type":"string"}},"required":["id","type","detailed_type"],"additionalProperties":false}},"edges":{"type":"array","items":{"type":"object","properties":{"from":{"type":"string"},"to":{"type":"string"},"label":{"type":"string"}},"required":["from","to","label"],"additionalProperties":false}}},"required":["nodes","edges"],"additionalProperties":false}
     """}, 
    {"role": "user", "content": """
-------Text begin-------
OpenAI is an American artificial intelligence (AI) research organization founded in December 2015 and headquartered in San Francisco, California. Its mission is to develop "safe and beneficial" artificial general intelligence, which it defines as "highly autonomous systems that outperform humans at most economically valuable work".[4] As a leading organization in the ongoing AI boom,[5] OpenAI is known for the GPT family of large language models, the DALL-E series of text-to-image models, and a text-to-video model named Sora.[6][7] Its release of ChatGPT in November 2022 has been credited with catalyzing widespread interest in generative AI.
-------Text end-------
"""}
] 

pipe = pipeline( 
    "text-generation", 
    model=model, 
    tokenizer=tokenizer, 
) 

generation_args = { 
    "max_new_tokens": 500, 
    "return_full_text": False, 
    "temperature": 0.0, 
    "do_sample": False, 
} 

output = pipe(messages, **generation_args) 
print(output[0]['generated_text'])

# Output:

# {
#     "nodes": [
#         {
#             "id": "OpenAI",
#             "type": "organization",
#             "detailed_type": "ai research organization"
#         },
#         {
#             "id": "GPT family",
#             "type": "technology",
#             "detailed_type": "large language models"
#         },
#         {
#             "id": "DALL-E series",
#             "type": "technology",
#             "detailed_type": "text-to-image models"
#         },
#         {
#             "id": "Sora",
#             "type": "technology",
#             "detailed_type": "text-to-video model"
#         },
#         {
#             "id": "ChatGPT",
#             "type": "technology",
#             "detailed_type": "generative ai"
#         },
#         {
#             "id": "San Francisco",
#             "type": "location",
#             "detailed_type": "city"
#         },
#         {
#             "id": "California",
#             "type": "location",
#             "detailed_type": "state"
#         },
#         {
#             "id": "December 2015",
#             "type": "date",
#             "detailed_type": "foundation date"
#         },
#         {
#             "id": "November 2022",
#             "type": "date",
#             "detailed_type": "release date"
#         }
#     ],
#     "edges": [
#         {
#             "from": "OpenAI",
#             "to": "San Francisco",
#             "label": "headquartered in"
#         },
#         {
#             "from": "San Francisco",
#             "to": "California",
#             "label": "located in"
#         },
#         {
#             "from": "OpenAI",
#             "to": "December 2015",
#             "label": "founded in"
#         },
#         {
#             "from": "OpenAI",
#             "to": "GPT family",
#             "label": "developed"
#         },
#         {
#             "from": "OpenAI",
#             "to": "DALL-E series",
#             "label": "developed"
#         },
#         {
#             "from": "OpenAI",
#             "to": "Sora",
#             "label": "developed"
#         },
#         {
#             "from": "OpenAI",
#             "to": "ChatGPT",
#             "label": "released"
#         },
#         {
#             "from": "ChatGPT",
#             "to": "November 2022",
#             "label": "released in"
#         }
#     ]
# }

Ethical Considerations

Users should be aware that this model is designed for entity relationship extraction from general text data and may not be suitable for other domains without further fine-tuning. The model's outputs should be reviewed and validated, especially when used in applications that may impact decision-making or public information dissemination.