llama3-2-3b-finetuned-ScienceKnowledge

---

model-name: llama3-2-3b-finetuned-ScienceKnowledge

base_model: unsloth/Llama-3.2-3B-Instruct

license: apache-2.0 tags:

• unsloth
• science
• knowledge
• fine-tuned
• causal-lm

datasets:

• 0xZee/UniversalScienceKownledge-finetome-top-20k

language:

• en library_name: transformers inference: true

---

llama3-2-3b-finetuned-ScienceKnowledge

This model is a fine-tuned version of unsloth/Llama-3.2-3B-Instruct, specifically adapted for enhanced performance in scientific knowledge tasks.

Model Details

• Base Model: unsloth/Llama-3.2-3B-Instruct
• Fine-tuning Dataset: 0xZee/UniversalScienceKownledge-finetome-top-20k
• Model Type: Causal Language Model
• License: Apache 2.0
• Finetuning Technique: The model was likely fine-tuned using techniques such as LoRA or QLoRA, which are efficient for smaller datasets.

Intended Use

This model is designed for tasks requiring scientific understanding, question answering, and knowledge retrieval in scientific contexts. It can be used for:

• Assisting researchers with scientific queries
• Educational tools in science domains
• Automated scientific knowledge discovery and summarization

Training Procedure

The model was fine-tuned on the UniversalScienceKownledge-finetome-top-20k dataset. Specific hyperparameters used in training are not provided, but typical fine-tuning approaches for LLMs include:

• Learning rate: Often around 2e-5 for Mistral-based models
• LoRA rank: Likely between 512 and 768
• Batch size: Small, possibly between 1 and 4

Limitations

• Performance is dependent on the quality and scope of the training data.
• May not generalize well to scientific domains outside of its training data.
• Potential for generating plausible-sounding but incorrect information (hallucinations).

Bias, Risks, and Limitations

As with all language models, this model may reflect biases present in its training data. Users should be aware of potential biases in scientific literature and exercise caution when using the model for sensitive applications.

Evaluation Results

Quantitative evaluation results are not provided. Users are encouraged to evaluate the model's performance on their specific tasks and domains of interest.

How to Use

You can use this model with the Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "0xZee/llama3-2-3b-finetuned-ScienceKnowledge"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "Explain the concept of quantum entanglement."
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
output = model.generate(input_ids, max_new_tokens=100)
print(tokenizer.decode(output, skip_special_tokens=True))