ernie-arduino-qa-unsloth-v2

ERNIE Arduino Q&A Model

Model Description

This is an ERNIE model fine-tuned on Arduino documentation using Unsloth for the AI Challenge. The model is specifically optimized to answer questions about Arduino programming, libraries, hardware, and best practices.

Training Details

• Base Model: baidu/ERNIE-4.5-0.3B-PT
• Fine-tuning Method: Unsloth with LoRA
• Dataset: Arduino training instruction dataset (17,944 samples)
• Hardware: NVIDIA T4 GPU
• Training Time: 7591.14 seconds
• Final Training Loss: 1.1063

Hyperparameters

• Learning Rate: 0.0005
• Epochs: 2
• Batch Size: 2
• Gradient Accumulation Steps: 8
• LoRA Rank: 32
• LoRA Alpha: 64
• Max Sequence Length: 1024
• Quantization: 4-bit

Usage

from unsloth import FastLanguageModel

# Load model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="NotIshaan/ernie-arduino-qa-unsloth-v2",
    max_seq_length=1024,
    dtype=None,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

# Generate response
instruction = "What is Arduino?"
prompt = f'''Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:
'''

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
response = tokenizer.batch_decode(outputs)[0]
print(response)

Example Outputs

Question: "How do I use digitalWrite in Arduino?"

Answer: The model provides detailed explanations of Arduino functions, syntax, and best practices.

Limitations

• Trained specifically on Arduino documentation
• May not generalize well to other programming domains
• Responses limited to information present in training data

Citation

If you use this model, please cite:

@misc{ernie-arduino-qa,
  author = {Ishaan Pandey},
  title = {ERNIE Arduino Q&A Model},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/NotIshaan/ernie-arduino-qa-unsloth-v2}
}

Acknowledgments

• Unsloth for efficient fine-tuning framework
• ERNIE team for the base model
• Arduino community for documentation