finetuned-llama

Model Card for Finetuned Llama 3.2 (ROS Query System)

This model is a finetuned version of Llama 3.2 specifically designed to answer questions related to the Robot Operating System (ROS). It was finetuned on Kaggle using domain-specific data scraped from GitHub repositories and Medium articles. The model powers a Retrieval-Augmented Generation (RAG) pipeline in our AI final project.

---

Model Details

Model Description

• Developed by: Krish Murjani (netid: km6520) & Shresth Kapoor (netid: sk11677)
• Project Name: CS-GY-6613 AI Final Project: ROS Query System
• Finetuned From: sentence-transformers/all-MiniLM-L6-v2
• Language(s): English
• License: Apache 2.0

---

Model Sources

• Repository: GitHub Repository

---

Uses

Direct Use

The model is used in a Retrieval-Augmented Generation (RAG) pipeline for answering questions related to the Robot Operating System (ROS). It integrates with a vector search engine (Qdrant) and MongoDB for efficient retrieval and query response generation.

Downstream Use

The model can be extended for other technical domains through additional finetuning or plug-in integration into larger AI systems.

Out-of-Scope Use

The model is not designed for tasks outside of technical documentation retrieval and answering ROS-related queries.

---

Bias, Risks, and Limitations

• Bias: The model may reflect biases inherent in the scraped ROS documentation and articles.
• Limitations: Responses are limited to the scraped and finetuned dataset. It may not generalize to broader queries.

Recommendations

• Use the model for educational and research purposes in robotics and ROS-specific domains.
• Avoid using the model in high-stakes applications where critical decisions rely on the accuracy of generated responses.

---

How to Get Started with the Model

from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("your-model-id")
tokenizer = AutoTokenizer.from_pretrained("your-model-id")

input_text = "How can I navigate to a specific pose using ROS?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model(**inputs)
print(outputs)

Training Details

Training Data

• Sources:
  • GitHub repositories related to the Robot Operating System (ROS).
  • Medium articles discussing ROS topics.

Training Procedure

• Preprocessing:

  • Data cleaning, text chunking, and embedding using Sentence-BERT (all-MiniLM-L6-v2).
  • Used ClearML orchestrator for ETL and finetuning pipelines.

• Training Framework:

  • Hugging Face Transformers, PyTorch

• Training Regime:

  • fp16 mixed precision (for efficiency and memory optimization)

---

Evaluation

Testing Data

• Dataset:
  • Internal evaluation dataset created from project-specific queries and generated question-answer pairs.

Factors & Metrics

• Metrics:

  • Query relevance, answer accuracy, and completeness.

• Evaluation Results:

  • Achieved high relevance and precision for domain-specific questions related to ROS.

---

Environmental Impact

• Hardware Type:

  • NVIDIA Tesla T4 (Kaggle)

• Hours Used:

  • Approximately 15-20 hours of training

• Compute Region:

  • US Central (Kaggle Cloud)

• Carbon Emitted:

  • Estimated using the Machine Learning Impact Calculator.

---

Technical Specifications

• Model Architecture:

  • Transformer-based language model (Llama 3.2)

• Compute Infrastructure:

  • Kaggle Cloud with NVIDIA Tesla T4 GPUs

• Frameworks:

  • Hugging Face Transformers, PyTorch, ClearML

---

Citation

@misc{kapoor2024rosquery,
  title={ROS Query System: A Retrieval-Augmented Generation Pipeline},
  author={Shresth Kapoor and Krish Murjani},
  year={2024},
  note={CS-GY-6613 AI Final Project, NYU Tandon School of Engineering}
}

Model Card Authors

• Krish Murjani (krishmurjani)
• Shresth Kapoor (shresthkapoor7)

Model Card Contact

For any inquiries, please contact us through our (GitHub Repository).