Dhenu2-In-Llama3.2-3B-Instruct

Dhenu2 India 3B

Model Overview

Model Name: Llama3.2-Dhenu2-In-3B-Instruct

Architecture: Llama3.2

Parameters: 3 Billion

Release Date: 24th October, 2024

License: Llama 3.2 Community License

Description

Dhenu2 India 3B offers a balanced performance, making it versatile for a wide range of agricultural applications. Built on the Llama3.2 architecture, this model is optimized to provide reliable conversational capabilities, ensuring a harmonious blend of knowledge depth and responsiveness. It is ideal for conversational applications that require accurate information and interactive engagement in the agricultural domain.

Intended Use

• Chatbots and Virtual Assistants: Develop interactive tools that assist farmers with daily agricultural queries and tasks.

Training Data

Dhenu2 India 3B was trained on a curated dataset that includes:

• Instruction Set: Over 1.5 million instructions from real and synthetic conversations.
• Synthetic Instructions: Generated to encompass more than 4,000 diverse agricultural topics.
• Data Sources: Mobile extension service logs, farmer feedback, agricultural package of practices, and localized studies.

Training Procedure

• Techniques: Utilized a combination of full fine-tuning and Low-Rank Adaptation (LoRA) to enhance model performance while conserving computational resources.
• Hardware: Trained on multi-GPU configurations with NVIDIA A100 GPUs, employing DeepSpeed for efficient distributed training.
• Optimization: Applied flash attention mechanisms to improve computational efficiency and reduce memory usage.

Evaluation

• Human Evaluation: Reviewed by agricultural experts for the accuracy and relevance of responses.
• Synthetic Evaluation: Conducted peer assessments using other LLMs to verify consistency and correctness.
• Performance Metrics: Measured based on domain-specific accuracy, response relevancy, and conversational fluency.

Limitations

Dhenu2 India 3B is tailored for agricultural applications and may not perform optimally outside this domain. It is essential to deploy this model within relevant agricultural contexts to maintain the accuracy and reliability of its responses.

API

Use our platform Dhenu with a generous free quota to start building your agriculture applications.

A note of gratitude

We want to thank our partners Microsoft and Microsoft for Startups for landing us compute. We would also like to thank our partner, Meta, for the open-source Llama models.

Contact

For more information, support, or collaboration inquiries, please contact us at [[email protected]].