phi_3_5

Phi 3.5 Mini Instruct

This repository hosts the Phi 3.5 Mini Instruct model, a fine-tuned version of the Phi-3.5 base model. It is optimized for instruction-following tasks and supports various downstream applications, such as question answering, summarization, and chatbot interactions.

Model Overview

• Base Model: Phi-3.5-mini-instruct
• Fine-Tuning Framework: LoRA (Low-Rank Adaptation)
• Quantization: 4-bit quantized for efficient inference
• Sequence Length: Supports sequences up to 2048 tokens
• Training Dataset: Alpaca-style instruction-following dataset

Features

• Optimized for instruction-following and conversational tasks
• Lightweight model leveraging 4-bit quantization
• Supports extended contexts via RoPE scaling (2048 tokens)

Intended Use

Applications

• Conversational AI: Develop chatbots for natural and engaging interactions
• Text Summarization: Generate concise summaries of input texts
• Question Answering: Provide accurate answers based on given context
• Instruction Following: Perform tasks based on detailed instructions

Limitations

• The model may not perform well on tasks requiring knowledge outside its training data.
• Outputs should be validated in critical applications, such as healthcare or legal advice.

Usage

Installation

To use this model, first install the necessary dependencies:

pip install transformers accelerate