DMaS-LLaMa-Lite-step-43.5k-instruct

DMaS-LLaMa-Lite-step-43.5k-instruct

This repository provides access to DMaS-LLaMa-Lite-step-43.5k-instruct, an instruction fine-tuned version of the DMaS-LLaMa-Lite model. Based on the 43,500-step checkpoint, this model has been further fine-tuned on question-answering datasets to enable instruction-following and dialogue capabilities.

It serves as a demonstration of the potential to fine-tune a pretrained LLaMa-based model for downstream tasks, such as question-answering, with improved alignment for human-like responses.

Model Overview

• Base Model: DMaS-LLaMa-Lite-step-43.5k
• Architecture: LLaMa-based
• Parameters: 1.7B (36 layers, 32 attention heads, RMSNorm)
• Tokenizer: GPT-2 tokenizer
• Fine-tuning Data: yahma/alpaca-cleaned
• Chat Template: Follows Vicuna 1.1 format
• Response End: Machine answers are terminated with <|endoftext|>
• Objective: Fine-tune the pretrained model to improve alignment with user instructions and dialogue-based question answering

This model showcases how a LLaMa-based pretrained checkpoint can be effectively adapted for instruction-following tasks, resulting in coherent, relevant, and contextually grounded completions.

Key Features

• Instruction Fine-tuning: Demonstrates alignment to question-answering and dialogue tasks.
• Chat Template Compatibility: Compatible with Vicuna 1.1-style formatting, ensuring easy integration with chat-based applications.
• Early Demonstration: Highlights the feasibility of fine-tuning pretrained models with minimal downstream data.

Chat Template

The model follows the Vicuna 1.1 chat format:

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.

USER: {user input}
ASSISTANT:{machine response}<|endoftext|>

Usage

You can load and use the model with Hugging Face's Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "McGill-DMaS/DMaS-LLaMa-Lite-step-43.5k-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Define the prompt in Vicuna 1.1 format
prompt = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.

USER: What are the Pyramids of Giza known for?
ASSISTANT:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)

# Decode the machine response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Citation

If you use this model or its training insights in your work, please cite the following paper:

@INPROCEEDINGS{li2025training,
  author={Li, Miles Q. and Fung, Benjamin C. M. and Huang, Shih-Chia},
  booktitle={2025 International Joint Conference on Neural Networks (IJCNN)},
  title={Training Dynamics of a 1.7B LLaMa Model: A Data-Efficient Approach},
  year={2025},
  volume={},
  number={},
  pages={1-10},
  keywords={Training;Analytical models;Refining;Benchmark testing;Throughput;Data models;Hardware;Stability analysis;Trajectory;Tuning},
  doi={10.1109/IJCNN64981.2025.11228044}}

License

This model and code are released under the Apache License 2.0. Please check the respective repositories for detailed terms.

Training Code

For details on pre-training processes, the scripts are available at:
📄 DMaS-LLaMa-Lite Training Code