cpt-demo-sft-mixed

CPT Demo SFT Mixed

cpt-demo-sft-mixed is an experimental, English-language, 135M-parameter-class assistant based on HuggingFaceTB/SmolLM-135M. It was adapted in two stages:

1. continual pretraining (CPT) on cleaned text extracted from recent AI and computer-science papers; and
2. supervised fine-tuning (SFT) on a mixture of scientific instructions and general conversations.

خلاصهٔ فارسی: این مدل یک دستیار آزمایشی انگلیسی‌زبان بر پایهٔ SmolLM-135M است. مدل ابتدا روی متن پاک‌سازی‌شدهٔ ۳۹۲ مقالهٔ علمی CPT شده و سپس با ۴۱٬۹۶۰ نمونهٔ آموزشی علمی و مکالمه‌ای SFT شده است. این نسخه هنوز مدل فارسی نیست و برای کاربردهای حساس یا پاسخ‌های کاملاً دقیق مناسب نیست.

Model details

This repository contains the merged mixed-SFT checkpoint. It does not require a separate LoRA adapter at inference time.

Training data

Stage 1: continual pretraining

The CPT corpus was built from 392 cleaned arXiv papers. The collection originally contained 400 downloaded files; eight malformed text extractions were removed before dataset creation.

The downloader targeted the following arXiv categories: cs.AI, cs.CL, cs.CV, cs.LG, cs.RO, cs.IR, cs.NE, cs.HC, cs.MA, and cs.CR.

Stage 2: mixed supervised fine-tuning

The SFT mixture combined scientific instructions with general conversations. Each source contributed 100 held-out validation examples.

Scientific examples use synthetic instruction/response data. Consequently, the model may inherit factual errors, omissions, formatting artifacts, or biases from the source papers and generation pipeline.

Training procedure

Both stages used parameter-efficient LoRA training with rank 32 and alpha 32 on the attention and MLP projection layers. The final weights were merged for distribution.

For mixed SFT, examples were packed and loss was applied only to assistant responses. The best validation checkpoint was loaded at the end of training.

Training metrics

These losses are internal validation metrics from different training stages and are not standardized benchmark scores. Formal benchmark results for this exact release are not currently reported, so these values should not be used to claim superiority over the base model or other models.

An experimental DPO checkpoint was also trained, but it was excluded from this release after qualitative regression testing. This repository intentionally publishes the mixed-SFT checkpoint.

Usage

Install the runtime dependencies:

pip install torch transformers accelerate safetensors

The model was trained with ChatML formatting. The following example constructs that prompt explicitly, so it does not depend on a tokenizer-side chat template:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "vlpMilad/cpt-demo-sft-mixed"

tokenizer = AutoTokenizer.from_pretrained(model_id)
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=dtype,
    device_map="auto",
)

system_prompt = (
    "You are a helpful, respectful, and honest assistant. "
    "Answer general questions naturally and directly. You are also knowledgeable "
    "about AI, deep learning, machine learning research, and their applications. "
    "If you do not know an answer, say so instead of making it up."
)
user_prompt = "Explain overfitting in neural networks in simple terms."

prompt = (
    f"<|im_start|>system\n{system_prompt}<|im_end|>\n"
    f"<|im_start|>user\n{user_prompt}<|im_end|>\n"
    "<|im_start|>assistant\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.eos_token_id,
    )

new_tokens = output[0, inputs.input_ids.shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

The system prompt is input context, not a guaranteed permanent identity. You can replace it with a different behavior or identity description, but a model of this size may not follow every instruction reliably.

Intended uses

This model is intended for:

• educational experiments with CPT, SFT, and small language models;
• lightweight English conversation;
• simple explanations of AI and machine-learning concepts; and
• research prototypes where outputs are reviewed by a human.

It is not intended to be an authoritative source or an autonomous decision-maker.

Limitations

• The model is English-only; this release was not trained as a Persian assistant.
• At roughly 135M parameters, its factual knowledge and reasoning ability are limited.
• It can hallucinate facts or give technically inaccurate explanations.
• It may repeat phrases, stop mid-answer, or fail strict formatting and counting instructions.
• Scientific coverage is limited to the selected corpus and does not guarantee that a particular question was present in training.
• It has not undergone a comprehensive safety evaluation.
• Do not rely on it for medical, legal, financial, security-critical, or other high-stakes decisions.

Users should validate important claims against reliable primary sources.

Licensing and attribution

The base model is released under the Apache License 2.0. HuggingFaceTB/smol-smoltalk is also tagged Apache-2.0. The scientific dataset card states that it is for educational purposes and asks users to ensure compatibility with the licenses and terms of the original source papers.

Because the training mixture includes third-party material with source-dependent terms, this model card does not declare a blanket license for all possible downstream uses. Users are responsible for reviewing the licenses and terms of the base model, datasets, and original papers before redistribution or commercial use.

Acknowledgements

This project builds on SmolLM and datasets published by Hugging Face, as well as the scientific instruction dataset published by paperbd. Thanks to their authors and contributors.