lainanon-0.6b

lainanon-0.6b

A fine tune of Qwen/Qwen3-0.6B, a 600M parameter causal language model. Fine-tuned on the rudyon/lainchan-alpaca dataset, a scraped collection of posts from lainchan.org, using LoRA fine-tuning with bfloat16 precision on a Vast.ai instance with a single RTX 5090.

usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B", torch_dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("rudyon/lainanon-0.6b")
model = PeftModel.from_pretrained(base, "rudyon/lainanon-0.6b")

def chat(instruction, max_new_tokens=200):
    prompt = f"### Instruction:\n{instruction}\n### Response:\n"
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_new_tokens=max_new_tokens)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

print(chat("How do I get started with Emacs?"))

prompt format

### Instruction:
Your instruction here
### Response: