Downloads · 30 days
5
7% of all-time downloads
potato-gang-gc
jungter/potato-gang-gc
potato-gang-gc is a text generation model from jungter. Use it when you need the model to write or continue text. It is set up for peft.
LoRA adapter fine-tuned on curated Discord group chat data for conversational style transfer. Just a for fun project.
All-time downloads
74
Public
Repo size
42.5 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt28.3 MB · 60%
At a glance
- Task
- Text Generation
- Library
- peft
- Access
- Public
- Created
- May 13, 2024
- Updated
- Mar 10, 2026
- SHA
- ab98d469
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
LFM2.5-1.2B-Base LoRA Fine-Tune (Checkpoint 19000)
LoRA adapter fine-tuned on curated Discord group chat data for conversational style transfer. Just a for fun project.
Model Details
Model Description
This is a LoRA adapter trained on top of LiquidAI/LFM2.5-1.2B-Base using the Unsloth + SFTTrainer pipeline. The training data was sourced from Discord message exports and processed through a multi-stage cleaning pipeline: heuristic-based filtering (bot removal, URL stripping, duplicate/echo detection, consecutive message merging, low-quality filtering), followed by LLM-based pair classification (Qwen3.5-35B-A3B via vLLM) to validate prompt-response relevance and conversation coherence.
- Developed by: Ted S
- Model type: Causal Language Model (LoRA adapter)
- Language(s): English
- License: Same as base model (LiquidAI/LFM2.5-1.2B-Base)
- Finetuned from model: LiquidAI/LFM2.5-1.2B-Base
Uses
Direct Use
Conversational text generation mimicking the style and tone of the Discord group used for training. Intended for personal/hobby use as a chatbot.
Out-of-Scope Use
Not intended for production applications, factual Q&A, or any use requiring reliable or safe outputs. This model adopts the casual, unfiltered style of the training data.
Training Details
Training Data
- Source: Discord group chat message export
- Preprocessing pipeline:
- Message normalization (bot filtering, content cleaning, timestamp parsing)
- Consecutive message merging (120s window)
- Echo/duplicate removal
- Low-quality message filtering (min 5 chars, min 2 words)
- Reply candidate collection (explicit Discord replies + adjacent message pairing)
- LLM-based classification using Qwen3.5-35B-A3B with structured JSON output to validate prompt-response pairs (confidence threshold: 0.65)
- Multi-turn conversation building + standalone pair extraction
- Format: Chat-template formatted text (
user/assistantroles) with Unsloth packing
Training Procedure
Training Hyperparameters
| Parameter | Value |
|---|---|
| Training regime | bf16 mixed precision |
| Base model | LiquidAI/LFM2.5-1.2B-Base |
| Method | LoRA (PEFT) |
| LoRA rank (r) | 64 |
| LoRA alpha | 64 |
| LoRA dropout | 0 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Max sequence length | 2048 |
| Batch size | 32–64 (adjusted mid-training) |
| Gradient accumulation | 1–2 (effective batch: 64) |
| Learning rate | 2e-5 |
| LR scheduler | Cosine |
| Optimizer | AdamW (full precision) |
| Weight decay | 0.01 |
| Warmup steps | 100 |
| Packing | Enabled |
| Gradient checkpointing | Unsloth optimized |
| Eval split | 5% |
| Checkpoint | Step 19000 |
Technical Specifications
Compute Infrastructure
Hardware
- Training GPU: NVIDIA B200, H100 (overkill but i didn't want to wait)
- Data cleaning GPU: NVIDIA H100 (vLLM inference for LLM-based pair classification)
Software
- Unsloth
- Transformers
- TRL (SFTTrainer)
- PEFT
- vLLM (data cleaning pipeline)
Framework Versions
- PEFT 0.18.1
- Task
- Text Generation
- Library
- peft
- Languages
- en
- Created
- May 13, 2024
- Updated
- Mar 10, 2026