vian_ai_shop_small

Model Card for Vian ai Fine-Tuned

Model Details

Model Description

• Developed by: Glavinnguyen
• Model type: Text Generation / Multimodal (Text/Vision/Audio Interface)
• Language(s) (NLP): Vietnamese (Primary), English (Primary)
• License: mit
• Finetuned from model: google/gemma-3-4b-it

Model Sources

• Repository: [More Information Needed]

Uses

Direct Use

This model is specifically fine-tuned on a small-scale dataset to standardize response formatting, style, and structure. It natively retains the advanced deep reasoning mechanisms (<|think|>) and inherent logical problem-solving capabilities of the base Gemma 3 architecture.

Out-of-Scope Use

The model should not be used for tasks requiring extensive broad-domain knowledge expansion outside the scope of the training dataset without human supervision. The fine-tuning process was focused on structuring behavioral output rather than massive knowledge injection.

Training Details

Training Data

The model was trained on a highly curated, high-quality alignment dataset.

Training Procedure

Training was conducted utilizing the QLoRA (Quantized Low-Rank Adaptation) method to minimize hardware resource consumption while aggressively preserving the base model's pre-trained weights.

Training Hyperparameters

The hyperparameters were carefully optimized for an ultra-small dataset to prevent catastrophic overfitting and achieve an ideal convergence point:

• Training regime: QLoRA (FP16/BF16 Mixed Precision)
• Learning Rate: 2e-4 to 1e-5 (Low Learning Rate)
• Per Device Train Batch Size: 1
• Gradient Accumulation Steps: 32 (Global Batch Size = 32)
• Number of Train Epochs: 1 to 2
• Max Length: 256 - 2048 tokens (Allocated to safeguard the generation of <|think|> tokens)
• Optimizer: AdamW
• LR Scheduler Type: Cosine / Constant
• LoRA Rank (r): 4 or 8
• LoRA Alpha ($\alpha$): 8 or 16

Speeds, Sizes, Times

• Final Training Loss: 0.69 (The sweet spot convergence for low-sample fine-tuning—balancing structural alignment with base intelligence retention).
• VRAM Consumption: ~2.2 GB (When running on a 4-bit Q4_0 execution profile).

Technical Specifications

Model Architecture and Objective

Built upon Google's next-generation Gemma 3 architecture, featuring integrated Quantization Aware Training (QAT) and an intrinsic multi-token prediction (MTP) engine. The model leverages an internal step-by-step reasoning loop before routing structural text outputs via designated generation tags.

Model Card Contact

• Contact: Glavinnguyen

Framework versions

• PEFT 0.19.1