Downloads · 30 days
56
7% of all-time downloads
nischay185/konkani-qwen2-1.5b-v3-full
konkani-qwen2-1.5b-v3-full is a text generation model from nischay185. Use it when you need the model to write or continue text. It is set up for transformers.
Konkani Qwen2-1.5B V3 Full is the standalone full-weight version of nischay185/konkani-qwen2-1.5b-v3.
Downloads · 30 days
56
7% of all-time downloads
All-time downloads
824
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.1 GB · 100%
From the Hugging Face model README
Konkani Qwen2-1.5B V3 Full is the standalone full-weight version of nischay185/konkani-qwen2-1.5b-v3.
It contains the same V3 model weights as the original V3 LoRA/PEFT release, but the adapter has been merged into the underlying Konkani SFT model and released as a standalone model.
Unlike the V3 adapter release, this repository does not require the original Konkani SFT model or PEFT to be loaded separately.
The model is designed to improve useful reasoning and task-following capabilities while retaining the Konkani language behavior learned during the original SFT stage.
V3 was not trained directly from the original Qwen2-1.5B base model.
The model lineage is:
Qwen/Qwen2-1.5B
↓
nischay185/konkani-qwen2-1.5b
Original Konkani SFT
↓
V3 LoRA / PEFT training
↓
nischay185/konkani-qwen2-1.5b-v3
V3 Adapter
↓
Adapter merged into base model
↓
nischay185/konkani-qwen2-1.5b-v3-full
Standalone V3 Full
The original V3 adapter is available as:
nischay185/konkani-qwen2-1.5b-v3
This repository is the standalone version:
nischay185/konkani-qwen2-1.5b-v3-full
The underlying model behavior and V3 training are the same.
The primary difference is the model format.
The adapter release contains only the LoRA adapter weights and requires:
nischay185/konkani-qwen2-1.5b
+
nischay185/konkani-qwen2-1.5b-v3
PEFT is required to attach the adapter to the original Konkani SFT model.
The full release contains the resulting standalone model weights:
nischay185/konkani-qwen2-1.5b-v3-full
It can be loaded directly with Transformers without separately loading the original Konkani SFT model or the V3 adapter.
The V3 Full model is therefore intended to provide a simpler deployment and inference experience.
The main objective of V3 is to improve the reasoning capability of a low-resource Konkani language model without replacing the language-specific knowledge and behavior learned during the original Konkani SFT stage.
The training approach was inspired by English-Pivoted Chain-of-Thought (CoT) reasoning for extremely low-resource languages.
The basic idea is:
Konkani question
↓
English reasoning trace
↓
Konkani final answer
English is used as an intermediate reasoning language because the underlying model has stronger English reasoning representations than Konkani.
The final response is still expected to be in Konkani.
V3 was trained using a mixed dataset containing both original Konkani SFT examples and reasoning-oriented CoT examples.
| Data source | Examples |
|---|---|
| Original Konkani SFT replay | 6,316 |
| English-Pivoted CoT | 3,926 |
| Corrected Targeted CoT | 2,390 |
| Total | 12,632 |
The final dataset contains approximately:
The SFT replay examples were included to help preserve the original model's Konkani behavior while introducing the new reasoning-oriented training examples.
V3 was trained using LoRA/PEFT on top of the original Konkani SFT model.
LoRA rank (r): 16
LoRA alpha: 32
LoRA dropout: 0.05
Target modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj
Learning rate: 2e-4
Scheduler: Cosine
Warmup ratio: 0.05
Weight decay: 0.01
Micro batch size: 2
Gradient accumulation: 8
Effective batch size: 16
Maximum sequence length: 512
Training epochs: 1
Optimizer: paged_adamw_8bit
Seed: 42
After training, the V3 adapter was merged with the original Konkani SFT model to produce this standalone release.
The V3 training run completed successfully with:
Global steps: 750
Training loss: 0.5795
Training runtime: ~0.83 hours
Validation loss was recorded during training:
Step 200: 0.592825
Step 400: 0.556194
Step 600: 0.537714
V3 was evaluated side-by-side against the original Konkani SFT model.
The comparison used:
Original SFT:
nischay185/konkani-qwen2-1.5b
V3:
nischay185/konkani-qwen2-1.5b
+
nischay185/konkani-qwen2-1.5b-v3
The V3 Full release contains the resulting standalone weights from the latter configuration.
Both models were given the same questions, system instruction, generation settings, and random seed.
The evaluation contained 39 questions across 13 categories.
These categories were used to check whether V3 retained the behavior of the original Konkani SFT model:
These categories were used to examine reasoning and task-following:
The evaluation was manually judged rather than using simple string matching.
Answers were considered based on factors such as:
The purpose of the evaluation is not only to ask whether V3 "reasons better", but whether it can improve useful reasoning while retaining the Konkani capabilities of the original SFT model.
Unlike the adapter release, the V3 Full model can be loaded directly.
pip install -U transformers accelerate torch
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_ID = "nischay185/konkani-qwen2-1.5b-v3-full"
tokenizer = AutoTokenizer.from_pretrained(
MODEL_ID,
trust_remote_code=True,
)
if tokenizer.pad_token is None:
tokenizer.pad_token = tokenizer.eos_token
dtype = torch.float16 if torch.cuda.is_available() else torch.float32
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID,
dtype=dtype,
device_map="auto",
trust_remote_code=True,
)
model.eval()
No separate PeftModel loading is required.
No separate download of nischay185/konkani-qwen2-1.5b is required for inference.
The evaluation used the following system instruction:
तुमी एक कोंकणी भाशेचो सहाय्यक,फक्त कोंकणी भाशेंत जाप दिवची,मराठी वा हिंदी वापरूं नाका.
The model was prompted using the Qwen chat format:
<|im_start|>system
[system instruction]
<|im_end|>
<|im_start|>user
[user question]
<|im_end|>
<|im_start|>assistant
For reasoning-oriented examples, V3 may produce an English reasoning trace followed by a Konkani answer.
A typical intended structure is:
<think>
English reasoning
</think>
Konkani final answer
The presence of a <think> section alone should not be interpreted as proof that the reasoning is correct. The final answer and task outcome should be evaluated for correctness.
Seed: 42
Max new tokens: 220
Temperature: 0.7
Top-p: 0.9
Repetition penalty: 1.1
Sampling: enabled
These settings are provided for reproducibility of the reported evaluation and are not necessarily optimal for every application.
This model is intended for:
This is a small 1.5B-parameter language model specialized for low-resource Konkani experimentation.
The model may:
The <think> section should not be treated as a guarantee of correct reasoning.
The reported evaluation is based on a relatively small manually evaluated set of 39 questions and should not be interpreted as a comprehensive benchmark of Konkani reasoning ability.
Base architecture:
Qwen/Qwen2-1.5B
Original Konkani SFT model:
nischay185/konkani-qwen2-1.5b
V3 adapter:
nischay185/konkani-qwen2-1.5b-v3
V3 standalone model:
nischay185/konkani-qwen2-1.5b-v3-full
V3 was developed as part of a research project investigating methods for improving low-resource language model capabilities using Goan Konkani as the test case.
Original Konkani SFT
nischay185/konkani-qwen2-1.5b
V3 LoRA Adapter
nischay185/konkani-qwen2-1.5b-v3
V3 Full Standalone
nischay185/konkani-qwen2-1.5b-v3-full
The adapter and full releases represent the same V3 model, provided in different formats for different deployment requirements.
If you use this model in research, please cite the associated work when available.
Please refer to the license and usage conditions of the original Qwen2-1.5B model and the original Konkani SFT model before redistributing or commercially using this model.
The V3 Full release contains merged weights derived from the original Konkani SFT model and the V3 LoRA training process. Users are responsible for complying with the applicable licenses and usage conditions of the underlying models.