Downloads · 30 days
8
1% of all-time downloads
Jaume-inLab/vocabulary_sliced_CA-ES-EN-qwen3-14B
vocabulary_sliced_CA-ES-EN-qwen3-14B is a machine learning model from Jaume-inLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model is a vocabulary-sliced version of the original Qwen/Qwen3-14B. The vocabulary has been pruned to retain only tokens relevant to Catalan, Spanish, and English, as well as essential special tokens and base by…
Downloads · 30 days
8
1% of all-time downloads
All-time downloads
610
Public
Parameters
14.2B
28.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors28.4 GB · 100%
From the Hugging Face model README
This model is a vocabulary-sliced version of the original Qwen/Qwen3-14B. The vocabulary has been pruned to retain only tokens relevant to Catalan, Spanish, and English, as well as essential special tokens and base bytes.
The primary goal of this pruning is to reduce the memory footprint (VRAM) during training and inference, specifically for tasks like Direct Preference Optimization (DPO) with long sequence lengths.
By reducing the vocabulary size from 152,064 to ~134,943 tokens, the size of the embed_tokens and lm_head matrices is significantly reduced. This provides:
This pruned model is 100% compatible with LoRA adapters trained on the original Qwen/Qwen3-14B model, provided the adapters only target the Attention and MLP layers (e.g., q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).
Because the adapter weights do not interact with the embed_tokens or lm_head matrices, they can be attached to this pruned base model without any shape mismatches or degradation in output quality.
The vocabulary was pruned using the following criteria:
áéíóúüñÁÉÍÓÚÜÑ¿¡àèòïçÀÈÒÏÇ·€) was kept.You can load this model just like the original Qwen3-14B model:
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "Jaume-inLab/vocabulary_sliced_CA-ES-EN-qwen3-14B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")