Downloads · 30 days
14
11% of all-time downloads
yowww1094/tourism-llm-fine-tuned-qwen2-1.5b-lora-merged
tourism-llm-fine-tuned-qwen2-1.5b-lora-merged is a question answering model from yowww1094. Use it when the input is a question plus a passage. It is set up for transformers. The card lists the license as mit.
A fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct trained on a custom tourism Q&A dataset generated through a RAG-grounded pipeline. Built as a personal end-to-end learning project covering data collection, dataset e…
Downloads · 30 days
14
11% of all-time downloads
All-time downloads
124
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.1 GB · 99%
From the Hugging Face model README
A fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct trained on a custom tourism Q&A dataset generated through a RAG-grounded pipeline. Built as a personal end-to-end learning project covering data collection, dataset engineering, supervised fine-tuning, and deployment.
⚠️ This model is not production-ready. It is a learning project. Outputs can be incorrect, incomplete, or inconsistent — especially on topics not well-represented in the training data. Do not rely on this model for real travel decisions.
This model was fine-tuned using FP16 LoRA (Low-Rank Adaptation) on a small custom dataset of ~150–200 tourism-focused question-answer pairs. The training examples were generated through a RAG pipeline: Reddit travel posts were scraped, embedded, and stored in a Qdrant vector store, then a locally-hosted Qwen2.5-7B (via Ollama) was used to generate grounded Q&A pairs from retrieved context chunks. The resulting dataset was formatted in ChatML and used to fine-tune this model.
The goal of fine-tuning was to adjust the model's behavioral style — making it more focused, concise, and consistently helpful for travel queries — rather than to inject new factual knowledge. Factual grounding at inference time is handled by a RAG pipeline backed by Qdrant Cloud.
This model can be used as a conversational assistant for general tourism and travel questions — destination information, logistics, visa guidance, packing advice, and similar topics. It works best when paired with a retrieval pipeline that provides relevant context at inference time.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "yowww1094/tourism-assistant-qwen2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [
{"role": "system", "content": "You are a helpful tourism assistant."},
{"role": "user", "content": "What are the must-visit places in Marrakech?"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)
This model is designed to be used as the generation component of a RAG pipeline. The recommended usage is:
all-MiniLM-L6-v2Pairing the model with retrieval significantly improves factual accuracy on specific travel queries compared to using the model standalone.
Dataset limitations — primary source of inaccuracy: The training dataset contains only ~150–200 examples sourced from Reddit travel communities. This introduces several compounding problems:
Model behaviour limitations:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "yowww1094/tourism-assistant-qwen2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto"
)
messages = [
{"role": "system", "content": "You are a helpful tourism assistant. Answer travel questions clearly and concisely."},
{"role": "user", "content": "What is the best time of year to visit Morocco?"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=300,
temperature=0.7,
top_p=0.9,
do_sample=True,
repetition_penalty=1.1
)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)
# retrieved_context = top-k chunks from your Qdrant vector store
system_prompt = "You are a helpful tourism assistant. Use only the provided context to answer. If the context does not contain enough information, say so."
user_prompt = f"""Context:
{retrieved_context}
Question: {user_query}"""
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
]
# then apply chat template and generate as above
The model was fine-tuned on yowww1094/tourism-llm-fine-tuning-dataset, a custom dataset generated through the following pipeline:
all-MiniLM-L6-v2 and indexed in a Qdrant vector storeDataset size: ~150–200 question-answer pairs
Format: JSON Lines
Split: 90% train / 10% validation
Quality note: Only ~65% of manually reviewed examples were rated acceptable. No automated quality filter was applied to the full dataset.
Each example was formatted using the Qwen2.5-Instruct chat template via tokenizer.apply_chat_template(). Sequences were truncated to a maximum length of 512 tokens. No data augmentation was applied.
| Parameter | Value |
|---|---|
| Training regime | FP16 mixed precision |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | q_proj, v_proj |
| Epochs | 3 |
| Batch size | 4 |
| Gradient accumulation steps | 4 (effective batch size: 16) |
| Learning rate | 2e-4 |
| LR scheduler | Cosine annealing |
| Warmup ratio | 0.03 |
| Max sequence length | 512 tokens |
| Optimizer | AdamW (default via transformers) |
Evaluation was performed on a held-out validation split of ~15–20 examples (10% of the total dataset). Due to the small size of this split, quantitative metrics should be interpreted with significant caution — they are not statistically reliable estimates of generalisation performance.
In addition to automatic metrics, a qualitative evaluation was performed by manually inspecting model outputs on ~20 held-out queries not present in the training data.
| Metric | Value | Notes |
|---|---|---|
| Training loss (epoch 1) | ~1.62 | |
| Training loss (epoch 3) | ~0.85 | Consistent decrease across epochs |
| Validation loss (epoch 1) | ~1.74 | |
| Validation loss (epoch 3) | ~1.29 | Slight divergence from train loss — mild overfitting |
| Validation perplexity | ~2.3 | On 15–20 examples only; not a reliable generalisation estimate |
What improved after fine-tuning:
What did not improve:
Fine-tuning on this small dataset improved behavioral style but not factual coverage. The model is more useful when paired with a retrieval pipeline than when used standalone. A dataset 10–20× larger with verified factual content would be needed to produce a genuinely reliable tourism assistant.
Carbon emissions were not formally measured. Estimated figures based on training setup:
q_proj and v_proj attention matrices, then merged into base weights before uploadsentence-transformers/all-MiniLM-L6-v2, CPU inferencetransformers>=4.40.0
peft>=0.10.0
trl>=0.8.0
accelerate>=0.28.0
sentence-transformers>=2.6.0
qdrant-client>=1.9.0
torch>=2.1.0
If you reference this project, please cite it as:
@misc{younes2026tourismllm,
author = {Younes},
title = {End-to-End LLM Pipeline for Tourism Assistant},
year = {2026},
publisher = {Hugging Face},
url = {[Model](https://huggingface.co/yowww1094/tourism-llm-fine-tuned-qwen2-1.5b-lora-merged)}
}
Full technical report (covering pipeline design, training decisions, limitations, and results in detail): docs/technical_report.pdf
GitHub repository with full pipeline code: Github
Training dataset: yowww1094/tourism-llm-fine-tuning-dataset
Younes AIT SI ABBOU — personal learning project, April 2026