qwen-mom-generator

Qwen2.5-3B MOM Generator

A LoRA adapter that generates structured Minutes of Meeting (MOM) from multilingual, code-switched meeting transcripts (Darija, French, English, MSA). This is stage 2 of a two-stage pipeline: it continues training from a Darija domain-adaptation adapter, then instruction-tunes on 500 self-generated meeting transcripts.

This end-to-end approach outperformed a sequential translate-then-generate pipeline across every metric measured (ROUGE-L +29.7 pts, BERTScore F1 +3.0 pts, schema compliance +4.1 pts).

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", torch_dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Chaima-KHENAFIF/qwen-mom-generator")
model = PeftModel.from_pretrained(base_model, "Chaima-KHENAFIF/qwen-mom-generator")

system_prompt = (
    "You are a professional secretary. Generate a structured Minutes of Meeting (MOM) "
    "in English. Output sections: SUMMARY, KEY DISCUSSIONS, DECISIONS, "
    "ACTION ITEMS, DEADLINES, NEXT STEPS."
)
prompt = f"<|im_start|>system\n{system_prompt}<|im_end|>\n<|im_start|>user\nTranscript:\n{{transcript}}<|im_end|>\n<|im_start|>assistant\n"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.1, do_sample=True, repetition_penalty=1.1)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Example

Input (code-switched English/French/Darija meeting transcript):

Ahmed: Salam, let's start. We are here to discuss the 'Corporate Monitoring Platform'... Chaima: Saha Ahmed. I wanted to update you, rani kemelt the web scraping scripts using Python... Sumaya: C'est parfait, Chaima. But wait, did you check the ethical constraints?... [continues]

Output:

### MINUTES OF MEETING

MEETING PARTICIPANTS: Ahmed (Project Manager), Chaima (Web Scraping Developer), Sumaya (Database Specialist)

#### SUMMARY
The team discussed the design of the 'Corporate Monitoring Platform'. Chaima confirmed
she had implemented web scraping scripts using Python while respecting ethical
constraints. The team agreed to use Oracle for structured data and Hadoop for log
analysis, and to address a WSL environment issue before Monday.

#### DECISIONS
- Use Oracle for structured data (student profiles).
- Use Hadoop for log analysis.
- Fix WSL crash issues by Monday.

#### ACTION ITEMS
- Chaima to assist Sumaya with WSL configuration.
- Sumaya to resolve the NameNode stability issue.

Example

Input transcript (code-switched English/French/Darija):

Ahmed: Salam, let's start. We are here to discuss the 'Corporate Monitoring Platform' for the internship project. Chaima: Saha Ahmed. I wanted to update you, rani kemelt the web scraping scripts using Python. Sumaya: C'est parfait, Chaima. But wait, did you check the ethical constraints? ...

Generated MOM:

### MINUTES OF MEETING

MEETING PARTICIPANTS: Ahmed (Project Manager), Chaima (Web Scraping Developer), Sumaya (Database Specialist)

#### SUMMARY
The team discussed the design of the 'Corporate Monitoring Platform' for an internship project.
The discussion revolved around the choice between Oracle and Hadoop databases, with Chaima
confirming she had implemented web scraping scripts using Python while respecting ethical
constraints. The team agreed to use Oracle for structured data and Hadoop for log analysis.

#### DECISIONS
- Use Oracle for structured data (student profiles).
- Use Hadoop for log analysis.
- Fix WSL crash issues by Monday.
- Prepare a complete presentation for the platform demo by Wednesday.

#### ACTION ITEMS
- Chaima to assist Sumaya with WSL configuration.
- Sumaya to resolve the NameNode stability issue.
- Ahmed to coordinate the project timeline.

#### IDENTIFIED PROBLEMS
- Ethical compliance risk during web scraping.
- Technical instability in Sumaya's development environment.
- Tight deadline for the platform demonstration.

Training

• Continues from a Stage 1 Darija domain-adaptation LoRA adapter (not frozen — trained further)
• QLoRA (4-bit), TRL SFTTrainer, 10 epochs, learning rate 3e-5, early stopping (patience 3)
• 500 self-generated meeting transcripts, code-switched across Darija, French, English, and MSA
• Output: structured MOM (participants, summary, decisions, action items, deadlines, identified problems)

Links

• Full code, training pipeline, and FastAPI deployment: GitHub repo
• Stage 1 (domain adaptation): qwen-darija-domain-adapted