job-searcher-qwen3-8B

Job Search (Qwen3-8B)

Two LoRA adapters distilled from DeepSeek V4 Pro onto Qwen/Qwen3-8B. Both adapters preserve the teacher's <think> reasoning traces.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-8B"
repo = "emrekuruu/job-search-lora"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype="bfloat16", device_map="auto")

# Load one task adapter
model = PeftModel.from_pretrained(model, repo, subfolder="query_gen")

# Or swap to the other task on the same base
model.load_adapter(repo, subfolder="fit_eval", adapter_name="fit_eval")
model.set_adapter("fit_eval")

Both tasks expect a chat-formatted prompt; see the reference prompts in the source repo.

Training

• Base: Qwen/Qwen3-8B, bf16, SDPA attention.
• LoRA: r=16, α=16, dropout=0, targets q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.
• Optimizer: AdamW, lr=2e-4, cosine schedule, weight_decay=0.01, warmup 20 steps.
• Sequence length: 16,384 (no truncation).
• Loss masking: assistant-only via TRL SFTConfig(assistant_only_loss=True).
• Selection: best eval_val_loss (early stopping patience=2), evaluated each epoch.
• Data: emrekuruu/job-search-distill — the query_gen_pairings config for query_gen/, the job_evals config for fit_eval/.
• Hardware: A100-40GB on Modal.

Training code: modal_apps/train.py.

Intended uses

• Resume-aware job-search assistants and ranking systems.
• Research baselines for reasoning-trace distillation from large teachers into ≤10B students.

Out-of-scope uses

• Production hiring decisions. The 5 fit-evaluation dimensions are author-defined and not a validated rubric. Treat total as an ordinal signal within a single candidate's shortlist, not as cross-candidate ground truth.
• High-stakes scoring without human review. Teacher errors propagate through distillation.

License

Apache-2.0, matching the Qwen3-8B base. Teacher labels used during training were generated via the DeepSeek API and are subject to DeepSeek's Open Platform Terms of Service.