resume-optimizer

Introduction

<!-- Provide a quick summary of what the model is/does. -->

The job market has become increasingly competitive, with applicant pools growing larger while hiring processes become more automated and less personal. Many applicant, particularly early-career professionals and recent graduates, struggle to present their skills effectively in thier resumes. Although large language models (LLMs) can assist in generating or editing content, most often have several limitations. They often produce generic, repetitive bullet points, or they lack alignment with the specific job description. They fail to incorporate the applicant’s actual experience, and frequently hallucinate responsibilities or achievements. As a result, resume bullet points may become misaligned, or uncompetitive against employer expectations.

This project aims to address these limitations by developing a model that is specifically resume focused, designed to rewrite a user’s resume bullet points so that they match the language and priorities expressed in a target job description. Unlike general writing assistants or chat based models, this model is fine-tuned specifically for resume rewriting tasks using a large scale, instruction formatted dataset developed from real resume entries and job descriptions. To further improve data clarity and relevance, I also generated synthetic teacher-model responses using Qwen2.5-3B-Instruct, producing a high-quality target output for each instruction. The final dataset includes over 96,000 cleaned and augmented resume bullet examples.

To build an optimized model, I finetuned Qwen2.5-1.5B-Instruct techniques and later merged the LoRA weights into a one model checkpoint for use. I then evaluated the model using both standard benchmarks (HellaSwag, PIQA, Winogrande), as well as more relevant metrics(ROUGE-L, BLEU, BERTScore). Across these metrics, the finetuned model demonstrated improved semantic similarity to target outputs and stronger rewrite alignment. Finetuning enhanced the model’s ability to generate job aligned resume content.

Ultimately, this project provides a resume optimization assistant capable of providing job-specific bullet points that incorporate user experience while matching the expectations of a given job description. This addresses a key limitation of general LLM tools usage in the highly specialized requirements of job applications.

Model Overview

The goal of this project was to train a large language model capable of rewriting resume bullet points so that they align closely with a target job description, while also preserving the user’s real life experience. To accomplish this, I began with Qwen2.5-1.5B-Instruct, a small but high quality instruction-tuned model known for its strong rewriting abilities. I then finetuned the model using LoRA to adapt it to the resume optimization task without modifying the full base model weights.

The fine-tuning process used a 96,658 example dataset, constructed by pairing real resume entries with job descriptions, and generating rewrite targets using a larger teacher model Qwen2.5-3B-Instruct. The dataset was cleaned to remove malformed samples, duplicated text, hallucinated content, and entries without valid responses. The final dataset was formatted into a consistent INSTRUCTION + RESPONSE structure used for finetuning.

I trained several LoRA configurations and selected a final setup that balanced quality, efficiency, and training stability. The best-performing configuration used:

• Rank(r): 32

• Alpha: 32

• Dropout: 0.05

• Batching: batch size of 32

• Max sequence length: 512 tokens

To evaluate performance, I benchmarked both the base model and finetuned model across a range of tasks. General language understanding benchmarks (HellaSwag, PIQA, Winogrande) were included to test whether finetuning affected core reasoning ability. I designed a resume rewrite specific evaluation set, using ROUGE-L, BLEU, and BERTScore to measure similarity between model rewrites and teacher-generated outputs. These metrics showed significant improvements in semantic alignment and content accuracy after finetuning, demonstrating that the model adapted well to the resume rewriting domain while also performing on general benchmarks.

The final model represents a compact, efficiently trained, and semantically aligned résumé optimization assistant capable of generating polished rewrites that align with job-specific requirements, reduce hallucination risk, and more accurately represent user experience.

Data

The training dataset for this project was constructed to reflect real-world resume rewriting needs. Users typically have a resume bullet point that describes their experience and a job description outlining the expectations for the role they want to apply for. To capture this interaction, I built a large paired dataset containing both of these elements along with a rewritten output.

Raw Data Source

The dataset combines multiple sources to reach a final size of 96,658 examples. The raw data combined an opensporks dataset along with a Kaggle dataset resume dataset.

Each raw record contains:

resume_entry: the original resume bullet

job_description: the target role’s description

Instruction: a templated instruction combining both

Response: the rewritten resume bullet (usihng synthetic generation)

The synthetic generation step was necessary because no raw dataset existed for this specific task.

Data Cleaning

Because the raw resumes and job descriptions contained a lot of noise, the following cleaning steps were used:

After cleaning, all rows had standardized strings and the dataset was reduced to 96,658 high-quality samples.

Instruction Formatting

To prepare data for fine-tuning, each example was converted into a standard format:

INSTRUCTION:
Rewrite the resume entry so it aligns with the job description...

RESPONSE:
model-generated rewritten bullet

Teacher Generated Response

Because the original dataset lacked labeled rewrite outputs, I used a large scale data generation pipeline. For this, I used the Qwen2.5-3B-Instruct model to generate rewrites for every instruction, which took over 4 hours. Any and all faulty generations were replaced with empty strings and filtered during cleanup. These responses were then used as the labels for finetuning.

Train/Validation Split

The cleaned dataset was split using a standard 90% train (86,992 data points) and 10% validation (9,666 data points) split. The validation set was used during hyperparameter tuning and validation loss tracking. A validation subsample of 500 examples was also used for quick benchmarking (ROUGE, BLEU, BERTScore) to reduce GPU compute time.

Dataset Limitations

Despite its massive scale and utility, the dataset has some pretty important limitations. Synthetic labels may contain bias from the teacher model. The dataset focuses on single bullets which reduces the utility of the model. There was also a large domain imbalance towqards finance, tech, operations, and healthcare roles.

Training Method

Qwen2.5-1.5B-Instruct was used as a base model. It offered a strong tradeoff between capability and compute efficiency, making it suitable for my project in terms of reliability. To adapt the base model to the task, I applied Low-Rank Adaptation (LoRA) using the cleaned dataset I had created earlier.

Training Pipeline

1. First I loaded the base model and tokenizer.
2. Tokenize dataset
3. Apply LoRA adapters

Lora was particularly beneficial for this task because Qwen does well with language generation prompts, and it is able to learn stylistic and structural shifts without overwriting the model’s core language abilities. MLP tuning helped adapt the tone.

Training Setup & Hyperparameters

Training was completed with the following parameters:

Hyperparameter Search and Results

To identify an effective configuration for adapting Qwen2.5-1.5B-Instruct to the résumé-rewriting task, I performed a structured hyperparameter search across LoRA parameters and training settings. Because the dataset contains 96,658 cleaned instruction–response pairs, even small hyperparameter differences can influence accuracy. Each run used the same train/validation split for comparison. The hyperparameter space was chosen to test were: LoRA rank sensitivity, dropout stability, learning rate effects, and adapter placement differences.

Even with constrained compute, this search provided enough coverage to identify trends without exceeding GPU time limits.

Each configuration was evaluated on: Validation loss, BERTScore, BLEU, ROUGE-L, and Output length ratio.

Hyperparameter Trial Summary

The first configuration used the following parameters: r=32, alpha=32, dropout=0.05, lr=5e-5

This configuration demonstrated the strongest semantic alignment and stylistic control while minimizing hallucinations. It became the final model.

The second configuration used the following parameters: r=64, alpha=128, dropout=0.15, lr=5e-5

The third configuration used the following parameters: r=16, alpha=16, dropout=0.0, lr=2e-5

The first configuration had the best trade-off across in metrics. It had the lowest validation loss and best Bert Score with an F1 of 0.842. It also performed well in terms of conciseness. In terms of computational efficiency, it was in the middle in terms of speed, it took 3 hours of run time vs the second which took 5 hours.

Evaluation and Metrics

To evaluate my model, I conducted two forms of evaluation. General reasoning benchmarks were measured using the LM Evaluation Harness (PIQA, HellaSwag, Winogrande). Task-specific rewriting metrics were evaluated using ROUGE, BLEU, and BERTscore on a subsampled validation set to reduce time.

LM Evaluation Harness

Because Qwen2.5-1.5B is an instruction-tuned model, it's important to verify that LoRA fine-tuning does not catastrophically degrade the model’s reasoning or comprehension abilities. This model was compared against the baseline, as well as another trained model:zephyr-7b-beta

The LoRA training caused only a minor decline (1–2%) in general reasoning benchmarks which can be expected for single-task. The finetuned model maintained strong commonsense ability, confirming that the LoRA configuration did not degrade overall model quality.

Task-Specific Resume Evaluation

General tasks do not measure the model’s rewriting accuracy, so I additionally created a task-specific evaluation suite aligned with rewriting objectives.

Task-Specific Metrics

ROUGE-L

BLEU

BERTScore

Output Length Stability

Qualitative Observations

The finetuned model noticeably had improved action-verb selection. It also reduced the number of filler words it used, while using lanugage derived from the job description. The tone also remained professional for the produced responses.

Summary of Benchmark Findings

Across all evaluation methods, the general reasoning ability was relatively unchanged. The task specific metrics showed strong improvement, while the qualitative observation showed that the language produced was improved!

Usage/Intended Usage

This model is intended to assist job applicants by generating polished resume bullet points tailored to specific job descriptions. It rewrites user provided bullet points so that they better align with job requirements. The primary users of this model are job applicants. Job seekers, especially early career applicants, would find this especially useful. This model is not intended to fabricate experience, misrepresent qualifications, or generate fraudulent claims. Users should carefully review all output for factual accuracy.

from transformers import AutoTokenizer, AutoModelForCausalLM

model = "vsr9awc/resume-optimizer"

tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModelForCausalLM.from_pretrained(model, device_map="auto")

Prompt Format

The model expects the instruction to contain the original resume bullet point, the job description, and the prompt to rewrite the bullet.

Rewrite the resume entry so it aligns with the given job description.

Resume Entry:
<user resume text>

Job Description:
<job description text>

RESPONSE:

Expected Output

When used as intended, the model should output a single rewritten bullet point, using a professional tone. Another constraint is that it will incorporate action verb phrasing with the inclusion of key skills/responsibilities present in the job description. There also should not be the presence of any hallucainations, information should be derived from the input.

Input Resume Entry:
"Assisted with weekly financial reporting for department operations."

Job Description Keyword:
"Seeking candidate with experience in financial analysis, variance tracking, and process optimization."

Model Output:
"Produced weekly financial reports and conducted variance analysis to support operational planning and improve budgeting accuracy."

Limitations

The model performs strongly on task alignment and clarity but still has several limitations. The first limitation is the risk of hallucinations. The model may occasionally generate achievements not present in the original entry. Users need to confirm responses prior to use. Tone bias is another limitation. The model may lean towards neutral tone and may homogenize writing style across industries, which might not be beneficial for some users. In highly specialized fields, the model may oversimplify achievements or terminology, minimizing previous experience. As previously mentioned, the training dataset overrepresents business/tech roles and underrepresents trades and creative roles. Therefore, users in these industries may need to be extra cautious when using this model. The model does not verify accuracy or truth prior to providing a response, and therefore can't validate claims. Especially considering it will take a user's input and provide an output without verifying truth. The model can only be used in English, since all of the training data was in English.