Human-Like-Qwen2.5-7B-Instruct

<div align="center"> <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/63da3d7ae697e5898cb86854/H-vpXOX6KZu01HnV87Jk5.jpeg" width="320" height="320" /> <h1>Enhancing Human-Like Responses in Large Language Models</h1> </div> <p align="center"> &nbsp&nbsp | 🤗 <a href="https://huggingface.co/collections/HumanLLMs/human-like-humanish-llms-6759fa68f22e11eb1a10967e">Models</a>&nbsp&nbsp | &nbsp&nbsp 📊 <a href="https://huggingface.co/datasets/HumanLLMs/Human-Like-DPO-Dataset">Dataset</a>&nbsp&nbsp | &nbsp&nbsp 📄<a href="https://arxiv.org/abs/2501.05032">Paper</a>&nbsp&nbsp | </p> <p align="center"> <b>📢 The paper associated with this model has been accepted to the <a href="https://personalizedllm.github.io/events/perfm-aaai26/">AAAI-26 Workshop on Personalization in the Era of Large Foundation Models (PerFM)</a>.</b> </p>

🚀 Human-Like-Qwen2.5-7B-Instruct

This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct, specifically optimized to generate more human-like and conversational responses.

The fine-tuning process employed both Low-Rank Adaptation (LoRA) and Direct Preference Optimization (DPO) to enhance natural language understanding, conversational coherence, and emotional intelligence in interactions.

The proccess of creating this models is detailed in the research paper “Enhancing Human-Like Responses in Large Language Models”.

🛠️ Training Configuration

• Base Model: Qwen2.5-7B-Instruct
• Framework: Axolotl v0.4.1
• Hardware: 2x NVIDIA A100 (80 GB) GPUs
• Training Time: ~2 hours 15 minutes
• Dataset: Synthetic dataset with ≈11,000 samples across 256 diverse topics

<details><summary>See axolotl config</summary>

axolotl version: 0.4.1

base_model: Qwen/Qwen2.5-7B-Instruct
model_type: AutoModalForCausalLM
tokenizer_type: AutoTokenizer

trust_remote_code: true

load_in_8bit: true
load_in_4bit: false
strict: false

chat_template: chatml
rl: dpo
datasets:
  - path: HumanLLMs/humanish-dpo-project
    type: chatml.prompt_pairs
    chat_template: chatml

dataset_prepared_path:
val_set_size: 0.05
output_dir: ./humanish-qwen2.5-7b-instruct

sequence_len: 8192
sample_packing: false
pad_to_sequence_len: true

adapter: lora
lora_model_dir:
lora_r: 8
lora_alpha: 4
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:

wandb_project: Humanish-DPO
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

hub_model_id: HumanLLMs/Humanish-Qwen2.5-7B-Instruct

gradient_accumulation_steps: 8
micro_batch_size: 2
num_epochs: 1
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0002

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
s2_attention:

warmup_steps: 10
evals_per_epoch: 2
eval_table_size:
eval_max_new_tokens: 128
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:

save_safetensors: true

</details><br>

💬 Prompt Template

You can use ChatML prompt template while using the model:

ChatML

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
{asistant}<|im_end|>

This prompt template is available as a chat template, which means you can format messages using the tokenizer.apply_chat_template() method:

messages = [
    {"role": "system", "content": "You are helpful AI asistant."},
    {"role": "user", "content": "Hello!"}
]
gen_input = tokenizer.apply_chat_template(message, return_tensors="pt")
model.generate(**gen_input)

🤖 Models

🔄 Quantizationed versions

GGUF @bartowski

• https://huggingface.co/bartowski/Human-Like-LLama3-8B-Instruct-GGUF

• https://huggingface.co/bartowski/Human-Like-Qwen2.5-7B-Instruct-GGUF

• https://huggingface.co/bartowski/Human-Like-Mistral-Nemo-Instruct-2407-GGUF

🎯 Benchmark Results

📊 Dataset

The dataset used for fine-tuning was generated using LLaMA 3 models. The dataset includes 10,884 samples across 256 distinct topics such as technology, daily life, science, history, and arts. Each sample consists of:

• Human-like responses: Natural, conversational answers mimicking human dialogue.
• Formal responses: Structured and precise answers with a more formal tone.

The dataset has been open-sourced and is available at:

• 👉 Human-Like-DPO-Dataset

More details on the dataset creation process can be found in the accompanying research paper.

📝 Citation

@misc{çalık2025enhancinghumanlikeresponseslarge,
      title={Enhancing Human-Like Responses in Large Language Models}, 
      author={Ethem Yağız Çalık and Talha Rüzgar Akkuş},
      year={2025},
      eprint={2501.05032},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.05032}, 
}