Downloads · 30 days
28
6% of all-time downloads
EganAI/Qwen3-4B-Instruct-2507-20250808-233922-0
Qwen3-4B-Instruct-2507-20250808-233922-0 is a text generation model from EganAI. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is a Qwen3-4B based model created through layer-wise merging of multiple fine-tuned variants to optimize performance on GPQA Diamond.
Downloads · 30 days
28
6% of all-time downloads
All-time downloads
499
Public
Parameters
4B
8.1 GB on disk
Likes
7
Public
Click a slice to open those files.
.safetensors8 GB · 100%
From the Hugging Face model README
This is a Qwen3-4B based model created through layer-wise merging of multiple fine-tuned variants to optimize performance on GPQA Diamond.
| Benchmark | Score | Description |
|---|---|---|
| MMLU (5-shot) | 0.7251 (72.51%) | Massive Multitask Language Understanding |
| GPQA Diamond (0-shot) | 0.4545 (45.45%) | Graduate-level Physics Q&A |


The following models were used in the layer-wise merge:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load the model
model = AutoModelForCausalLM.from_pretrained(
"EganAI/Qwen3-4B-Instruct-2507-20250808-233922-0",
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("EganAI/Qwen3-4B-Instruct-2507-20250808-233922-0")
# Example: MMLU-style question
prompt = '''Question: The study of the distribution and determinants of health and disease in populations is:
A) Epidemiology
B) Ecology
C) Etiology
D) Endocrinology
Answer:'''
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_length=150,
temperature=0.7,
do_sample=True
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
from vllm import LLM, SamplingParams
llm = LLM(model="EganAI/Qwen3-4B-Instruct-2507-20250808-233922-1")
sampling_params = SamplingParams(temperature=0.7, top_p=0.95, max_tokens=256)
prompts = ["Question: Explain quantum entanglement in simple terms."]
outputs = llm.generate(prompts, sampling_params)
This model uses a layer-wise merging approach where each transformer layer is selected from different source models based on optimization criteria. This technique allows combining strengths from multiple fine-tuned models.
If you use this model, please cite the original source models and egan.ai
This model is provided for research purposes. Always validate performance on your specific use case before deployment.