Qwen-3-8B-RHEA-property-predictor

Model Overview

This repository contains the weights for the Qwen-3-8B-RHEA-property-predictor, fine-tuned for refractory high entropy alloys property prediction and phase classification tasks.

Training Details

Prompt Template

Training prompts follow the template:

property prediction task

“You are a materials science expert. Predict the {property name} for the following refractory high entropy alloy.”

property name includs density, hardness, compressive yield strength at room temperature, compressive strain at room temperature, compressive yield strength at 1073K, and compressive yield strength at 1273K

phase classification task

“You are a materials science expert. Determine whether the given refractory high entropy alloy has {single solution phase/intermetallic phase}.”

Each query was formatted as:

“{composition} alloy prepared by {process description text}”

Hyperparameters & Settings

• Task: Binary classification / Regression
• GPU: 4 × NVIDIA GeForce RTX 3090
• Seed: 42
• Final Epoch: 4
• per-device batch size: 2
• gradient accumulation: 8
• Training Objective: Full fine-tuning with CrossEntropyLoss
• Sequence Length: 1024 tokens
• Dataset:
  • Train: train_data-all.jsonl
  • Validation: val_data-all.jsonl
• Dataset Source: The datasets are available at (https://huggingface.co/datasets/tianchuang/RHEA-mechanical-property)

---

Validation Metrics

Metrics are from model-calling evaluation.

Property prediction (val)

Phase classification (val)

---

How to Use

You can load and run inference with this model using the transformers library. The model uses the ChatML prompt format.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig

# 1. Load model and tokenizer
model_id = "tianchuang/Qwen-3-8B-RHEA-property-predictor"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# 2. Prepare your input data
instruction = "Is the material BaTiO3 likely synthesizable? Answer with P (positive) or N (negative)."
input_text = ""  # Leave empty if no additional context is needed

# 3. Format the input using the ChatML template
if input_text:
    prompt = f"<|im_start|>system\nYou are a materials science expert.<|im_end|>\n<|im_start|>user\n{instruction}\n{input_text}<|im_end|>\n<|im_start|>assistant\n"
else:
    prompt = f"<|im_start|>system\nYou are a materials science expert.<|im_end|>\n<|im_start|>user\n{instruction}<|im_end|>\n<|im_start|>assistant\n"

# 4. Tokenize and generate response
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

generation_config = GenerationConfig(
    max_new_tokens=64,
    do_sample=True,
    temperature=0.6, 
    top_p=0.9, 
    top_k=50,
    pad_token_id=tokenizer.eos_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

outputs = model.generate(**inputs, generation_config=generation_config)

# 5. Decode and parse the prediction
full_response = tokenizer.decode(outputs[0], skip_special_tokens=False)
assistant_response = full_response.split("<|im_start|>assistant")[-1]
clean_response = assistant_response.replace("<|im_end|>", "").strip()

print(f"Prediction: {clean_response}")