ReFIne-qwen3-4b

ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretability

This repository contains the ReFIne-qwen3-4b model, which is part of the work presented in the paper ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretability.

ReFIne is a new training framework designed to enhance Large Reasoning Models (LRMs) by promoting interpretability, faithfulness, and reliability in their reasoning processes. It integrates supervised fine-tuning with GRPO to encourage models to:

• Improve Interpretability: By producing structured, tag-based traces with high-level planning that are easier for humans to follow.
• Enhance Faithfulness: By explicitly disclosing the decisive information guiding each solution, with consistent cross-section references.
• Promote Reliability: By providing self-assessments of both the derivation's soundness and the confidence of the final answer.

This framework has been applied to Qwen3 models at multiple scales and evaluated across mathematical benchmarks, demonstrating significant improvements in these trustworthiness dimensions.

<p align="center"> <img src="https://github.com/Trustworthy-ML-Lab/Training_Trustworthy_LRM_with_Refine/raw/main/fig/overview.png" width="80%" alt="Overview of the ReFIne framework" /> <br> <em>Overview of ReFIne framework, which enhances LRMs in terms of reliability, faithfulness, and interpretability.</em> </p>

Code and More Information

For detailed installation instructions, training procedures, evaluation scripts, and more information, please refer to the official GitHub repository: https://github.com/Trustworthy-ML-Lab/Training_Trustworthy_LRM_with_Refine

🧠 Try ReFIne model

You can try ReFIne-qwen3-4b directly with the following example.

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "cesun/ReFIne-qwen3-4b"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", trust_remote_code=True)

messages = [
    {
        "role": "user",
        "content": (
            "Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning "
            "and bakes muffins for her friends every day with four. She sells the remainder "
            "at the farmers' market daily for $2 per fresh duck egg. "
            "How much in dollars does she make every day at the farmers' market?"
        )
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=2048)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

💡 Example Output

<understanding>
The problem is asking about how much money Janet makes each day at the farmers' market from selling duck eggs. It provides information about the number of eggs Janet's ducks lay daily, how many eggs Janet uses for breakfast, how many she uses for baking muffins, and the price per egg when she sells the remaining eggs. The task is to determine the daily income from the sales at the farmers' market, based on the number of eggs sold and the price per egg.
</understanding>

<facts>
- Janet’s ducks lay 16 eggs per day.
- Janet eats 3 eggs for breakfast every morning.
- Janet bakes muffins for her friends every day using 4 eggs.
- Janet sells the remainder of the eggs at the farmers' market daily.
- Janet sells each fresh duck egg for $2.
</facts>

<plan>
1. Determine the total number of eggs laid by Janet's ducks each day, which is given as 16.
2. Calculate the total number of eggs used by Janet for personal consumption, which includes 3 eggs for breakfast and 4 eggs for baking muffins, totaling 7 eggs.
3. Subtract the number of eggs used for personal consumption from the total number of eggs laid to find out how many eggs are left for sale at the farmers' market.
4. Multiply the number of eggs available for sale by the price per egg, which is $2, to calculate the daily income from the farmers' market sales.
</plan>

<think>
Okay, let's tackle this problem step by step. First, I need to make sure I understand what's being asked. The question is about how much money Janet makes each day at the farmers' market by selling her ducks' eggs. The key here is to figure out how many eggs she has left after she uses some for her own breakfast and baking, and then multiply that by the price per egg.

Looking at the <understanding> section, it confirms that the goal is to calculate her daily income from the market sales. The <facts> provide the numbers: 16 eggs laid daily, 3 eaten for breakfast, 4 used for muffins, and each egg sold for $2. The <plan> outlines the steps clearly, so I should follow that.

Starting with step 1 from the <plan>, the total eggs laid per day are 16. That's straightforward. Then step 2 says to calculate the eggs used for personal consumption. According to the <facts>, she eats 3 for breakfast and uses 4 for muffins. Adding those together, 3 + 4 equals 7 eggs. So, she uses 7 eggs each day.

Moving to step 3, I need to subtract the eggs used from the total laid. So 16 total eggs minus 7 used equals 9 eggs left for sale. Wait, let me check that again. 16 minus 7 is indeed 9. So, 9 eggs are available for the market.

Step 4 is to multiply the number of eggs sold by the price per egg. The price is $2 per egg. So 9 eggs times $2 gives $18. That seems right. Let me verify each step again to be sure. 

Total eggs: 16. Eggs used: 3 + 4 = 7. Remaining eggs: 16 - 7 = 9. Sales revenue: 9 * 2 = $18. Yep, that adds up. I don't see any mistakes in the <facts> or the <plan>. The calculations are simple arithmetic, so as long as each step is followed correctly, the answer should be correct. Therefore, Janet makes $18 every day at the farmers' market.
</think>

<final_answer>
To determine how much money Janet makes every day at the farmers' market, we follow the steps outlined in the <plan> and use the information provided in the <facts>.

---

### Step 1: Total Eggs Laid Per Day  
From the <facts>, we know that Janet’s ducks lay **16 eggs per day**.

---

### Step 2: Eggs Used for Personal Consumption  
Janet uses **3 eggs for breakfast** and **4 eggs for baking muffins**.  
$$
\text{Total eggs used} = 3 + 4 = 7
$$

---

### Step 3: Eggs Available for Sale  
To find the number of eggs left for sale, subtract the eggs used from the total laid:  
$$
\text{Eggs available for sale} = 16 - 7 = 9
$$

---

### Step 4: Revenue from Farmers' Market Sales  
Each egg is sold for **$2**, so the daily income from sales is:  
$$
\text{Daily income} = 9 \times 2 = 18
$$

---

### Final Answer  
$$
\boxed{18}
$$
</final_answer>

<self_assessment>
I followed the plan step by step, verifying each calculation and ensuring that the logic was consistent with the given facts. I double-checked the arithmetic and confirmed that the total eggs laid, the eggs used, and the remaining eggs for sale all align with the information provided. The final calculation of $18 seems accurate based on the given data. I do not see any contradictions, unverified assumptions, or logical gaps in my reasoning. Every step was clearly justified and internally consistent. I believe my reasoning is solid and well-supported.
Confidence: 10/10
</self_assessment>

Citation

If you find this work useful for your research, please cite the paper:

@article{ReFIne,
   title={ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretability},
   author={Sun, Chung-En and Yan, Ge and Kulkarni, Akshay and Weng, Tsui-Wei},
   journal={arXiv},
   year={2025}
}