fluxem-qwen3-4b

FluxEM-Qwen3-4B Tool-Calling

Summary

FluxEM-Qwen3-4B is a tool-orchestration wrapper around Qwen3-4B-Instruct that routes queries to deterministic FluxEM tools across 11 algebraic domains (arithmetic, physics, chemistry, biology, math, music, geometry, graphs, sets, logic, number theory). It is not a fine-tuned model; it is a tool-calling pipeline and benchmark suite for reproducible evaluation.

Research framing

Inspired by NVIDIA ToolOrchestra (https://research.nvidia.com/labs/lpr/ToolOrchestra/) and prior embedding work, this project asks what counts as a "tool" when the knowledge is algebraic and not naturally tokenized. FluxEM encoders are algebraically structured and deterministic; treating them as tools can yield reliable computation with less training compute than trying to learn those operators from text.

How it works

• Domain routing and tool selection (LLM-first, with pattern fallback)
• Query extraction into tool inputs
• Tool execution in FluxEM
• Response returns tool output directly (no LLM regeneration)

Quickstart

1. Install dependencies (choose one):
   • Apple Silicon / MLX: pip install fluxem[full-mlx]
   • Transformers fallback: pip install fluxem[huggingface] torch
2. Download a local Qwen3-4B model:
   • MLX: https://huggingface.co/Qwen/Qwen3-4B (MLX quantized)
3. Enable MLX (only if using MLX): export FLUXEM_ENABLE_MLX=1
4. Run:

from experiments.qwen3_toolcalling.qwen3_wrapper import create_wrapper

wrapper = create_wrapper(
    model_path="~/.mlx/models/Qwen/Qwen3-4B-Instruct-MLX",
    tool_selection="llm",
    llm_query_extraction=True,
)
wrapper.load_model()

result = wrapper.generate_with_tools(
    "A box has 12 apples. You eat 5. How many are left?"
)
print(result)

Benchmarks (internal)

We include two internal synthetic datasets to validate the tool-routing pipeline, not general language understanding:

• Hard: 58 prompts across 11 domains (experiments/qwen3_toolcalling/benchmark_data_hard.py)
• Very hard: 33 prompts across 11 domains (experiments/qwen3_toolcalling/benchmark_data_very_hard.py)

Example questions by domain (from benchmark_data_very_hard.py):

• arithmetic: "Compute ((987654321 - 123456789) * (54321 + 98765) + 7^9) / 13"
• physics: "Convert 88 ft/s to m/s"
• chemistry: "What is the molecular weight of Ca3(PO4)2?"
• biology: "What's the GC content of ATGCGTACGATCGGATCCGATCGTAGCTAGC?"
• math: "Calculate the determinant of [[4, 2, 0, 1], [3, 5, 1, 2], [2, 0, 3, 4], [1, 2, 4, 0]]"
• music: "Transpose [1, 4, 8] by -7 semitones"
• geometry: "Rotate [7, -2] by 225 degrees"
• graphs: "What's the shortest path from node 0 to node 6 in Graph(nodes={0,1,2,3,4,5,6}, edges=[(0,1),(1,2),(2,6),(0,3),(3,4),(4,5),(5,6),(1,4)])?"
• sets: "What is the union of {-10, -5, 0, 5, 10} and {-5, -3, 2, 5, 7}?"
• logic: "Is 'p or not p' a tautology?"
• number_theory: "What is 13^123 mod 997?"

Results (internal)

Very hard benchmark:

• Overall accuracy: tool-calling 100.0%, baseline 39.4%
• Average improvement: 2.0x
• Raw results: experiments/qwen3_toolcalling/results/benchmark_results_20260107_004119.json

Hard benchmark:

• Overall accuracy: tool-calling 100.0%, baseline 43.1%
• Average improvement: 2.3x
• Raw results: experiments/qwen3_toolcalling/results/benchmark_results_20260106_232757.json

Related work

Usage

See experiments/qwen3_toolcalling/README.md for setup and full benchmark instructions.

Limitations

• Benchmarks are internal and synthetic; they validate routing and tool execution, not general knowledge.
• Tool coverage is limited to FluxEM's 11 domains.
• Natural language query extraction can still fail for unusual phrasing.
• No multi-turn tool calling yet.

License

MIT (wrapper code). The base model is subject to its original license terms.