IndustryLLM

IndustryLLM: Failure-Driven LLM for Industrial Procurement

<p align="center"> <a href="https://huggingface.co/alibaba-multimodal-industrial-ai/IndustryLLM/resolve/main/IndustryLLM_paper.pdf"><b>[Technical Report (PDF)]</b></a> | <a href="https://huggingface.co/alibaba-multimodal-industrial-ai"><b>[Alibaba Multimodal & Industrial AI Team]</b></a> </p>

IndustryLLM is an open-weight industrial language model trained from Qwen/Qwen3.5-35B-A3B-Base, developed by the Multimodal and Industrial AI Team at Alibaba. It features 35B total parameters with approximately 3B activated per token (8 routed + 1 shared expert out of 256), combining deep industrial engineering knowledge with ultra-efficient inference.

---

Key Features

• Authoritative Engineering & Standards Grounding: Continually pre-trained on a curated ≈100B-token industrial corpus, injecting 5B tokens of national standards (e.g., GB/T) and technical specifications, 10B tokens of authentic enterprise B2B transaction and inquiry records, and 60B tokens of general replay.
• Failure-Driven Data Reconstruction: Overcomes register mismatch and factual brittleness through multi-register rewriting across 10 genres and 8 writing styles, confidence-routed minimal factual editing, and error-targeted QA synthesis.
• Robust Procurement Query Structuring: Bridges colloquial buyer jargon, phonetic typos (e.g., resolving 42络钼 → 42CrMo), and truncated standard codes (16674 → GB/T 16674), while intercepting contradictory geometric specifications before catalog dispatch.
• Reasoning-Enabled Mode (Think): Activates deep multi-step engineering reasoning for complex analytical and constraint-satisfaction tasks.
• Direct-Response Mode (No-Think): Strips intermediate CoT for latency-critical production environments (under 2s SLA).

---

Model Details

• Architecture: Hybrid-attention Mixture-of-Experts (Qwen3_5MoeForConditionalGeneration)
• Total / Activated Parameters: 35.95B total parameters / $\approx$3B activated per token
• Weight Format: BF16 safetensors (16 shards)
• Context Window: 262,144 tokens architectural limit (fine-tuned on packed 12,288 sequences)
• Training Adaptation: Language-model parameters adapted via CPT and full-parameter SFT with task-filtered teacher targets; vision tower and multimodal projection modules remain strictly frozen to isolate text-based engineering reasoning.
• Chat Template: Native Qwen3.5 thinking / non-thinking chat templates.

---

Quickstart & Usage

We recommend using transformers>=5.2.0 with native Qwen3.5 MoE support.

pip install "transformers>=5.2.0" accelerate

1. Direct-Response Mode (No-Think, Recommended for Low-Latency Query Structuring)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "alibaba-multimodal-industrial-ai/IndustryLLM"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Example: Resolving colloquial buyer inquiry with conflicting specs
prompt = "帮我找 42络钼 螺栓 16674 M10*35 但要50长"

messages = [
    {"role": "system", "content": "你是一位专业的工业品采销专家。请将买家的采购意图进行参数结构化归一化，指出潜在的规格冲突或缺失项。"},
    {"role": "user", "content": prompt}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,  # Direct response for fast catalog integration (<2s)
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(**inputs, max_new_tokens=512, temperature=0.2)

completion_ids = output_ids[0, inputs["input_ids"].shape[-1] :]
print(tokenizer.decode(completion_ids, skip_special_tokens=True))

2. Reasoning-Enabled Mode (Think, for Deep Engineering Analysis)

# Pass enable_thinking=True to activate test-time reasoning traces
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,  # Activates chain-of-thought
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(**inputs, max_new_tokens=2048, temperature=0.6)

completion_ids = output_ids[0, inputs["input_ids"].shape[-1] :]
print(tokenizer.decode(completion_ids, skip_special_tokens=True))

---

System Boundary & Evidence-Gated Evaluation

As formalized in our technical report, category relevance is not product eligibility:

1. Demand Track: IndustryLLM is engineered to power upstream demand interpretation, canonical property extraction, unit normalization, and conflict identification.
2. Product Track: Final physical eligibility must be evaluated downstream via an evidence gate under three-valued logic (Satisfied, Unknown, Violated): unverified product claims or missing supplier evidence strictly evaluate to unknown rather than satisfied ("unknown is not satisfied").

---

Limitations & Responsible Use

• Text-Only Adaptation: Although the architecture inherits multimodal vision weights from the base model, the vision encoder remained frozen; the model is not evaluated for engineering blueprints, CAD files, or defect image classification.
• Safety-Critical Operations: Outputs should not be taken as certified engineering guarantees for high-risk equipment (e.g., pressure vessels, explosive gas seals) without independent expert validation and official manufacturer test certificates.
• Deployment Auditing: Deployers are responsible for domain-specific safety checks, sandboxing code/tool executions, and enforcing data privacy policies.

---

Citation

If you find IndustryLLM useful for your research or enterprise applications, please cite our technical report:

@article{industryllm2026,
  title   = {IndustryLLM: Failure-Driven LLM Training for Industrial Procurement},
  author  = {Multimodal and Industrial AI Team, Alibaba},
  journal = {Technical Report},
  year    = {2026},
  url     = {https://huggingface.co/alibaba-multimodal-industrial-ai/IndustryLLM/resolve/main/IndustryLLM_paper.pdf}
}

Please also cite the upstream Qwen3.5 foundation model:

@misc{qwen3_5,
  title  = {Qwen3.5: Towards Native Multimodal Agents},
  author = {Qwen Team},
  year   = {2026},
  month  = {February},
  url    = {[https://qwen.ai/blog?id=qwen3.5](https://qwen.ai/blog?id=qwen3.5)}
}