hts-nemotron-8b-lora-v1

HTS-Nemotron-8B-LoRA-v1

A LoRA adapter for nvidia/Llama-3.1-Nemotron-Nano-8B-v1 fine-tuned for Harmonized Tariff Schedule (HTS) classification. Given a product description (and optionally materials, intended use, and country of origin), the model returns the full HTS hierarchy: chapter → heading → subheading → 10-digit HTS code, plus reasoning and the relevant "provides for" language.

This is v1 — an initial proof-of-concept trained on a single H100 in under 14 hours. Treat it as a research artifact; it has not been formally evaluated on a held-out test set yet.

Model Details

• Base model: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
• Adapter type: LoRA (PEFT)
• Parameters trained: ~83.9M (1.03% of 8.1B)
• License: Inherits the NVIDIA Open Model License from the base model.

LoRA configuration

Training

Note on overfitting

Eval loss bottomed out around step 7000 (epoch ~1.87) at 0.2596 and slowly crept up to 0.2729 by step 11000. This is mild overfitting in the back half of epoch 2 and through epoch 3. The adapter weights uploaded here are the final step-11214 weights. A future v1.1 release may use the step-7000 checkpoint or shorten training to ~2 epochs.

Prompt format

This adapter requires Nemotron's "thinking off" prompt format. The system message must start with detailed thinking off, and the assistant response begins with an empty <think> block:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

detailed thinking off

You are an expert in the U.S. Harmonized Tariff Schedule (HTS).
Given a product description, return the chapter, heading, subheading,
and 10-digit HTS code with reasoning.<|eot_id|><|start_header_id|>user<|end_header_id|>

Product: <product description>
Materials: <optional>
Use: <optional>
Country of origin: <optional><|eot_id|><|start_header_id|>assistant<|end_header_id|>

<think>
</think>

The model then generates:

Chapter NN: <description>
Heading NN.NN: <description>
Subheading NNNN.NN: <description>
HTS Code: NNNN.NN.NNNN

Reasoning: <free text>

Provides for: <relevant statutory language>

For items it cannot classify, it begins with Cannot classify: <reason>.

Quick start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "nvidia/Llama-3.1-Nemotron-Nano-8B-v1"
adapter_id = "<this repo>"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

messages = [
    {"role": "system", "content": "detailed thinking off\n\nYou are an expert in the U.S. Harmonized Tariff Schedule..."},
    {"role": "user", "content": "Product: insulated copper electrical wire, 12 AWG\nUse: residential wiring\nCountry of origin: Mexico"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
prompt += "<think>\n</think>\n\n"  # Nemotron thinking-off prefix

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False, repetition_penalty=1.05)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations

• Not yet formally evaluated. v1 is a research artifact. Held-out test set evaluation is pending.
• Spot-checks show the model gets the chapter and heading right on common electronics/wiring products, but can confuse adjacent subheadings.
• Trained only on English product descriptions.
• HTS classification is a legal determination. Do not use this model for binding customs filings without expert review.

Framework versions

• PEFT 0.13.2
• Transformers 4.46.3
• Accelerate 0.34.2
• Torch 2.4.1