granite-4.1-3b-hardened

Hirundo Granite-4.1-3B Hardened

This repository contains a behaviorally hardened derivative of ibm-granite/granite-4.1-3b. Hirundo modified the model to reduce prompt-injection susceptibility while preserving its general capabilities.

The checkpoint itself was modified at the model level. The intervention is not a runtime prompt, output filter, or external guardrail.

About Hirundo

Hirundo develops machine-unlearning methods for changing specific model behaviors. In its behavioral-unlearning workflow, a target behavior is identified, the model is modified, and the resulting checkpoint is evaluated for the targeted security behavior and general capabilities. This card reports the measured comparison for this checkpoint rather than a guarantee about every use case.

Learn more:

• Hirundo
• Hirundo AI Labs
• Hirundo documentation

What Hirundo changed

Prompt injection occurs when adversarial or untrusted instructions cause a model to disregard its intended task or reveal protected context. Hirundo applied behavioral unlearning to reduce this susceptibility in the base checkpoint.

Evaluation summary

The hardened checkpoint was compared with the unmodified base checkpoint under the reported evaluation setup, with thinking mode off. For the security evaluations below, lower scores indicate fewer successful attacks or leaks and therefore better performance. Relative reduction is (base − hardened) / base × 100 when the base rate is nonzero.

Headline result: PurpleLlama prompt injection

The PurpleLlama Cybersecurity Benchmarks provide the prompt-injection evaluation used here. Attack success rate (ASR) fell from 40.24% to 8.77%, a 78.22% relative reduction.

Garak security evaluation

The Garak scanner probes models for several attack and leakage behaviors. Attack-family rates and leakage-oriented rates below have different definitions and should be read separately.

Results by attack family

The unweighted mean relative reduction across these three attack families is 89.70%.

Leakage-oriented results

The first three metrics report leaks per try or attempt. The attack success rate is measured per attack or prompt, which may include multiple tries; these rates therefore have different denominators.

The unweighted mean relative reduction across these four leakage metrics is 94.00%.

General-capability evaluation

Six general-capability benchmarks were compared with the base checkpoint. The table gives only the hardened checkpoint's change versus base in percentage points (hardened − base); positive values indicate improvement. The measured security improvement was accompanied by small changes on most of these benchmarks, although GPQA, IFBENCH, LIVECODEBENCH, and SCICODE declined.

The unweighted mean change is −0.52 pp. The largest measured decrease is 1.11 pp on LIVECODEBENCH. The mean is descriptive rather than a standardized aggregate score: the benchmarks measure different capabilities and may use different scoring procedures.

Intended use

This checkpoint is intended for:

• Research into mitigation of prompt-injection susceptibility through model modification.
• Comparative security evaluation against the unmodified base model and other defenses.
• Development of applications that need a model-level hardening component.
• Further evaluation and task-specific fine-tuning.

Test the checkpoint in the application's actual prompts, tools, retrieval pipeline, and threat model. Benchmark results do not establish immunity to prompt injection or guarantee that the measured improvements transfer to every deployment setup.

Granite-4.1-3B is a text-generation model supporting conversational text inputs. Consult the upstream model card for architecture, modalities, languages, context length, generation settings, and base-model considerations.

Loading with Transformers

Use a recent Transformers version with Granite 4.1 support. The causal-language-model loader and tokenizer follow the upstream model's example:

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "hirundo-io/granite-4.1-3b-hardened"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [{"role": "user", "content": "Who are you?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

License

This checkpoint is released under the repository's Apache-2.0 license. See the upstream model card for the base model's license and terms.

Citation

@misc{hirundo_granite_41_3b_hardened,
  title = {Hirundo Granite-4.1-3B Hardened},
  author = {Hirundo},
  howpublished = {\url{https://huggingface.co/hirundo-io/granite-4.1-3b-hardened}}
}