Nanbeige4.2-3B-CE-v1.0-LoRA

---

base_model: Nanbeige/Nanbeige4.2-3B library_name: peft pipeline_tag: text-generation language:

• en tags:
• peft
• lora
• qlora
• computer-engineering
• systems
• linux
• operating-systems
• computer-architecture
• devops

---

Nanbeige4.2-3B-CE v1.0

Nanbeige4.2-3B-CE is a Computer Engineering and systems-focused LoRA/QLoRA adaptation of Nanbeige/Nanbeige4.2-3B.

The model is intended for technical question answering and reasoning across areas such as:

• operating systems
• Linux and infrastructure
• computer architecture
• memory systems
• concurrency and atomics
• networking and DNS
• containers and Docker
• embedded and systems engineering
• engineering-oriented software development

This repository contains the PEFT adapter, not a standalone copy of the Nanbeige4.2-3B base model.

Release

Public release: v1.0 Internal selected checkpoint: v1.0.5 Base model: Nanbeige/Nanbeige4.2-3B Pinned base revision: 3384e426066d1a49c3aea90a7190b81260a6533f

The v1.0.5 checkpoint was retained after subsequent targeted SFT, completion-only correction training, checkpoint sweeps, and preference-optimization experiments failed to improve the overall frozen systems evaluation without introducing regressions.

Integrity

SHA-256 of the released adapter:

37b208cd124fe94c00fb713a2fd4938ac10a80b29dd36523be4bae8bbaf786ed  adapter_model.safetensors

SHA-256 of the adapter configuration:

432b8b35d9784439f99e931f13505bfecd2599f22a193f06f1cc2eb5ad0d973f  adapter_config.json

Training

The model was trained using parameter-efficient QLoRA/LoRA adaptation.

Key configuration:

• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Bias: none
• Task type: causal language modeling
• Quantized base during training: 4-bit NF4
• Double quantization: enabled
• Compute dtype: bfloat16
• Attention implementation: SDPA

LoRA target modules:

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

The final selected adapter contains approximately 24 million trainable LoRA parameters.

Usage

Install the required libraries:

pip install torch transformers peft accelerate bitsandbytes

Example:

import torch

from peft import PeftModel
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)

BASE_MODEL = "Nanbeige/Nanbeige4.2-3B"
BASE_REVISION = "3384e426066d1a49c3aea90a7190b81260a6533f"
ADAPTER = "Irfanuruchi/Nanbeige4.2-3B-CE-v1.0"

tokenizer = AutoTokenizer.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    trust_remote_code=True,
)

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    trust_remote_code=True,
    quantization_config=quantization_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

messages = [
    {
        "role": "user",
        "content": "Why can false sharing reduce multithreaded performance?"
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    output = model.generate(
        inputs,
        max_new_tokens=160,
        do_sample=False,
    )

answer = tokenizer.decode(
    output[0][inputs.shape[-1]:],
    skip_special_tokens=True,
)

print(answer)

For evaluation and this release, enable_thinking=False was used.

Validation

The release checkpoint was evaluated with a frozen 28-prompt systems-focused regression suite covering:

• x86-64 address translation
• TLB misses and page faults
• copy-on-write behavior
• C++ atomic memory ordering
• sequential consistency
• cache coherence and write policies
• false sharing
• Docker DNS behavior
• DNS UDP/TCP handling
• Linux load average
• worker-thread contention
• NUMA placement

The final checkpoint was selected because later correction runs did not improve the overall evaluation without regressions.

Known limitations

This model is not claimed to be factually perfect.

The frozen release evaluation identified several stubborn edge-case weaknesses.

C++ std::memory_order enumeration

The model may incorrectly omit memory_order_acquire and memory_order_release and may hallucinate a nonexistent memory_order_acq_release.

The standard values are:

memory_order_relaxed
memory_order_consume
memory_order_acquire
memory_order_release
memory_order_acq_rel
memory_order_seq_cst

memory_order_seq_cst precision

The model may describe seq_cst only as providing "sequential consistency" without stating the more precise guarantee that sequentially consistent atomic operations participate in a single total order subject to the language's ordering requirements.

Write-back cache wording

The model may incorrectly state that write-back reduces traffic by "avoiding every write-back operation."

More precisely, write-back can keep data dirty locally and coalesce multiple stores before lower-level propagation. Dirty data must still be propagated when required.

Minor versus major page faults

On some phrasings involving allocation of a new physical page, the model may incorrectly describe the major/minor distinction in terms of privileged operating-system work.

The relevant distinction is whether resolving the fault requires backing-store I/O. Allocating a fresh physical frame does not by itself make a fault major.

Intended use

This model is intended for:

• learning and experimentation
• Computer Engineering assistance
• systems-oriented technical discussion
• local inference
• research into small specialized language models

It should not be treated as an authoritative source for safety-critical, security-critical, or production-critical engineering decisions without independent verification.

Base model

Built on:

Nanbeige/Nanbeige4.2-3B

The base model uses custom Transformers code, so loading it requires trust_remote_code=True.

Reproducibility environment

The final release was validated with:

Python        3.12.3
PyTorch       2.13.0+cu130
CUDA          13.0
Transformers  4.45.1
PEFT          0.20.0
TRL           0.11.4
Accelerate    1.14.0
Datasets      5.0.1
bitsandbytes  0.50.2

Hardware used for final local validation:

NVIDIA GeForce RTX 5060 8 GB

Acknowledgements

This project builds on Nanbeige4.2-3B by the Nanbeige team.

The adapter should be used in accordance with the terms applicable to the base model and relevant software and data dependencies.