portal-mistral-7b

PorTAL refit for Mistral 7B v0.3

This is a native PorTAL artifact refitted onto mistralai/Mistral-7B-v0.3. Its 14-task latent table and canonical LoRA-generating core were learned jointly from Qwen3-1.7B and Qwen3-4B and frozen during refitting. Only a fresh Mistral alignment was trained.

The artifact records the 64 exact q/v projection targets across the 32 decoder layers and generates rank-8 LoRA factors only for those paths.

Evaluation

One seed was evaluated on the pinned 14-task validation suite using continuation log-probability divided by character length (acc_norm). Gold continuation token-mean NLL was tracked separately.

The selected checkpoint was epoch 2. The independently minimum-NLL checkpoint was epoch 1 with macro gold NLL 2.4824.

These are research benchmark results for this exact artifact and evaluation recipe, not a general performance guarantee. The complete epoch history and diagnostics are in metrics.json.

Refit recipe

• Base: mistralai/Mistral-7B-v0.3 at caa1feb0e54d415e2df31207e5f4e273e33509b1
• Frozen source carrier: RampPublic/[email protected]
• Dataset: RampPublic/portallib-tasks at ffc3c0e44f529bf64a5ae62ed5db090952db97ea
• Refit data: deterministic seeded sample of up to 1,000 examples per task from the full pool
• Evaluation data: leading 1,000 validation examples per task, or all rows for shorter tasks
• Optimization: 2 epochs, batch size 4, alignment LR 2e-5, linear decay with 10% warmup, norm-equalized task gradients, character-normalized choice-loss weight 3, seed 0
• Trainable parameters: target-base alignment only; task latents and canonical core remained frozen
• Checkpoint selection: maximum macro validation acc_norm, with lower gold NLL as the tie-breaker
• Architecture: exact q/v targets, rank 8, alpha 16, task latent 256, layer embedding 32, hidden 512, canonical width 1024

The checked-in recipe is examples/configs/refits/mistral-7b.toml.

Usage

Install portallib>=0.2.1, then load the immutable artifact revision:

from portallib import PortalModel

portal = PortalModel.from_pretrained(
    "RampPublic/portal-mistral-7b",
    revision="v0.2.1",
)
portal.export_peft("rte", "./portal-rte-mistral-7b")

The exported directory is an ordinary PEFT LoRA adapter for the exact base revision recorded above. The artifact is Apache-2.0; the benchmark dataset contains components under multiple upstream licenses documented on its dataset card.