portal-gemma-4-e2b

PorTAL refit for Gemma 4 E2B

This is a native PorTAL artifact refitted onto google/gemma-4-E2B. Its 14-task latent table and canonical LoRA-generating core were learned jointly from Qwen3-1.7B and Qwen3-4B and frozen during refitting. Only a fresh Gemma 4 alignment was trained.

Gemma 4 alternates sliding- and full-attention projection widths and reuses KV states in later layers. The artifact records the exact sparse q/v projection targets and generates rank-8 LoRA factors only for projections that exist in the text decoder.

Evaluation

One seed was evaluated on the pinned 14-task validation suite using continuation log-probability divided by character length (acc_norm). Gold continuation token-mean NLL was tracked separately for checkpoint selection.

The selected checkpoint was epoch 4. Across the same tasks, independently trained rank-16 full-text-decoder LoRA baselines averaged 0.7084. Relative to the frozen base, PorTAL retained 120.6% of their aggregate lift.

These are research benchmark results for this exact artifact and evaluation recipe, not a general performance guarantee. Full selected-epoch results are in metrics.json.

Refit recipe

• Base: google/gemma-4-E2B at d29ff6b45f081a49ee2733a859c9c9c2d95d1a6f
• Frozen source carrier: RampPublic/[email protected]
• Dataset: RampPublic/portallib-tasks at ffc3c0e44f529bf64a5ae62ed5db090952db97ea
• Refit data: deterministic nested sample of up to 1,000 examples per task
• Optimization: 5 epochs, batch size 4, alignment LR 1e-3, linear decay with 10% warmup, seed 0
• Trainable parameters: target-base alignment only; task latents and canonical core remained frozen
• Checkpoint selection: maximum macro validation acc_norm, with lower gold NLL as the tie-breaker
• Architecture: q/v targets, rank 8, alpha 16, task latent 256, layer embedding 32, hidden 512, canonical width 1024

Usage

Install portallib>=0.2.0, then load the immutable artifact revision:

from portallib import PortalModel

portal = PortalModel.from_pretrained(
    "RampPublic/portal-gemma-4-e2b",
    revision="v0.2.0",
)
portal.export_peft("rte", "./portal-rte-gemma-4-e2b")

See the release recipe for the full task list, evaluation definition, target topology, and refitting procedure. The artifact is Apache-2.0; the benchmark dataset contains components under multiple upstream licenses documented on its dataset card.