LLaVA-llama32-3b-Tfull-s42-s2a

LLaVA-llama32-3b-Tfull-s42-s2a — T-cap

T-cap arm of a controlled VL-vs-text training grid built on LLaVA-NeXT. Arm definition: stripped S2a. Text twin of VL-cap.

Backbone llama32-3b (meta-llama/Llama-3.2-3B-Instruct), seed 42, output of S2a (caption stage).

This is a text twin: the exact same examples in the exact same order, with <image>/<video> tokens stripped and the image field removed, so only the language model is tuned. Vision-tower and projector weights are carried in the checkpoint at their initial values and are not used. It exists to isolate what VL training changes relative to consuming the same text.

What the grid controls

Every arm sees the same mixtures, the same example order for a given seed, the same optimizer settings and the same restarted LR schedule. Arms differ only in data, order, and which modules are trainable — so differences between arms are attributable, not confounded.

Because Tfull runs its stages sequentially with the seed inherited, the S2a checkpoint is the cap-only arm at that seed. Both stages are published separately for exactly that reason.

Training

Training loss 2.712 -> 1.345 (mean of last 50 logged steps: 1.340), 0 non-finite losses across 5859 logged steps. The run reached a full epoch; trainer_state.json is included in this repo with the complete per-step loss / grad-norm / LR history.

Usage

These weights use the LlavaLlamaForCausalLM class from the LLaVA-NeXT repo, which is not part of transformers. Load them with that repo checked out:

from llava.model.builder import load_pretrained_model

tokenizer, model, image_processor, ctx_len = load_pretrained_model(
    "BAJUKA/LLaVA-llama32-3b-Tfull-s42-s2a", None, "llava_llama"
)

Prompt template: llama_v3.

Caveats

• Research artifact from a controlled comparison, not a tuned or safety-aligned release. It inherits the licence and the limitations of meta-llama/Llama-3.2-3B-Instruct.
• The intermediate (s2a) checkpoint is a legitimate arm of the grid, not a "best" checkpoint — no early stopping or checkpoint selection was performed. Each stage simply runs one full epoch.
• Optimizer / DeepSpeed / RNG state is not published; these are inference weights.