Llama-3.2-1B-rtx4090-slim

---
library_name: pytorch
tags:
  - llama
  - causal-lm
  - text-generation
  - pruning
  - knowledge-distillation
  - speedup
license: apache-2.0
dataset: slimpajama-test
pipeline_tag: text-generation
---

hawada/Llama-3.2-1B-rtx4090-slim

This repository contains two variants:

• Gated student (with learned pruning gates) – requires custom code.
• Slim student (post-prune/export) – loads with standard HF APIs plus this repo’s custom code.

Inference (LLaMA slim)

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained('hawada/Llama-3.2-1B-rtx4090-slim')
mdl = AutoModelForCausalLM.from_pretrained('hawada/Llama-3.2-1B-rtx4090-slim', torch_dtype='auto').eval()
x = tok('Hello', return_tensors='pt')
print(tok.decode(mdl.generate(**x, max_new_tokens=16)[0]))

Notes

• The gated repo includes lightweight custom code (adapters/…, core/…) needed to attach/load gates.
• The slim model is exported for efficient inference.

Training metadata

{
  "base_id": "meta-llama/Llama-3.2-1B",
  "variant": "slim-export",
  "repo_slim": "hawada/Llama-3.2-1B-rtx4090-slim"
}