gopro-AI-gemma3-4b

GoPro.gg — fine-tuned Gemma 3 4B - Final Year Project

Merged bfloat16 coaching model for GoPro.gg (Valorant, LoL, EA FC, eFootball).

Inference Endpoints (vLLM)

Uses Transformers v4-style rope_theta / rope_scaling (not v5 rope_parameters) for vLLM 0.18 compatibility.

Tokenizer/processor metadata synced from unsloth/gemma-3-4b-it (revision 5551a22).

Tesla T4 16GB — not supported for Gemma 3 + vLLM

Gemma 3 on vLLM is stuck between two hard limits on T4:

| Option | Result on T4 |

|--------|----------------|

| --dtype bfloat16 | T4 lacks bfloat16 (compute 7.5, needs 8.0+) |

| --dtype half / float16 | vLLM rejects it: "gemma3 does not support float16 — numerical instability" |

| --dtype float32 | ~17 GB weights alone → OOM on 16 GB VRAM |

This is a known vLLM limitation, not a config bug in this repo. See vLLM forum.

Use L4 24GB, A10G 24GB, or newer (Ampere+, compute ≥ 8.0).

L4 / A10G 24GB+ (recommended)

--dtype bfloat16 --max-model-len 4096 --max-num-seqs 1 --gpu-memory-utilization 0.92 --trust-remote-code --language-model-only

T4 workarounds (if L4 unavailable)

1. Different GPU region on HF Inference Endpoints — try AWS/GCP regions with L4 or A10G.

2. GPTQ 4-bit — quantize the model (~2 GB VRAM) so T4 can serve it; AWQ is known broken for Gemma 3 (use GPTQ).

3. Keep Gemini/Groq as primary in dashboards-pfe/ai-service — HF endpoint is optional until Ampere+ GPU is available.