Downloads · 30 days
11
16% of all-time downloads
pepsisodayoda/gemma-4-sharded-E2B-it-Opus-4.6-Reasoning
gemma-4-sharded-E2B-it-Opus-4.6-Reasoning is a image-text-to-text model from pepsisodayoda. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: filipwx - License: apache-2.0 - Finetuned from model : unsloth/gemma-4-E2B-it
Downloads · 30 days
11
16% of all-time downloads
All-time downloads
70
Public
Parameters
5.1B
10.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors10.2 GB · 100%
From the Hugging Face model README
This gemma4 model was trained 2x faster with Unsloth and Huggingface's TRL library.
license: apache-2.0 tags:
This is a locally sharded and VRAM-optimized deployment version of the original filipwx/gemma-4-E2B-it-Opus-4.6-Reasoning repository.
The original repository hosts the entire model inside a single massive 10.2 GB model.safetensors file. On consumer hardware with lower VRAM pools (like a 4GB NVIDIA RTX 3050 Laptop GPU), loading a single shard causes two critical points of failure:
transformers defaults to using device_map="auto", which routinely chokes on a single 10GB shard, offloading critical token embeddings to a placeholder meta device and triggering: RuntimeError: Tensor.item() cannot be called on meta tensors.accelerate frequently miscalculates available space, outputs the warning does not fit any GPU's remaining memory, and leaves the model completely unutilized in host System RAM (0% VRAM usage).This repository breaks the weights down into optimized 2 GB safe serialization shards (model-00001-of-00005.safetensors, etc.).
When combined with a direct root device mapping (device_map={"": 0}), the shards completely bypass the broken automated memory-checking loops, allowing the 4-bit quantized layers to anchor directly into your dedicated NVIDIA graphics memory.
Use this exact configuration block to load the model seamlessly. This bypasses the deprecated torch_dtype warning, forces dtype=torch.bfloat16, and mandates direct GPU execution:
from transformers import AutoProcessor, AutoModelForImageTextToText, BitsAndBytesConfig
import torch
model_id = "pepsisodayoda/gemma-4-sharded-E2B-it-Opus-4.6-Reasoning"
print("Initializing 4-bit bitsandbytes VRAM routing...")
processor = AutoProcessor.from_pretrained(model_id, fix_mistral_regex=True)
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_quant_type="nf4"
)
# Hard override targeting the primary NVIDIA card directly ({"": 0})
# This forces the weights out of system RAM and onto the graphics card hardware
model = AutoModelForImageTextToText.from_pretrained(
model_id,
quantization_config=quantization_config,
dtype=torch.bfloat16,
device_map={"": 0},
low_cpu_mem_usage=True
)
print("Gemma-4 Shards successfully forced onto Dedicated GPU VRAM!")