Param-1-7B-GodMode-4bit

Param-1-7B God Mode Hybrid (4-bit / 16-bit)

A highly optimized, hybrid-quantized version of the bharatgenai/Param-1-7B model. This model utilizes a custom 4-bit/16-bit "God Mode" configuration to drastically reduce VRAM usage while preserving high mathematical accuracy by selectively keeping critical attention and routing mechanisms in 16-bit precision.

Model Details

Model Description

This model was created to enable consumer-hardware inference for the Param-1-7B architecture without suffering the severe degradation often associated with pure 4-bit quantization. It uses bitsandbytes NF4 quantization with a highly sensitive skip-list, keeping the vocabulary projector, context engine, attention aggregators, and Mixture-of-Experts (MoE) routers in bfloat16.

• Developed by: Arun Venkat Jonna
• Affiliation: Sustainable, Advanced, Robust Computing Systems (SARCS) Lab
• Model type: Causal Language Model (Quantized / Mixture of Experts)
• Language(s) (NLP): English
• License: Inherited from bharatgenai/Param-1-7B
• Quantized from base model: bharatgenai/Param-1-7B

Model Sources

• Base Model Repository: bharatgenai/Param-1-7B

Uses

Direct Use

This model is intended for developers, researchers, and hobbyists who want to run the Param-1-7B model on GPUs with extremely limited VRAM (5 GB or more). Because the quantization configuration is baked directly into the config.json, it can be loaded natively with the transformers library without explicitly defining a BitsAndBytesConfig.

How to Get Started with the Model

Use the code below to get started with the model. Note that you do not need to define the skip-list; it is handled automatically.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "arunvenkat17/Param-1-7B-GodMode-4bit"

# Load Tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Load Optimized Model
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True,
    attn_implementation="sdpa"
)

print("Model loaded successfully!")
print(f"VRAM Used: {model.get_memory_footprint() / (1024 ** 3):.2f} GB")