gemma-2b-trader-8bit

Gemma-2-2B Trading Summarizer (8-bit Quantized)

Model Description

This is an 8-bit quantized version of the fine-tuned Gemma-2-2B trading journal summarizer. It offers ~50% reduction in model size and memory usage with minimal quality loss.

Quantization Details

• Method: bitsandbytes 8-bit quantization
• Original Precision: fp16
• Quantized Precision: int8
• Size Reduction: ~50%
• Quality Impact: Typically <2% degradation

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "./gemma-2b-trader-8bit",
    load_in_8bit=True,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("./gemma-2b-trader-8bit")

# Same usage as fp16 version

When to Use This Version

• Limited GPU memory (<8GB VRAM)
• Faster loading times needed
• Deployment on edge devices
• When inference speed is more important than marginal quality

When to Use FP16 Version

• Maximum quality required
• Sufficient GPU memory available
• Fine-tuning or further training needed