gemma-2b-trader-fp16

Gemma-2-2B Fine-tuned for Trading Journal Summarization

Model Description

This model is a fine-tuned version of google/gemma-2-2b, specifically trained to generate structured bullet-point summaries of trading journal entries and financial documents.

Training Details

• Base Model: google/gemma-2-2b
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• LoRA Rank: 16
• LoRA Alpha: 32
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Training Epochs: 3
• Learning Rate: 2e-05
• Batch Size: 2 (effective: 16)
• Training Precision: fp16

Performance Metrics

• Training Loss: 0.4682
• Validation Loss: 0.4266
• Perplexity: 1.53

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "./gemma-2b-trader-fp16",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("./gemma-2b-trader-fp16")

prompt = '''### Instruction:
Summarize the following trading journal entry using structured bullet points and precise trading terminology.

### Input:
[Your trading entry here]

### Summary:
'''

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Data

Fine-tuned on the EDGAR-CORPUS-Financial-Summarization dataset for financial text domain adaptation.

Limitations

• Optimized for trading/financial context
• Best results with English text
• May require domain-specific prompting for optimal output

Citation

If you use this model, please cite the original Gemma model and this fine-tuned version.