Terraguide_q8.gguf

Model Card for TerraGuideAI

Model Description

TerraGuideAI is a fine-tuned version of Google's Gemma-2-2B model, specialized for providing FEMA-grounded, actionable guidance in disaster situations. It operates as an offline-first emergency response assistant powered by Offlyn.ai, with strong emphasis on branding compliance ("TerraGuideAI powered by Offlyn.ai").

• Developed by: Offlyn.ai Team
• Model type: Causal Language Model (Fine-tuned with LoRA)
• Language(s) (NLP): English
• License: Gemma License (same as base model)
• Finetuned from model: mlx-community/Gemma-2-2B

Model Sources

• Repository: TerraGuideAI GitHub
• Demo: Interactive demo available in project tests

Uses

Direct Use

TerraGuideAI is designed for direct use in emergency response scenarios, providing guidance on disaster preparedness, response, and recovery based on FEMA protocols.

Out-of-Scope Use

Not intended for medical advice, legal decisions, or non-emergency applications. Avoid use in high-stakes scenarios without human oversight.

Bias, Risks, and Limitations

• Biases: Trained on FEMA data which may reflect US-centric views.
• Risks: AI-generated advice should not replace professional emergency services.
• Limitations: Offline-first but requires sufficient device resources; may hallucinate in edge cases.

Recommendations

Always verify AI responses with official sources. Use in conjunction with emergency services.

How to Get Started with the Model

Use MLX-LM to load and generate:

from mlx_lm import load, generate

model, tokenizer = load("path/to/enhanced_terraguide_adapters")
response = generate(model, tokenizer, prompt="What should I do during an earthquake?", max_tokens=200)

Training Details

Training Data

• Source: Enhanced dataset from FEMA documents (e.g., guides on disaster preparedness).
• Composition: 40% identity/branding examples, 40% emergency responses, 20% general FEMA content.
• Size: 116 examples (98 train, 18 validation).
• Preprocessing: JSONL format with system/user/assistant prompts.

Training Procedure

Preprocessing

Data augmented with synthetic examples using MLX-LM for better branding compliance.

Training Hyperparameters

• Training regime: LoRA fine-tuning with MLX-LM
• Optimizer: AdamW
• Learning rate: 2e-5
• Epochs/Iterations: 1200 iterations
• Batch size: 2
• LoRA Parameters:
  • Rank: 16
  • Scale: 32.0
  • Dropout: 0.05
  • Target layers: self_attn.[qkv]_proj, mlp.[gate_up_down]_proj
• Mixed Precision: Enabled
• Gradient Checkpointing: Enabled

Speeds, Sizes, Times

• Training Time: ~45 minutes
• Peak Memory: 7.576 GB
• Model Size: Original ~4.9GB (GGUF F16); Quantized versions available (e.g., Q8_0 ~2.6GB)

Evaluation

Testing Data, Factors & Metrics

Testing Data

Validation split from enhanced FEMA dataset.

Factors

• Branding compliance
• Response accuracy
• FEMA protocol adherence

Metrics

• Final Training Loss: 0.016
• Validation Loss: 0.016
• Branding Compliance: 60% (from tests)

Results

The model shows strong identity retention and emergency guidance quality. See ENHANCED_MODEL_REPORT.md for details.

Model Examination

• Perplexity: Not explicitly measured, but low loss indicates good performance.
• Zero-shot/Few-shot: Performs well in zero-shot emergency queries.

Environmental Impact

• Hardware Type: Apple Silicon (M-series)
• Hours used: ~0.75 hours per training run
• Carbon Emitted: Minimal (local training)

Technical Specifications

Model Architecture and Objective

Gemma-2-2B with LoRA adapters for causal language modeling.

Compute Infrastructure

Apple M-series GPU/CPU.

Hardware

Apple Silicon Mac.

Software

MLX-LM framework.

Citation

BibTeX:

@article{terraguideai2024,
  title={TerraGuideAI: Fine-tuned Gemma-2-2B for Emergency Response},
  author={Offlyn.ai Team},
  year={2024}
}

Citation for base model:

See Gemma-2-2B model card.

Model Card Contact

Contact: [email protected] (based on workspace info)

How to Get Started With the Model

See the code snippets in the "How to Get Started" section above.