solarhive-26b-a4b-lora

SolarHive 26B A4B LoRA — Community Solar Energy Intelligence

LoRA fine-tuned adapters for Gemma 4 26B A4B, specialized in community solar energy management with native function calling and multimodal visual question answering.

Built for the Gemma 4 Good Hackathon (Google DeepMind x Kaggle).

---

Model Overview

SolarHive is an AI energy advisor for community solar microgrids. It helps suburban neighborhoods collectively optimize distributed solar generation and shared battery storage through natural language conversation, visual inspection, and live data integration.

This is the cloud inference model. It powers the live demo with full multimodal VQA and native function calling. For edge deployment via Ollama (privacy-first, no internet required), see the companion SolarHive E4B Ollama.

This repository contains LoRA adapters only — you need the base Gemma 4 26B A4B model to use them. The adapters add domain expertise in solar energy, battery management, grid optimization, and community coordination while preserving the base model's general capabilities.

What These Adapters Add

• Domain expertise in solar production, battery management, grid pricing, panel inspection, and community energy coordination
• Improved function calling for four energy-specific tools (weather, solar production, battery state, grid status)
• Visual question answering for sky condition analysis, panel health inspection, and neighborhood aerial assessment
• Grounded responses that reference real data from live APIs rather than hallucinating numbers

---

Benchmark Results

Evaluated on held-out questions not seen during training:

Domain Q&A (5/5)

Tool Calling (1/3)

Note: The isolated benchmark (single-turn) scores 8/8. In the full agentic loop, the model also scores 8/8 — see below.

Production Benchmark (8/8) — Inference Agentic Loop

When evaluated using generate_with_tools() with tool schemas in context, the model scores 8/8 (5/5 Q&A + 3/3 tool calling):

Q&A (5/5) — same questions, same correct answers as above.

Tool Calling (3/3):

The difference: the agentic loop passes tool schemas via apply_chat_template(tools=[...]), giving the model the function signatures it was trained on. The isolated benchmark tests raw generation without tool context.

Multi-Variant Deployment Validation (Final Run, May 2026)

The 26B A4B LoRA + base is the baseline of the multi-variant comparison. Score on the 10-question parity benchmark (5 Q&A + 5 tool):

Score: 5/5 Q&A + 4/5 tool = 9/10

The single FAIL is the lenient multi-call probe — "Compare today's irradiance forecast across Ann Arbor, Phoenix, and Seattle" (min_calls=2) — where this A4B LoRA returned no tool call. 4 of 5 ran variants share the same multi-call failure mode; only the E4B LoRA + base variant chained the multi-city calls (3 × get_weather). Pattern is reproducible across runs — systematic, not stochastic.

Inference-time When2Call Validation — A4B LoRA scores 3/3 (directly measured)

Three held-out probes from Ross et al. (2025), When2Call: When (not) to Call Tools, arXiv:2504.18851. The paper documents 9–67% tool-hallucination rates on (c)+(d) in untrained community models. The A4B LoRA passes all three probes (3/3, directly measured in the May 2026 inference run), confirming that the SolarHive fine-tune — which includes 16 explicit unable-to-answer + follow-up clarification examples following the When2Call taxonomy — handles refusal + follow-up behaviors correctly:

Compare to the E4B family (solarhive-e4b-lora and solarhive-e4b-ollama) which both score 2/3 on the same probes (pass (b)+(d), fail (c) by auto-filling location instead of asking back). The +1/3 W2C delta between the A4B family (3/3 across LoRA + merged + NF4, all measured) and E4B family (2/3 across LoRA + merged) is the empirical signature of size-vs-refusal scaling. A4B outperforming E4B on these reasoning-heavy probes was the pre-stated hypothesis going in, not a discovery — per the official Google Gemma 4 Core docs "Parameter sizes and quantization" section: "Models with higher parameters and bit counts (higher precision) are generally more capable, but are more expensive to run." This 26B A4B variant accesses ~25B total knowledge capacity (3.8B active per token via MoE sparsity) and a ~550M vision encoder — vs E4B's 8B total / 4.5B effective / ~150M vision encoder. The When2Call paper documents the same size-vs-refusal scaling empirically. A4B is the right deployment target for under-specified or out-of-scope queries; E4B handles the well-specified-routing volume at the edge.

Quantitative reinforcement from Unsloth's published Gemma 4 benchmarks:

The 45.8 pp AIME gap (math reasoning) + 21 pp MMMU Pro gap predict the SolarHive When2Call (c)/(d) regression directly — refusal/follow-up behavior is a reasoning task, and the published reasoning-benchmark delta scales cleanly into the 2-of-3 behavioral regression we observed.

---

Precision Note — BF16 is Gemma 4's Native Release Format

This repository contains LoRA adapter weights only — apply them on top of Google's open-source google/gemma-4-26b-a4b-it base via Unsloth's FastVisionModel.from_pretrained(...) at inference time. Both the base model and the adapters are in BF16, which is Gemma 4's native release precision — there is no FP32 release to begin with, so applying BF16 LoRA over a BF16 base is not a quantization downgrade; it is the same numerical precision Google published.

All three variants are derived from the same fine-tuning run; the LoRA delta in this repo is the canonical source. The merged and NF4 variants exist for deployment convenience.

---

Training Details

Hyperparameters

Training Data — 1,727 Examples

The canonical training corpus is solarhive-community-solar-multimodal — 1,727 rows (1,713 text + 14 image-grounded). The full hand-crafted portion is preserved verbatim in solarhive_datagen.py Cell 7a (LEGACY_DATA + LEGACY_TOOL_CALL_DATA), and the API-grounded portion is reproducible at training time via _fetch_api_examples().

Three complementary sources ensure both breadth and depth:

413 hand-crafted Q&A spanning 15+ US cities and 9 energy domains:

• Sky conditions and cloud impact on production
• Battery management and charge/discharge strategy
• Panel health diagnostics and maintenance
• Consumption optimization and load shifting
• Community and grid coordination strategy
• Emergency resilience and outage planning
• Seasonal planning and weather adaptation
• Multi-step reasoning across multiple data sources
• Alternative storage (fuel cells, thermal)

~1,117 API-grounded Q&A generated from live data:

• Open-Meteo (GHI, DNI, DHI, low/mid/high cloud cover), PVWatts, OpenWeatherMap, EIA APIs
• Joined on (location, hourly timestamp) so each multi-source example carries co-occurring grounding
• Locations: Ann Arbor, MI and San Mateo, CA
• Every numeric claim traces back to a real API response

183 tool-calling examples trained with the When2Call taxonomy — 106 should-call, 53 should-not-call, 10 unable-to-answer, 6 follow-up clarification, 8 failure-recovery — so the model learns when to call tools, when to refuse politely, when to admit a tool can't answer, and when to ask a clarifying question.

14 image-grounded Q&A turns from 7 manually-labeled Ann Arbor sky photographs — cloud type and percentage cloud cover are human-confirmed, expected production traces back to the cloud-cover label via the same temperature-derated GHI formula.

Training Loss

Hardware

• GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition (102 GB GDDR7 total, 94.97 GB max usable per Unsloth)
• Platform: Google Colab Pro (G4 VM)
• Precision: BF16 (no quantization during training)

---

How to Use

Loading with Unsloth (Recommended)

Standard PEFT cannot handle Gemma 4's Gemma4ClippableLinear layers. Use Unsloth's FastVisionModel for reliable adapter loading:

from unsloth import FastVisionModel
import torch

# Load base model + LoRA adapters
model, processor = FastVisionModel.from_pretrained(
    model_name="google/gemma-4-26b-a4b-it",
    adapter_name="Truthseeker87/solarhive-26b-a4b-lora",  # This repo
    dtype=torch.bfloat16,
    device_map="auto",
)
FastVisionModel.for_inference(model)

Two-Step Tokenization (Required)

Single-step apply_chat_template(tokenize=True) crashes in transformers 5.5.x on messages without a "content" key (e.g., tool_calls messages). Use this two-step pattern:

messages = [
    {"role": "system", "content": "You are SolarHive, an AI energy advisor..."},
    {"role": "user", "content": "How will today's weather affect our solar production?"},
]

# Step 1: render text (tokenize=False)
text = processor.apply_chat_template(
    messages, tools=tools,
    add_generation_prompt=True,
    enable_thinking=False,
    tokenize=False,
)

# Step 2: tokenize separately
inputs = processor(text=text, images=None, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=1.0, top_p=0.95, top_k=64)
response = processor.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)

Native Function Calling

Define tools as Python functions with Google-style docstrings. Gemma 4 autonomously decides which to invoke:

def get_weather(location: str) -> dict:
    """Get current weather conditions for a location.

    Args:
        location: City name, e.g. 'Ann Arbor, MI'

    Returns:
        dict with temp_f, clouds_pct, wind_mph, humidity, sunrise, sunset
    """
    # Your API call here
    ...

def get_solar_production(clouds_pct: int, temp_f: float) -> dict:
    """Get estimated community solar production using GHI irradiance data.

    Args:
        clouds_pct: Cloud cover percentage (0-100)
        temp_f: Temperature in Fahrenheit

    Returns:
        dict with production_kw, capacity_kw, efficiency_pct, ghi_wm2
    """
    ...

tools = [get_weather, get_solar_production, get_battery_state, get_grid_status]

text = processor.apply_chat_template(
    messages, tools=tools,
    add_generation_prompt=True,
    enable_thinking=False,
    tokenize=False,
)

The model emits tool calls as call:fn_name{arg: "value"} in its output, parsed via regex r'call:(\w+)\{([^}]*)\}'.

---

Core Capabilities

1. Multimodal Visual Question Answering (3 Modes)

2. Native Function Calling (5 Tools — all 3 keyed APIs wired)

Tool results feed back as a 2-message sequence matching the training distribution:

{"role": "assistant", "tool_calls": [{"function": {"name": ..., "arguments": ...}}, ...]}
{"role": "tool",      "name": "<fn>", "content": json.dumps(result)}  # one per tool call

This format is shared across solarhive_datagen.py (training-data generation), solarhive_finetune.py (SFT preprocessing + schema validation), solarhive_inference.py Cell 4, and test_ollama_tools.py Solution B — inference matches the training distribution exactly.

3. Selective Tool Reasoning

The model reasons about which tools are relevant — not blindly calling everything:

"What time does peak pricing start?"
→ Calls: get_grid_status() only

"Is today's production above typical for January?"
→ Calls: get_solar_production() + get_nrel_pvwatts_baseline()

"Should I run my pool heater now?"
→ Calls: get_weather() + get_solar_production() + get_battery_state() + get_grid_status()

"What are general maintenance tips?"
→ Calls: none (answers from training knowledge)

4. Inference-time When2Call Validation

Three held-out probes validate coverage of 3 of the 4 failure-mode categories from Ross, H., Mahabaleshwarkar, A. S., & Suhara, Y. (2025). When2Call: When (not) to Call Tools. arXiv:2504.18851. The paper documents 9–67% tool-hallucination rates in untrained community models on (c) and (d):

Models trained without explicit unable-to-answer and follow-up clarification examples typically fail (c) + (d). The SolarHive corpus includes 16 such examples (10 unable-to-answer + 6 follow-up clarification) following the When2Call taxonomy; this A4B LoRA passes all 3 probes (3/3, directly measured in the May 2026 inference run).

---

Live Demo Output

Five interactive queries from the April 15, 2026 demo capture with real sky photos from Ann Arbor:

Query 1 — Sky photo uploaded (VQA Mode 1)

Q: "How is current cloud cover affecting potential household solar production?"

A: "The image shows heavy, uniform overcast conditions with thick gray cloud cover obscuring the sun completely. This type of overcast sky typically reduces solar production to 10-25% of clear-sky capacity. At this moment, expect roughly 10-25 kW community output from your 72 kW array. This is a good time to conserve battery for evening peak hours and defer non-essential loads."

Query 2 — Tool calling (solar production)

Q: "What was estimated solar production at 9.00am Eastern time?"

Tools called: get_solar_production

A: "At 9:00am in Ann Arbor, MI, the community produced 40.4 kW out of 72 kW capacity (56.1% efficiency). GHI was 660.0 W/m² with 10% cloud cover and no temperature derating at 65°F. This is a solid early morning output — production will ramp up toward the noon peak."

Query 3 — Sky photo + energy planning

Q: "How should I plan my household solar production and energy storage for the next two hours?"

Tools called: none (direct visual answer)

A: "The image shows heavy, broken cumulus and stratocumulus clouds covering approximately 80-90% of the sky. Production will be highly variable — cycling between 15-30% of capacity during cloud gaps and dropping to 5-10% under the thicker gray patches. If your battery is below 50%, prioritize essential loads only."

Query 4 — Tool calling (grid status)

Q: "What is current grid capacity now?"

Tools called: get_grid_status

A: "Peak pricing at $0.28/kWh with 30.3% renewable grid mix. CO2 intensity is 381.5 kg/MWh. Peak rates typically run 2-7pm — prioritize battery discharge and minimize grid imports."

Query 5 — Tool calling (weather)

Q: "What is the weather forecast going to be like this week in Ann Arbor?"

Tools called: get_weather

A: "Current conditions in Ann Arbor: 67°F with 100% cloud cover and mist. With 100% cloud cover, expect significantly reduced solar output — run only essential loads and preserve battery for evening peak."

---

Data Pipeline Diagnostics

Training data quality validated with 14 diagnostic charts generated from live API data:

Solar Irradiance and Production

Environmental Correlations

Cross-Validation and Grid Analysis

Atmospheric Decomposition

---

Community Model

---

Companion Repositories

---

Versions — v2 Update (Text-Only Training on the Multimodal-Capable Corpus)

The repository was refreshed on April 30, 2026 with a v2 LoRA produced by re-training on the consolidated training corpus (1,727 rows = 1,713 text + 14 image-grounded). The v2 fine-tune trains on the text subset only; image rows are skipped at the data-prep layer. Multimodal training is deferred post-hackathon — a real image corpus and a held-out VQA benchmark would be prerequisites. The base Gemma 4 26B A4B model retains full multimodal capability regardless of which corpus subset is used in any given fine-tune run, so VQA at inference time continues to work on the saved adapters.

The shipped dataset uses the project archive only — fewer images than originally planned, but every label is human-confirmed and every paired Q&A traces back to the same GHI / temperature derating formula used elsewhere. Image-source planning had earlier rejected the SWIM corpora (NUS — CC BY-NC licensing) and NREL SRRL (legacy MIDC SkyCam archive ended May 2017).

The v2 fine-tune is pre-aligned with the official Unsloth Gemma 4 documentation (train guide, bug fixes & tips): explicit loader arguments (max_seq_length, dtype, full_finetuning=False), explicit SFTConfig arguments (weight_decay, lr_scheduler_type), text-only data path (finetune_vision_layers=False, dataset_text_field="text", TRL default text collator, train_on_responses_only wrapper for assistant-only loss masking).

---

Technical Notes

• System prompt repetition: The system prompt is repeated twice in the message format. This technique improves instruction following in causal LLMs, winning 47/70 benchmark-model tests with zero losses (Leviathan et al., 2025, Google Research).
• PEFT incompatibility: Standard PEFT cannot handle Gemma 4's Gemma4ClippableLinear layers. Use Unsloth's FastVisionModel for adapter loading.
• VRAM requirements: ~48 GB in BF16, ~16 GB in NF4 (4-bit). T4 x2 cannot run this model.
• Sampling: temperature=1.0, top_p=0.95, top_k=64 (Kaggle-recommended defaults).

---

Limitations

• Prototype tested on a single community model (12 homes, Ann Arbor). Real-world deployment requires validation across diverse geographies and community sizes.
• The model occasionally uses "60 kW" instead of the correct 72 kW community capacity in direct VQA responses (without tool calls). This is a base model tendency that additional fine-tuning examples will address.
• Tool responses depend on external API availability. Open-Meteo and EIA have rate limits. OpenWeatherMap free tier allows 1,000 calls/day.
• The battery state simulator is deterministic for demonstrations. Real deployment requires integration with actual battery management systems.

---

Future Iteration — Multi-Token Prediction (MTP) Drafters

Not in the measured numbers above. Google announced Gemma 4 MTP drafters on May 5, 2026 (blog, overview, HF collection, Kaggle, @GoogleGemma) — after this artifact's final benchmark was captured. The benchmarks above reflect standard autoregressive decoding only. MTP integration is documented here as future iteration; no measured speedup is claimed in this release.

Theoretical foundation. Speculative decoding (Leviathan, Kalman & Matias, Fast Inference from Transformers via Speculative Decoding, ICML 2023, arXiv:2211.17192) accelerates generation without changing the output distribution under argmax decoding: a smaller drafter proposes γ candidate tokens, the target verifies all γ in a single parallel forward pass, accepted tokens are kept, and any rejection is resampled from a corrected distribution. The output distribution is preserved exactly regardless of drafter quality; only acceptance rate α, and therefore walltime speedup, varies.

What Google released on May 5, 2026. Paired drafter checkpoints for all four IT-tuned Gemma 4 variants — gemma-4-E2B-it-assistant, gemma-4-E4B-it-assistant, gemma-4-26B-A4B-it-assistant, gemma-4-31B-it-assistant — discoverable via the google/gemma-4 Hugging Face collection and on Kaggle Models. The drafters share the input embedding table with their paired target and consume the target's last-layer activations (architecture per the MTP overview). For this target the paired drafter is google/gemma-4-26B-A4B-it-assistant (~0.4 B params). Google reports up to 3× decode speedup with no quality degradation on the 26B-A4B configuration, and ~2.2× on Apple Silicon at batch sizes 4–8. Tested runtimes named in the blog: LiteRT-LM, MLX, Hugging Face Transformers, vLLM, SGLang, Ollama.

Integration cost is one kwarg in Hugging Face Transformers:

target_base = AutoModelForCausalLM.from_pretrained("google/gemma-4-26B-A4B-it", dtype=torch.bfloat16, ...)
target = FastVisionModel.from_pretrained("Truthseeker87/solarhive-26b-a4b-lora", ...)  # apply LoRA on top
assistant = AutoModelForCausalLM.from_pretrained("google/gemma-4-26B-A4B-it-assistant", dtype=torch.bfloat16, ...)
target.generate(**inputs, assistant_model=assistant)  # MTP enabled

The integration ships as a gated future-iteration cell (§14, _RUN_MTP_DEMO = False) in solarhive_inference.py; reviewers can flip the flag to reproduce a baseline-vs-MTP comparison under argmax decoding.

Open question specific to this LoRA-adapter target. Per the 2023 speculative-sampling guarantee, correctness is invariant to drafter quality — the target's verification step preserves the exact output distribution regardless of what the drafter proposes. What varies is acceptance rate α, since Google's released drafter was trained against the base gemma-4-26B-A4B-it, not against this LoRA-adapter-on-top target. Measured α and the resulting walltime speedup on this target are the planned post-hackathon contribution.

---

Citation

@misc{solarhive2026,
  title={SolarHive: AI-Powered Community Solar Energy Intelligence},
  author={Youshen Lim},
  year={2026},
  url={https://github.com/youshen-lim/the-gemma4-good-hackathon-solarhive},
  note={Gemma 4 Good Hackathon submission — Google DeepMind x Kaggle}
}

---

Links

• GitHub: youshen-lim/the-gemma4-good-hackathon-solarhive
• Kaggle: The Gemma 4 Good Hackathon
• Base Model: google/gemma-4-26b-a4b-it

Built with Gemma 4 in Ann Arbor, Michigan. May 2026.

Gemma is a trademark of Google LLC.