Downloads · 30 days
6
6% of all-time downloads
beaunix/aegis-geo-mind-qwen2.5-7b-bnb-4bit
aegis-geo-mind-qwen2.5-7b-bnb-4bit is a text generation model from beaunix. Use it when you need the model to write or continue text. The card lists the license as mit.
A QLoRA fine-tune of Qwen2.5-7B-Instruct specialized in geology and petroleum geology, covering general geology, petroleum systems, sedimentary basin analysis, sequence stratigraphy, and well log interpretation.
Downloads · 30 days
6
6% of all-time downloads
All-time downloads
99
Public
Parameters
7.6B
5.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.5 GB · 100%
How the weights are stored.
U86.5B · 86%
From the Hugging Face model README
A QLoRA fine-tune of Qwen2.5-7B-Instruct specialized in geology and petroleum geology, covering general geology, petroleum systems, sedimentary basin analysis, sequence stratigraphy, and well log interpretation.
This repository hosts the 4-bit (bitsandbytes, nf4) version of the model, optimized for inference on lower-VRAM hardware (e.g. a T4 GPU).
Aegis-Geo-Mind is fine-tuned to explain and reason about geological and petroleum-geology concepts in fluent, domain-appropriate language, including correct technical terminology across:
The model was built to address a gap in publicly available geoscience LLMs: none of the existing open geology-focused models specialize in petroleum geology.
This is a research preview. See Limitations below before relying on any specific factual or numeric claim.
daven3/geosignal (K2 corpus), reduced from
39,749 to 18,054 rows through five rounds of quality filtering (removal
of generic instruction filler, noisy NER categories, bibliography-only
answers, broken template artifacts, and empty-field placeholders).The curated pairs were oversampled ×5 within the training split (never in validation) to offset their small share of the combined corpus. Final training set: 21,639 rows. Validation set: 940 rows, held disjoint from training at every merge and filtering step.
Full dataset available at beaunix/geo-mind-qa.
| Parameter | Value |
|---|---|
| Base model | Qwen2.5-7B-Instruct |
| Method | QLoRA, 4-bit |
| LoRA rank / alpha | 16 / 16 |
| Target modules | q/k/v/o_proj, gate/up/down_proj |
| Effective batch size | 32 (per-device 8 × grad. accumulation 4) |
| Epochs | 3 (best checkpoint restored from step 1,250) |
| Learning rate | 2e-4, cosine schedule |
| max_seq_length | 1,024 tokens (covers 99.85% of the training corpus) |
| Early stopping | patience 3 on validation loss |
Batch configuration was selected via a dedicated throughput benchmark (examples/second, not steps/second) across multiple batch/accumulation combinations, rather than assumed.
Best validation loss: 1.560 (step 1,250 of 2,031).
A validation loss in this range should not, by itself, be read as evidence of a weak model. Much of the training data consists of open-ended explanatory answers where multiple correct phrasings exist — token-level loss penalizes deviation from one reference wording even when the model's own answer is equally correct. This produces an irreducible loss floor tied to the dataset's phrasing diversity, not a direct measure of geological competence.
Evaluated on 60 held-out Q/A pairs (disjoint from training). Test perplexity: 6.12.
Manual review of generated answers found:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
model_id = "beaunix/aegis-geo-mind-qwen2.5-7b-bnb-4bit"
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, quantization_config=bnb_config, device_map="auto"
)
messages = [
{"role": "system", "content": "You are an expert geologist specializing in Earth Sciences."},
{"role": "user", "content": "What are growth faults and why are they important in sedimentary basins?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.9)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
This model is a supervised fine-tune without retrieval augmentation. It reliably reproduces the style and reasoning structure of expert geological explanation, but can state specific facts, numbers, and ratios incorrectly with high confidence. Confirmed weak areas from evaluation:
For any factual, numeric, or operational use, verify claims against authoritative references (e.g. Tissot & Welte, Selley, Schlumberger logging references). A retrieval-augmented (RAG) version addressing these gaps is planned.
Not for operational decisions. This model must not be used as a sole basis for exploration, drilling, reservoir, or any other operational geological decision.
MIT.
If you use this model, please cite the repository:
@misc{aegis-geo-mind-2026,
author = {beaunix},
title = {Aegis-Geo-Mind: A QLoRA Fine-tune of Qwen2.5-7B for Geology and Petroleum Geology},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/beaunix/aegis-geo-mind-qwen2.5-7b-bnb-4bit}
}