Downloads · 30 days
160
22% of all-time downloads
asferrer/gemma-4-E2B-it-oceanguard-marine-debris
gemma-4-E2B-it-oceanguard-marine-debris is a image-to-text model from asferrer. Use it when you need a caption or text from an image. It is set up for peft. The card lists the license as apache-2.0.
A LoRA fine-tune of google/gemma-4-E2B-it that adapts the model to open-vocabulary detection of anthropogenic marine debris on consumer mobile hardware. The adapter is the training artefact behind the OceanGuard AI An…
Downloads · 30 days
160
22% of all-time downloads
All-time downloads
728
Public
Repo size
12.2 GB
Likes
0
Public
Click a slice to open those files.
.litertlm5.2 GB · 59%
From the Hugging Face model README
A LoRA fine-tune of google/gemma-4-E2B-it
that adapts the model to open-vocabulary detection of anthropogenic marine debris on
consumer mobile hardware. The adapter is the training artefact behind the
OceanGuard AI Android application, a fully
offline marine-debris intelligence toolkit built for the Kaggle Gemma 4 Good Hackathon
(Global Resilience track), and builds on two peer-reviewed publications on underwater debris
detection (Sánchez-Ferrer et al., PRL
2023;
IbPRIA 2022).
Headline result. On TEST-200, a 200-image evaluation subset whose images share no group with any training image, the adapter raises the [email protected] of the unmodified base model from 0.1381 to 0.2941 (paired difference +0.156, 95 % bootstrap interval [0.045, 0.178], i.e. about 2.1 times higher). The adapter was selected as the best of three LoRA configurations by its mAP on a separate 100-image subset (VAL-100); on TEST-200 the three configurations are statistically indistinguishable (see Evaluation).
| Artefact | Purpose | Consumed by |
|---|---|---|
adapter_model.safetensors + adapter_config.json | LoRA adapter (about 119 MB, r=16, α=32) on top of google/gemma-4-E2B-it | PEFT / Unsloth pipelines |
gemma-4-E2B-it-oceanguard-Q4_K_M.gguf | Adapter merged into the base model and quantized to Q4_K_M (about 3.4 GB, 5.88 bits per weight; the 262k-token vocabulary dominates the size). Text decoder only, llama.cpp compatible | OceanGuard Android app via LlamaTextEngine, or any llama.cpp client |
gemma-4-E2B-it-oceanguard.litertlm | Adapter merged into the base model and exported to LiteRT-LM with int8 quantization (5,245,718,144 bytes, about 5.0 GiB). Text decoder and vision encoder, 140 vision tokens, no speculative decoding. SHA-256 992dc19b3b72f847fdbf42c69f60995cd670c0f065d8e0855e37d78bf2b53e11 | OceanGuard Android app (LiteRTTextEngine) |
tokenizer.json, tokenizer_config.json, chat_template.jinja, processor_config.json | Tokenizer, chat template and processor configuration saved with the adapter | Transformers / PEFT / Unsloth |
OceanGuard AI runs Gemma 4 E2B fully on-device for open-vocabulary analysis of marine debris.
This adapter aligns the model with the structured output contract that the Android application
consumes: a JSON array of {box_2d, label, material} objects, where box_2d is
[y_min, x_min, y_max, x_max] on a 0-1000 grid and label is one of eight snake_case classes:
plastic_bottle, metal_can, fishing_net, glove, face_mask, metal_debris,
plastic_debris, tire. The application maps these labels onto its ecological-impact families
for the health score and report generation.
The adapter is a standard PEFT LoRA (rank 16, α = 32, no dropout) attached to the attention
(q/k/v/o_proj) and MLP (gate/up/down_proj) projections. It was trained on the 4-bit
quantized base model (QLoRA) with Unsloth in bf16.
In-scope.
Out-of-scope.
| Part | Images | Instances | Notes |
|---|---|---|---|
| Real TRAIN (CleanSea, Neural_Ocean, Ocean_garbage) | 10,245 | 21,123 | all images of the TRAIN partition (767 CleanSea / 3,129 Neural_Ocean / 6,349 Ocean_garbage) |
| Synthetic | 5,000 | 19,500 | random sample (seed 42) of a clean batch of copy-paste composites made with the authors' synthetic data generator |
| Total | 15,245 | 33 % synthetic |
Real images come from three public marine-debris datasets, including the CleanSea corpus introduced by the model author (IbPRIA 2022):
| Source | License | Annotations |
|---|---|---|
| CleanSea | Non-profit research or educational use (per dataset page) | COCO boxes, mapped to 8 classes |
| Ocean_garbage | Roboflow Universe per-dataset terms | COCO boxes, mapped to 8 classes |
| Neural_Ocean | Roboflow Universe per-dataset terms | COCO boxes, mapped to 8 classes |
The corpus is partitioned by group (for example, all images of a CleanSea dive, or sharing a source-file name key, go to the same partition), so no group of the evaluation subsets appears in training. Training instances per class: Bottle 4,333, Can 412, Fishing_Net 2,812, Glove 2,582, Mask 3,164, Metal_Debris 102, Plastic_Debris 3,957, Tire 3,761 (real images only).
Geographic coverage. The three sources were collected predominantly in coastal waters around Japan and the wider north-western Pacific, so the training distribution is biased towards the debris types, water clarity and lighting of that region. Validate on a local hold-out before operational use elsewhere.
Re-distributors of the training corpus must comply with the original dataset licenses.
Unsloth FastVisionModel, one workstation GPU, bf16 compute over a 4-bit base model. LoRA is
attached to the attention and MLP projections selected by module name, which matches both the
language tower and the vision encoder: the saved adapter has non-zero low-rank updates in both
(245 pairs in the language tower, 112 in the vision encoder). The vision_lora switch of the training
configuration had no effect on which modules were adapted (see Limitations).
| Hyper-parameter | Value |
|---|---|
| Base model | unsloth/gemma-4-E2B-it (4-bit load) |
LoRA r / α / dropout | 16 / 32 / 0.0 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Optimiser | adamw_8bit, learning rate 1e-4, cosine schedule, weight decay 0.01, warm-up ratio 0.1, max grad norm 1.0 |
| Batch size | 4 per device × 8 gradient accumulation = 32 effective |
| Epochs / optimisation steps | 3 / 1,431 |
| Sequence length / image token budget | 1,024 / 64 |
| Seed | 42 |
| Loss | causal-LM over the assistant JSON answer (box_2d + label + material) |
Mean training loss over the run: 0.851 (the synthetic images raise it relative to a real-only run). Wall-clock 12,287 s (3.4 h) on a single NVIDIA RTX 5090 (32 GiB), Windows 11, CUDA 12.8; peak allocated GPU memory 20.6 GB.
Software: unsloth 2026.5.2, transformers 5.5.0, peft 0.19.1, torch 2.12 (nightly, CUDA 12.8), bitsandbytes.
The adapter file holds 786 LoRA tensors (393 A/B pairs, about 31 M parameters; see Limitations for
their distribution across towers).
asferrer/oceanguard-marine-debris-eval-1000
(annotations only, no images).pycocotools) is also reported.| Configuration | Adapted modules | Training data | Steps |
|---|---|---|---|
| Base | none | none | none |
LoRA, real only (exp10g) | language + vision attention/MLP | 10,245 real | 963 |
LoRA, real only, second run (exp12g) | same as exp10g | 10,245 real | 963 |
LoRA, real + synthetic (exp11g) — this repository | same as exp10g | 10,245 real + 5,000 synthetic | 1,431 |
exp12g was launched with vision_lora: true, but its adapter has exactly the same structure as
exp10g (the switch had no effect), so it is a second run of the real-only configuration with the same
seed; the gap between the two runs is an indication of run-to-run variability.
| Configuration | TEST-200 [email protected] | 95 % interval | VAL-100 [email protected] | JSON valid (TEST) | Predictions (TEST) | Latency (s, TEST) |
|---|---|---|---|---|---|---|
Base gemma-4-E2B-it | 0.1381 | [0.106, 0.187] | 0.1081 | 0.990 | 183 | 3.19 |
LoRA, real only (exp10g) | 0.2948 | [0.227, 0.335] | 0.2570 | 0.935 | 300 | 9.03 |
LoRA, real only, second run (exp12g) | 0.2825 | [0.211, 0.319] | 0.2063 | 0.975 | 342 | 7.75 |
LoRA, real + synthetic (exp11g, this repo) | 0.2941 | [0.209, 0.320] | 0.2820 | 0.980 | 313 | 7.04 |
With the COCO-style evaluator the TEST-200 [email protected] is 0.1073 (base), 0.2780, 0.2704 and 0.3003 for the three LoRA rows in the order above (VAL-100: 0.0942, 0.2461, 0.2017, 0.2792).
Paired differences on TEST-200 (95 % basic bootstrap interval):
| Comparison | Difference in [email protected] |
|---|---|
exp10g − base | +0.157 [0.075, 0.190] |
exp12g − base | +0.144 [0.056, 0.177] |
exp11g − base (this repo) | +0.156 [0.045, 0.178] |
exp11g − exp10g | −0.001 [−0.072, 0.026] |
exp12g − exp10g | −0.012 [−0.054, 0.021] |
exp12g − exp11g | −0.012 [−0.043, 0.049] |
All three adapters improve clearly over the base model, and the intervals of the differences between
adapters include zero on TEST-200: adding synthetic data did not change the global mAP, and the two
runs of the real-only configuration differ by 0.012 on TEST-200 but by 0.051 on VAL-100. exp11g was chosen because it had the best mAP on VAL-100
(0.2820 against 0.2570 and 0.2063), although the spread between the two real-only runs on VAL-100
shows that this choice is within run-to-run variability; it also has the highest JSON validity and the lowest (indicative) latency
among the adapters.
n is the number of ground-truth instances of the class in TEST-200.
| Class | n | Base | exp10g | exp12g | exp11g (this repo) | Δ (exp11g − base) |
|---|---|---|---|---|---|---|
| Bottle | 98 | 0.190 | 0.411 | 0.353 | 0.393 | +0.203 |
| Can | 35 | 0.000 | 0.045 | 0.061 | 0.091 | +0.091 |
| Fishing_Net | 52 | 0.294 | 0.436 | 0.377 | 0.406 | +0.113 |
| Glove | 63 | 0.118 | 0.426 | 0.511 | 0.451 | +0.334 |
| Mask | 67 | 0.091 | 0.251 | 0.250 | 0.343 | +0.253 |
| Metal_Debris | 15 | 0.227 | 0.091 | 0.091 | 0.000 | −0.227 |
| Plastic_Debris | 58 | 0.139 | 0.488 | 0.453 | 0.491 | +0.352 |
| Tire | 45 | 0.045 | 0.210 | 0.164 | 0.177 | +0.132 |
The adapter improves seven of the eight classes. Metal_Debris (15 instances in TEST-200, 102 in
training) is the only class where the base model is better, and the released adapter does not detect it
(0.000, the other adapters reach 0.091). Can (35 instances) is still weak. Per-class figures on so few
instances are very noisy; the 5,000 synthetic images (2,041 Metal_Debris and 2,344 Can instances) did not
rescue these two classes.
| Source | n images | Base | exp10g | exp12g | exp11g (this repo) | exp11g / base |
|---|---|---|---|---|---|---|
| Global | 200 | 0.138 | 0.295 | 0.282 | 0.294 | 2.13x |
| CleanSea | 28 | 0.163 | 0.260 | 0.202 | 0.177 | 1.09x |
| Neural_Ocean | 64 | 0.089 | 0.321 | 0.278 | 0.357 | 4.01x |
| Ocean_garbage | 108 | 0.121 | 0.299 | 0.291 | 0.346 | 2.85x |
The gain concentrates in Neural_Ocean and Ocean_garbage (Roboflow-exported imagery); on CleanSea
(28 images) the released adapter improves only by +0.014. The per-source and per-class tables can be
recomputed from the cached predictions with scripts/compute_per_source_map.py in the
evaluation dataset repository.
System: You are a marine debris detection system. Output ONLY valid JSON arrays. Never include explanations, markdown, or text outside the JSON.
User (image + text):
Detect ALL marine debris and litter in this image. Output ONLY a JSON array.
For each object, provide:
- "box_2d": bounding box as [y_min, x_min, y_max, x_max], integers 0-1000
- "label": ONE of these 8 labels (snake_case, lowercase): plastic_bottle, metal_can, fishing_net, glove, face_mask, metal_debris, plastic_debris, tire
- "material": Plastic, Metal, Glass, Rubber, Fabric, Latex
Output format:
[{"box_2d": [y_min, x_min, y_max, x_max], "label": "object_type", "material": "material_type"}]
If no debris is found, output: []
| Step | Tooling | Status |
|---|---|---|
| Train LoRA | Unsloth FastVisionModel | Done |
| Publish adapter | Hugging Face Hub | Done (this repository) |
| Use adapter via Transformers / PEFT / Unsloth | model.load_adapter(...) | Done |
| Merge adapter into the base model (float16) | manual W + (α/r)·B·A merge | Done |
| Convert merged model to GGUF f16 and quantize to Q4_K_M | llama.cpp convert_hf_to_gguf.py, llama-quantize | Done: gemma-4-E2B-it-oceanguard-Q4_K_M.gguf |
| Wire the GGUF to the Android app | LlamaTextEngine + llama.cpp | Done (downloaded from this repository by the app) |
Export merged model to LiteRT-LM (.litertlm, int8) | litert-torch | Done: gemma-4-E2B-it-oceanguard.litertlm |
| On-device latency of the adapted model | Samsung Galaxy S22 Ultra | Done: 9.15 s per photo on average (five photos) |
The GGUF contains the merged text decoder only (no vision encoder or multimodal projector). To
send images through llama.cpp you need a multimodal projector (mmproj) for Gemma 4 E2B, which is not
part of this repository.
curl -L -o gemma-4-E2B-it-oceanguard-Q4_K_M.gguf \
"https://huggingface.co/asferrer/gemma-4-E2B-it-oceanguard-marine-debris/resolve/main/gemma-4-E2B-it-oceanguard-Q4_K_M.gguf"
./llama-cli -m gemma-4-E2B-it-oceanguard-Q4_K_M.gguf -n 256 --temp 0.0 -p "..."
gemma-4-E2B-it-oceanguard.litertlm runs text and vision on the device with LiteRT-LM.
| Model, 448 px / 140 vision tokens, [email protected] | VAL-100 | TEST-200 |
|---|---|---|
| This LiteRT-LM build (int8) | 0.2482 | 0.2632 |
| Base Gemma 4 E2B, LiteRT-LM | 0.1547 | 0.1556 |
These figures were measured on a PC, not on the phone. On a Samsung Galaxy S22 Ultra (Exynos 2200, GPU) detection took 9.15 s per photo on average over five photos without annotations, so that test measures time and not accuracy. Loading the engine took 24.4 s with the compilation cache and 39.5 s the first time, and memory use was 4.9 to 5.2 GB.
Metal_Debris (15 instances), Can (35) and CleanSea (28 images),
are noisy.Metal_Debris is not detected (mAP 0.000) and Can is weak (0.091). Operators should expect
silent misses on these classes.from unsloth import FastVisionModel
model, tokenizer = FastVisionModel.from_pretrained(
"unsloth/gemma-4-E2B-it",
load_in_4bit=True,
)
model.load_adapter(
"asferrer/gemma-4-E2B-it-oceanguard-marine-debris",
adapter_name="oceanguard",
)
model.set_adapter("oceanguard")
FastVisionModel.for_inference(model)
Use the detection prompt shown in Evaluation and greedy decoding
(temperature = 0, max_new_tokens = 256), then parse the answer as a JSON array. An end-to-end
example is in the OceanGuard repository.
See LiteRT-LM build. The OceanGuard Android app downloads this file and offers it as the fine-tuned Gemma 4 variant.
See GGUF download. The OceanGuard Android app downloads and loads this file through
its "Gemma 4 FT (GGUF)" provider using LlamaTextEngine (llama.cpp).
google/gemma-4-E2B-it, subject to the Gemma Terms of Use. The merged GGUF and
LiteRT-LM files are derivatives of the base model and are also subject to those terms.If you use this adapter, please cite the adapter, the application and the article:
@misc{sanchezferrer2026oceanguard_adapter,
author = {S{\'a}nchez-Ferrer, Alejandro},
title = {{Gemma 4 E2B OceanGuard Marine Debris LoRA Adapter}},
year = {2026},
howpublished = {Hugging Face Model Hub},
note = {LoRA adapter for google/gemma-4-E2B-it. Apache 2.0.},
url = {https://huggingface.co/asferrer/gemma-4-E2B-it-oceanguard-marine-debris}
}
@misc{sanchezferrer2026oceanguard_app,
author = {S{\'a}nchez-Ferrer, Alejandro},
title = {{OceanGuard AI: Fully Offline Marine Debris Intelligence with Gemma 4 and Two-Phase Tool Calling}},
year = {2026},
howpublished = {Kaggle Gemma 4 Good Hackathon (Global Resilience track)},
url = {https://github.com/asferrer/oceanguard-app}
}
@misc{sanchezferrer2026synthetic,
author = {S{\'a}nchez-Ferrer, Alejandro and Gallego, Antonio Javier and Calvo-Zaragoza, Jorge},
title = {{When Does Synthetic Data Help Underwater Debris Detection? Gap Metrics, Data Regimes and Detector Paradigms}},
year = {2026},
howpublished = {Neurocomputing},
note = {Manuscript in preparation}
}
FastVisionModel implementation that makes a Gemma 4 vision-language
fine-tune tractable on a single workstation GPU.