OneOcean

<p align="center"> <img src="https://huggingface.co/Jady-Zhao/OneOcean/resolve/main/OneOcean.png" width="80%"> </p> <h1 align="center">🐚 OneOcean : 多模态深海大模型 </b></h1> </p>

We have released OneOcean, a multimodal large model specialized for the marine domain, to help researchers and engineers explore intelligent understanding of deep-sea imagery.<br> If you are interested in extending OneOcean for specific marine applications, feel free to contact us for collaboration.<br> 我们已发布 OneOcean 多模态海洋领域大模型，旨在支持科研人员与工程师探索深海图像的智能理解。<br> 如果您希望将 OneOcean 应用于特定的海洋研究或工程场景，欢迎与我们联系合作。<br> <br>

🔔 Important<br> We will continue to update and optimize the OneOcean multimodal large model.<br> The model’s capabilities may vary across versions, and your feedback is highly appreciated to advance the use of multimodal LLMs in marine science.<br> 🔔 重要提示<br> 我们将持续更新和优化OneOcean多模态大模型。<br> 模型的性能可能随版本变化而不同，欢迎您提出宝贵反馈，共同推动多模态大模型在海洋科学领域的应用发展。<br>

<details> <summary><b>📘 Table of Contents / 目录</b></summary>

• ℹ️ About the model / 关于模型
• ✨ Features / 功能
• 📊 Training summary / 训练摘要
• 🗂️ Training datasets / 训练数据
• ⚙️ Framework versions / 框架版本
• 🚀 Quickstart / 快速使用
• 🔍 Inference / 推理
• 🌻 Acknowledgement / 致谢
• ⚠️ Limitations / 模型限制
• 🚩 Citation & Contact / 引用与联系

</details> <br>

ℹ️ About the model / 关于模型

We fine-tuned the Qwen2.5-VL-72B-Instruct foundation model to create OneOcean, a multimodal model specialized for the marine domain. OneOcean supports functionalities such as marine organism detection and geological feature recognition.<br> 基于 Qwen2.5-VL-72B 大模型进行微调，得到 OneOcean 多模态海洋领域大模型，支持 生物检测 和 地质识别 等功能。

<br>

✨ Features / 功能

• Marine organism detection (生物检测)
• Geological feature recognition (地质识别)
• Multimodal understanding of underwater imagery (多模态海底图像理解)

<br>

📊 Training summary / 训练摘要

• Base model: Qwen2.5-VL-72B-Instruct
• Fine-tuning method: LoRA via LLaMA-Factory
• Final training loss: 0.3548

<br>

🗂️ Training datasets / 训练数据

• ~300K+ image-text pairs for marine organisms (30多万条生物图文对)
• ~100K+ image-text pairs for geological features (10多万条地质图文对)
• LoRA fine-tuning via LLaMA-Factory

<br>

⚙️ Framework versions / 框架版本

• PyTorch 2.4.1+cu121
• Transformers 4.49.0.dev0
• PEFT 0.15.2
• Datasets 3.6.0
• Tokenizers 0.21.1

<br>

🚀 Quickstart / 快速使用

conda create -n py3.11 python=3.11
conda activate py3.11
pip install -r requirements.txt

Download the model

Download from HuggingFace

# use git lfs
git lfs install
git clone https://huggingface.co/Jady-Zhao/OneOcean

🔍 Inference / 推理

🌋 Geological inference / 地质推理

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from PIL import Image
import torch

model_name = "Jady-Zhao/OneOcean"

# Load model and processor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(model_name, device_map="auto", trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)

image = Image.open("example_underwater_image.jpg")
question = "请判断这张图的地质类型。"  # Please determine the geological type of this image

inputs = processor(
    text=[question],
    images=[image],
    padding=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=50)
prediction = processor.batch_decode(outputs, skip_special_tokens=True)[0]

print("Predicted geological type:", prediction)

🦑 Biological inference / 生物推理

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from PIL import Image
import torch

model_name = "Jady-Zhao/OneOcean"

# Load model and processor
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(model_name, device_map="auto", trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)

image = Image.open("example_underwater_image.jpg")
question = "请输出这张图中的深海生物类型。"  # Please identify the deep-sea organism in the image

inputs = processor(
    text=[question],
    images=[image],
    padding=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=50)
prediction = processor.batch_decode(outputs, skip_special_tokens=True)[0]

print("Predicted biological type:", prediction)

<br>

🌻 Acknowledgement / 致谢

We would like to thank:<br> 谨向以下人员致谢：

• The developers of Qwen2.5-VL-72B for providing the foundation model;
• Qwen2.5-VL-72B 基础模型的开发者;
• The creators of LLaMA-Factory / PEFT / Transformers / BitsAndBytes for their excellent open-source libraries;
• LLaMA-Factory / PEFT / Transformers / BitsAndBytes 等优秀开源库的贡献者；
• The contributors of the Oceanus_Bio and Oceanus_Geo datasets;
• Oceanus_Bio 与 Oceanus_Geo 数据集的提供者; <br>

⚠️ Limitations / 模型限制

• The model is specialized for marine organisms and geological features and may not generalize to other domains;
• 模型专注于 海洋生物和地质特征，可能无法推广到其他领域;
• Predictions may be inaccurate for low-quality, noisy, or unusual images.
• 对低质量、噪声或特殊图像的预测可能不准确;
• Outputs can vary slightly depending on prompt wording.
• 输出结果可能受 提示语 影响而略有差异;
• The model is not optimized for real-time processing of large video streams.
• 模型尚未针对大规模视频流进行实时优化

<br>

🚩 Citation & Contact / 引用与联系

This model (OneOcean) has not been formally published yet.<br> 本模型 (OneOcean) 尚未正式发布。 <br>

If you would like to use it in your work, please contact the author via email:<br> 如果希望在研究或工作中使用，请 发送邮件联系作者:

• [email protected]
• [email protected]
• [email protected]

@misc{oneocean2025,
  title = {OneOcean: A Multimodal Marine Foundation Model},
  author = {Zhejianglab},
  year = {2025},
  howpublished = {\url{https://huggingface.co/Jady-Zhao/OneOcean}},
  note = {Accessed: 2025-11-03}
}

<br> <h1 <b> 🏆 Contributors </b></h1> COMPUTATIONAL SENSING RESEARCH CENTER, ZHEJIANG LAB