ocean-fingerprint

Ocean Fingerprint

Acoustic vessel-identity fingerprinting pipeline: preprocessing → mel/CQT features → triplet-loss embedding network → fingerprint database → drift/mismatch detection → dashboard.

This is a research/demo prototype built around the architecture in Ocean Fingerprint — Full System Architecture. It's designed to train on public underwater acoustic datasets (DeepShip, ShipsEar) and to simulate "drift" (mechanical wear) and "spoofing" (identity mismatch) via audio augmentation, since adversarial recordings of real spoofing attempts don't exist publicly.

1. Install

pip install -r requirements.txt

Python 3.10+ recommended. Needs torch, torchaudio, librosa, soundfile, scikit-learn, numpy, pandas, matplotlib, streamlit.

2. Get data

This repo does NOT ship any audio. Download one of:

• DeepShip: https://github.com/irfankamboh/DeepShip (or the original authors' repo — search "DeepShip underwater acoustic dataset")
• ShipsEar: http://atlanttic.uvigo.es/underwaternoise/ (registration may be required)

Unzip so you end up with a folder of .wav files and a metadata/label file (ship class or vessel ID per file). Then build a manifest CSV:

python src/build_manifest.py --audio_dir /path/to/wavs --labels_csv /path/to/labels.csv \
    --out data/manifest.csv

The manifest schema matches the architecture doc:

vessel_id,timestamp,filepath,condition_label,sea_state,depth,speed,ambient_noise_db

If you don't have real labels yet, build_manifest.py can also auto-generate one from a folder structure like data/raw/<vessel_id>/<clip>.wav — see --infer_from_dirs.

3. Preprocess: segment + extract features

python src/preprocessing.py --manifest data/manifest.csv --out_dir data/features \
    --window_sec 3.0 --overlap 0.5 --feature mel

This denoises (spectral subtraction), chops each clip into overlapping windows, and saves log-mel spectrograms as .npy files, plus an updated manifest (data/features/windows_manifest.csv) with one row per window.

4. Simulate drift & spoofing (for demo purposes)

python src/augment.py --manifest data/features/windows_manifest.csv \
    --out_dir data/features_augmented

Produces additional rows labeled drift_early, drift_severe, and spoofed by applying pitch-shift / noise-injection / time-stretch to existing clips, exactly as described in the architecture doc (section 8).

5. Train the embedding model

# Recommended: ECAPA-TDNN backbone + AAM-Softmax (ArcFace-style) loss
python src/train.py --manifest data/features_augmented/manifest.csv \
    --epochs 30 --batch_size 32 --embedding_dim 192 \
    --backbone ecapa_tdnn --loss aam_softmax --out_dir checkpoints/

# Original triplet-loss setup is still available
python src/train.py --manifest data/features_augmented/manifest.csv \
    --epochs 30 --batch_size 32 --embedding_dim 192 \
    --backbone tdnn --loss triplet --out_dir checkpoints/

Backbones: tdnn (x-vector style), resnet_lite, or ecapa_tdnn (SE-Res2Blocks + multi-layer feature aggregation + attentive statistics pooling — current standard architecture for speaker/acoustic verification, recommended default).

Losses: triplet (original) or aam_softmax (angular-margin classification loss, recommended — trains faster and produces a more separated embedding space; see src/losses.py docstring). aam_softmax also applies SpecAugment automatically (src/augment.py::spec_augment, disable with --no_spec_augment). Both loss modes support mixed-precision training on CUDA via --amp (on by default).

See src/model.py, src/dataset.py, src/losses.py.

5b. Calibrate detection thresholds

config.yaml's drift_threshold / identity_threshold are starting points, not tuned values. Once you have a trained checkpoint and a held-out manifest (vessels or time windows not used in training/enrollment), calibrate them properly from an EER (Equal Error Rate) analysis instead of guessing:

python src/calibrate_thresholds.py --manifest data/holdout/manifest.csv \
    --checkpoint checkpoints/best.pt --condition baseline \
    --update_config config.yaml

Prints the EER and writes calibrated drift_threshold / identity_threshold back into config.yaml. See src/calibrate_thresholds.py docstring for the methodology.

6. Enroll vessels into the fingerprint database

python src/enroll.py --manifest data/features_augmented/manifest.csv \
    --checkpoint checkpoints/best.pt --condition baseline --out data/fingerprint_db.json

Computes centroid + variance embeddings per vessel from baseline clips only, and stores them (see src/fingerprint_db.py).

7. Run inference / comparison engine

python src/comparison_engine.py --checkpoint checkpoints/best.pt \
    --db data/fingerprint_db.json --clip /path/to/new_clip.wav --claimed_vessel INS_0001

Prints similarity score and one of: normal, drift, mismatch.

8. Drift tracking over time

src/drift_detection.py maintains embedding_history[vessel_id] and runs an Isolation Forest / rolling control chart over the distance-from-baseline series to distinguish sudden jumps (likely damage/event) from slow steady drift (maintenance scheduling signal).

9. Dashboard

streamlit run dashboard/app.py

Simple dashboard: vessel list, live similarity score, drift trend graph, alert log — reads from data/fingerprint_db.json and data/alert_log.csv.

Project layout

ocean_fingerprint/
├── README.md
├── requirements.txt
├── config.yaml
├── data/                     # you populate this
├── checkpoints/              # trained models land here
├── src/
│   ├── build_manifest.py
│   ├── preprocessing.py      # denoise, segment, mel/CQT features
│   ├── augment.py            # drift + spoof simulation
│   ├── dataset.py            # triplet + classification-style PyTorch Datasets
│   ├── calibrate_thresholds.py  # EER-based drift/identity threshold calibration
│   ├── model.py              # TDNN/ResNet-lite embedding network
│   ├── losses.py             # triplet / contrastive / AAM-Softmax (ArcFace) loss
│   ├── train.py              # training loop
│   ├── fingerprint_db.py     # enrollment DB (JSON-backed)
│   ├── enroll.py             # CLI to build the DB
│   ├── comparison_engine.py  # inference: distance + thresholds
│   ├── drift_detection.py    # Isolation Forest / control chart over embedding history
│   └── utils.py
└── dashboard/
    └── app.py                # Streamlit dashboard

Notes / honest caveats (carried over from the architecture doc)

• Environment vs. identity confound is the hardest part. Sea state, depth, and speed all shift the acoustic signature of the same vessel. The environmental_metadata fields exist so you can eventually condition the model on them or hold them out as a robustness test — this codebase logs them but does not yet fully solve the confound. Expect to spend real iteration time here.
• Drift vs. mismatch are different alerts for different teams (maintenance vs. security) — the dashboard keeps them as separate alert streams rather than one feed.
• This is a demo/prototype, not a production naval system. Real deployment would need hardened edge hardware, SATCOM/mesh integration, and a lot more validation on real spoofing scenarios (which don't really exist in public data — the augment.py simulation is a stand-in, clearly labeled as such).