Downloads · 30 days
0
JinchengPeng520QMUL/msc-urban-sound-tagging-models
msc-urban-sound-tagging-models is a audio classification model from JinchengPeng520QMUL. Use it for the audio classification task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as bsd-3-clause.
This repository is the supporting material for an MSc project on multi-label urban sound tagging. It contains the complete retained Python source code, all 13 trained CNN and Audio Spectrogram Transformer (AST) checkp…
Downloads · 30 days
0
Access
Public
Updated Aug 18, 2026
Repo size
2 GB
Likes
0
Public
Click a slice to open those files.
.pt1.8 GB · 90%
From the Hugging Face model README
This repository is the supporting material for an MSc project on multi-label urban sound tagging. It contains the complete retained Python source code, all 13 trained CNN and Audio Spectrogram Transformer (AST) checkpoints, derived label tables, normalization statistics, saved predictions, training histories, and evaluation tables.
Raw SONYC-UST audio and generated feature arrays are deliberately not redistributed because they are large and are available from the official dataset source. The instructions below reproduce the project from the official audio release.
The reported experiments used:
All pinned Python packages are listed in requirements.txt. In the PyCharm Terminal opened at the repository root, create and activate an environment and install them:
conda create -n sonyc-ust-reproduction python=3.10.20 -y
conda activate sonyc-ust-reproduction
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
The exact CUDA environment is recommended for training reproduction. CPU execution is possible for some preprocessing, evaluation, and inference steps, but training all models on CPU is impractical. The first AST run also downloads the BSD-3-Clause base model MIT/ast-finetuned-audioset-10-10-0.4593 from Hugging Face, and the CNN code downloads ImageNet-pretrained MobileNetV2 weights through torchvision; therefore, internet access is required on the first run.
All commands below assume that the PyCharm Terminal working directory is this repository root:
cd "<path-to-this-repository>"
Do not run the commands from inside the data or baseline_code folders because the project uses paths relative to the repository root.
Dataset: SONYC Urban Sound Tagging / DCASE 2019 Task 5.
Official task and download page:
https://dcase.community/challenge2019/task-urban-sound-tagging
On that page, go to Download and select SONYC Urban Sound Tagging (2.1 GB, version 0.2.0). The permanent Zenodo record is:
https://doi.org/10.5281/zenodo.3233082
The official release separates the audio into these two archives:
audio-dev.tar.gz # development audio: both the training set and validation set
audio-eval.tar.gz # official evaluation/test-set audio
Therefore, audio-eval.tar.gz is the file that must be downloaded for reproducing this project's final test-set predictions. After extraction, its WAV files must be available under data/audio-eval/. The audio-dev.tar.gz archive is not test data: it supplies both data/train/ and data/validate/ for model training and validation.
The development annotations are provided separately as annotations-dev.csv. In the original DCASE 2019 evaluation release, the public evaluation package supplied the test audio but did not publicly disclose the evaluation ground-truth labels. The consensus evaluation annotation tables retained in this repository are used by the supplied 11-evaluate-*-test.py scripts to reproduce the reported test metrics.
In the original PyCharm project, the dataset root directory was named exactly data. The local archive and metadata names were:
data/
|-- audio-dev.tar.gz
|-- audio-eval.tar.gz
|-- annotations-dev.csv
|-- dcase-ust-taxonomy.yaml
After extraction, the scripts expect this layout:
data/
|-- train/ # development training WAV files
|-- validate/ # development validation WAV files
|-- audio-eval/ # evaluation WAV files
|-- annotations-dev.csv
|-- dcase-ust-taxonomy.yaml
The development release contains 2,351 training recordings and 443 validation recordings. The project evaluation/test directory contained 274 WAV files. The archives themselves and the extracted WAV files are not included in this repository.
Download the development (audio-dev.tar.gz) and evaluation/test (audio-eval.tar.gz) audio from the official page. Releases may use archive or annotation filenames different from those used in this PyCharm project. Reproducers should use the filenames supplied by their actual downloaded release, then either:
./data/... path constants in the scripts to match their actual names.If the downloaded archives have been placed at data/audio-dev.tar.gz and data/audio-eval.tar.gz, extract them from the PyCharm Terminal with:
python extract_audio.py
The included data/README.md records the dataset description and CC BY 4.0 conditions. Do not use the included derived annotation tables as a substitute for acquiring the official audio release.
All retained CNN experiment scripts are included, notably:
07-train-system-2.py: original medium-resolution System-2 CNN.07-train-system-2-logmel-multiscale-baseline.py: independent short/long CNN baselines.07-train-system-2-temporal-attention.py: temporal-attention experiment.07-train-system-2-residual-temporal-attention.py: residual temporal-attention experiment.07-train-system-2-temporal-logit-residual.py: short/medium/long temporal logit-residual experiments.The data directory contains all 13 trained checkpoints:
model_system2
model_system2_short
model_system2_long
model_system2_temporal_attention
model_system2_residual_temporal_attention
model_system2_temporal_logit_residual
model_system2_temporal_logit_residual_short
model_system2_temporal_logit_residual_long
model_ast_audioset.pt
model_ast_audioset_short.pt
model_ast_audioset_long.pt
model_ast_end_to_end_gated_multires.pt
model_ast_cross_resolution_attention.pt
Their upload-integrity hashes are recorded in MODEL_SHA256SUMS.txt. From the repository root, they can be recomputed in the PyCharm PowerShell terminal with:
Get-FileHash -Algorithm SHA256 .\data\model* | Format-Table Hash, Path
These total approximately 1.89 GiB. Derived predictions, gates, pooling weights, metrics, label tables, normalization statistics, and loss histories are included beside them.
Run these commands once after arranging the official dataset as described above.
Parse the development annotations:
python 01-parse-annotations.py
This creates data/metadata.pkl.
Create the 37-column soft training and validation label tables used by both the CNN and AST experiments:
python 06-scale-annos-system-2.py
This creates:
data/train_scaled.pkl
data/valid_scaled.pkl
Precomputed copies are included so that the supplied GUI and checkpoints can be inspected without repeating label preparation.
Create the original medium-resolution development and evaluation features:
python 02-compute-log-mel.py
Create the aligned short- and long-window features. The existing medium features are reused:
python 02-compute-log-mel-multiscale.py --scales short long --splits development evaluation
Generated directories are:
data/logmelspec/
data/logmelspec-eval/
data/logmelspec-short/
data/logmelspec-short-eval/
data/logmelspec-long/
data/logmelspec-long-eval/
Train the original medium-resolution System-2 CNN:
python 07-train-system-2.py
Train the independent short- and long-resolution CNN baselines:
python 07-train-system-2-logmel-multiscale-baseline.py --scale short --seed 42
python 07-train-system-2-logmel-multiscale-baseline.py --scale long --seed 42
The original medium script produces data/model_system2. The short and long commands produce data/model_system2_short and data/model_system2_long. Re-running a training command overwrites its corresponding best checkpoint, so preserve the supplied checkpoint first if it must remain unchanged.
Temporal-attention CNN:
python 07-train-system-2-temporal-attention.py --init-from-system2 ./data/model_system2 --batch-size 64 --lr 0.0005
Residual temporal-attention CNN:
python 07-train-system-2-residual-temporal-attention.py --init-from-system2 ./data/model_system2 --batch-size 64 --base-lr 0.00005 --temporal-lr 0.0005
Temporal logit-residual CNNs at all three resolutions:
python 07-train-system-2-temporal-logit-residual.py --scale short --batch-size 64 --base-lr 0.00002 --temporal-lr 0.0005 --seed 42
python 07-train-system-2-temporal-logit-residual.py --scale medium --batch-size 64 --base-lr 0.00002 --temporal-lr 0.0005 --seed 42
python 07-train-system-2-temporal-logit-residual.py --scale long --batch-size 64 --base-lr 0.00002 --temporal-lr 0.0005 --seed 42
Original System-2 CNN:
python 08-generate-valid-set-preds-system-2.py
Temporal-attention variants:
python 08-generate-valid-set-preds-system-2-temporal-attention.py
python 08-generate-valid-set-preds-system-2-residual-temporal-attention.py
Temporal logit-residual variants:
python 08-generate-valid-set-preds-system-2-temporal-logit-residual.py --scale short --split validation --tta-rounds 10 --seed 42
python 08-generate-valid-set-preds-system-2-temporal-logit-residual.py --scale medium --split validation --tta-rounds 10 --seed 42
python 08-generate-valid-set-preds-system-2-temporal-logit-residual.py --scale long --split validation --tta-rounds 10 --seed 42
Fuse the short/medium/long temporal logit-residual probabilities with the validation-selected weights used in the project:
python 09-ensemble-system-2-logmel-multiscale.py --split validation --weights 0.2 0.6 0.2
Evaluate all three scales and their late fusion with the official DCASE metrics:
python 10-evaluate-system-2-logmel-multiscale.py
Individual CNN prediction files can also be evaluated directly. For example:
python baseline_code/evaluate_predictions.py data/valid-set-preds-system-2.csv data/annotations-dev.csv data/dcase-ust-taxonomy.yaml
python baseline_code/evaluate_predictions.py data/valid-set-preds-system-2-temporal-attention.csv data/annotations-dev.csv data/dcase-ust-taxonomy.yaml
python baseline_code/evaluate_predictions.py data/valid-set-preds-system-2-residual-temporal-attention.csv data/annotations-dev.csv data/dcase-ust-taxonomy.yaml
python 08-generate-valid-set-preds-system-2-temporal-logit-residual.py --scale short --split evaluation --tta-rounds 10 --seed 42
python 08-generate-valid-set-preds-system-2-temporal-logit-residual.py --scale medium --split evaluation --tta-rounds 10 --seed 42
python 08-generate-valid-set-preds-system-2-temporal-logit-residual.py --scale long --split evaluation --tta-rounds 10 --seed 42
python 09-ensemble-system-2-logmel-multiscale.py --split evaluation --weights 0.2 0.6 0.2
python 11-evaluate-system-2-logmel-multiscale-test.py --label-version leaderboard-v0.3
python 11-evaluate-system-2-logmel-multiscale-test.py --label-version corrected-v2.3
The leaderboard-v0.3 option reproduces the original DCASE 2019 label version. The corrected-v2.3 option uses the later corrected machinery-impact labels. The required annotation CSVs and the derived consensus tables are included in data.
Create the standard 25 ms (medium) AST features for all splits:
python 02-compute-ast-audioset-fbank.py --splits train validation evaluation
Create the 10 ms short and 50 ms long AST features for all splits:
python 02-compute-ast-audioset-multiscale-fbank.py --scales short long --splits train validation evaluation
The feature directories created are:
data/ast-fbank/{train,validation,evaluation}/
data/ast-fbank-short/{train,validation,evaluation}/
data/ast-fbank-long/{train,validation,evaluation}/
An optional diagnostic plot can be produced after feature extraction:
python 03-plot-ast-multiscale-fbank.py --split train --separate
python 07-train-ast-audioset.py --scale short --batch-size 8 --gradient-accumulation 4 --epochs 100 --patience 25 --seed 42
python 07-train-ast-audioset.py --scale medium --batch-size 8 --gradient-accumulation 4 --epochs 100 --patience 25 --seed 42
python 07-train-ast-audioset.py --scale long --batch-size 8 --gradient-accumulation 4 --epochs 100 --patience 25 --seed 42
If GPU memory is insufficient, reduce --batch-size and increase --gradient-accumulation to retain a similar effective batch size. Add --gradient-checkpointing when required.
Validation predictions:
python 08-generate-valid-set-preds-ast-audioset.py --scale short --batch-size 4
python 08-generate-valid-set-preds-ast-audioset.py --scale medium --batch-size 4
python 08-generate-valid-set-preds-ast-audioset.py --scale long --batch-size 4
Evaluation predictions:
python 08-generate-eval-set-preds-ast-audioset.py --scale short --batch-size 8
python 08-generate-eval-set-preds-ast-audioset.py --scale medium --batch-size 8
python 08-generate-eval-set-preds-ast-audioset.py --scale long --batch-size 8
Select late-fusion weights on validation data, then reuse the saved weights on evaluation data:
python 09-ensemble-ast-audioset-multiscale.py --split validation --search-step 0.02
python 09-ensemble-ast-audioset-multiscale.py --split evaluation
Evaluate the medium and fused validation predictions:
python 10-evaluate-ast-audioset.py --prediction data/valid-set-preds-ast-audioset.csv --metrics-output data/valid-metrics-ast-audioset.csv
python 10-evaluate-ast-audioset.py --prediction data/valid-set-preds-ast-audioset-multiscale.csv --metrics-output data/valid-metrics-ast-audioset-multiscale.csv
Evaluate the medium and fused evaluation predictions against both available label versions:
python 11-evaluate-ast-audioset-test.py --label-version leaderboard-v0.3 --prediction data/eval-set-preds-ast-audioset.csv
python 11-evaluate-ast-audioset-test.py --label-version corrected-v2.3 --prediction data/eval-set-preds-ast-audioset.csv
python 11-evaluate-ast-audioset-test.py --label-version leaderboard-v0.3 --prediction data/eval-set-preds-ast-audioset-multiscale.csv
python 11-evaluate-ast-audioset-test.py --label-version corrected-v2.3 --prediction data/eval-set-preds-ast-audioset-multiscale.csv
The same evaluation commands may be applied to the short and long prediction CSVs by changing --prediction.
This is the checkpoint used by the included GUI.
Train:
python 07-train-ast-end-to-end-gated-multires.py --batch-size 2 --gradient-accumulation 16 --epochs 100 --patience 25 --seed 42
Generate validation and evaluation predictions and gate values:
python 08-generate-preds-ast-end-to-end-gated-multires.py --split validation --batch-size 2
python 08-generate-preds-ast-end-to-end-gated-multires.py --split evaluation --batch-size 2
Evaluate:
python 10-evaluate-ast-audioset.py --prediction data/valid-set-preds-ast-end-to-end-gated-multires.csv --metrics-output data/valid-metrics-ast-end-to-end-gated-multires.csv
python 11-evaluate-ast-audioset-test.py --label-version leaderboard-v0.3 --prediction data/eval-set-preds-ast-end-to-end-gated-multires.csv --metrics-output data/test-metrics-ast-end-to-end-gated-multires-leaderboard-v0.3.csv
python 11-evaluate-ast-audioset-test.py --label-version corrected-v2.3 --prediction data/eval-set-preds-ast-end-to-end-gated-multires.csv --metrics-output data/test-metrics-ast-end-to-end-gated-multires-corrected-v2.3.csv
Train with the retained project configuration:
python 07-train-ast-cross-resolution-attention.py --batch-size 2 --gradient-accumulation 16 --epochs 100 --patience 25 --intra-layers 4 --cross-layers 4 --joint-layers 4 --tokens-per-scale 256 --adapter-dim 64 --aux-loss-weight 0.1 --diversity-loss-weight 0.01 --seed 42
Generate validation and evaluation predictions and pooling diagnostics:
python 08-generate-preds-ast-cross-resolution-attention.py --split validation --batch-size 2
python 08-generate-preds-ast-cross-resolution-attention.py --split evaluation --batch-size 2
Evaluate:
python 10-evaluate-ast-audioset.py --prediction data/valid-set-preds-ast-cross-resolution-attention.csv --metrics-output data/valid-metrics-ast-cross-resolution-attention.csv
python 11-evaluate-ast-audioset-test.py --label-version leaderboard-v0.3 --prediction data/eval-set-preds-ast-cross-resolution-attention.csv --metrics-output data/test-metrics-ast-cross-resolution-attention-leaderboard-v0.3.csv
The GUI uses:
app_ast_gated.pydata/model_ast_end_to_end_gated_multires.ptdata/valid_scaled.pkl for the exact 37-label orderassets/qmul-logo.pngThese files are all included. Internet access is required on the first run to obtain the underlying AudioSet-pretrained AST configuration/weights from Hugging Face.
From the PyCharm Terminal at the repository root:
python app_ast_gated.py
The application opens http://127.0.0.1:7860 in the default browser. Upload a WAV file, choose a probability threshold, and select Analyze audio. Only WAV input is accepted. The GUI divides longer recordings into 10-second segments and reports fine- and coarse-level labels.
For a headless launch or a different port:
python app_ast_gated.py --no-browser --port 7860
The included CSV and NPY files allow the reported outputs to be inspected without retraining. Examples include:
Full retraining of all 13 checkpoints is computationally expensive. Exact floating-point equality is not guaranteed across GPU models, CUDA/cuDNN versions, or library builds, even with the supplied seeds. Reproduction should therefore compare the documented pipeline, label order, output files, and metrics within normal numerical variation.
Scripts such as ast_audioset.py, ast_multiscale.py, ast_end_to_end_gated.py, ast_cross_resolution_attention.py, multiscale_logmel.py, temporal_models.py, utils.py, and RandomErasing.py are imported helper modules and are not intended to be run directly.
The baseline_code directory retains the official DCASE metric implementation and evaluator used by this project. The upstream repository's legacy VGGish/TensorFlow feature-extraction and logistic-regression training scripts are not part of this project's CNN or AST pipelines and are therefore not included here; the upstream source is linked for provenance.
LICENSE.MIT/ast-finetuned-audioset-10-10-0.4593, BSD 3-Clause.baseline_code: MIT License; see baseline_code/LICENSE.md.data/README.md and the official DCASE page.THIRD_PARTY_NOTICES.md.The raw dataset audio is not included in this repository.