hf-avro-poc

Apache Avro benign parser-DoS PoC

This PoC demonstrates a scanner/runtime mismatch for Avro artifacts in ML-adjacent pipelines:

• modelscan does not treat .avro as a supported model file type and skips it.
• Python avro and fastavro fully decompress an Avro object-container block before yielding the first record.
• A small compressed .avro file can therefore trigger disproportionate memory and CPU work when used for ML dataset shards, metadata bundles, or checkpoint-adjacent binary blobs.

Files

• generate_poc.py - builds a benign .avro artifact with a highly compressible tensor_bytes payload
• verifier.py - reads the artifact with both Python runtimes, captures working-set deltas, and records modelscan output
• artifacts/avro_bzip2_dos.avro - staged benign artifact
• results/results.json - summarized run output

Reproduce

cd C:\Users\Pragnyan\dev\huntr-exp1\avro
.\.venv\Scripts\python.exe .\hf_avro_poc\generate_poc.py
.\.venv\Scripts\python.exe .\hf_avro_poc\verifier.py

Expected outcome

• Artifact size is tiny relative to the uncompressed payload.
• Both readers report tensor_bytes_len equal to the large embedded payload.
• modelscan reports the file as skipped or unsupported, leaving the runtime behavior unanalyzed.