Downloads · 30 days
0
ericblackgachara/arrow-decompress-parquet-oom-poc
arrow-decompress-parquet-oom-poc is a machine learning model from ericblackgachara. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
0
Access
Public
Updated May 31, 2026
Repo size
207 KB
Likes
0
Public
Click a slice to open those files.
.pdf207 KB · 89%
From the Hugging Face model README
| Field | Value |
|---|---|
| Platform | huntr.com |
| Program | Parquet (Model File Format) |
| Affected version | Apache Arrow latest (commit ca8a194) |
| Confirmed library | pyarrow 23.0.1 |
| Language | C++ (pyarrow Python bindings) |
Uncontrolled Memory Allocation in SerializedPageReader::DecompressIfNeeded()
The Parquet C++ decompression path resizes an internal buffer using uncompressed_page_size from the page header with no upper bound validation. A 595-byte SNAPPY-compressed Parquet file with a forged uncompressed_page_size = INT32_MAX triggers Resize(2,147,483,647) — a 2 GB allocation — before any decompression is attempted.
cpp/src/parquet/column_reader.cc — SerializedPageReader::DecompressIfNeeded()
if (compressed_len < 0 || uncompressed_len < 0) ... // ONLY check: negatives
decompression_buffer_->Resize(uncompressed_len, false); // <- UNBOUNDED ALLOC
Note: column_writer.cc:288 validates this on the write side. The reader has no matching check — a classic read/write asymmetry.
DATA_PAGE Thrift header: uncompressed_page_size = 2,147,483,647 (5-byte varint: FE FF FF FF 0F)
compressed_page_size = 139 (139 actual bytes)
Validation: if (uncompressed_len < 0) → PASSES (2,147,483,647 is positive)
DecompressIfNeeded(): decompression_buffer_->Resize(2,147,483,647) ← CRASH
| Offset | Original | Patched | Effect |
|---|---|---|---|
| [7, 9) | ce 25 | fe ff ff ff 0f | uncompressed_page_size: 2407 → 2,147,483,647 (+3 bytes) |
| [13, 15) | 9c 02 | 96 02 | compressed_page_size: 142 → 139 (compensates stream budget) |
The two patches cancel out in the stream budget — total_compressed_size in the footer is unchanged, no offset errors.
pip install pyarrow
ulimit -v 524288
python3 poc_arrow_decompress.py
[+] MemoryError: malloc of size 2147483648 failed
[+] DecompressIfNeeded triggered unbounded Resize(2,147,483,647)!
[+] CWE-789: Uncontrolled Memory Allocation — CONFIRMED
| File | Purpose |
|---|---|
poc_arrow_decompress.py | Working PoC — writes + patches + triggers |
submission.md | Plain-prose huntr submission (no markdown formatting) |
poc-evidence.html | Self-contained HTML evidence page with terminal output |
README.md | This file |
| Finding #1 | Finding #2 (this) | |
|---|---|---|
| Field | num_values (DICTIONARY_PAGE) | uncompressed_page_size (any page) |
| File | decoder.cc | column_reader.cc |
| Function | DictDecoderImpl::DecodeDict | SerializedPageReader::DecompressIfNeeded |
| Requires | Dict-encoded column | Compressed column (SNAPPY/GZIP/etc.) |
| Max alloc | ~16 GB | ~2 GB (int32_t limit) |
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H = 7.5 HIGH
Reported 2026-05-31 | Eric Gachara