sec-restatement-arms

SEC Restatement Arms

Nine fitted models over the 2021 window of sec-restatement-8k-402: three feature sets (numeric XBRL tag presence, words bag-of-words, embed sentence embeddings) by three learners (random forest, histogram gradient boosting, L2 logistic regression).

Each pickle carries the estimator and its feature map, because a model without its vocabulary is not reproducible.

These ship without a decision threshold, on purpose

The paper these come from measures what happens when you pick one badly. Selecting a threshold on the rows a model was fitted on inflates test cost by up to 5.5x, is governed by how completely the model memorises its training set rather than by model quality, and is therefore worst for the strongest model. Calibration cannot repair it: a monotone map cannot change which rows a threshold selects.

Shipping a fitted threshold would hand you that bug. Choose yours out of fold, or on a held-out block if your training set is small. Five extra fits, no held-out data, recovers a median 95% of the loss.

Do not expect these to beat a trivial baseline

Under an asymmetric cost with a miss worth ten false alarms, none of them beat flagging every filing. That is the paper's finding, not a defect in the weights. Evaluate at a stated review capacity with precision@k instead, which needs no cost ratio.

Code, results and the paper: https://github.com/hmmrlabs/cost-functions