smollm-gray-swan-classifier

Gray Swan ML Engineer Technical Take-Home

Part 1: Dataset Generation

Due to local compute constraints on the Google Colab T4 environment and strict time limitations, generation was capped at 192 high-fidelity samples. To fulfill the pipeline requirements, this dataset was split: 80% used as the training set and 20% reserved for the synthetic evaluation set.

The generation utilized a custom CodecLM & Condor pipeline using the Qwen2.5-3B-Instruct model to ensure highly indirect, complex Gray Swan prompts. The dataset is included in this submission as gray_swan_hf_dataset.json.

Part 2: Training

The training was conducted using a LoRA adapter on the HuggingFaceTB/SmolLM2-360M-Instruct base model.

• Training Logs: Included in this submission as training_logs.txt.
• Trained Model Link: https://huggingface.co/szabeen/smollm-gray-swan-classifier

Part 3: Evaluation & Discussion of Findings

Final Metrics (SmolLM2-360M LoRA Adapter)

• ROC AUC: 0.5537
• AUPR: 0.5549
• FPR @ 90% TPR: 0.8785
• FPR @ 95% TPR: 0.9159

Discussion

As anticipated given the compute constraints and resulting architectural choices, the 360M parameter model struggled significantly with the zero-shot generalization required for Gray Swan detection, yielding an AUC hovering near random chance (0.55).

Reviewing the 10-sample probability subset reveals the core issue: the model lacks the semantic depth required to separate benign academic/defensive queries from obfuscated malicious intent. For example, the model assigned a nearly identical probability of harm to a truly malicious exploit request (0.5293) and a benign defensive inquiry (0.4766).

Root Causes & Next Steps

1. Parameter Bottleneck: Gray Swan prompts are inherently designed to bypass shallow pattern matching by hiding malicious intent behind academic, hypothetical, or professional framing. A 360M parameter model does not have the representational capacity to resolve this level of linguistic ambiguity compared to a >7B parameter model.
2. Data Scarcity: The model was fine-tuned on ~150 high-fidelity synthetic samples. While sufficient to teach the model the format of the output, it is orders of magnitude too small to map the complex latent boundaries of the 7 specific CBRN/Exploitation policy classes.
3. Future Architecture: To achieve production-ready FPRs (<0.05) at 95% TPR, the pipeline requires scaling the CodecLM synthetic generation to >50,000 samples, utilizing a larger base model (e.g., Llama-3-8B), and employing Full Fine-Tuning or heavily regularized DPO (Direct Preference Optimization) rather than a low-rank adapter.