cyb001-baseline-classifier

CYB001 Baseline Classifier

Multi-class network flow classifier trained on the CYB001 synthetic network traffic sample. Predicts BENIGN, MALICIOUS, or AMBIGUOUS from per-flow features.

Baseline reference, not for production use. This model demonstrates that the CYB001 sample dataset is learnable end-to-end and gives prospective buyers a working starting point to evaluate against their own pipelines. It is not an intrusion detection system. See Limitations.

Model overview

Two model artifacts are published. They are designed to be used together — disagreement between them is itself a useful triage signal:

• model_xgb.json — gradient-boosted trees, primary recommendation
• model_mlp.safetensors — PyTorch MLP in SafeTensors format

Quick start

pip install xgboost torch safetensors pandas huggingface_hub

from huggingface_hub import hf_hub_download
import json, numpy as np, torch, xgboost as xgb
from safetensors.torch import load_file

REPO = "xpertsystems/cyb001-baseline-classifier"

# Download artifacts
paths = {n: hf_hub_download(REPO, n) for n in [
    "model_xgb.json", "model_mlp.safetensors",
    "feature_engineering.py", "feature_meta.json", "feature_scaler.json",
]}

# Make feature pipeline importable
import sys, os
sys.path.insert(0, os.path.dirname(paths["feature_engineering.py"]))
from feature_engineering import transform_single, load_meta, INT_TO_LABEL

meta = load_meta(paths["feature_meta.json"])
xgb_model = xgb.XGBClassifier(); xgb_model.load_model(paths["model_xgb.json"])

# Predict (see inference_example.ipynb for full single-record example)
X = transform_single(my_flow_record_dict, meta)
proba = xgb_model.predict_proba(X)[0]
print(INT_TO_LABEL[int(np.argmax(proba))])

See inference_example.ipynb for a full copy-paste demo including the MLP load path and a batch run on 200 rows from the public sample.

Training data

Trained on the public sample of CYB001, 9,770 flows with:

Split: 70 / 15 / 15 stratified by label, seed 42.

Class imbalance was addressed with class_weight='balanced' (XGBoost sample_weight) and weighted cross-entropy (MLP). Stratified splitting preserves the proportion in each fold.

Dataset calibration anchors

The CYB001 sample is calibrated to 12 named industry signatures. The features that surface most prominently in the baseline correspond to these anchors:

Full benchmark table in the dataset card.

Feature pipeline

The bundled feature_engineering.py is the canonical feature recipe. The training script and the inference example both call into it.

Three columns are deliberately excluded because they leak the label:

• traffic_category — perfectly deterministic of label (every attack_* category is 100% MALICIOUS, etc.).
• attack_subcategory — non-null iff label is MALICIOUS.
• attacker_capability_tier — generator metadata labeled per flow including benign flows; not a real-world observable at inference time.

Five session-level features were kept after a per-label leakage audit (payload_entropy_mean, retransmission_rate, protocol_violation_count, c2_beacon_flag, session_risk_score) because their distributions overlap meaningfully across labels (i.e. they behave like detector outputs, not oracles). Three were dropped (exfil_volume_bytes, scan_probe_count, lateral_move_flag) because they are zero for all non-MALICIOUS rows.

Engineered features (each encodes a stated domain hypothesis, see source for the one-line rationale per feature):

• iat_cv — inter-arrival-time coefficient of variation. C2 beacon signature.
• fwd_bwd_byte_ratio — exfiltration signature.
• bytes_per_packet_fwd, payload_density — flow shape.
• tcp_flag_anomaly_score — RST/URG/FIN density. Scan and protocol-misuse signature.
• hour_of_day, is_off_hours — diurnal pattern. APT and insider tiers are off-peak biased in the dataset calibration.
• is_well_known_dest_port, is_ephemeral_src_port — port observables.

Evaluation

Test-set metrics (n = 1,466, stratified)

XGBoost

MLP

Confusion matrices and per-class precision/recall are in validation_results.json.

Ablation: contribution of session-level features

To check whether the model is genuinely reading the flow-level signal or leaning on session aggregates, the same XGBoost configuration was trained with all five session-aggregate features removed:

The session join contributes about +1.0 pp of accuracy and +0.02 macro-F1. The model is not session-dominated; the flow-level features carry the bulk of the signal. The full numbers for both configurations are in ablation_results.json.

Architecture

XGBoost: multi-class gradient boosting (multi:softprob, 3 classes), hist tree method, class-balanced sample weights, early stopping on validation macro-F1.

MLP: n_features → 128 → 64 → 3, each hidden layer followed by BatchNorm1d → ReLU → Dropout(0.3), weighted cross-entropy loss, AdamW optimizer, early stopping on validation macro-F1.

Training hyperparameters (learning rate, batch size, n_estimators, early-stopping patience, weight decay, class-weighting strategy) are held internally by XpertSystems and are not part of this release.

Limitations

This is a baseline reference, not an intrusion detection system.

1. Performance is inflated by synthetic structure. The numbers above reflect performance on calibrated synthetic data where the BENIGN and attack categories sit on distinct statistical signatures by construction. A real production IDS facing live traffic must contend with concept drift, adversarial evasion, encrypted-traffic ambiguity, and a much fatter long tail of benign behaviour. Expect substantial degradation when transferring to real CICIDS-style datasets or in-the-wild traffic.

2. Sample size for AMBIGUOUS is small. Only 117 test examples; the per-class F1 has wide confidence bands. The full CYB001 product (~62k AMBIGUOUS flows out of ~500k) supports more reliable estimation.

3. Trained on the public 1/60th sample only. The full product contains additional traffic categories, longer sequences, and richer adversary behaviour. A model trained on the full dataset would perform differently — likely lower headline accuracy with better calibration and generalisation. The intent of this release is reference, not state-of-the-art.

4. Topology features are static labels, not signals. Fields like defender_architecture and firewall_policy are descriptive categorical attributes of the network segment, not learned defender responses. They help the model condition on context but do not simulate real adversarial dynamics.

5. MLP brittleness on OOD inputs. With ~7k training rows, the MLP can produce confidently-wrong predictions on hand-crafted records whose feature combinations are far from the training manifold. The inference notebook demonstrates this. XGBoost is more robust here. In practice, use both and treat disagreement as a signal for review.

6. Class imbalance handling is straightforward. Class-balanced weights work for this sample but production-scale rare-class detection (e.g. APT C2 at < 0.1% of traffic) needs more careful threshold calibration, ranking metrics, and likely calibrated probabilities rather than argmax classification.

Intended use

• Evaluating fit of the CYB001 dataset for your IDS / NDR research
• Baseline reference for new model architectures on synthetic network traffic
• Teaching and demo for tabular classification on flow-level features
• Feature engineering reference for CICFlowMeter-compatible fields

Out-of-scope use

• Production intrusion detection on real network traffic
• Forensic attribution of real attacks
• Adversarial robustness evaluation (the dataset is not adversarially generated)
• Any safety-critical decision

Reproducibility

Outputs above were produced with seed = 42, stratified 70/15/15 split, on the published sample (xpertsystems/cyb001-sample, version 1.0.0, generated 2026-05-16). The feature pipeline in feature_engineering.py is deterministic and the trained weights in this repo correspond exactly to the metrics above.

The training script itself is private to XpertSystems. The published artifacts contain the feature pipeline, model weights, scaler, metadata, and validation results — sufficient to reproduce inference but not training.

Files in this repo

Contact and full product

The full CYB001 dataset contains ~685,000 rows across four files with calibrated A+ benchmark validation. The full XpertSystems.ai synthetic data catalogue spans 41 SKUs across Cybersecurity, Healthcare, Insurance & Risk, Oil & Gas, and Materials & Energy.

• 📧 [email protected]
• 🌐 https://xpertsystems.ai
• 🗂 Dataset: https://huggingface.co/datasets/xpertsystems/cyb001-sample

Citation

@misc{xpertsystems_cyb001_baseline_2026,
  title  = {CYB001 Baseline Classifier: XGBoost and MLP for Synthetic Network Flow Classification},
  author = {XpertSystems.ai},
  year   = {2026},
  url    = {https://huggingface.co/xpertsystems/cyb001-baseline-classifier},
  note   = {Baseline reference model trained on xpertsystems/cyb001-sample}
}