nids-model-cnn-bilstm-ae

Hybrid CNN-BiLSTM Autoencoder for Network Intrusion Detection System (NIDS)

This model is a Hybrid CNN-BiLSTM Autoencoder designed for anomaly-based Network Intrusion Detection. It extracts spatial representations from packet features using 1D Convolution layers and captures temporal sequences of network flows using Bidirectional LSTM layers.

The model was adapted to real-world traffic using a Three-Phase Training scheme (Pre-training on CIC-IDS 2017 -> Distillation & Margin adaptation on CSE-CIC-IDS 2018 -> Domain Adaptation on 24h Real Benign Traffic).

Model Details

• Developer: Akwan Cakra Tajimalela
• Model Type: Unsupervised Anomaly Detector (Autoencoder)
• Input Dimension: 25 network flow features, sequenced with a temporal window of T = 10 flows.
• Output: Reconstruction of input features (Reconstruction MAE is used as anomaly score).
• Framework: TensorFlow 2.10 (Keras backend)

---

How to Use

To use this model for inference, download the weights hybrid_nids_v13_fixed_v2.keras and the preprocessing parameters scaler_params.pkl.

import joblib
import numpy as np
import pandas as pd
import tensorflow as tf

# 1. Load model and scaler
model = tf.keras.models.load_model("hybrid_nids_v13_fixed_v2.keras")
scaler = joblib.load("scaler_params.pkl")

# 2. Define the 25 features required
FEATURES = [
    "Init Fwd Win Byts", "Bwd Pkts/s", "Init Bwd Win Byts", "Pkt Size Avg", 
    "Dst Port", "Bwd Pkt Len Max", "Bwd IAT Tot", "Flow IAT Min", 
    "Flow IAT Mean", "Fwd Pkts/s", "Flow IAT Std", "Fwd Seg Size Avg", 
    "TotLen Fwd Pkts", "Flow Byts/s", "Bwd Pkt Len Min", "Bwd Pkt Len Std", 
    "Fwd IAT Min", "FIN Flag Cnt", "SYN Flag Cnt", "Fwd IAT Std", 
    "Subflow Bwd Byts", "Down/Up Ratio", "Pkt Len Std", "Fwd Pkt Len Std", 
    "Fwd Act Data Pkts"
]

# 3. Create dummy flow data (minimum 10 rows for sequence context)
df = pd.DataFrame(np.random.rand(15, len(FEATURES)), columns=FEATURES)
df["Dst Port"] = np.random.randint(0, 65535, size=15)

# 4. Preprocess: scale features
X_scaled = scaler.transform(df[FEATURES].values)

# 5. Create sequences (Window size T = 10)
T = 10
n_seq = len(X_scaled) - T + 1
X_seq = np.empty((n_seq, T, len(FEATURES)), dtype=np.float32)
for i in range(n_seq):
    X_seq[i] = X_scaled[i : i + T]

# 6. Run inference & calculate reconstruction error
reconstruction = model.predict(X_seq)
errors = np.mean(np.abs(X_seq - reconstruction), axis=(1, 2))

# 7. Apply Dual Thresholds
THR_WARNING = 0.05831   # Warning: FPR ~1.0%
THR_CRITICAL = 0.07033  # Critical: FPR ~0.17%

for idx, error in enumerate(errors):
    flow_idx = idx + T - 1
    alert = "NORMAL"
    if error > THR_CRITICAL:
        alert = "CRITICAL"
    elif error > THR_WARNING:
        alert = "WARNING"
        
    print(f"Flow {flow_idx} -> Reconstruction Error: {error:.5f} | Status: {alert}")

---

Training Data & Procedure

Features (25 selected)

The model was trained on 25 network features selected from 77 canonical features of CIC datasets using a robust pipeline of:

1. Variance threshold filtering.
2. Correlation filter.
3. LightGBM proxy importance ranking.

Three-Phase Training Scheme

1. Phase 1 (Pre-training): Trained on benign network flows from CIC-IDS 2017 (~200k flows) to learn benign behaviors.
2. Phase 2 (Distillation & Boundary adaptation): Fine-tuned on CSE-CIC-IDS 2018 using Knowledge Distillation and an Attack Replay Margin Loss to handle distribution shift while preserving anomaly bounds.
3. Phase 3 (Domain Adaptation): Decoders adapted to 24h Real Benign Traffic (~106k flows) while freezing the encoder. Objective: minimize benign reconstruction error while retaining attack sensitivity via a hinge adaptation loss.

---

Evaluation Results

Calibrated thresholds evaluated on real-world validation data:

• WARNING Threshold (0.05831): Calibrated for balanced detection. FPR ~1.0% on real benign validation set.
• CRITICAL Threshold (0.07033): Calibrated for high precision. FPR ~0.17% on real benign validation set.

---

Citation

If you use this model or code in your research, please cite:

@thesis{nids_hybrid_cnn_bilstm_ae_2026,
  author       = {Akwan Cakra Tajimalela},
  title        = {Anomaly-based Network Intrusion Detection System using Hybrid CNN-BiLSTM Autoencoder and Domain Adaptation},
  school       = {Universitas Pendidikan Indonesia},
  year         = {2026},
  type         = {Undergraduate Thesis}
}