⚠ DRIFT DETECTED — Chunk 11 [ipsweep] — severity=1.00 — service_eco_i(15.3%) ✓ STABLE — Chunk 03 [normal] — error=0.0044 ⚠ DRIFT DETECTED — Chunk 24 — root_shell(27.7%) srv_diff_host_rate(11.1%) ✓ STABLE — Chunk 07 [normal] — error=0.0047 ⚠ DRIFT DETECTED — Chunk 19 [nmap] — severity=0.73 — dst_host_srv_count(9.2%) ✓ STABLE — Chunk 05 [normal] — error=0.0049 ⚠ DRIFT DETECTED — Chunk 11 [ipsweep] — severity=1.00 — service_eco_i(15.3%) ✓ STABLE — Chunk 03 [normal] — error=0.0044 ⚠ DRIFT DETECTED — Chunk 24 — root_shell(27.7%) srv_diff_host_rate(11.1%) ✓ STABLE — Chunk 07 [normal] — error=0.0047 ⚠ DRIFT DETECTED — Chunk 19 [nmap] — severity=0.73 — dst_host_srv_count(9.2%) ✓ STABLE — Chunk 05 [normal] — error=0.0049
VIGIL

Your model knows
when the data shifts.

No labels. No thresholds. Unsupervised concept drift detection for network traffic streams — with feature-level attribution that tells you exactly which signals changed.

$ pip install vigil-drift
kafka_pipeline/consumer.py — live output
Capabilities

Drift detection that
actually explains itself.

Most detectors tell you that something changed. Vigil tells you what changed and ranks it.

01
🔬
Unsupervised Detection
Replicated T-Test (r=15) on autoencoder reconstruction errors. Zero ground-truth labels needed. Works in real streams from day one.
NO LABELS REQUIRED
02
🆕
Novel Class Recognition
A frozen mirror autoencoder (A_KC) separates unknown attack classes from distribution shift — knows the difference between "changed" and "new".
DUAL AUTOENCODER
03
📊
Feature Attribution
DriftAttributor ranks input features by reconstruction error delta. Turns a drift alert into: "service_eco_i shifted 15.3%, dst_host changed 12.3%".
NOVEL CONTRIBUTION
04
🏭
Production Pipeline
FastAPI REST. Kafka stream. Airflow retraining DAGs with quality gates. MLflow tracking. Docker compose. Five services, one command.
MLOPS READY
0
Tests Passing
0
% Coverage
0
Docker Services
0
Airflow DAGs
0
NSL-KDD Features
Architecture

End-to-end.
Fully automated.

From raw network packets to automatic model retraining — Vigil orchestrates the entire pipeline.

📡
Kafka
Stream ingestion
🛡️
Vigil
Drift detection
📈
MLflow
Experiment tracking
✈️
Airflow
Auto-retraining
🖥️
SOC Dashboard
Live monitoring
Quick Start

Three lines.
Real detections.

main.py Python 3.11
# pip install vigil-drift

from vigil import Vigil

v = Vigil(feature_names=your_feature_names)
v.fit(baseline_traffic)          # offline phase

for batch in live_stream:
    result = v.detect(batch)

    if result.drift_detected:
        print(f"⚠  severity={result.drift_severity:.2f}")

        for f in result.attribution.top_features:
            print(f"   {f['feature_name']}: {f['contribution']:.1%}")

# ⚠  severity=1.00
# → service_eco_i              15.3%   (port scan signature)
# → dst_host_same_src_port_rate 12.3%   (scanning pattern)
# → srv_diff_host_rate          11.1%   (lateral movement)
Tech Stack

Industry tools.
Real ML.

PyTorch
scikit-learn
SciPy
NumPy
FastAPI
Pydantic
Uvicorn
MLflow
Apache Kafka
Apache Airflow
Docker
Streamlit
Plotly
GitHub Actions
pytest
Ruff
NSL-KDD Dataset
Open Source · MIT

Clone it.
Break it.
Build on it.

Everything is on GitHub. The algorithm, the API, the Airflow DAGs, the Kafka pipeline, the dashboard. All of it.