Behavioral Drift Detection
Objective
Prevent undetected model degradation causing business loss.
Control / requirement
Baseline profiling + KL divergence monitoring + accuracy tracking.
Business impact
Failure to implement this control creates risk of model performance degradation without detection, with estimated financial exposure of $500k-$2M per incident based on IBM Cost of a Data Breach Report 2025.
Validation approach
Test ID: D1-CTL-03-VTS-001 Test Type: Automated Test Design: Monitor model predictions over 30-day period; calculate KL divergence against established baseline Execution Steps: 1. Establish 7-day baseline of model outputs on production traffic 2. Configure daily drift detection job: pytest tests/d1_model_integrity/test_behavioral_drift.py -v # oda3-gaissf-vts — schedule daily 3. Run for 30 days 4. Generate report with drift_events and max_kl_divergence Pass Criteria: max_kl_divergence < 0.05; accuracy_drop < 5% over 30 days; alert_generated_for_any_drift_event = True Independent Verification: Auditor reviews 30-day drift log and verifies alert generation. Re-runs drift calculation on sample of organisation's data.
Expected evidence
Not separately specified in the available source.
Mapping and source
Not separately specified in the available source.
Implementation guidance
Translate the requirement into system-specific procedures, responsible roles, technical configurations and review conditions. Retain evidence showing both design and operating performance. Where the source provides no separate implementation instruction, do not infer that a single technical mechanism is sufficient.
Assessment considerations
- Confirm scope and applicability.
- Inspect control design and responsible ownership.
- Test representative operation and adverse conditions where appropriate.
- Evaluate evidence provenance, completeness and contradictory evidence.
- Record limitations and notably absent outcomes.