Non-Invasive Fairness Drift Monitor for Machine Learning Models Using Conformance Constraints
Microsoft Teams
Dr. Gokhan Kul
gkul@umassd.edu
https://teams.microsoft.com/meet/245292205330763?p=xBhMejmhVBcZHQGSk6
Thesis Advisor: Dr. Gökhan Kul, Computer and Information Science
Committee Members: Dr. Joshua Carberry, Computer and Information Science and Dr. Yuchou Chang, Computer and Information Science
Abstract: Machine learning models deployed in consequential domains can become unfair toward protected subgroups as the data they receive drifts over time, yet the protected attributes needed to measure fairness directly are often unavailable at runtime due to privacy regulation and operational constraints. This creates a gap: existing fairness toolkits require protected labels and perform one-time audits, while generic drift detectors monitor continuously but cannot localize which subgroup a shift harms. This thesis develops a non-invasive fairness drift monitor that addresses this gap by repurposing Conformance Constraints, a data-profiling primitive, as a temporal fairness signal. The monitor learns per-subgroup distributional profiles at baseline, using protected attributes only once, and thereafter tracks violation of those profiles over incoming data batches without any runtime access to protected attributes. Across three fairness benchmarks, two classifiers, and nineteen controlled drift scenarios, the conformance-constraint violation signals track fairness degradation more closely than KS and KL detectors under global drift, and they remain competitive with them under group-targeted drift. Subgroup localization provides its clearest advantage under global drift, where a minority-subgroup signal substantially outperforms both aggregate signals and the baselines. The correlations are modest in absolute terms, indicating that the monitor functions as a screening instrument that flags fairness degradation for closer investigation rather than as a precise estimator. The approach offers privacy-preserving, subgroup-aware fairness monitoring suited to regulated deployment settings.
For further information please contact Dr. Gokhan Kul at gkul@umassd.edu.