Database Agnostic Regex & AI Powered Personal Identifying Information Anonymization Pipeline
Microsoft Teams
Dr. Gokhan Kul
gkul@umassd.edu
https://teams.microsoft.com/meet/225470078366318?p=4hWIV8w4Us9lVFoi5g
Thesis Advisor: Dr. Gokhan Kul - Computer & Information Science
Committee Members:
- Dr. Iren Valova - Computer & Information Science/Associate Dean, College of Engineering
- Dr. Joshua Carberry - Computer & Information Science
Abstract: Survey research depends on respondents disclosing information that is identifying by design. Health studies require diagnoses and medication histories, labor studies require employer names and income, and social science studies require demographic and immigration status. This necessity creates the protection problem. Virtually all federally funded human subjects research is governed by the Common Rule (45 CFR §46) and IRB oversight, with sector-specific statutes such as HIPAA, GDPR Article 9, CCPA, FERPA, GINA, GLBA, and others, layering additional obligations depending on institutional context. A single survey spanning health, financial, and demographic questions may trigger several frameworks at once, and even absent a specific statute, research ethics principles require protecting respondents from re-identification. Anonymization resolves this by preserving the analytical utility while removing identifying elements, but existing tools force a poor choice between regex pattern matching that misses contextual and combinatorial risk, and cloud-hosted AI that cannot legally or ethically process PHI-adjacent content.
This thesis presents a database-agnostic anonymization pipeline that evaluates three detection methods under controlled, reproducible conditions: a regex-only detector, an AI-only detector using a locally hosted Ollama model for contextual risk assessment, and a hybrid detector that merges both signals via an escalation-only design, always selecting the higher-risk classification. Medical and PHI-adjacent content is routed exclusively to local models; the pipeline operates uniformly across MongoDB, SQL, and file-based sources through a shared interface; and every classification maps to a four-tier anonymization-action framework (suppress, pseudonymize, generalize, keep) grounded in U.S. privacy law rather than abstract sensitivity alone.
Evaluated against a 300-question ground-truth dataset spanning PII, medical, and benign content, and validated against two independent external AI annotators (Claude and GPT, which agreed with each other on 88.3% of labels, kappa = 0.850), the three pipeline detectors showed vastly different performance profiles. The regex-only detector achieved the highest overall accuracy among pipeline strategies (57.0%) and near-perfect benign recall, but systematically under-classified RELAXED and MODERATE content and under-flagged 31.5% of high-risk fields. The local AI-only detector (llama3.1:8b) reached 47.3% overall accuracy and under-flagged 57.5% of high-risk fields, the worst of the three, but demonstrated complementary value by catching contextual risk regex missed, including two STRICT financial identifiers regex scored only MODERATE. The hybrid escalation only detector reached 46.7% overall accuracy while reducing high risk under-flagging to 26.0%, the lowest of any pipeline detector, validating the escalation only design principle. External annotators substantially outperformed all three pipeline detectors (76.7% and 76.0% overall accuracy, with only 9.6% and 13.7% high-risk under-flagging), with the largest gap concentrated in medical content (16–20% versus 48–50%)—confirming that the models best suited to sensitive content are precisely the ones that cannot legally be used on it.
The resulting pipeline is intended for researchers, institutional review boards, and data stewards who must anonymize survey data before storage or sharing but cannot rely on cloud-hosted AI for regulatory or ethical reasons. Because detectors are interchangeable behind a common interface, institutions can adopt regex-only, AI-only, or hybrid mode as a configuration decision—trading speed and infrastructure cost against detection sensitivity—rather than a redesign.
For further information please contact Dr Gokhan Kul at gkul@umassd.edu.