Assumption-Aware Audits: Non-Parametric Standards for Algorithmic Impact Assessments

Propose and evaluate an “assumption-aware” audit protocol requiring agencies and platforms to test distributional assumptions (normality, homoscedasticity, independence) in performance, bias, and drift analyses. If assumptions are violated, non-parametric or robust alternatives must be used and methodology impacts disclosed. This idea transfers insights from cybersecurity about invalid normality assumptions to governance audits, adding explicit assumption checks, pre-registered analysis plans with fallback tests (e.g., permutation tests, rank-based methods), and interpretability of effect sizes under non-parametric regimes. This improves the statistical integrity of user-impact studies, reduces false assurances of fairness or effectiveness, and tightens accountability. The impact includes adoption by standards bodies and regulators, with audit reports including assumption diagnostics and sensitivity analyses to improve trust and reproducibility.

References:

1. Challenging Assumptions of Normality in AES s-Box Configurations under Side-Channel Analysis. Clay Carper, Stone Olguin, Jarek Brown, Caylie Charlton, Mike Borowczak (2023). Journal of Cybersecurity and Privacy.