Beyond Black-Box Audits: Creating Modular, Explainable ‘White-Box’ Audit Kits for Open-Source AI

Casper et al. (2024) highlight the limitations of black-box audits and the need for “white-” and “outside-the-box” access for rigorous oversight. Yet, most open-source projects (Chakraborti et al., 2024) lack the resources or incentives to implement robust audit trails. This research proposes modular, open-source audit toolkits that provide explainable, reproducible audit routines (e.g., fairness, bias, data lineage, and performance drift) and are easy to plug into common open-source development workflows (such as those on Hugging Face). By standardizing “auditability by design” and making it accessible to resource-constrained projects, this could democratize best-practice auditing and close the accountability gap in rapidly evolving open ecosystems. The innovation here is in lowering the barrier for robust, explainable audits—supporting both technical rigor and societal transparency.

References:

1. Black-Box Access is Insufficient for Rigorous AI Audits. Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Ben Bucknall, Andreas A. Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, Lee Sharkey, Satyapriya Krishna, Marvin von Hagen, Silas Alberti, Alan Chan, Qinyi Sun, Michael Gerovitch, David Bau, Max Tegmark, David Krueger, Dylan Hadfield-Menell (2024). Conference on Fairness, Accountability and Transparency.
2. Responsible AI in Open Ecosystems: Reconciling Innovation with Risk Assessment and Disclosure. Mahasweta Chakraborti, Bert Joseph Prestoza, Nicholas Vincent, Seth Frey (2024). arXiv.org.