Glass-Box Hypothesis Testing: Integrating Model Internals with External Evaluation

Huang et al. (2024) demonstrate the promise of “self-evaluation” using model-internal features, but this is largely decoupled from explicit hypothesis testing about reasoning processes. This proposal bridges that gap: for each evaluation hypothesis (e.g., “the model reasons via deductive logic on this task”), the framework not only scores outputs but also probes internal activation patterns, attention maps, or softmax distributions for alignment with expected reasoning signatures. For instance, tasks hypothesized to require inductive reasoning should correspond to distributed, pattern-seeking representations, while deductive tasks might show more focused, rule-based activations. This cross-modal hypothesis testing creates a much richer evaluation signal and could help explain why LLMs succeed or fail on certain tasks, moving beyond output “black boxes” and towards interpretable, scientifically grounded model assessments.

References:

1. Self-Evaluation of Large Language Model based on Glass-box Features. Hui Huang, Yingqi Qu, Jing Liu, Muyun Yang, Tiejun Zhao (2024). Conference on Empirical Methods in Natural Language Processing.