Mapping the "Blind Spots": Visualizing and Correcting Positional Faithfulness Biases in Long Legal Summaries

Building on Wan et al. (2024), which uncovers a "U-shaped" pattern of faithfulness—where model summaries tend to be more faithful to the beginning and end of long documents while neglecting the middle—this project proposes a novel diagnostic and correction tool. The idea is to design an interactive visualization platform that highlights which sections of the source legal document are underrepresented or misrepresented in the summary (with a focus on the neglected "middle"). The system would analyze LLM outputs, provide feedback to both users and models, and even suggest targeted re-prompting or chunk re-ordering strategies (inspired by HERA, Li et al. 2025) to address detected blind spots. Unlike prior work, which primarily quantifies the bias or tests mitigation heuristics, this approach empowers both model developers and legal practitioners to iteratively correct and understand these positional faithfulness failures. The impact could be significant in both improving summary reliability and providing much-needed transparency for high-stakes legal settings.

References:

1. On Positional Bias of Faithfulness for Long-form Summarization. David Wan, Jesse Vig, Mohit Bansal, Shafiq R. Joty (2024). North American Chapter of the Association for Computational Linguistics.
2. HERA: Improving Long Document Summarization using Large Language Models with Context Packaging and Reordering. Taiji Li, Hao Chen, Fei Yu, Yin Zhang (2025). arXiv.org.