Hierarchical Recursive Memory: Fusing Brain-Inspired Memory Architectures with Recursive Language Models

TL;DR: What if RLMs could remember like the human brain, not just by splitting prompts, but by storing and recalling at different abstraction levels? Let’s try to build a recursive LLM that, at each recursion, stores summarized memories at multiple granularities, just like a Hierarchical Memory Transformer (HMT), and see if it helps with even longer or more complex prompts. An initial experiment would compare standard RLM decomposition with a version that recursively builds and queries hierarchical memory banks, measuring both accuracy and memory footprint.

Research Question: Can integrating hierarchical, multi-level memory architectures into Recursive Language Models improve long-context understanding and efficiency, particularly for tasks requiring information synthesis across distant or abstract segments of input?

Hypothesis: RLMs equipped with a hierarchical memory structure, inspired by the Hierarchical Memory Transformer (He et al., 2024), will outperform traditional flat-memory RLMs on tasks that demand multi-level abstraction and cross-segment reasoning, while maintaining or improving efficiency.

Experiment Plan: - Setup: Extend an RLM framework to build and maintain a hierarchical memory bank: for each recursive call, summaries are created at different levels (sentence, paragraph, section).

• Data: Use long-context benchmarks such as NoCha (Karpinska et al., 2024) and LooGLE (Li et al., 2023).
• Measurements: Compare task accuracy, memory usage, and retrieval efficiency versus standard RLMs and HMTs.
• Expected Outcomes: Hierarchical memory RLMs should show improved synthesis and reasoning over global document context, as well as lower memory overhead for similarly accurate results.

References:

• He, Z., Cao, Y., Qin, Z., Prakriya, N., Sun, Y., & Cong, J. (2024). HMT: Hierarchical Memory Transformer for Efficient Long Context Language Processing. North American Chapter of the Association for Computational Linguistics.
• Karpinska, M., Thai, K., Lo, K., Goyal, T., & Iyyer, M. (2024). One Thousand and One Pairs: A “novel” challenge for long-context language models. Conference on Empirical Methods in Natural Language Processing.
• Li, J., Wang, M., Zheng, Z., & Zhang, M. (2023). LooGLE: Can Long-Context Language Models Understand Long Contexts? Annual Meeting of the Association for Computational Linguistics.