CPAMA: Cross-Precision Adaptive Matrix Algorithms for Dynamic Hardware-Software Co-Optimization

While mixed-precision LU factorization (2023) showed benefits of using multiple precisions, current approaches use static precision assignments. CPAMA would create algorithms that continuously monitor numerical error accumulation and dynamically adjust precision at the granularity of individual matrix blocks or even operations. This goes beyond the mixed-precision survey (2021) by making precision adaptation truly dynamic rather than predetermined. The system would use lightweight error estimators inspired by the Riemann-Hilbert perturbation theory (2021) to determine when higher precision is needed. Unlike Amber's (2024) fixed hardware acceleration, CPAMA would work with existing heterogeneous systems, intelligently routing operations to appropriate compute units based on precision requirements. The innovation lies in treating precision as a first-class resource that can be allocated and deallocated dynamically, similar to how PARALiA (2023) treats computational resources. This could reduce memory usage by 30-50% while maintaining accuracy, particularly valuable for memory-bound operations on large-scale systems.

References:

1. PARALiA: A Performance Aware Runtime for Auto-tuning Linear Algebra on Heterogeneous Systems. Petros Anastasiadis, Nikela Papadopoulou, G. Goumas, N. Koziris, Dennis Hoppe, Li Zhong (2023). ACM Transactions on Architecture and Code Optimization (TACO).
2. A survey of numerical linear algebra methods utilizing mixed-precision arithmetic. A. Abdelfattah, H. Anzt, E. Boman, E. Carson, T. Cojean, J. Dongarra, Alyson Fox, Mark Gates, N. Higham, X. Li, J. Loe, P. Luszczek, S. Pranesh, S. Rajamanickam, T. Ribizel, Barry Smith, K. Swirydowicz, Stephen J. Thomas, S. Tomov, Y. Tsai, U. Yang (2021). The international journal of high performance computing applications.
3. Amber: A 16-nm System-on-Chip With a Coarse- Grained Reconfigurable Array for Flexible Acceleration of Dense Linear Algebra. Kathleen Feng, Taeyoung Kong, Kalhan Koul, Jackson Melchert, Alex Carsello, Qiaoyi Liu, Gedeon Nyengele, Maxwell Strange, Kecheng Zhang, Ankita Nayak, Jeff Setter, James J. Thomas, Kavya Sreedhar, Po-Han Chen, Nikhil Bhagdikar, Zachary Myers, Brandon D'Agostino, Pranil Joshi, Stephen Richardson, Christopher Torng, Mark Horowitz, Priyanka Raina (2024). IEEE Journal of Solid-State Circuits.
4. A Riemann-Hilbert approach to the perturbation theory for orthogonal polynomials: Applications to numerical linear algebra and random matrix theory. Xiucai Ding, T. Trogdon (2021). International mathematics research notices.
5. Mixed precision LU factorization on GPU tensor cores: reducing data movement and memory footprint. Florent Lopez, Théo Mary (2023). The international journal of high performance computing applications.