HPMF: Hierarchical Performance Modeling Framework for Cross-Architecture Linear Algebra Optimization

Current performance modeling approaches like Kaizen Programming (2022) and the Cray XE models (2014) focus on specific levels or architectures. HPMF would create a unified framework with models at multiple abstraction levels: from analytical communication models (inspired by Ballard et al.'s work) to machine learning-based performance predictors (extending Laberge et al.'s supervised learning approach). The key innovation is the hierarchical integration - higher-level models guide lower-level ones, creating a coherent prediction system. Unlike existing approaches that either use pure analytical models or pure machine learning, HPMF would combine both, using analytical models to provide constraints and machine learning to capture complex behaviors. The framework would be extensible to new architectures like Amber's (2024) CGRA and would include uncertainty quantification for its predictions. This could improve prediction accuracy from 70-80% (current state) to over 95% while providing insights into why certain performance behaviors occur, enabling more informed algorithmic choices.

References:

1. Amber: A 16-nm System-on-Chip With a Coarse- Grained Reconfigurable Array for Flexible Acceleration of Dense Linear Algebra. Kathleen Feng, Taeyoung Kong, Kalhan Koul, Jackson Melchert, Alex Carsello, Qiaoyi Liu, Gedeon Nyengele, Maxwell Strange, Kecheng Zhang, Ankita Nayak, Jeff Setter, James J. Thomas, Kavya Sreedhar, Po-Han Chen, Nikhil Bhagdikar, Zachary Myers, Brandon D'Agostino, Pranil Joshi, Stephen Richardson, Christopher Torng, Mark Horowitz, Priyanka Raina (2024). IEEE Journal of Solid-State Circuits.
2. Minimizing Communication in Numerical Linear Algebra. Grey Ballard, J. Demmel, Olga Holtz, O. Schwartz (2009). SIAM Journal on Matrix Analysis and Applications.
3. Kaizen Programming for predicting numerical linear algebra operations performance. J. Ferreira, E. Dufrechou, M. Pedemonte (2022). Latin American Conference on Computational Intelligence.
4. Constructing Performance Models for Dense Linear Algebra Algorithms on Cray XE Systems. J. González-Domínguez, E. Georganas, Yili Zheng, María J. Martín (2014). arXiv.org.
5. Scheduling Optimization of Parallel Linear Algebra Algorithms Using Supervised Learning. Gabriel Laberge, S. Shirzad, Patrick Diehl, Hartmut Kaiser, S. Prudhomme, Adrian S. Lemoine (2019). Workshop on Machine Learning in High Performance Computing Environments.