Hardware-ML Co-Optimization for Unexpected Bottlenecks

Fayyazi et al.’s ARCO framework and Liu et al.’s unified buffer show promise in hardware/software co-design, but neither addresses unexpected bottlenecks (e.g., memory fragmentation from dynamic shapes). This idea extends ARCO by training a multi-agent RL system where one agent optimizes compiler passes (e.g., fusion, quantization) while another tunes hardware parameters (e.g., buffer sizes). Crucially, the agents would be penalized for variance in performance across batches—not just average latency—addressing the instability observed in BladeDISC’s dynamic shape handling. This differs from TVM’s static cost models by adapting to runtime irregularities, inspired by the unexpected temperature build-up in Song et al.’s FCHEV work. The result: a resilient compiler that smooths out performance cliffs before they occur.

References:

1. Safety and Longevity-Enhanced Energy Management of Fuel Cell Hybrid Electric Vehicle With Machine Learning Approach. Ruoyang Song, Xinghua Liu, Zhongbao Wei, Fengwen Pan, Yanbo Wang, Hongwen He (2024). IEEE Transactions on Transportation Electrification.
2. ARCO:Adaptive Multi-Agent Reinforcement Learning-Based Hardware/Software Co-Optimization Compiler for Improved Performance in DNN Accelerator Design. Arya Fayyazi, M. Kamal, M. Pedram (2024). arXiv.org.
3. Unified Buffer: Compiling Image Processing and Machine Learning Applications to Push-Memory Accelerators. Qiaoyi Liu, Jeff Setter, Dillon Huff, Maxwell Strange, Kathleen Feng, M. Horowitz, Priyanka Raina, Fredrik Kjolstad (2022). ACM Transactions on Architecture and Code Optimization (TACO).
4. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. Tianqi Chen, T. Moreau, Ziheng Jiang, Haichen Shen, Eddie Q. Yan, Leyuan Wang, Yuwei Hu, L. Ceze, Carlos Guestrin, A. Krishnamurthy (2018).
5. BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach. Zhen Zheng, Zaifeng Pan, Dalin Wang, Kai Zhu, Wenyi Zhao, Tianyou Guo, Xiafei Qiu, Minmin Sun, Junjie Bai, Feng Zhang, Xiaoyong Du, Jidong Zhai, Wei Lin (2023). Proc. ACM Manag. Data.