Reproducible Performance Engineering for Heterogeneous Accelerators

This methods review examines reproducible performance engineering for heterogeneous accelerators. The organizing question is which experimental controls make performance comparisons portable across hardware and software stacks. Ten related scholarly sources are synthesized through a decision-centered framework spanning problem definition, mechanism, measurement, evaluation, implementation, and governance. The review does not invent experiments, pooled estimates, or unreported quantitative results. It instead evaluates the strength and transferability of the available evidence, with particular attention to presenting a narrowly tuned speedup as a generally reproducible systems result. The resulting framework links technical or empirical performance to explicit use conditions and identifies tests that should precede wider adoption in high-performance and scientific computing evaluations.

1. Abadal, S., Jain, A., Guirado, R., López-Alonso, J., & Alarcón, E. (2021). Computing Graph Neural Networks: A Survey from Algorithms to Accelerators. ACM Computing Surveys, 54(9), 1-38. https://doi.org/10.1145/3477141 DOI
2. Bridges, R. A., Imam, N., & Mintz, T. M. (2016). Understanding GPU Power. ACM Computing Surveys, 49(3), 1-27. https://doi.org/10.1145/2962131 DOI
3. Caviedes-Voullième, D., Morales-Hernández, M., Norman, M. R., & Özgen-Xian, I. (2023). SERGHEI (SERGHEI-SWE) v1.0: a performance-portable high-performance parallel-computing shallow-water solver for hydrology and environmental hydraulics. Geoscientific model development, 16(3), 977-1008. https://doi.org/10.5194/gmd-16-977-2023 DOI
4. Deelman, E., Peterka, T., Altintas, I., Carothers, C. D., Dam, K. K. V., Moreland, K., Parashar, M., Ramakrishnan, L., Taufer, M., & Vetter, J. (2017). The future of scientific workflows. The International Journal of High Performance Computing Applications, 32(1), 159-175. https://doi.org/10.1177/1094342017704893 DOI
5. Humble, T. S., McCaskey, A., Lyakh, D. I., Gowrishankar, M., Frisch, A., & Monz, T. (2021). Quantum Computers for High-Performance Computing. IEEE Micro, 41(5), 15-23. https://doi.org/10.1109/mm.2021.3099140 DOI
6. Pandey, M., Fernandez, M., Gentile, F., Isayev, O., Tropsha, A., Stern, A. C., & Cherkasov, A. (2022). The transformational role of GPU computing and deep learning in drug discovery. Nature Machine Intelligence, 4(3), 211-221. https://doi.org/10.1038/s42256-022-00463-x DOI
7. Shalf, J. (2020). The future of computing beyond Moore’s Law. Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences, 378(2166), 20190061. https://doi.org/10.1098/rsta.2019.0061 DOI
8. Silva, C. A., Vilaça, R., Pereira, A., & Bessa, R. J. (2023). A review on the decarbonization of high-performance computing centers. Renewable and Sustainable Energy Reviews, 189, 114019. https://doi.org/10.1016/j.rser.2023.114019 DOI
9. Varghese, B., Wang, N., Bermbach, D., Hong, C. H., Lara, E. D., Shi, W., & Stewart, C. (2021). A Survey on Edge Performance Benchmarking. ACM Computing Surveys, 54(3), 1-33. https://doi.org/10.1145/3444692 DOI
10. Ye, Z., Gao, W., Hu, Q., Sun, P., Wang, X., Luo, Y., Zhang, T., & Wen, Y. (2023). Deep Learning Workload Scheduling in GPU Datacenters: A Survey. ACM Computing Surveys, 56(6), 1-38. https://doi.org/10.1145/3638757 DOI