Title: Unlocking the Power of Mixed-Precision Spatial Compute in the AMD Ryzen™ AI NPU
Speakers:
Gagandeep Singh, Kristof Denolf & Alireza Khodamoradi, AMD, Research and Advanced Development Group: https://www.amd.com/en/corporate/rese...

SAFARI-EFCL Live Seminar Talk: https://safari.ethz.ch/safari-efcl-se...

Slides (pdf): 
Slides (pptx): 

Abstract: In today’s tech landscape, specialized hardware accelerators, such as Neural Processing Units (NPUs) in consumer laptops with AMD Ryzen™ AI CPUs, are widely available. The NPU on AMD Ryzen™ AI features an AI Engine array composed of VLIW vector processors, data movement accelerators (DMAs), and adaptable interconnect. The latest generation, Strix, enhances NPU capabilities by adding support for the MXINT8 datatype. Our work leverages these advanced features through IRON (Interface Representations for hands-ON programming of Fast and Efficient AIE designs), an open-source toolkit that provides close-to-metal programming via Python language bindings around the mlir-aie dialect. IRON empowers performance engineers to maximize the potential of these NPUs by simplifying the creation of fast and efficient designs.

This presentation will provide insights into the AI Engine’s compute and data movement capabilities supported in our tool flow; and demonstrate performance optimizations of increasingly complex designs, including machine learning models, weather forecasting simulations, and genome sequencing. We emphasize the importance of balancing workloads across available processing resources while selecting the optimal datatype is crucial to achieve high performance and accuracy on spatial architectures.

Speaker Bios:
Gagandeep Singh is a Researcher at AMD’s Research and Advanced Development Group, focusing on hardware acceleration and performance modeling. Prior to joining AMD, he was a Postdoctoral Researcher at ETH Zürich in SAFARI Research Group. He received his Ph.D. from TU Eindhoven in collaboration with IBM Research Zürich in 2021. In 2017, Gagan received a joint M.Sc. degree with distinction in Integrated Circuit Design from TUM, Germany, and NTU, Singapore. He did his master’s thesis at IMEC, Belgium, on architecture modeling and design space exploration for 3D stacked interconnect technology, for which he received the best thesis award. Gagan was also an R&D Software Developer at Oracle, India. He has published several research papers in prestigious conferences and journals, including ISCA, MICRO, IEEE Micro, Genome Biology, and Bioinformatics. He is passionate about computer architecture, hardware acceleration, and machine learning.

Kristof Denolf is a Fellow in AMD’s Research and Advanced Development group where he is working on energy-efficient computer vision and video processing applications to shape future AMD devices. He earned an M.Eng. in electronics from the Katholieke Hogeschool Brugge-Oostende (1998), now part of KULeuven, an M.Sc. in electronic system design from Leeds Beckett University (2000), and a Ph.D. from the Technical University Eindhoven (2007). He has over 25 years of combined research and industry experience at IMEC, Philips, Barco, Apple, Xilinx, and AMD. His main research interests are all aspects of the cost-efficient and dataflow-oriented design of video, vision, and graphics systems.

Alireza Khodamoradi is a senior member of AMD Research and Development Group, working on energy-efficient signal processing and AI applications to shape future AMD devices. He received a B.Sc. in electrical engineering from Azad University, Iran, in 2002, M.Sc. in electrical engineering from the University of Kerman, Iran, in 2008, MAS in wireless embedded systems from UC San Diego in 2015, and his Ph.D. in computer engineering from UC San Diego in 2021. His main research interests are signal processing, hardware acceleration for vision and AI applications, and neural model compression techniques.

Past SAFARI Live Seminars: https://safari.ethz.ch/safari-seminar...