Abstract
In this talk, we examine how high-performance computing has changed over the last 10 years and look at trends in the future. These changes have had and will continue to impact our software significantly. Some of the software and algorithm challenges have already been encountered, such as management of communication and memory hierarchies through a combination of compile-time and run-time techniques, but the increased scale of computation, depth of memory hierarchies, range of latencies, and increased run-time environment variability will make these problems much harder.
Mixed precision numerical methods are paramount for increasing the throughput of traditional and artificial intelligence (AI) workloads beyond riding the wave of the hardware alone. Reducing precision comes at the price of trading away some accuracy for performance (reckless behaviour) but in noncritical segments of the workflow (responsible behaviour) so that the accuracy requirements of the application can still be satisfied.
