EPFL algorithm in the world's most popular deep learning software

Artificial Intelligence technology © 2021 4X-image - PyTorch is used for countless AI applications ranging from Tesla's Autopilot to Facebook's translation software and its latest version, just launched, features an EPFL communication-efficient training algorithm that even helps the planet. Deep learning is part of a broader family of machine learning methods, based on artificial neural networks, and has allowed us to develop everything from voice and image recognition tools, enhance drug discovery and toxicology, and improve financial fraud detection. As the applications of machine learning become bigger, more complex and increasingly ubiquitous in our modern, digital age, neural networks have grown tremendously in size, consisting of trillions of connections. To train these models faster, researchers typically distribute the training effort over many computers or Graphics Processing Units yet, just like humans that collaborate to solve a task, collaborating computers also suffer from communication overhead. "Because the neural networks that are trained are so large, the communication required between computers to achieve an accurate model can amount to many PetaBytes. Researchers have long been trying to find ways to compress the bandwidth needed while still allowing accurate training," said Martin Jaggi, Head of the Machine Learning and Optimization Laboratory (MLO), part of the School of Computer and Communications Sciences (IC). New EPFL algorithm developed  PowerSGD is an algorithm developed by PhD students, Thijs Vogels and Sai Praneeth Karimireddy, who work with Professor Jaggi.