Every year, the computers and devices in our lives become smaller, faster, more versatile and affordable. And for many years, the pace of that innovation has been measured by the capabilities of computing hardware — the speed of the processor, the amount of memory, pixel density or the size and weight of the device.
But the sum of the computing experience is about more than hardware. It also demands software that enables amazing and useful experiences. And for software to do that, it needs to be able to take advantage of new coding techniques and software libraries — as well as new features built into hardware — whether that’s utilizing multi-core processors, taking advantage of the GPU to process visual data or enabling new power-efficiency and security features.
When software developers fail to optimize their code for the latest technology, it results in something called bit-rot — the idea that software gets increasingly out of date and seemingly slower as technology advances.
“It’s really hard for developers to keep up with the pace of technology evolution in CPUs and GPUs,” notes Shoaib Kamil, former research scientist at MIT now working as a senior research scientist in Adobe’s Creative Technology Lab. That’s especially true for a high-performance, multi-faceted software product like Photoshop, with a 25-year legacy. Optimizing a complex software product might take months and require thousands of developer hours to update millions of lines of code.
But thanks to new technology being developed at MIT in collaboration with Adobe, called Project Helium, software developers everywhere may soon have a new way to update and improve the performance of software just as quickly as the hardware evolves.
The goal of Project Helium is to see if complex software can be analyzed and optimized automatically, without requiring the intervention of software engineers. “We wanted to see if we could solve the problem of bit-rot,” says Sylvain Paris, principal research scientist at Adobe. “We wanted to automate the process of keeping software up-to-date.”
“We also wanted to see if a highly optimized program like Photoshop could be made to go even faster,” notes Shoaib.
To test the idea, Adobe and MIT researchers decided to focus on the optimization of Photoshop filters. But rather than analyze the entirety of the source code of the filters themselves, they decided to analyze the execution tracing — or only the instructions that are actually run in the CPU when a filter is applied — to identify which operations are actually used, and which were being applied multiple times or inefficiently.
By comparing the execution trace of the filter to the actual pixel changes that take place on screen when the filter is applied, the researchers were able to construct a high-level representation of original binary code and recreate it in Halide, a modern programming language geared toward image processing. The Halide code is then run through an auto-tuner to optimize it for the latest hardware and recompiled.
It’s a groundbreaking idea. In fact, Charith Mendis, one of the student researchers with the project, was recently awarded the 2015 William A. Martin Memorial Thesis Award for outstanding thesis in Computer Science for their work on the project. The results are even more impressive. “On average, we’re able to make the filters run about 75 percent faster, without human intervention at any point,” says Sylvain.
Although the technology used in Project Helium has only been applied to image processing thus far, it represents a powerful proof-of-concept for software optimization with the potential to be applied to many different areas of software development.
Sylvain muses, “In the future, this kind of software optimization has the potential to perform months of traditional software optimization in just minutes.”