CoCoPIE: A software solution for putting real artificial intelligence in smaller spaces
Bit by bit, byte by byte, artificial intelligence has been working its way into public consciousness and into everyday computer use.
High-profile computer victories over human competitors such as IBM’s Watson’s victory on Jeopardy! and Deep Blue’s defeat of chess champion Garry Kasparov are for the history books. Artificial intelligence and deep learning have been deeply woven into more and more aspects of end-user computing.
Smartphones and other mobile devices use AI as well. Up until now, the artificial intelligence work has been done in the cloud, but a new approach to software design aims to arm mobile devices with real artificial-intelligence capability.
“A mobile device is very resource-constrained,” explained William & Mary computer scientist Bin Ren. “Its CPU and GPU are much smaller than the ones that can be accessed by the cloud.”
Ren is a member of a collaboration that has developed a software framework that they call CoCoPIE. Ren explains that tablets, smartphones and internet-of-things devices such as smart thermostats that run software enriched with CoCoPIE can use real-time AI applications without extensive hardware modifications of the devices.
Ren is an assistant professor in William & Mary’s Department of Computer Science. Other members of the CoCoPIE collaboration are Yanzhi Wang of Northeastern University, Xipeng Shen of North Carolina State University and Shaoshan Liu of PerceptIn Inc.
Jason McDevitt, William & Mary’s director of technology transfer, says CoCoPIE has excellent commercial potential. The university, in partnership with Northeastern, has a group of patents pending on CoCoPIE technology.
Ren explains that CoCoPIE works on the principle of co-design, hitching the functions of compression and compiler together to create artificial-intelligence capability. He teaches courses in compiling, which essentially is a translation from a human-written programming language into binary code. The CoCo, or compression/compiling concept, is simple in theory, Ren explained, but a challenge to make happen.
“We designed a novel compression technique, for one thing,” he said. “It’s a pruning technique, and our work requires pattern-based pruning. The compiler cannot benefit from patterns by default, but we figured out a way to make them work together.”
The collaborators tried out their software-only AI approach on an off-the-shelf Samsung Galaxy S10 smartphone. They ran a battery of image-enhancement tests against other software deep neural network acceleration frameworks and hardware solutions. Ren said they found CoCoPIE outperformed all of these solutions in terms of speed and power efficiency, for the first time achieving real-time deep neural network execution on mobile devices.
The tests involved manipulation of a set of images that were subjected to manipulations common among users of smartphone users — colorization, style transfer and sharpening through digital super resolution. Ren says the idea isn’t just to benefit casual phone camera users. For one thing, he said native AI in mobile and IoT devices could help police identify and apprehend perpetrators of crimes.
“And there’s the bandwidth,” he said. “AI on mobile devices will save a lot of bandwidth.”