Rapid advances in processing, led by domain-specific silicon and better use of resources, are placing pressure on memory systems to provide enough data to keep these engines running at top speed. Among the ideas receiving increased attention are moving processing closer to the data, and applying new architectures to eliminate bottlenecks in memory and networks. A number of technical challenges must be addressed before the industry can bring these solutions to market.
Introduction: Computing Bottlenecks are a Moving Target
Processing, memory, storage, and networking are the four pillars of enterprise-class computing. Ideally, these critical subsystems should be in balance for optimum performance and efficiency. Of course, the reality is never that simple; each of these leverages different technologies that follow their own innovation and development trajectories, with the result that improvements in one subsystem inevitably shifts the balance of a system, leaving others needing to catch up.
Today, CPUs and GPUs have become so fast that these processors often must wait for data from memory before they can continue working. Recent advances in processing have exacerbated the decades-old “Processor-Memory Gap,” with modern systems often left struggling to move data from storage and memory to the processors, across interfaces that have limited bandwidth.
Particularly challenging are workloads that involve filtering large quantities of data to look for a small number of items – akin to finding a “needle in a haystack.” These include certain types of database operations like search, which are computationally untaxing but involve the movement of vast amounts of data over already stressed interconnections. Moving so much data from memory and storage to the processor, only to realize that you don’t need most of it, wastes considerable time and energy. In some cases, the amount of energy spent moving data to the processor can be about 100 times larger than the amount of energy spent doing simple operations on the data.
Next: Big data demand