Analog and digital circuits for machine learning: Page 4 of 6

July 24, 2018 // By Avi Baum
Avi Baum, chief technology officer of Hailo (Tel Aviv, Israel), compares the underlying principles and energy considerations behind analog and digital approaches to neural network implementation and machine learning circuits.

Performance

When looking into the performance of the various methods, it is clear that while digital solutions are well established they are limited by the barriers of CMOS technology. These are typically transistor level threshold voltage at ~0.4V, a process-dependant maximum clock frequency for standard cells at less than 3GHz and duty-cycle limitation. This results in a lower bound for the processing node that is roughly around ~100fJ for a single 8bit multiply-and-add operation [1]

Analog circuitry, in contrast, is theoretically bounded by thermal noise that is roughly at 0.01fJ. This is four orders of magnitude lower than the digital option. Hence the interest in building circuits based on an analog compute fabric. Yet, practical deployment is challenged by various issues such as delivering data into a large array of compute elements as described, parasitic effects related to their connectivity, storing the output efficiently and finally the ability to translate into large-scale design flows and mass production techniques. In practice, the reported information indicates achievable energy for the compute element in the ballpark of of 1 to 10fJ [2]. In these implementations, indeed the compute element energy becomes negligible, however, the overall energy is largely dominated by the surrounding circuitry and storage elements. All in all a practical efficiency of x10 to x100 on top of digital-based building blocks is achievable at small scale but rapidly drops away when scaling up the number of elements.

[1] Computing’s Energy Problem (and what we can do about it), M. Horowitz, 2014

[2] Energy-Efficient Time-Domain Vector-by-Matrix Multiplier for Neurocomputing & Beyond, Bavandpour et. al, 2017

Figure 6: Relevant domain of operation illustration

Figure 6 is a qualitative description of the different approaches. Efficiency loss of analog circuits is primarily due to implementation loss (i.e. the detector circuit has some internal noise which degrades signal-to-noise ratio and requires better margin). In this case, a spiking approach has a lower detection threshold. When scaling up analog solutions, noise coupling is observed. This effect grows with solution scale (it is more dominant in continuous approaches). Digital approaches suffer less from this coupling effect. Indeed the energy gap from analog to digital is attributed to higher voltage levels and operational  frequency, which is much higher in the analog case.

Practically, large-scale circuit design has matured throughout the last few decades and the overall acquired industry experience cannot be easily dismissed. Therefore, the combination of the scalability issue and productization aspects largely limit the ability to make analog-based solutions a dominant approach for the general problem. Furthermore, at the system level, the secondary contributors cannot be overlooked. Once the compute element contribution is lowered down to a reasonable level, further improvement becomes less important. 

Next: Comparison


Vous êtes certain ?

Si vous désactivez les cookies, vous ne pouvez plus naviguer sur le site.

Vous allez être rediriger vers Google.