Implementation: analog and digital
Various methods for the neuron implementation need to address two basic aspects, namely (i) processing - the part that takes care of computing the output out of inputs and weights and (ii) data transfer - the part that takes care of data delivery and storage.
While digital implementation is more common in modern, large-scale IC design, recent approaches involve analog implementations. A digital realization of a neuron is based on a multiply-and-accumulate circuit. Each operation involves reading an input and a weight, and results in an intermediate result. This procedure is repeated multiple times. After the summation ends nonlinearity needs to be applied to the resulting value and the result is rendered the neuron output. A result is available once for every N cycles This result should be stored thereafter.
Figure 3: Digital building block
Analog implementations leverage the continuous nature of signals to express the sum of some physical level (e.g. sum of voltage potentials or sum of currents) and get a continuous signal that is exempt of finite world length representation issues.
Figure 4: Analog building block (continuous operation)
Another variant of an analog circuit is a spiking-based circuit that leverages the concept of pulse train of constant amplitude. Excitation level is rate dependant in this case. This concept is the one that mostly resembles brain neuronal activity.
Figure 5: analog building block (spiking operation)
In the analog case, data storage is a non-trivial challenge. It can be addressed by translation to the digital domain, implying a need for some kind of an analog-to-digital conversion while storing data and digital-to-analog conversion while fetching it. Alternatively, the output can directly feed the next stage, thus avoiding any storage. The latter approach is highly efficient provided that the design is capable of supporting the needed bandwidth. Some capacitance may be applied to allow bandwidth control if needed. (Note: Implementation diagrams 3, 4 & 5 show one option for implementing each of the formerly mentioned approaches and don’t carry all implementation details.)