Current AI solutions are quite resources/energy-hungry
and still time-consuming. In fact, today DNNs (as other AI models) still rely
on Boolean algebra transistors to do an enormous amount of digital computations
over huge data sets. The roadblock is that chipsets technologies aren’t getting
faster at the same pace as AI software solutions are progressing in serving
markets’ needs.
Will this energy consuming trend be really sustainable
in the long term?
We remind that in basic functioning of a DNN, each high-level
layer learns increasingly abstract higher-level features, providing a useful,
and at times reduced, representation of the features to a lower-level layer.
This similarity is suggestin, the intriguing possibility that DNNs principles
are deeply rooted in Quantum Field Theory and Quantum Electromagnetics. This
aspect is, perhaps, offering a way to bypass above roadblocks: developing AI
technologies based on photonic/optical computing systems which are much faster
and much less energy-consuming that current ones.
As a matter of fact, while, in line with the Moore’s
law, electronics starts facing physically fundamental bottlenecks,
nanophotonics technologies are considered promising candidates to overcome
electronics future limitations. Consider that DNNs operations are mostly matrix
multiplications, and nanophotonic circuits can make such operations almost at
the speed of light and very efficiently due to the nature of photons. In simple
words, photonic/optical computing uses electromagnetic signals (e.g., via laser
beams) to store, transfer and process information. Optics has been around for
decades, but until now, it has been mostly limited to laser transmission over
optical fiber. Today technologies, using optical signals to do computations and
store data, would accelerate AI computing by orders of magnitude in latency,
throughput and power efficiency.
Matrix multiplication is the most power hungry and
time-consuming operations in AI algorithms. Extrapolating current trends, speed
of electronics components performing matrix calculations is likely to
insufficient for supporting future AI applications, at least in the long term.
Reducing the electric energy consumptions is also another strict requirement
for sustainability.
Neuroscience may hold the solution. Our brain is not
digital, it’s analogue, and it makes calculations all the time using
electromagnetics signals, consuming just 30 W.
The advantage of using light processing to do matrix
multiplication plays significantly in speeding up calculations and power
savings. In fact, instead of using streams of electrons, the calculations are
performed by beams of photons that interact with one another, in a medium, and
with optical resonators and guiding components. To make it simple: unlike
electrons, photons have no mass, travel at light-speed and draw no additional
power once generated.
Interesting prototypes of an all-optical DNN are
already available. For example, in this paper [1] shows the feasibility study
of an all-optical diffractive DNN. The prototype is made of a set of
diffractive layers, where each point (equivalent to a neuron) acts as a
secondary source of an electromagnetic wave directed to the following layer.
The amplitude and phase of the secondary wave are
determined by the product of the input wave and the complex-valued transmission
or reflection coefficient at that point (following the laws of transformation
optics). The transmission/reflection coefficient of each point of a layer is a
learnable network parameter, which is iteratively adjusted during the training
process (e.g., performed in a computer) using a classical error
back-propagation method. After the training, the design of the layer is fixed
as the transmission/reflection coefficients of all the neurons of all layers
are determined.
Other examples of prototypes are based on the metamaterials,
information meta-surface, and optical field-programmable gate array based on
Mach-Zehnder Interferometers.
[1] Lin, Xing, et
al. "All-optical machine learning using diffractive deep neural networks.
Science 361.6406 (2018): 1004-1008.
