I’m interested in Quantum Neural Networks, specifically how to actually build the things. Any input would be greatly appreciated on this one. This is open notebook science in an extreme sense: I’m discussing here something I’d like to go into eventually, it may be several years down the line, but it’s worth thinking about it in the meantime.
The first point I’d like to address is the Superconducting Neuron model – this is an approach which attempts to build real life biologically inspired neural nets from superconducting hardware. I’ll discuss some other approaches to utilising the ‘quantum’ aspect of QNNs more efficiently in subsequent posts, for now this discussion is limited to this one hardware model.
Here are some papers I’ve been reading on the subject:
Mizugaki et al., IEEE Trans. Appl. Supercond., 4, (1), 1994
Rippert et al., IEEE Trans, Appl. Supercond., 7, (2), 1997
Hidaka et al., Supercond. Sci. Technol., 4 (654-657), 1991
There are several advantages to using SC hardware to build NNs. The RSFQ framework makes it much easier to implement, for example, fan-in and fan-out systems. Flux pulses can correspond directly to nerve-firings. The circuit elements dissipate much less power than their silicon counterparts. And you could simulate factors such as neurotransmitter levels and polarity using flux-couplers and bias leads, which (I believe) seems to be a much more natural way of doing things than trying to invent a way to mimic this in semiconductor technology.
What I understand about this field so far: In the 1990’s a couple of Japanese groups tried to demonstrate principles of superconducting neuron circuits. They built a few, and they even worked up to a point. So what has happened to this research?
1.) Well one school of thought is that the device tolerance is just not up to scratch. It is true that when you make Josephson junction circuits, the tolerances on the individual components tends not to be better than ~5%. However, is this really a problem? I can’t see that being the case, I’m sure that the similarity between biological neurons can’t be that good.
2.) Another potential problem is that research into neural networks generally has diminished (partly due to the so-called AI winter). If people using supercomputers can’t get their simulated neural networks to do anything *that* interesting, why bother with building the things in hardware? Such realizations would have far fewer neurons anyway! I guess the answer is that simulating superconducting circuits is still quite hard, and there could be some real advantages to building the things – similar to the reasons for building modern ASICs.
3.) A third problem is device integration level. Even with the best fab facilities available, superconducting circuits can only be made to low level VLSI (10,000’s of junctions). Again my point is – well why not try something on this scale? Unfortunately, cell libraries for RSFQ design probably don’t natively support the kind of realisations you need to build superconducting neurons. (For example, you need a great deal of FAN-IN FAN-OUT). So you’d probably have to go fully custom, but that’s just a design challenge.
4.) And then there’s a theoretical problem that has been bugging me for a while now. Although you can simulate any level of connectivity in highly abstracted models of NNs (given enough processing power and memory), if you actually want to build one, are you limited by the current 2-dimensional planar nature of the fabrication process? In a 3-dimensional interconnected system such as a real human brain, you are able to connect distant regions via UNIQUE, DIRECT links. In a 2D system, you are limited by the circuit layout and can (essentially) only make nearest neighbour connections. I’m pretty sure there’s a graph theory proof pinging somewhere around the edge of my mind here about connectivity in different-dimensional systems. The question is, does this limitation mean that it is theoretically impossible to build biologically inspired neural networks in planar hardware?
The field of RSFQ / Superconducting digital electronics is suffering low funding at the moment from ‘lack of applications’ syndrome. The number of people investigating applications of RSFQ circuits and Josephson logic seems to be much lower than the number of people working on the fundamental Physics of the devices. It’s a problem with the way research is funded. No-one will fund mid-term technology development, it’s either fundamental Physics or applications breakthroughs.
There may well be research being done in this area that I am unaware of, and I would be most intrigued to learn of any progress, and whether there are there problems in addition to the four presented here. However, if the research is not being done, why not? And would it be possible to get funding for projects in this area…