Singularity Summit 2009

I’ll be attending the Singularity Summit this year.
Anyone who is interested in attending should register here:

singularity summit - yey

The list of speakers looks something like this:

Randal Koene, Itamar Arel, Ben Goertzel, Stuart Hameroff, Ned Seeman, Jürgen Schmidhuber, Gary Drescher, David Chalmers, Ed Boyden, Marcus Hutter, William Dickens, Béla Nagy, Gary Marcus, Gary Wolf, Michael Nielsen, Robin Hanson, Brad Templeton, Ray Kurzweil, Anders Sandberg, Peter Thiel
Eliezer Yudkowsky, Aubrey de Grey.

The link also takes you to the website where you can find out more information on this unusual yet inspiring conference.

See you there!

Digital lab books

Should I digitise my lab books?

The answer to this is almost definitely yes. Other than it taking a long time, there aren’t really any disadvantages to doing so. It would mean I had access to all my experimental settings in digital form, and I would have a built in cross-reference of the plots that are currently stuck in the lab books and also on my hard drive with the experimental notes (I already save the copies of the plots with the date but then you have to go and get the lab book to look up the date and find what sample the data was from and the experimental settings for that date).

In addition, if I decide to go completely Open Notebook, I could put the digital versions online. Although I’d need a lot of server space for this – Rob, I’m looking at you 😉 It would also be cool to build the digital lab books into an Experimental Physics Wiki I’ve been meaning to get around to writing. I could easily navigate around previous experiments by tag too, rather than my current, somewhat prehistoric, method of ‘post-it notes’ stuck to various pages.

Having a digital backup would also put my mind somewhat to rest over the matter that if I lost my lab books I would be totally screwed. All my data analysis and experimental development information would all be lost, and you can’t plot the data without the precious settings which are meticulously noted down in the lab book everytime the experiment is run.

I’m also quite proud of my lab books as they are quite neat and pretty, and are often used to show students a ‘good example’ so it would be a shame to lose them.

So, I was trying to work out what method to use. At first I thought about scanning the books, but I think this will take too long. I have about 8 books with 100+ pages in each. I thought about photocpoying the pages (our photocopier has a function that allows you to e-mail the copy to yourself) but my current favourite is the idea of taking hi-res digital photographs of each page. If I could set up a little ‘photo-shoot’ with good lighting and camera tripod, and a stand for the books, I could probably get them done pretty fast.

I’ve also always wanted an excuse to buy a lectern. 😀 Like this one:

eaglelecturn2

Quantum Neural Networks 1 – the Superconducting Neuron model

I’m interested in Quantum Neural Networks, specifically how to actually build the things. Any input would be greatly appreciated on this one. This is open notebook science in an extreme sense: I’m discussing here something I’d like to go into eventually, it may be several years down the line, but it’s worth thinking about it in the meantime.

The first point I’d like to address is the Superconducting Neuron model – this is an approach which attempts to build real life biologically inspired neural nets from superconducting hardware. I’ll discuss some other approaches to utilising the ‘quantum’ aspect of QNNs more efficiently in subsequent posts, for now this discussion is limited to this one hardware model.

Here are some papers I’ve been reading on the subject:

Mizugaki et al., IEEE Trans. Appl. Supercond., 4, (1), 1994

Rippert et al., IEEE Trans, Appl. Supercond., 7, (2), 1997

Hidaka et al., Supercond. Sci. Technol., 4 (654-657), 1991

There are several advantages to using SC hardware to build NNs. The RSFQ framework makes it much easier to implement, for example, fan-in and fan-out systems. Flux pulses can correspond directly to nerve-firings. The circuit elements dissipate much less power than their silicon counterparts. And you could simulate factors such as neurotransmitter levels and polarity using flux-couplers and bias leads, which (I believe) seems to be a much more natural way of doing things than trying to invent a way to mimic this in semiconductor technology.

What I understand about this field so far: In the 1990’s a couple of Japanese groups tried to demonstrate principles of superconducting neuron circuits. They built a few, and they even worked up to a point. So what has happened to this research?

Four Problems

1.) Well one school of thought is that the device tolerance is just not up to scratch. It is true that when you make Josephson junction circuits, the tolerances on the individual components tends not to be better than ~5%. However, is this really a problem? I can’t see that being the case, I’m sure that the similarity between biological neurons can’t be that good.

2.) Another potential problem is that research into neural networks generally has diminished (partly due to the so-called AI winter). If people using supercomputers can’t get their simulated neural networks to do anything *that* interesting, why bother with building the things in hardware? Such realizations would have far fewer neurons anyway! I guess the answer is that simulating superconducting circuits is still quite hard, and there could be some real advantages to building the things – similar to the reasons for building modern ASICs.

3.) A third problem is device integration level. Even with the best fab facilities available, superconducting circuits can only be made to low level VLSI (10,000’s of junctions). Again my point is – well why not try something on this scale? Unfortunately, cell libraries for RSFQ design probably don’t natively support the kind of realisations you need to build superconducting neurons. (For example, you need a great deal of FAN-IN FAN-OUT). So you’d probably have to go fully custom, but that’s just a design challenge.

4.) And then there’s a theoretical problem that has been bugging me for a while now. Although you can simulate any level of connectivity in highly abstracted models of NNs (given enough processing power and memory), if you actually want to build one, are you limited by the current 2-dimensional planar nature of the fabrication process? In a 3-dimensional interconnected system such as a real human brain, you are able to connect distant regions via UNIQUE, DIRECT links. In a 2D system, you are limited by the circuit layout and can (essentially) only make nearest neighbour connections. I’m pretty sure there’s a graph theory proof pinging somewhere around the edge of my mind here about connectivity in different-dimensional systems. The question is, does this limitation mean that it is theoretically impossible to build biologically inspired neural networks in planar hardware?

The field of RSFQ / Superconducting digital electronics is suffering low funding at the moment from ‘lack of applications’ syndrome. The number of people investigating applications of RSFQ circuits and Josephson logic seems to be much lower than the number of people working on the fundamental Physics of the devices. It’s a problem with the way research is funded. No-one will fund mid-term technology development, it’s either fundamental Physics or applications breakthroughs.

There may well be research being done in this area that I am unaware of, and I would be most intrigued to learn of any progress, and whether there are there problems in addition to the four presented here. However, if the research is not being done, why not? And would it be possible to get funding for projects in this area…

Lifelogging

I’ve been looking into some options for life logging software recently. Evernote is one option.

However, I still have several reservations about using third party software for this task, similar to the objections raised here. I worry about loss of data, I worry about future-proofing, and I worry about privacy.

I’d like a system that had some kind of intuitive support for importing and exporting blog entries. I’d like a system that could export the lifelog as a database structured similarly to a filesystem, for example containing only .txt files and .jpeg files. Until I find such a system I think I’m going to try and make my own. I’m thinking of basing it on html and running it something like a wiki. My own ‘LifeWiki’.

I’ve had bad experiences with LiveJournal in the past, for example without a premium member subscription you can’t do any advanced searching, sorting or filtering of the entries. So one of the first things I’ll have to do do is to pull all my old LJ entries and get them into a much simpler LifeWiki thing.

Another good reason for taking this approach is that (unless I wish it otherwise) the data will all be stored on my personal space and I am wholly responsible for backing up. The data format will (at least in some way) be futureproofed. Again it’s up to me to convert the data into new formats, but if the system is kept simple this should not be a problem, and if I am used to managing the system I will be expecting to have to do this. I don’t like the idea of the format of all my data being at the whim of a third party.

Obviously the down side of this endeavour is that LifeWiki won’t have any of the advanced options, it won’t have graphic text recognition searching, it won’t natively support tagging (I’ll have to sort out my own system for that), and it might just end up getting too big to handle in this primitive way.

RedNotebook seems to be the kind of thing I might be looking for, but I’ll have to read more about this and other options. But I’ve always been good at organising things, so it might just be worth a stab at doing my own version for now.