The Physics World is my Oyster

Physics and Cake got a mention in Physics World this month! 🙂 As a long time reader of Physics World, I’m really happy to see this! I guess this means I’ll have to blog more about Physics and less about the speculative promises and hidden possibilities of Artificial General Intelligence… (especially as AGI apparently didn’t make the transcription below). Though I ‘m afraid I cannot currently shake my desire to explore the intersection between AGI and Physics!

Hmm, looking at this post in the browser is oddly fractal! Though not quite enough to become a Strange Loop. (H/T Douglas Hofstadter, you are awesome).

Transhumanism and objectivity: An introduction

I have been involved in the transhumanism community for a fair while now, and I have heard many arguments arising from both proponents and skeptics of the ‘movement’. However, many of these arguments seem to stem from instinctive reactions rather than critical thinking. Transhumanism proponents will sometimes dogmatically defend their assumptions without considering whether or not what they believe may actually be physically possible. The reasoning behind this is fairly easy to understand: Transhumanism promises escape from some of humanity’s deepest built in fears. However, the belief that something of value will arise if one’s assumptions are correct can often leave us afraid to question those assumptions.

I would currently class myself as neither a proponent or a skeptic of the transhumanism movement. However I do love to explore and investigate the subject, as it seems to dance close to the very limits of our understanding of what is possible in the Universe. Can we learn something from analyzing the assumptions upon which this philosophical movement is based? I would answer not only yes, but that to do so yields one of the most exciting applications of the scientific method that we have encountered as a society.

I find myself increasingly drawn toward talking about how we can explore transhumanism from a more rational and objective point of view. I think all transhumanists should be obliged to take this standpoint, to avoid falling into a trap of dogmatic delusion. By playing devil’s advocate and challenging some of the basic tenets and assumptions, I doubt any harm can be done. At the least those tenets and assumptions will have to be rethought. But moreover, we may find that the lessons learned from encountering philosophical green lights and stop signs may inform the way we steer our engineering of the future.

I’ve thus decided to shift the focus of this blog a little towards some of these ideas. In a way I have already implemented some of this shift: I have written a couple of essays and posts before. But from now on, expect to see a lot more of this in the future. A blog format is an excellent way of disseminating information on this subject: It is dynamic, and can in principle reach a large audience. I also think that it fits in well with the Physics and Cake ethos – applying the principles of Physics to this area will form a large part of the investigations. And, of course, everything should always be discussed over coffee and a slice of cake! Another advantage is that this is something that everyone can think about and contribute to. You don’t need an expensive lab or a PhD in theoretical Physics to muse over these issues. In a lot of cases, curiosity, rationality, and the patience to follow an argument is all that is necessary.

AGI is number 1!

Interesting article on the Lifeboat Foundation website about top-ten transhumanist technologies:

Top Ten Tranhumanist Technologies

Ooh, the alliteration 🙂 I notice that AGI is at number 1. Let’s hope that we can actually work towards a good definition of AGI and solve some foundational issues along the way to keep it up there! Interestingly I think a few of these technologies have interesting crossovers, such as virtual reality, mind uploading and cybernetics. The more we advance progress in technology, the more these disciplines will become indistinguishable.

Anyway, I think this is a nice article as it gives an introduction to many of the things that transhumanists talk about over coffee and cake (or water and fruit if you are a Paleo).

Humanity+ Conference 2010 Caltech

I gave a presentation yesterday at the H+ conference at Caltech. The session in which I spoke was the ‘Redefining Artificial Intelligence’ session. I’ll try to get the video of the talk up here as soon as possible along with slides.

Other talks in this session were given by Randal Koene, Geordie Rose, Alex Peake, Paul Rosenbloom, Adrian Stoica, Moran Cerf and Ben Goertzel.

My talk was entitled ‘Pavlov’s AI: What do superintelligences really want?’ I discussed the foundations of AGI, and what I believe to be a problem (or at least an interesting philosophical gold-seam) in the idea of building self-improving artificial intelligences. I’ll be writing a lot more on this topic in the future, hopefully in the form of essays, blogposts and papers. I think it is very important to assess what we are trying to do in the area of AI, what the overall objectives are, and looking at what we can build from an objective point of view is helpful in framing our progress.

The conference was livestreamed, which was great. I think my talk had around 500 viewers. Add to that the 200 or so in the lecture hall; 700 is a pretty big audience! Some of talks had over 1300 remote viewers. Livestreaming really is a great way to reach a much bigger audience than is possible with real-life events alone.

I didn’t get to see much of the Caltech campus, but the courtyard at the Beckman Institute where the conference was held was beautiful. I enjoyed the fact that coffee and lunch was served outside in the courtyard. It was very pleasant! Sitting around outside in L.A. in December was surprisingly similar to a British summer!

I got to talk to some great people. I enjoy transhumanism-focused conferences as the people you meet tend to have many diverse interests and multidisciplinary backgrounds.

I was very inspired to continue exploring and documenting my journey into the interesting world of AGI. One of the things I really love doing is looking into the fundamental science behind Singularity-focused technologies. I try to be impartial to this and give both an optimistic account of the promise of future technologies whilst maintaining a skeptical curiosity about whether such technologies are fundamentally possible, and what roadmaps might lead to their successful implementation. So stay tuned for more Skepto-advocate Singularity fun!

New scheduling for ‘Thinking about the Hardware of thinking’

I was scheduled to give a live virtual seminar, streamed to the Transvision conference in Italy on October 23rd. Unfortunately I was not able to deliver the presentation due to technical problems at the conference venue.

But the good news is, I will be giving the talk this weekend instead!

Here is the abstract (slightly updated as the talk will be a little longer than originally planned)

Thinking about the hardware of thinking:
Can disruptive technologies help us achieve uploading?

S. Gildert,
Teleplace, 28th November 2010
10am PST (1pm EST, 6pm UK, 7pm continental EU).

We are surrounded by devices that rely on general purpose silicon processors, which are mostly very similar in terms of their design. But is this the only possibility? As we begin to run larger and more brain-like emulations, will our current methods of simulating neural networks be enough, even in principle? Why does the brain, with 100 billion neurons, consume less than 30W of power, whilst our attempts to simulate tens of thousands of neurons (for example in the blue brain project) consumes tens of KW? As we wish to run computations faster and more efficiently, we might we need to consider if the design of the hardware that we all take for granted is optimal. In this presentation I will discuss the recent return to a focus upon co-design – that is, designing specialized software algorithms running on specialized hardware, and how this approach may help us create much more powerful applications in the future. As an example, I will discuss some possible ways of running AI algorithms on novel forms of computer hardware, such as superconducting quantum computing processors. These behave entirely differently to our current silicon chips, and help to emphasize just how important disruptive technologies may be to our attempts to build intelligent machines.

Here is a link to the Teleplace announcement.

Hope to see you there!

The ‘observer with a hammer’ effect

Here is another short essay about quantum mechanics-related stuff. It’s a very high level essay, so any practising quantum physicists probably shouldn’t read it 😉 It is more aimed at a general audience (and news reporters!) and talks about the ‘spooky’ and ‘weird’ properties of superposition and decoherence that people seem to like to tie in with consciousness, cats, and ‘the observer effect’. It doesn’t really go into entanglement directly, I think that should be an issue for a separate post! It is also a fun introduction to some issues when trying to perform experimental quantum computing and quantum physics in general.

I’ve also put this essay in the Resources section as a permanent link.

.
The not-so spooky after all ‘observer-with-a-hammer’ effect

S. Gildert November 2010

I’m so sick of people using phrases like this:

“Looking at, nay, even thinking about a quantum computer will destroy its delicate computation. Even scientists do not understand this strange and counter-intuitive property of quantum mechanics”

or worse:

“The act of a conscious observer making a measurement on a quantum computer whilst it is performing a calculation causes the wavefunction to collapse. The spooky nature of these devices means that they just don’t work when we are looking at them!”

ARGGHHHHH!!!!!!!!

These kind of phrases spread like viral memes because they are easy to remember and they pique people’s curiosity. People like the idea of anthropomorphizing inanimate systems. It makes them seem unusual and special. This misunderstanding, the idea that a quantum system somehow ‘cares’ or is emotionally sensitive to what a human is doing, is actually what causes this meme to perpetuate.

So I’m going to put a new meme out there into the-internet-ether-blogosphere-tubes. Maybe someone will pick up on this analogy and it will become totally viral. It probably won’t, because it seems pretty dull in comparison to spooky ethereal all-seeing quantum systems, but if it flicks a light switch in the mind of but a single reader, if on contemplating my words someone’s conceptual picture of quantum mechanics as a mystical, ever elusive resource is reduced even by the tiniest amount, then my work here will be done.

Memetic surgery

Let’s start by cutting the yukky tumorous part from this meme and dissecting it on our operating table:

“Looking at a quantum system changes it.”

Now I don’t necessarily disagree with this statement, but I think you need to define what you mean by ‘looking’….

Usually when physicists ‘look’ at things, they are trying to measure something to extract information from it. To measure something, you need to interact with it in some way or other. In fact, everything in the world interacts with many other things around it (that’s why Physics is interesting!). Everything one could ever wish to measure is actually sitting in a little bath of other things that are constantly interacting with it. Usually, we can ignore this and concentrate on the one thing we care about. But sometimes this interacting-background property can cause unwanted problems.

Measuring small things

Brownian motion can give us a nice example of a nasty background interaction. Imagine that a scientist wanted to investigate the repulsion (or attraction) of some tiny magnetic particles in a solution that had just precipitated out of an awesomely cool chemical reaction. (I don’t know why you’d want to do this, but scientists have some weird ideas). So she starts to take measurements of the positions of the little magnetic particles over time, and finds that they are not obeying the laws of magnetism. How dare they! What could be wrong with the experiment? So our good scientist takes the solution in her beaker and you start to adjust various parameters to try and figure out what is going on. It turns out that when she cools the solution, the particles start to behave more in line with what is expected. She figures that the Brownian motion – all the other molecules jostling and wiggling around near the magnetic particles – are actually kicking the experiment around, ruining the results. But by lowering the temperature, it is possible to stop the environment in which the particles sit from disturbing them as much.

In this example, the scientist was able to measure the positions of the particles with something like a ruler or a laser or some other cool technique, and it was fairly easy, even though the environment had become irritatingly convolved with our experiment. Once she had got around how to stop the interaction with the environment, then our experiment worked well.

Quantum systems are small, and small things are delicate. But quantum systems are so small that the environment, the ‘background-interaction’ around them, is no longer something that they, or we, can ignore. It pushes them around. In order to have a chance at engineering quantum systems, researchers have to isolate them carefully from the environment (or at least the bits of the environment that kick them around). Scientists spend a lot of time trying to stop the environment from interacting with their qubits. For example, superconducting processors need to be operated at very cold temperatures, in extremely low magnetic field environments. But I won’t digress into the experimental details. The main idea is that no matter how you build your quantum computer, you will have to solve this problem in some way or other. And even after all this careful engineering, the darn things still interact with the environment to some degree.

It gets worse

But with quantum systems, there is an extra problem. The problem is not just the environment. To illustrate this problem, I’ll propose another little story of the striving scientists.

Imagine that our scientists have developed a technique to measure the diameter of bird eggs using a robotic arm. The arm has a hand that grasps the eggs, measures them, and then displays the diameter on a neat built-in display. (Alternatively, you can Bluetooth the results to your iPhone, so the scientists tell me). Anyway, this robotic arm is so ridiculously precise that it can measure the diameter of eggs more accurately than any pair or vernier calipers, any laser-interferometer array or any other cool way of measuring eggs that has ever existed. The National Standards laboratories are intrigued.

However, there is a slight problem. Every time the robot tries to measure an egg, it breaks the darn thing. There is no way to get around this. The scientific breakthrough relating to the accuracy of the new machine comes from the fact that the robot squeezes the egg slightly. Try and change the way that the measurement is performed, and you just can’t get good results anymore. It seems that we just cannot avoid breaking the eggs. The interaction of the robot with the egg is ruining our experiment.

Of course, a robot-egg measuring system like this sounds ridiculous, but this is exactly the problem that we have with quantum systems. The measuring apparatus is huge compared to the quantum system, and it interacts with it, just like the pesky environment does. It pushes and squeezes our quantum system. The result is that anything huge that we use to try to perform a delicate measurement will break it. And worse still, we can’t just try to ‘turn it off completely’ like we could with the environment surrounding the particles in the solution. By the very nature of what we are trying to do, we need the measurement apparatus to interact with the qubits, otherwise how can we measure them? What a pain. We end up measuring a kind of qubit-environment-combination mess, just like trying to measure the diameter of a broken egg whose contents are running all over our robotic measurement apparatus.

I can’t stress enough how comparatively big and clumsy quantum measurement apparatus is. Whilst scientists are trying to build better measurement techniques that don’t have such a bad effect on quantum systems, ultimately you just can’t get around this problem, because the large-scale things that we care about are just not compatible with the small-scale of the quantum world.

This doesn’t mean that quantum computers aren’t useful. It just means that the information we can extract from such systems is not neat, clean and unique to the thing we were trying to measure. We have to ‘reconstruct’ information from the inevitable conglomerate that we get out of a measurement. In some cases, this is enough to help us do useful computations.

Hammering the message home

Nowhere here does one need to invoke any spookiness, consciousness, roles of the observer, or animal cruelty involving cats and boxes. In fact, the so-called ‘observer’ effect could perhaps be more appropriately termed the ‘observer-with-a-hammer’ effect. We take for granted that we can measure large classical systems, like the 0 or 1 binary states of transistors, without affecting them too much. But measuring a quantum system is like trying the determine the voltage states of a single transistor by taking a hammer to the motherboard and counting the number of electrons that ended up sticking to the end of it. It kind of upsets the computation that you were in the middle of. It’s not the observer that’s the problem here, it’s the hammer.

So, the perhaps-not-so-viral phraseology for one to take away from my relentless ranting is thus:

“When you try and measure a delicate quantum system with clumsy apparatus, you actually end up with a messy combination of both!”

Alternatively, you could say ‘you can’t make a quantum measurement without breaking a few eggs’ – But if that terrible pun sticks then I will forever be embarrassed.

Building more intelligent machines: Can ‘co-design’ help?

Here is a little essay I wrote in response to an article on HPCWire about hardware-software co-design and how it relates to D-Wave’s processors. I’ve also put this essay in the Resources section as a permanent link.

.
Building more intelligent machines: Can ‘co-design’ help?

S. Gildert November 2010

There are many challenges that we face as we consider the future of computer architectures, and as the type of problem that people require such architectures to solve changes in scale and complexity. A recent article written for HPCwire [1] on ‘co-design’ highlights some of these issues, and demonstrates that the High Performance Computing community is very interested in new visions of breakthrough system architectures. Simply scaling up the number of cores of current technologies seems to be getting more difficult, more expensive, and more energy-hungry. One might imagine that in the face of such diminishing returns, there could be innovations in architectures that are vastly different from anything currently in existence. It seems clear that people are becoming more open to the idea that something revolutionary in this area may be required to make the leap to ‘exascale’ machines and beyond. The desire for larger and more powerful machines is driving people to try to create more ‘clever’ ways of solving problems (algorithmic and software development), rather than just increasing the speed and sheer number of transistors doing the processing. Co-design is one example of a buzzword that is sneakily spreading these memes which hint at ‘clever’ computing into the HPC community.

Generalization and specialization

I will explain the idea of co-design by using a colorful biological analogy. Imagine trying to design a general purpose animal: Our beast can fly, run, swim, dig tunnels and climb trees. It can survive in many different environments. However, anyone trying to design such an animal would soon discover that the large wings prevented it from digging tunnels effectively; that the thick fur coat to survive the extreme cold was not helpful in achieving a streamlined, fast swimmer. Any animal that was even slightly more specialized in one of these areas would quickly out-compete our general design. Indeed, for this very reason, natural selection causes specialization and therefore great diversity amongst the species that we see around us. Particular species are very good at surviving in particular environments.

How does this tie in with computer processing?

The problems that processors are designed to solve today are mostly all very similar. One can view this as being a bit like the ‘environmental landscape’ that our general purpose creatures live in. If the problems that they encounter around their environment on a day-to-day basis are of the same type, then there is no reason to diversify. Similarly, a large proportion of all computing resources today address some very similar problems, which can be solved quite well using general purpose architectures such as Intel Centrino chips. These tasks include the calculations that underlie familiar everyday tasks such as word-processing, and displaying web pages. But there do exist problems that have been previously thought to be very difficult for computers to solve, problems which seem out of reach of conventional computing. Examples of such problems are face-recognition, realistic speech synthesis, the discovery of patterns in large amounts of genetic data, and the extraction of ‘meaning’ from poetry or prose. These problems are like the trees and cliffs and oceans of our evolutionary landscape. The general purpose animals simply cannot exploit these features, they cannot solve these problems, so the problems are typically ignored or deemed ‘too hard’ for current computing platforms.

But there are companies and industries that do care about these problems. They require computing power to be harnessed for some very specific tasks. A few examples include extracting information from genetic data in the biotechnology companies, improving patient diagnosis and medical knowledge of expert systems in the healthcare sector, improving computer graphics for gaming experiences in entertainment businesses, and developing intelligent military tools for the defense industry. These fields all require the searching and sorting of data in parallel, and the manipulation of data on a much more abstract level for it to be efficient and worthwhile. This parallel operation and abstraction is something that general purpose processors are not very good at. They can attempt such a feat, but it takes the power of a supercomputer-size machine to tackle even very small instances of these specialized problems, using speed and brute force to overwhelm the difficulty. The result is very expensive, very inefficient, and does not scale well to larger problems of the same type.

It is this incorporation of variety and structure, the addition of trees, cliffs and oceans, into our computational problems causes our general-purpose processors to be very inefficient at these tasks. So why not allow the processors to specialize and diversify, just like natural selection explores the problem environment defined by our biological needs?

Following nature’s example

Co-design attempts to address this problem. It tries to design solutions around the structure of the problem type, resulting in an ability to solve that one problem very well indeed. In practice this is done by meticulous crafting of both software and hardware in synchrony. This allows software which complements the hardware and utilizes subtleties in the construction of the processor to help speed things up, rather than software which runs on a general architecture and incurs a much larger overhead. The result is a blindingly fast and efficient special purpose architecture and algorithm that is extremely good at tackling a particular problem. Though the resulting processor may not be very good at certain tasks we take for granted using general-purpose processors, solving specialized problems instead can be just as valuable, and perhaps will be even more valuable in the future.

A selection of processors which are starting to specialize are discussed in the HPCwire article. These include MDGRAPE-3, which calculates inter-atomic forces, and Anton, a system specifically designed to model the behaviour of molecules and proteins. More common names in the processor world are also beginning to explore possible specializations. Nvidia’s GPU based architectures are gaining in popularity, and FPGA and ASIC alternatives are now often considered for inclusion in HPC systems, such as some of Xilinx’s products. As better software and more special purpose algorithms are written to exploit these new architectures, they become cheaper and smaller than the brute-force general purpose alternatives. The size of the market for these products increases accordingly.

The quantum processors built by D-Wave Systems [2] are a perfect example of specialized animals, and give an insightful look into some of the ideas behind co-design. The D-Wave machines don’t look much like regular computers. They require complex refrigeration equipment and magnetic shielding. They use superconducting electronics rather than semiconducting transistors. They are, at first inspection, very unusual indeed. But they are carefully designed and built in a way that allows an intimate match between the hardware and the software algorithm that they run. As such they are very specialized, but this property allows them to tackle very well a particular class of problems known as discrete optimization problems,. This class of problems may appear highly mathematical, but looks can be deceiving. It turns out that once you start looking, examples of these problems are found in many interesting areas of industry and research. Most importantly, optimization forms the basis of many of the problems mentioned earlier, such as pattern recognition, machine learning, and meaning analysis. These are exactly the problems which are deemed ‘too hard’ for most computer processors, and yet could be of incredible market value. In short, there are many, many trees, cliffs and oceans in our problem landscape, and a wealth of opportunity for specialized processors to exploit this wonderful evolutionary niche!

Co-design is an important ideas in computing, and hopefully it will open people’s minds to the potential of new types of architecture that they may never have imagined before. I believe it will grow ever more important in the future, as we expect a larger and more complex variety of problems to be solved by our machines. The first time one sees footage of a tropical rainforest, one can but stare in awe at the wonders of never-before-seen species, each perfectly engineered to efficiently solve a particular biological problem. I hope that in the future, we will open our eyes to the possibility of an eco-sphere of computer architectures, populated by similarly diverse, beautiful and unusual creatures.

[1] http://www.hpcwire.com/features/Compilers-and-More-Hardware-Software-Codesign-106554093.html

[2] http://www.dwavesys.com/