Pavlov’s AI – What did it mean?

So recently I gave a talk at the H+ Summit in Los Angeles. However, I got the impression that the talk, which was about the fundamentals of Artificial General Intelligence (something I decided to call ‘foundations of AGI’) was not fully understood. I apologize to anyone in the audience who didn’t quite ‘get’ it, as the blame must fall upon the speaker in such instances. Although, in my defense, I had only 15 minutes to describe a series of arguments and philosophical threads that I had been musing over for a good few months 🙂

If you haven’t seen the talk, and would like to watch it, here it is:

However, this article is written as a standalone resources, so don’t worry if you haven’t seen the talk.

What I would like to do is start exploring some of those issues on this blog. So, here is my attempt to describe the first of the points that I set out to try and explore in the talk. I’ve used a slightly modified argument, to try and complement the talk for those who have already seen it.


Pavlov’s AI:
What do superintelligences really want?

S. Gildert November 2010

(Photo © Thomas Saur)


Humans are pretty intelligent. Most people would not argue with this. We spend a large majority of our lives trying to become MORE intelligent. Some of us spend nearly three decades of our lives in school, learning about the world. We also strive to work together in groups, as nations, and as a species, to better tackle the problems that face us.

Fairly recently in the history of man, we have developed tools, industrial machines, and lately computer systems to help us in our pursuit of this goal. Some particular humans (specifically some transhumanists) believe that their purpose in life is to try and become better than human. In practice this usually means striving to live longer, to become more intelligent, healthier, more aware and more connected with others. The use of technology plays a key role in this ideology.

A second track of transhumanism is to facilitate and support improvement of machines in parallel to improvements in human quality of life. Many people argue that we have also already built complex computer programs which show a glimmer of autonomous intelligence, and that in the future we will be able to create computer programs that are equal to, or have a much greater level of intelligence than humans. Such an intelligent system will be able to self-improve, just as we humans identify gaps in our knowledge and try to fill them by going to school and by learning all we can from others. Our computer programs will soon be able to read Wikipedia and Google Books to learn, just like their creators.

A perfect scientist?

But the design of our computer programs can be much more efficient in places where we, as humans, are rather limited. They will not get ‘bored’ in mathematics classes. They will work for hours on end, with no exhaustion, no fatigue, no wandering thoughts or daydreams. There would be no need for such a system to take a 2-hour lunch break, to sleep, or to worry about where its next meal will come from. The programs will also be able to analyze data in many more interesting ways than a human could, perhaps becoming a super-scientist. These programs will be far greater workers, far greater scholars, perhaps far greater citizens, than we could ever be.

It will be useful in analyzing the way such a machine would think about the world by starting with an analysis of humans. Why do humans want to learn things? I believe it is because there is a reward for doing so. If we excel in various subjects, we can get good jobs, a good income, and time to spend with others. By learning about the way the world works and becoming more intelligent, we can make our lives more comfortable. We know that if we put in the hard work, eventually it will pay off. There seem to be reward mechanisms built into humans, causing us to go out and do things in the world, knowing that there will be a payoff. These mechanisms act at such a deep level that we just follow them on a day-to-day basis – we don’t often think about why they might be there. Where do these reward mechanisms come from? Let’s take an example:

Why do you go to work every day?
To make money?
To pay for the education of your children?
To socialize and exchange information with your peers?
To gain respect and status in your organization?
To win prizes, to achieve success and fame?

I believe that ALL these rewards – and in fact EVERY reward – can be tied back to a basic human instinct. And that is the instinct to survive. We all want to survive and live happily in the world, and we also want to ensure that our children and those we care about have a good chance of surviving in the world too. In order to do this, and as our society becomes more and more complex, we have to become more and more intelligent to find ways to survive, such as those in the list above. When you trace back through the reasoning behind each of these things, when you strip back the complex social and personal layers, the driving motivations for everything we do are very simple. They form a small collection of desires. Furthermore, each one of those desires is something we do to maximize our chance at survival in the world.

So all these complex reward mechanisms we find in society are built up around simple desires. What are those desires? Those desires are to eat, to find water, to sleep, to be warm and comfortable, to avoid pain, to procreate and to protect those in our close social group. Our intelligence has evolved over thousands of years to make us better and better at fulfilling these desires. Why? Because if we weren’t good at doing that, we wouldn’t be here! And we have found more and more intelligent ways of wrapping these desires in complex reward mechanisms. Why do we obfuscate the underlying motivations? In a world where all the other members of the species are trying to do the same thing, we must find more intelligent, more complex ways of fulfilling these desires, so that we can outdo our rivals. Some of the ways in which we go about satisfying basic desires have become very complex and clever indeed! But I hope that you can see through that veil of complexity, to see that our intelligence is intrinsically linked to our survival, and this link is manifested in the world as these desires, these reward mechanisms, those things that drive us.

Building intelligent machines

Now, after that little deviation into human desires, I shall return to the main track of this article! Remember earlier I talked about building machines (computer systems) that may become much more intelligent than we are in the future. As I mentioned, the belief that this is possible is a commonly held view. In fact, most people not only believe that this is possible, but that such systems will self-improve, learn, and boost their own intelligence SO QUICKLY that once they surpass human level understanding they will become the dominant species on the planet, and may well wipe us out in the process. Such scenarios are often portrayed in the plotlines of movies, such as ‘Terminator’, or ‘The Matrix’.

I’m going to argue against this. I’m going to argue that the idea of building something that can ‘self-improve’ in an unlimited fashion is flawed. I believe there to be a hole in the argument. That flaw is uncovered when we try to apply the above analysis of desires and rewards in humans to machine intelligences. And I hope now that the title of this article starts to make sense – recall the famous experiments done by Pavlov [1] in which a dog was conditioned to expect rewards when certain things happened in the world. Hence, we will now try to assess what happens when you try to condition artificial intelligences (computer programs) in a similar way.

In artificial intelligence, just as with humans, we find that the idea of reward crops up all the time. There is a field of artificial intelligence called reinforcement learning [2], which is the idea of teaching a computer program new tricks by giving it a reward each time it gets something right. How can you give a computer program a reward? Well, just as an example, you could have within a computer program a piece of code (a mathematical function) which tries to maximize a number. Each time the computer does something which is ‘good’, the number gets increased.

The computer program therefore tries to increase the number, so you can make the computer do ‘good things’ by allowing it to ‘add 1’ to its number every time it performs a useful action. So a computer can discover which things are ‘good’ and which things are ‘bad’ simply by seeing if the value of the number is increasing. In a way the computer is being ‘rewarded’ for a good job. One would write the code such that the program was also able to remember which actions helped to increase its number, so that it can take those actions again in the future. (I challenge you to try to think of a way to write a computer program which can learn and take useful actions but doesn’t use a ‘reward’ technique similar to this one. It’s actually quite hard.)

Even in our deepest theories of machine intelligence, the idea of reward comes up. There is a theoretical model of intelligence called AIXI, developed by Marcus Hutter [3], which is basically a mathematical model which describes a very general, theoretical way in which an intelligent piece of code can work. This model is highly abstract, and allows, for example, all possible combinations of computer program code snippets to be considered in the construction of an intelligent system. Because of this, it hasn’t actually ever been implemented in a real computer. But, also because of this, the model is very general, and captures a description of the most intelligent program that could possibly exist. Note that in order to try and build something that even approximates this model is way beyond our computing capability at the moment, but we are talking now about computer systems that may in the future may be much more powerful. Anyway, the interesting thing about this model is that one of the parameters is a term describing… you guessed it… REWARD.

Changing your own code

We, as humans, are clever enough to look at this model, to understand it, and see that there is a reward term in there. And if we can see it, then any computer system that is based on this highly intelligent model will certainly be able to understand this model, and see the reward term too. But – and here’s the catch – the computer system that we build based on this model has the ability to change its own code! (In fact it had to in order to become more intelligent than us in the first place, once it realized we were such lousy programmers and took over programming itself!)

So imagine a simple example – our case from earlier – where a computer gets an additional ‘1’ added to a numerical value for each good thing it does, and it tries to maximize the total by doing more good things. But if the computer program is clever enough, why can’t it just rewrite it’s own code and replace that piece of code that says ‘add 1’ with an ‘add 2’? Now the program gets twice the reward for every good thing that it does! And why stop at 2? Why not 3, or 4? Soon, the program will spend so much time thinking about adjusting its reward number that it will ignore the good task it was doing in the first place!
It seems that being intelligent enough to start modifying your own reward mechanisms is not necessarily a good thing!

But wait a minute, I said earlier that humans are intelligent. Don’t we have this same problem? Indeed, humans are intelligent. In fact, we are intelligent enough that in some ways we CAN analyze our own code. We can look at the way we are built, we can see all those things that I mentioned earlier – all those drives for food, warmth, sex. We too can see our own ‘reward function’. But the difference in humans is that we cannot change it. It is just too difficult! Our reward mechanisms are hard-coded by biology. They have evolved over millions of years to be locked into our genes, locked into the structure of the way our brains are wired. We can try to change them, perhaps by meditation or attending a motivational course. But in the end, biology always wins out. We always seem to have those basic needs.

All those things that I mentioned earlier that seem to limit humans – that seem to make us ‘inferior’ to that super-intelligent-scientist-machine we imagined – are there for a very good reason. They are what drive us to do everything we do. If we could change them, we’d be in exactly the same boat as the computer program. We’d be obsessed with changing our reward mechanisms to give us more reward rather than actually being driven to do things in the world in order to get that reward. And the ability to change our reward mechanisms is certainly NOT linked to survival! We quickly forget about all those things that are there for a reason, there to protect us and drive us to continue passing on our genes into the future.

So here’s the dilemna – we either hard code reward mechanisms into our computer programs – which means they can never be as intelligent as we are – they must never be able to see or adjust those reward mechanisms or change them. The second option is that we allow the programs full access to be able to adjust their own code, in which case they are in danger of becoming obsessed with changing their own reward function, and doing nothing else. This is why I refer to as humans being self-consistent – we see our own reward function but we do not have access to our own code. It is also the reason why I believe super-intelligent computer programs would not be self-consistent, because any system intelligent enough to understand itself AND change itself will no longer be driven to do useful things in the world and to continue improving itself.

In Conclusion:

In the case of humans, everything that we do that seems intelligent is part of a large, complex mechanism in which we are engaged to ensure our survival. This is so hardwired into us that we do not see it easily, and we certainly cannot change it very much. However, superintelligent computer programs are not limited in this way. They understand the way that they work, can change their own code, and are not limited by any particular reward mechanism. I argue that because of this fact, such entities are not self-consistent. In fact, if our superintelligent program has no hard-coded survival mechanism, it is more likely to switch itself off than to destroy the human race willfully.


As this analysis stands, it is a very simple argument, and of course there are many cases which are not covered here. But that does not mean they have been neglected! I hope to address some of these problems in subsequent posts, as including them here would make this article way too long.

[1] – Pavlov’s dog experiment –

[2] – Reinforcement Learning –

[3] – AIXI Model, M Hutter el el. –

A particularly bad attack on the Singularity

Whilst I am not a raging advocate of ‘sitting back and ‘waiting’ for the Singularity to happen (I prefer to get excited about the technologies that underlie the concept of it), I feel that I have a responsibility to defend the poor meme in the case where an argument against is is actually very wrong, such as in this article from Science Not Fiction:

Genomics Has Bad News For The Singularity

The basic argument that the article puts forward is that the cost of sequencing the human genome has fallen following a super-exponential trend over the past 10 years. And yet, we do not have amazing breakthroughs in drug treatment and designer therapies. So how could we expect to have “genuine artificial intelligence, self-replicating nanorobots, and human-machine hybrids” even though Moore’s law is ensuring that the cost of processing power is falling? And it is falling at a much slower rate than genome sequencing costs!

The article states:

“In less than a decade, genomics has shown that improvements in the cost and speed of a technology do not guarantee real-world applications or immediate paradigm shifts in how we live our day-to-day lives.”

I feel however, that the article is somewhat comparing apples and oranges. I have two arguments against the comparison:

The first is that sequencing the genome just gives us data. There’s no algorithmic component. We still have little idea of how most of the code is actually implemented in the making of an organism. We don’t have the protein algorithmics. It’s like having the source code for an AGI without a compiler. But we do have reasonable physical and algorithmic models for neurons (and even entire brains!), we just lack the computational power to simulate billions of neurons in a highly connected structure. We can simulate larger and larger neural networks as hardware increases in speed, connectivity, and efficiency. And given that the algorithm is ‘captured’ in the very structure of the neural net, the algorithm advances as the hardware improves. This is not the case in genome sequencing.

The second argument is that sequencing genomes is not a process that can be bootstrapped. The very process of knowing a genome sequence isn’t going to help us sequence genomes faster or help you engineer designer drugs. But building smart AI systems – or “genuine artificial intelligence” as the article states – CAN enable you to bootstrap the process, as you will have access to copyable capital for almost no cost: Intelligent machines which can be put to the task of designing more intelligent machines. If we can build AIs that pass a particular threshold in terms of being able to design improved algorithmic versions of themselves, why should this be limited by hardware requirements at all? Moore’s law really just gives us an upper bound on the resources necessary to build intelligent systems if we approach the problem using a brute-force method.

We still need people working on the algorithmic side of things in AI – just as we need people working on how genes are actually expressed and give rise to characteristics in organisms. But in the case of AI, we already have an existence proof for such an object – the human brain, and so even with no algorithmic advances, we should be able to build one in-silico. Applications for genomics do not have such a clearly defined goal based on something that exists naturally (though harnessing effetcs like the way in which cancer cells avoid apoptosis might be a good place to start).

I’d be interested in hearing people thoughts on this.

An adiabatic tragedy of advocates and sceptics

I think the recent musings around the blogosphere and beyond are completely missing a huge and fundamental point about why building AQC and AQO-based machines will be not only useful, but something that we will wonder how we lived without. I’m not going to talk about the specifics of implementing superconducting technology for cool applications (I can defer that to subsequent posts). Here I just want to explain a little about why we should do it, and how the main hurdle to progressing such technology is nothing to do with our ability to build and understand the technology itself. In fact, it is a rather sad story.

Let’s think for a second: Do we really have the right computational substrates for realising concepts such as highly connected neural networks, and thinking machines? Is the Von Neumann architecture really the best way to support such things? We are currently solving the problem of simulating intelligent systems by throwing more and more computational power at them. Though there’s something very odd about this, we have little choice, as we have become very good at designing processors that behave in a particular way, that talk to memory in a particular way, and that have a small number of cores. The problem with the commonly used architectures is that they just cannot embed things like neural nets, and other highly parallel structures, very efficiently.

Could adiabatic quantum processors be used for neural net and highly parallel processing purposes? Of course! The architectures are very compatible. AQO processors are very similar to Boltzmann machines, which underlie the operation of way many pattern-recognising systems, such as our own brain. There are other delicious fruits of superconducting logic, for example the ease with which we can implement reversible logic, and the exceedingly low power dissipation of such circuits. These systems also exhibit macroscopic quantum effects, which may be a great resource in computation, or it may not. But even if it does not, we should not ignore the fact that actually building such machines is the only way to answer this, and many other questions.

I think that superconducting processors are offering us a wonderful gift here, and yet we refuse to take advantage of it. Why? The reasons are a little hard to stomach.

It seems that while we’d rather just get going building some cool things, we end up spending a large proportion of our time and effort debating issues like whether or not quantum computers can solve NP-Complete problems in polynomial time. What?! I hear you say. My answer: Exactly. Obscure as they are, these questions are the most important thing in the world to a vanishingly small fraction of the population. Yet these seemingly lofty theoretical debates are casting a shadow over the use of superconducting QC architectures in every area of research, including things like the novel implementations of hardware-based neural nets, which may prove to be an even more fruitful avenue than optimization.

It will take a large amount of financing to commercialize general purpose superconducting processors, and an unimaginably large effort on behalf of the scientific researchers and engineers who devote their time to trying to progress this technology. The step from fundamental research to commercialisation cannot and will not work in an academic environment. Why? Because in order to fabricate integrated circuits of any kind you need investments of hundreds of millions of dollars into foundries. Robust and scalable technologies can only be realised in an industrial setting. Sure, small scale systems can be realised in many labs, but there are no industrial uses for devices produced in this setting: Anything that they demonstrate can be outperformed with a standard computing system. And it will stay that way until we realise that we need to take a risk as a society and really invest now in a technology of the future. I’ve seen distressing things happening at RSFQ roadmapping meetings. The conclusion of the roadmapping somewhat tragically boils down to ‘make another roadmap’ – because there are no applications beyond a few incredibly specialised circuits that can be realised within the scope of government research grants (~$10K-5M. Oh and by the way they aren’t very reproducible either). There is no funding on the necessary scale, and therefore whatever money is put into this distributed and broken field rather swiftly evaporates, even though if considered cumulatively it may have been enough to get a single, well focused effort going.

So, my plea to the academic world is to realise that there comes a point where quarrelling over things should be secondary to our solidarity as a community who really want to see something achieved. I’d rather try and fail than sit smugly in my ivory tower proclaiming that something is 20, or 50 years away (read – therefore not my problem guys!) Day after day people (with little or no knowledge in the field) exclaim that these things will never work, you are not building anything useful, your results are fraudulent, your devices are vapour-ware….etc etc.

This is more than cautious scientific scepticism, this is sheer hatred. It seems very unusual, and from my point of view, very detrimental to the entire field. I’d first advise such people to read some of our peer-reviewed papers to get their facts straight. I’d then say the following:

If those working in the fields of superconducting electronics, flux qubits, and quantum information really cared about the future of a technology that they had helped show had huge theoretical promise, they should rally along with us rather than spitefully burning their own bridges. As a community, we should promote the potential of our research and try to convince whoever needs convincing that these endeavours, these ‘Manhattan’ style projects are the only way in which we can bring disruptive technologies to fruition. And even if such projects fail, even if there is no other possible spin off that comes out of them, think about this: Such projects cost about the same amount as a couple of ICBMs. I know which I’d rather have more of in this world.

Responses to some post-singularity Physics comments

So the post I wrote on Post-Singularity Physics got linked a couple of times, and although people are very reluctant to comment on original source material these days, there were many many discussions over at io9.

Here are some points:

svenhoek likes guns: “A ceiling? Wow, I think we have heard this kind of speak all through history. People that say this always are made to look stupid by history. plus, there will still be scientists needed to analyze the data and figures the machines bring up.”

But why shouldn’t there be a ceiling, just because there hasn’t been one up until now? 😉 To let someone else do the work for me, I think that this comment summarises nicely how I feel about this ‘ceiling’ argument:

Derek Pegritz: “I think one of the reasons there hasn’t been a major breakthrough in theoretical physics since the early days of quantum mechanics is simply that the baseline human mind is no longer capable of crunching the multidimensional data required to view the universe in its totality. As we are now, humans are perfectly capable of observing, analyzing, and describing the way in which the 4-dimensional submanifold our minds exist in primarily. Once you start adding dimensions and reduce physics to dealing with incredibly fine, emphemeral particle and field transactions, the 1.0 human mind is simply cannot envision the higher orders of logic which may obtain. This is not to say that we can’t see and understand certain *glimpses* of higher-order multidimensional physics–string theory, M theory, et. al. indicate that we can certainly comprehend it to *some* degree–but to really get to the heart of post-quantum physics, we’ll probably need a measure of intelligence amplification. Combining a machine-mind’s speed and data-crunching capacity with the human brain’s excellent pattern-matching faculties could very well lead to a true unified TOE.”

(There were several other comments along this line)
I do agree that humans mostly work by analogy and building up a mental picture of the world to whatever level of detail is necessary for us to survive. So the argument goes that we just aren’t evolved to be able to make logical arguments in N-dimensions – it doesn’t come naturally to us, so we use tools to help, but we are still at a disadvantage in that the links are no longer ‘obvious’ to us. How often have you heard the phrase ‘quantum mechanics is weird’ or ‘spooky’ – even Physicists don’t like QM because it is non-intuitive (it is also incomplete, but let’s not go there – it’s pretty good, and a better theory that closed some of those loopholes probably wouldn’t look *much* more ‘intuitive’ to us.)

Makidian: “I feel like only part of this will ever be true. I don’t think physicists will ever want to give up the aspect of their job that causes them to ask fundamental questions of everything. It would counterproductive and against what they have been working for to just have a computer do it for them. These futurists are so in love witht the idea of the singularity that they just start posing theories without considering that humans may want to do it and still work out the answer for themselves.
I’m all about making a better human to a certain degree, but not at the cost of losing my own humanity because isn’t that really the point!?”

I think the point here is not that they wouldn’t still want to do their job, but that that they would HAVE to give it up if something could do it much better than them, as no-one would pay them (at a higher rate) to continue. Physicists don’t sit around pondering the great problems of the Universe for fun – most do it to earn a living. At the least they would have to relegate it to a hobby (in the same way that people still build radios from discrete components). If machines do something better than people, they will be replaced, unless something about our global socio-economic system changes radically in the next few decades (which I won’t rule out completely). This is also assuming that human-physicists remain the same throughout this entire process, which I also do not believe will happen. We are already starting to augment ourselves as a community (how many of us read arXiv on smart-phones on the train in the morning?) There are many ways in which we can continue this trend, hence my point in the original post about experimentalists possible undergoing a human-machine merger just before their jobs get stolen 😉

artum: “Enough of this Singularity BS. It’s just another get-immortal-quick scheme for morons. Also research of any kind like this would be a horrible idea because there’d be no one to confirm it as with ordinary science, only other science-bots”

This comment made me smile. Gotta love that bio-Luddite ad-hominem attack touch. Anyway, I think that the machines would very much enjoy sharing the information with each other, if their hardwired goal was to discover a model with which to predict the behaviour of the Universe, it would make them very happy! Much happier than it would humans, who tend to think: ‘Damn, I’ve been scooped again by my competitor’.

RandomThought: “As for the rest of your article, I personally have come to the opinion, that in general more stays the same than changes. People have always thought they were heading towards the big everything-changing event and somehow life mostly just goes on. “

The Singularity meme is a double edged sword. It gives people an idea of what I am talking about without me having to describe it in minute detail, but then again I have to accept all the baggage that comes along with it. So yes, I admit, I was being lazy, and I wasn’t necessarily referring to the Singularity in all its ‘wonderful glory’ but just rather the part where AI technologies start to become better at doing Physics than we are. Personally, this really does radically affect my life, seeing as I am a professional Physicist for a living 😉

Just in case you missed the links:
Original post
Discussion over at io9

Life Logging – an urge to create a sparse but useful dataset?

I have a strange urge and desire to life-log, which I am unable to explain. Since I was very young, I have always kept a journal of some form or another. More recently I have moved to a digital journal format.

In my very young days, I would keep a diary because I was told to. Later, in high single-digit and early teen years I would keep a diary because it was somewhere I could write private thoughts, fulfilling the role of what some might have thought of as an ‘imaginary friend’ – I very much talked to my journal in a ‘dear diary’ style, as though it understood my concerns about the world.

My recent reasons for logging have generally been because I’m very busy, I’m enjoying life, and I’m doing a lot of things. I find it great to read back over my journal entries and relive the experiences. I especially like comparing the anticipation of an event with the memories of how it went and what I learnt from it. It really reinforces the idea of events which you may be nervous about never being as bad as you expect. It can be a really insightful thing to do.

Last weekend I spent a lot of time scanning old photographs into digital format. It’s amazing how each photograph opens up an entire set of memories, thoughts, and feelings. I’m also scanning my entire back archive of paper artwork (hundreds of a4 and a5 images). I like the idea of having all this stuff in a digital format such that it may eventually be tagged and have a proper semantic referencing system, when an appropriate framework for this kind of thing is developed.

However, I have a slightly more practical (and somewhat more controversial) reason for lifelogging, which I would like to explore in the next few years (or maybe decades).

Creating an upload from an extended lifelog

I like the idea of creating an AI that could take all this data and infer things from it. It could perhaps infer what kind of a person I was, and what kind of a person I am now. It might be a useful dataset for an AGI trying to understand human development, or developing itself.

An even more interesting idea is to create a virtual version of yourself by giving it access to all this information and a timeline. (You’d effectively be giving it memories).

One currently in vogue lifelogging technique is recording your entire set of experiences using an on-person video camera with built-in audio. However I feel that this method has its flaws. The stream obviously only records external input. You would ideally have a technique which also monitors streams such as internal reasoning, understanding, feelings and personal thoughts. Some of this could be automatically recorded using secondary effects, for example heart rate, hormone levels, blood sugar levels. But even those techniques just can’t capture that oh-so-elusive personal subjective experience.

Journal keeping is one way to do get around this problem, but you have to learn to write your journal in a very specific way. So something like “I listened to some music today” would be pretty information lean, whereas “I listened to song X today and it made me feel rather melancholy because it reminded me of the time when I first heard it, I was doing Y, and that inspired me to draw this piece of artwork Z. Now everytime I hear that piece of music I’m inspired to create more artwork”. In addition, I think that tagging stuff will be easier in text and image formats than in a video stream.

A dream diary can also contribute to the dataset, as it could give an AI more data about how the subjective experience during sleep can be different to normal.

In short, there’s no way to create an exhaustive dataset, but a sparse one may still be useful. I guess I’ll continue doing it as long as I find it fun.

AGI-10 Monday

Last day of the AGI-10 conference. As usual the live-blogging attempt failed 🙂 but that was kind of to-be expected. I blame the tiny netbook keyboard, which makes it very hard to type. Additionally, I found myself taking quite a lot of notes.

So what have I learnt from this conference? (I’ll probably go into all these ideas in much more detail in subsequent posts, but for now I’m just putting down some thoughts.)

* AGI is a young field, with many disputes and disagreements, which makes the conference both interesting and useful.

* People seem very passionate about the subject, which manifests both as optimism about the field and fierce debates over the problems anticipated, and already being encountered.

* There is a wide range of people here with very diverse backgrounds. I’ve spoken to computer scientists, physicists, mathematicians, philoshophers, neuroscientists, software programmers, entrepreneurs, and many others.

* There is an interesting split between the theoretical (understanding, defining and bounding what AGI is) and the experimental (building candidate systems). It actually strikes me as being similar to the QIP community, except QIP has had about 20 extra years for the theory to race ahead of the experimental verification. I worry that the same might happen to AGI.

* There is another split, which is a bit more subtle, between those that believe that bio/brain inspired investigation can help push AGI forward, and those that believe it won’t – or even worse, that it might cause the field to go backward, by ‘distracting’ researchers who would be working on other potential areas.

* The major problem is that people still can’t agree on a definition of intelligence, or even if there is, or can be one.

* There is also a problem in that the people actually trying to build systems do not know what cognitive architectures will support full AGI, so lots of people are trying lots of different architectures, basically ‘stamp collecting’, until more rigorous theories of cognitive architecture emerge. Some (most) of the current architectures that are being used are bio-inspired.

* There were a few presentations that I thought were much closer to narrow AI than AGI, especially on the more practical side. I guess this is to be expected, but I didn’t get the feeling that the generalization of these techniques was being pursued with vigour.

Quantum brains

I’m going to talk about quantum brains. But before I do, I have to take a bit of a philosophical detour. So bear with me and we’ll get onto the meaty quantum bits (qubits?) soon.

Disclaimer 1: This is a very general introduction article – it is probably not suitable for QIP scientists who may attempt to dispose of me (probably with giant lasers) for lack of scientific rigor…. *ducks to avoid flying qubits*

We need to think about what we are trying to build. Say we want to build a brain (in silicon, for arguments sake). Well, for a start that’s not really enough information to get on with the task. What we actually want is a mind in a box. We want it to think, and do human-like things. So we run into a problem here because the mind is a pretty vague and fuzzy concept. So for the purpose of this argument, I’m going to use Penrose’s definition of 4 viewpoints of how the mind might be connected to the physical brain, which is given in his book Shadows of the Mind, but I will summarise here for those who are not familiar with the definitions:

There are basically 4 different ways you can interpret the way the mind is related to the actual signals buzzing around and the physics going on in that wet, squishy 3lb lump that sits in your skull. Here they are:

(A) – The ‘mind’ just comes about through electro-chemical signals in the brain. You could fully reproduce a ‘mind’ in any substrate using standard computer providing you could encode and simulate these signals accurately enough. It would think and be conscious and self-aware in exactly the same way as a human being.

(B) – The workings of the brain can be simulated in exactly the same way as in (A) but it would never be conscious or have self-awareness, it would just be a bunch of signals that ‘seemed’ to be behaving like a human from the outside. It would effectively be a zombie, there would be no ‘mind’ arising from it at all.

(C) – There’s no way you can simulate a mind with a standard computer because there’s some science going on that creates the ‘mind’ that we don’t yet know about (but we might discover it in the future).

(D) – There’s no way you can ever simulate a mind because our minds exist outside the realm of physical science. Period. Even that science which we are yet to discover. (This is a somewhat mystical / spiritual / religious argument).

Interestingly Penrose goes for C – mainly because he believes that there are quantum processes occurring in the brain, and the quantum mechanics going on in there cannot be simulated using a conventional computer. So it’s not that we don’t understand the science yet, but we can’t build computers that are able to take that science into account (i.e. model the quantum mechanics correctly). Or can we… don’t we have, like quantum computers now?

Now back to the quantum braaains…

What do I think is the most exciting prospect for quantum computers? Forget factoring, what about building quantum brains? Note: I’m using the phrase ‘brain’ here in a rather unscientific sense to mean a large collection of interconnected agents – essentially a large neural network.

I am a supporter of (A) – which is a variant of the Strong AI hypothesis. That is, a human-level intelligence could be fully simulated on an alternate substrate using a standard, ‘classical’ computer and actually BE conscious and self-aware. However, with this point of view, one might wonder what a similar level of integration would be capable of if it could use some aspects of quantum mechanics as an integral part of its operation.

My viewpoint conveniently makes my argument for the further development of QCs pretty watertight. If quantum computers ARE required to simulate the human brain, (which I do not believe to be the case), then we should probably develop them anyway. If they are NOT required, but are believed (at least by some) to be fundamentally more efficient for certain computational tasks, then wouldn’t it be a cool experiment to make a brain which could harness that extra computational power? I mean… it would be a fundamentally different type of intelligence. Doesn’t that sound cool? Doesn’t that just make you smile and make the hairs on the back of your neck stand on end? Or maybe that’s just me…

Attentive readers may note that I have subtley disregarded option D here. That’s because D stands for Deepak Chopra, who is much better at explaining how QM ties in with that viewpoint than I am.

Quantum Neural Networks have already been explored theoretically. (See here, here, here for just a taste). I think very small QNNs could be realised experimentally at present. If they can be shown to work in principle, they can be scaled up and investigated further.

Adiabatic Quantum Systems based on the Ising model are perfect for this task. Their structure and behaviour resembles a spin-glass, which is mathematically equivalent to certain types of neural network. A spin glass can store patterns in ‘stable’ configurations of spins, just as the brain stores memories as patterns in configurations of the synaptic strengths between neurons (a simplistic model but it’s kinda the main point).

Of course there’s always the problem of decoherence – and it most likely will be a problem in large scale quantum systems. There’s probably some puddles of coherence around the place, maybe they overlap, maybe they don’t. No-one really knows. Could those puddles of local coherence provide any extra computational power? How connected (or perhaps disconnected) would they have to be? Can we design scalable solid state systems with larger puddles?

Again, that sounds to me like something we should investigate.

In conclusion

We should be able to simulate anything that the brain is doing (even if we need quantum computers). If the brain IS using large scale coherence in its operation, it shows us that it IS possible to build large scale coherent quantum systems (if nature can do it then so can we). This would be useful for all sorts of things, like simulating protein folding. In fact this would arguable be the best outcome. I kinda hope Roger Penrose is right…

However, I don’t believe he is right, as I currently believe the level of large-scale quantum coherent phenomena in the brain is very close to ZERO. But that means we can only IMPROVE the level by which quantum mechanics could be leveraged in brain-like systems, by building huge and densely connected NNs using quantum devices such as superconducting qubits. We can explore completely new territory in the building of intelligent systems…

Thus we have a win-win situation 🙂

In other words, QCs are cool and we should build them.
And we need more money *ahem*

Note: I argue this and a bunch of other stuff in my QC & AI lecture. Here is the link to my post about that

Disclaimer 2: This topic has also probably been debated to death and back on various places around the internet but it’s always good to exhume it once more for a guest appearance. In fact if I wasn’t feeling so lazy (and cold, the heating in here appears to be broken at the moment) I might have bothered to dig up some references. It’s also a useful place to send people to if they want to know my point of view on this.

EDIT: To perfectly illustrate both my points that a.) there’s loads of stuff on the internet + I’m lazy and b.) software systems are surprisingly intelligent already (WordPress helpfully pointed out the link for me) here’s some stuff that Geordie wrote about this a while ago:

Can an artificial general intelligence arise from a purely classical software system?

Media coverage, skew, and general not-in-the-Christmas-spirit ranting

Bah, humbug.

A little earlier I nonchalantly and lazily microblogged using the oh-so-professional channel of my Facebook status about something that slightly irritated me whilst watching the news. Here was what I wrote:

One attempted act of terrorism: Almost 24/7 news coverage. Thousands of scientific breakthroughs every day: Not even a few minutes of airtime. No wonder the general public feel isolated from science, in fear of technological advancement and generally depressed at the state of the world…

Anyway, seeing as the comment sparked quite an interesting conversation, I thought I’d relay these thoughts on this, my slightly more traditional soap-box haunt.

Obviously my comment pertained to the recent attempt to bring down a plane travelling from Amsterdam to Detroit. How much about this could there possibly be to report? Hours worth of television time, apparently.

It just depresses me that the news programs, day after day, cover crime, terrorism, war and political unrest as their main stories, then attempt to lighten this doom-riddled cake of hopelessness with a cherry in the form of a ridiculous human factor story about some family’s cat being rescued from a tree (or something equally banal). Earlier today I watched a story about freak weather in the UK – the seemingly important aspect of this being a woman who gave birth in an ambulance because it was stuck in the snow on a sliproad on the A14. Oh, and then there’s 10 minutes of football news!! If I were sat at a desk I would somewhat non-figuratively be slamming my head against it at this point. Luckily for me I’m on a sofa.

I’m not just going to have a long rant here about what I think the news SHOULD be covering, I think that much is pretty obvious from my list of interests ~superconducting flux qubits, yey!~ <-excuse the voices in my head.
Moreover, I'm wondering: What can we actually DO about it?

During my time in a University setting, I think I've only once been invited to a seminar by a person from the Press (in this case it was Radio 4). I think that the press just do not engage enough with scientists. There is so much cool research just waiting to be explored and popularised. There is also a large body of enthusiastic, young PhD students who would be willing to talk about what they are doing, which would not only help popularise their respective subjects, but also break that 'mad professor / scientist' stereotype which seems to still be hanging around a couple hundred years after it was actually representative.

However, until the 'media approaching scientists' kind of thing reaches a critical mass, it will be up to the scientists to shout louder and more ingeniously to make people take note. It will be up to them to chase down the media opportunities.

And I believe that there is a somewhat insidious problem here – primarily that it's not considered a worthwhile activity amongst science/engineering peers to communicate and popularise your research. Carl Sagan et al. are very much the exception rather than the rule. Popularisation is certainly not taught alongside science, or encouraged, even though there are lots of external grants available for this kind of thing. We rely on the rebellious defector amongst academics to propagate the enthusiasm. Instead, we should be supporting those who wish to act as spokespeople for their research.

A lot of media types are also looking for the ‘scare story’ angle. They will try every trick on the book to hype the negative angle of your research, especially of you are working in disruptive or controversial technology areas. I think that academics should be trained to answer media questions a bit like politicians: Get across the positive impact at all costs (short of actually, you know, lying..). And for goodness sake don’t mention Skynet. Or nano greygoo*ahem* tech. It’s ‘submicron’ or ‘molecular’ engineering, guys!

Ultimately, the goverment hands out the money to the funding councils, and therefore if we as scientists aren’t in the forefront of their minds (I mean look, we’re competing with the NHS, the education system, the war in Afghanistan, etc….), if they don’t see our science and go “Wow, you know – that’s not only the future of our country, but it’s actually pretty interesting too”, then funding for our research will indeed be cut.

So go out there and tell people that your research, be it Physics or whatever, is awesome. In any way you can. Ignore anyone who tells you that it isn’t worthwhile. And if you do it right – if you do it *really* right – then those people might just mention it to their friends the next time they have tea and cake 😉


Note: On this topic with a slightly more transhumanist slant, inspired by the conversation, Stuart has also written a blogpost.

Merry Christmas by the way people.

The low hanging fruit of the ivory towers

For the last couple of days I’ve been on a business course for researchers. We’ve been putting the Acumen into Academic, sorting the Dragons from the Deadwood, and converting PhD into USP. We’ve been taught networking skills, including how to analyse and survey a room full of people, the kind of groups that people form and why, how to handshake properly and how to say our names properly amongst other amusing team activities.

We also had workshop sessions on IP, the patent system, commercializing research, licensing, spin-off companies, and we had several presentations on case studies of companies (mainly with a biotech theme). The course of patents was incredibly interesting, it taught me not to give away any really good ideas at conferences etc., as the information would then be in the public domain and impossible to patent.

Is it important for academics to learn about business? I would say yes. I think it’s a great idea to see your research being put into application and practice. It’s also good to talk to people from industry, they have a vastly different point of view which can make you reconsider your work in a new light and with a more open mind. I’d recommend this kind of course to anyone. Even if the material hadn’t been up to scratch (which was not the case here), I met some very interesting people and made useful contacts with people of a similar mindset but working in diverse disciplines.

Physics is a little more difficult to commercialize than biotech and chemical engineering, but there are still opportunities there and we shouldn’t let them slip by because of some ancient ivory tower stigma.

By the way I’m now in New York but more on that later…. 🙂