The Robot Artist Project

I’ve been interested in art for as long as I can remember. But lately I have been also extremely interested in robotics. And yet another interest I have is in AGI (artificial general intelligence) and specifically machine creativity. So I’ve been trying to combine these interests in a fun and unique project. As you do.

This project isn’t very far along yet, but I thought I’d start talking about it anyway… I have several projects on the go and talking about them will help me keep working on them! I’m going to separate projects with different logos and tags, so people can easily find them.

Background:

One of the things that we (and by we I mean the Artificial Intelligence/Machine Learning communities in general) are not so very good at is demonstrating cool algorithmic results in a way that people can understand and connect to. For example, this:

L0_sparse_coding_eqn

Is pretty exciting if you’re presenting at a machine learning conference. However, this:

robot_awesomeness

Is more exciting (or at least thought provoking) for nearly everyone else.

At work we’d been musing for a while over the idea that we needed to craft more cool demos. And so I made a little deal with 2 of my co-workers that we’d each try to come up with a cool, creative, robot-based project to work on in our spare time, power it with some serious AI algorithms, and pitch the resulting entities against one another to achieve the highest level of nerdy robot coolness.

The challenge: Build a creative robot demo powered by an advanced machine Learning algorithm (in your spare time).

I code-named my attempt The Robot Artist Project. The robot I would like to build would be an embodied cognitive system with a ‘savant-like’ ability in the graphic arts, specifically canvas painting (more on this later). Once I had decided I liked the idea of building a robot artist, I had a look around the internet for drawing and painting robots. You can get quite a long way towards your goal by learning from successes and mistakes made in previous attempts to do similar things. I didn’t find that many socially-challenged but creatively gifted savant painting robots in my search, but this project stood out:

The thing I like especially about this is the artist’s story and reasoning behind building the robot – he suddenly had an overwhelming feeling that he could not produce art anymore, had lost his creativity, and needed to create something to do the work for him. I sympathize with this artist’s viewpoint. He is a troubled, angst-ridden creative soul and I can relate to that. Personally, my desire to build artist robots comes from slightly different reasoning. I have a strong negative reaction to anthropocentrism, and as such I strive to build systems and objects that blur the boundary between humans and other entities. Through building intelligent systems, I wish to unveil some of the contradictions evident in people’s kin-selecting, anthropocentric belief systems. Anyway, that’s veering more into the philosophy of AGI, and I came here to talk about robots. So, back to the build!

I wanted to get to something that drew stuff on paper AS FAST AS POSSIBLE, so I didn’t really care how it looked or to what MacGyverish extent I had to take things to get the robot drawing.

Hardware:

I had a couple of servos lying around, some lego mindstorms NXT pieces and an Arduino, which I purchased last summer. The first thing I did was connect the servos to a makeshift frame up using some pieces of Lego Mindstorms. Interestingly the NXT brick wasn’t used for any active control; but it did prove useful as a weight to anchor the robot arm! I also used some custom parts (picture hanging hooks) to attach the lego to the servo-rotor. I ended up with two position-controllable joints in the system, one for the arm of the artist (moving the hand in an arc) and one which lifted the pen up and down (A bit like turtle http://en.wikipedia.org/wiki/Turtle_graphics)

robot_artist_project_1

Software:

I used the servo sketch in my Sparkfun electronics kit as a template to get the Arduino to control the servo. Next I needed to get the whole thing hooked up to my PC to enable higher level control from Python, so I installed pySerial in order to send simple serial commands over USB to the Arduino. The Arduino listens to the serial ports, and adjusts the position of the servo based on what Python sends to that port.

Getting the loop delays sorted out was a little fiddley. When controlling servos in this way, I found that you have to carefully balance two delays, an inner delay (which sets the duty cycle of the Pulse Width Modulated signal the servo interprets) and an outer delay which gives the servo enough time to reach the desired position before trying to move to the next position.

robot_artist_project_2

Testing:

The first thing the robot did was manoeuvre the pen off the edge of the paper and onto the surface of my table. Supposedly a dry-wipe board pen, it kind of had the effect of a permanent marker. I chastised the robot; frowned at the table-marks. Then I thought how awesome it was that I had created a robot artist and the first thing it had done was scribbled angstily on my furniture. Sweet.

Here are some results from the first tests.

robot_artist_project_3

robot_artist_project_4

I knew that the little proto-artist would be dissembled and used for other builds in the future; it was way too hacked together to be kept intact. But it was fun to play with and I was inspired to continue with this project. There’s no machine learning or creativity in there yet, but there will be soon…

(Image credits: Shutterstock 114077734)

Gothic Fall – English edition available from Heavy Metal

For something a bit different; some light relief from AGI, quantum computing and whatnot…

The new edition of my Gothic Fall book is now available!

For those who have no idea what I am talking about, let me explain:
One of my main hobbies is fantasy and dark art (which is mainly in the form of digital painting, although I do paint with acrylics too) and I have published a book of these works along with accompanying poetry. The first edition of the book was published in Spanish, but now the English version is available! Quote from the site:

“If you never leave home without wearing black you will enjoy this potent combination of black magic, graveyards, knee high boots, chains, candelabras, wings, cathedrals and masks. Suzanne Gildert is a UK based artist, specialising in fantasy, gothic and dark art. She combines traditional drawing and painting methods with more modern digital techniques to establish a unique blend of elements and an easily recognisable style.”

Here is the link to the book:
Buy GF book online!

I’m really happy that this edition has finally been released. it has given me a new motivation do do more artwork 🙂 My next challenge is to get the book on the shelves in Chapters!

Enjoy!

BANG! The Universe Verse

I was asked to review this rather cute book:

BANG! The Universe Verse (Book I). The book is a portrayal of how the laws of Physics as we know them today arose in the short period of time after the Big Bang. The book also explains how matter forms, and how nuclear fusion and stellar activity plays a significant role in explaining why the Universe appears as it does at present.

But the cool thing about the book is that is is presented in a comic book format, with two cute characters guiding you through the science. Here is an excerpt:

“The proton in the centre may not be alone
As another has access to this VIP Zone
The neutron may not be quite as attractive
But it is quiet, well mannered, and rarely reactive”

This would be great to read to kids 🙂

You can read the PDF version online or support the author and buy the book.

Another amazing video

Another stunningly beautiful video visually describing our place in the universe.

The bit that struck me the most was how small our sphere of entire radio wave transmission is, compared to the size of the galaxy. Pretty obvious when you think about it but putting it into this visualisation really helps to grasp the sense of scale.

I love the mapping of all the earth’s satellites too.

Science visualisations

This is very cool:


Bio Visions – Harvard

Check out the video ‘The inner life of the cell’.

I probably learnt more about biology in watching this 8 minute video than I would have done in a whole university module. It’s a great project; we need more like it. It got me thinking about visual learning.

Humans seem to be primarily visual animals, and as such we can learn a lot from watching a good visualisation of something. When I read a text book or try to understand a new concept in Physics, I try to mentally imagine a model of the world. I’m using analogies and manipulations of 3-dimensional abstract objects, but I’m always painting a picture. I wonder if having that picture painted for us can help us learn faster.

One could argue that different people learn by different methods, which I’m sure they do. Some people are audio, textual or mathematical learners. But I feel that the standard, classical lecture-style learning in which we are taught most of science is probably also the most inefficient way of getting the information from A to B (i.e. it resonates strongly with very few learners). I think a visual method may not suit everyone, but it would resonate more strongly with a larger number of learners.

In physics, I’ve seen a few 3d visualisations of solid state phenomena, but they certainly aren’t commonplace. I really think that visualisations can help with understanding some pretty abstract concepts, such as the transition of electrons from a classical ‘wave packet’ or ‘particle’ description to a macroscopic quantum wavefunction during a condensation through the critical temperature. Try explaining that to someone who has never encountered any of the concepts. Now imagine that you could project the little video that you run inside your head everytime you visualise the process, and talk through that instead. I’m pretty sure it would make explaining things a lot easier.

There’s a slightly subtle point here: Does the very building of your OWN internal visualisation of an effect help you to truly understand it? If you were just shown someone else’s idea you might not have shared the same thought process and building of the model to get to the final ‘understanding’ stage. Could it even worsen the situation, rendering you biased against developing your own different yet valid visual interpretation?

Is taking away the extra stage of building a world model through internal visualisation really something that would deprive people of insight and deep understanding? Or would that insight just come about all the more quickly?

I’m surprised that the traditional style of lecturing has perservered for so long. One might have thought that powerful and beautiful visualisations of Physics would have pervaded our attempts at conveying a model of the world to students and colleagues.

I think that two hurdles stand in the way.

1.) Existing teachers and lecturers are not accustomed to using new techniques ‘The way you learnt is the way you teach’.

2.) There is an activation barrier to using these new technlogies because they are pretty hard to get going in the first place.

In a subsequent post I might discuss possible ways in which we can do something about this instead of just going off on a complete brain-ramble which is what I seem to have done here 🙂

The obligatory book post

You all knew it was coming, so here it is:
My art book is now available… which means I’m a published artist, and one step up from a starving artist 🙂

This version is in Spanish, I’m working on getting an English version published but that may take a little while. But the pretty pictures really are the main focus of the book, the text is just an accompaniment, and there are translations available on my website.

I apologise to those readers who have already been inundated with GF paraphernalia.

You can buy the book on Amazon:
Gothic Fall – S Gildert AMAZON

or direct from the publisher (much cheaper):
Gothic Fall – S Gildert NORMA EDITORIAL

Steampunk fun

Cool. I love steampunk stuff. Especially as quite a lot of the random junk equipment in my lab looks vaguely like it could belong in this genre. So here’s an exhibition you can go and see in Oxford:

Tech Know: Fast forward to the past

“The growing number of artists and amateurs who have built steampunk devices has led the Oxford Museum of the History of Science to mount an exhibition of them. The show runs until February 2010.

Browse the exhibits and spend time with steampunks and it becomes obvious how to spot members of that distinguished breed – they are the ones with the swagger and buckets of style”

Here are some more Pretty pictures

There’s also a blog:
Steampunk Art @ Oxford, The Museum of the History of Science, Oxford University

Learning to draw by understanding human vision processing

I’m feeling in an artsey mood pending the release of my ART BOOK <- Yes that is a shameless plug. So I got thinking about what our brain actually does when we try to draw things.

Specifically, why is drawing lifelife objects difficult, especially from memory?

I like to draw, and I often come across people who say “I wish I had that talent/skill”. I personally don’t believe it is an instrinsic talent, I think that anyone can learn to draw with the appropriate tuition. For the most part it involves following a fairly simple set of rules. Deciding what to draw is the ‘creative’ part 🙂 When I was originally interested in drawing, I was told by teachers to always ‘draw what you see’. This is an interesting statement, as what we see and how we interpret the information are two very different things.

Try to draw a face without looking at a reference. You’ll get it wrong. I suspect there are only a handful of people that can draw realistic faces without a reference (or without some rule of thumb/mathematically based guidelines). I personally tried to perfect this skill for years, before I actually realised that it was damn near impossible. And it wasn’t a lack of general artistic ‘skill’ – I could draw almost perfectly from a reference.

The problems associated with drawing from memory are to do with our hard-wired vision processing architecture. Our brain interprets data by using heavy compression algorithms. Lines are built up into shapes, shapes into blocks of shapes commonly found together, and these blocks of shapes along with colour information form concepts. Each level of complexity is recognised at a different level in the cortical hieracy, and each level is a compressed version of concepts from the previous one. In other words, our brain works by vector image processing.

So when we hold an ‘idea’ of a picture in our mind, it actually looks nothing like the real-world version. It is very interesting to watch children try to draw – they construct everything out of lines and basic shapes, because this is how the concepts of those objects are stored in the brain.

So how can we break this instinctive behaviour in a bid to become more artistically fluent?

I think that it requires turning off your instinctive mind’s-eye view of an object. There are several ‘tips’ that one can follow to attempt to bypass the brain’s heavy image processing. Most of these are things that each artist personally rediscovers by trial and error over many years.

1.) Draw based on tonality, not shape.
If you see an area of shadow, shade in this region. Try to view the subject as blocks of colour, light and dark areas, not as a predefined ‘object’ such as a bowl of fruit or a face.

2.) Never draw lines!
Lines hardly ever exist in real life. They are created by your brain as a way of compressing information about a step change in brightness, saturation or colour between two areas. If you simply must draw lines, draw them faintly and then come back to it later and match the tone of the line with a filled (crosshatched/shaded, whatever) area on one side.

3.) Turn the reference image upside down.
This technique works amazingly well. Your immediately kill some of the recognition pathways of the brain just by inverting the object. Your brain doesn’t store patterns (concepts) like ‘an upside down horse’ because we don’t often see upside down horses.

4.) Use a ‘window’ referencing technique
Cut a window out of a piece of paper and draw from the reference only what you see through the window (works best with a printed reference image)

5.) Flip the image
If you are working digitally, flip (mirror about x co-ordinate) both your image and the reference image quite often. You often spot mistakes immediately, and this prevents you from becoming ‘accustomed’ to any major errors in perspective/anatomy etc in your piece. You can do the same thing non-digitally as a check if the image is on a single side of paper – by holding it up to the light or using a lightbox. (You can’t edit the other side though, this is a read-only technique!). Interestingly this method works in somewhat of an opposite way to 3.) because the flipped image is (usually) still consistent with reality, therefore you actually use your brain’s ability to pattern match to spot the inconsistencies very quickly, as opposed to reducing the pattern matching it is able to perform by making the image non-realistic.

6.) Observe the image from a greater distance
This allows you to spot certain compositional errors without your brain automatically finding the focal point of the image and ‘ignoring’ the rest, as it does if you are close enough to see the detail.

I guess the GOLDEN rule is to always use references where possible. I had an issue with this for many years as it seemed like ‘cheating’. It’s not cheating, it’s just that our internal references don’t look anything like reality, so if you want to draw something realistically, don’t rely on your brain! (In fact save it for the creative part).

Out of interest my book is now available on Amazon. N.B: This first publication run is IN SPANISH. For translations of the writings accompanying the artwork, you’ll have to visit my website until I get the English version released. The prices and availability also seems to have screwed up a bit. I think these are just teething troubles. I’ll probably put a dedicated post about it when everything has actually been sorted out properly 😀